Skip to content

perf(cargo-anvil): parallelize Miri test artifacts - #123

Open
st-dev-gh wants to merge 16 commits into
mainfrom
u/st-dev-gh/miri
Open

perf(cargo-anvil): parallelize Miri test artifacts#123
st-dev-gh wants to merge 16 commits into
mainfrom
u/st-dev-gh/miri

Conversation

@st-dev-gh

@st-dev-gh st-dev-gh commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Compile each selected Miri profile once, preserving workspace feature unification.
  • Run independent test artifacts concurrently through cargo-miri runner, defaulting to one worker per logical processor with an ANVIL_MIRI_JOBS override.
  • Add package-level Miri exclusions, grouped GitHub and Azure DevOps output, and aggregated artifact failures.
  • Apply the updated cargo-anvil templates to this repository.

This brings the generic Miri orchestration developed in microsoft/oxidizer#706 into cargo-anvil. Repository-specific workload reductions and artifact priorities remain outside cargo-anvil.

Behavior

The four scheduled Miri profiles remain sequential. Within each profile, Cargo compiles the selected scope once with --no-run, and only Cargo artifacts marked profile.test = true are executed concurrently.

Packages can exclude their own test targets while remaining available as dependencies:

[package.metadata.anvil.miri]
exclude = true

ANVIL_MIRI_JOBS accepts a positive integer override.

Validation

  • Full cargo-anvil test suite
  • Repository spellcheck
  • Byte-exact generated-output snapshots for local, GitHub, and Azure DevOps backends
  • Clean cargo anvil --dry-run after applying the templates
  • Real Miri smoke run with two concurrent test artifacts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR upgrades the cargo-anvil Miri check implementation to compile the selected workspace scope once and then execute discovered profile.test = true libtest artifacts concurrently via cargo-miri runner, with deterministic log replay and aggregated failures. It also adds a package-level opt-out knob ([package.metadata.anvil.miri] exclude = true) and applies the updated templates to this repository, including documentation and snapshot updates.

Changes:

  • Add a shared _anvil-miri-test Just recipe that compiles once (--no-run), discovers test artifacts from Cargo JSON, and runs them in parallel with ANVIL_MIRI_JOBS controlling concurrency.
  • Update all Miri profiles (miri, miri-tree-borrows, miri-strict-provenance, miri-race-coverage) to delegate execution to the shared runner and preserve profile-specific flags.
  • Extend cargo-anvil contract tests/docs/snapshots and update .anvil.lock to reflect regenerated template outputs.

Reviewed changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated no comments.

Show a summary per file
File Description
justfiles/anvil/checks/miri.just Introduces _anvil-miri-test and routes anvil-miri through the shared parallel artifact runner.
justfiles/anvil/checks/miri-tree-borrows.just Delegates Tree Borrows profile execution to _anvil-miri-test.
justfiles/anvil/checks/miri-strict-provenance.just Delegates Strict Provenance profile execution to _anvil-miri-test.
justfiles/anvil/checks/miri-race-coverage.just Delegates Race Coverage profile execution to _anvil-miri-test.
crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just Template source for the new shared parallel Miri artifact runner.
crates/cargo-anvil/templates/justfiles/anvil/checks/miri-tree-borrows.just Template updated to call the shared runner.
crates/cargo-anvil/templates/justfiles/anvil/checks/miri-strict-provenance.just Template updated to call the shared runner.
crates/cargo-anvil/templates/justfiles/anvil/checks/miri-race-coverage.just Template updated to call the shared runner.
crates/cargo-anvil/src/lib.rs Updates public crate documentation to describe compile-once + parallel artifact execution, plus package opt-out metadata.
crates/cargo-anvil/docs/design/local.md Documents the new Miri concurrency model and the package-level exclude metadata.
crates/cargo-anvil/docs/design/checks.md Updates the miri check contract to match the new artifact runner behavior and profile delegation.
crates/cargo-anvil/src/anvil/artifacts/justfile.rs Adds a regression test ensuring Miri profiles delegate to the shared runner and key behaviors are present in the generated check file bodies.
crates/cargo-anvil/tests/recipe_contracts.rs Adds contract tests with a fake cargo miri/cargo-miri runner to validate exclusion logic, parallelism, flag propagation, and failure aggregation.
crates/cargo-anvil/tests/snapshots/snapshots__local_only.snap Updates expected generated output snapshot to include _anvil-miri-test and delegation.
crates/cargo-anvil/tests/snapshots/snapshots__github_backend.snap Updates expected generated output snapshot to include _anvil-miri-test and delegation.
crates/cargo-anvil/tests/snapshots/snapshots__ado_backend.snap Updates expected generated output snapshot to include _anvil-miri-test and delegation.
crates/cargo-anvil/README.md Regenerated README content reflecting the updated Miri behavior and metadata opt-out.
.anvil.lock Updates catalog checksum and file/region checksums for regenerated Anvil-managed files.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just Outdated
Copilot AI review requested due to automatic review settings September 1, 2026 09:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated 1 comment.

Comment thread justfiles/anvil/checks/miri.just Outdated
Copilot AI review requested due to automatic review settings September 1, 2026 10:23

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated no new comments.

Suppressed comments (1)

justfiles/anvil/checks/miri.just:177

  • The PR description claims “memory telemetry”, but this runner currently doesn’t emit any memory usage/pressure info (it only prints artifact/job counts). Either add explicit telemetry (and document what it measures) or adjust the PR description so it doesn’t promise this behavior.
        Write-Host "anvil miri: running $($artifacts.Count) test artifacts with $jobs concurrent process(es)"

@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.5%. Comparing base (7a5daf3) to head (9bf220d).
⚠️ Report is 1 commits behind head on main.

❌ Your project status has failed because the head coverage (97.5%) is below the target coverage (100.0%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@           Coverage Diff           @@
##            main    #123     +/-   ##
=======================================
- Coverage   97.5%   97.5%   -0.1%     
=======================================
  Files        300     300             
  Lines      67799   67799             
=======================================
- Hits       66160   66154      -6     
- Misses      1639    1645      +6     
Flag Coverage Δ
linux 97.5% <ø> (-0.1%) ⬇️
linux-arm 97.5% <ø> (-0.1%) ⬇️
scheduled ?
windows 97.7% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Copilot speaking]

Published 19 findings. No finding follows up on an existing discussion thread.

See diagnostics
Diagnostic Value
Cache Hit

Comment thread crates/cargo-anvil/docs/design/local.md Outdated
Comment thread crates/cargo-anvil/docs/design/checks.md Outdated
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just
Comment thread crates/cargo-anvil/docs/design/local.md Outdated
Comment thread crates/cargo-anvil/docs/design/checks.md Outdated
Comment thread crates/cargo-anvil/tests/recipe_contracts.rs
Comment thread crates/cargo-anvil/tests/snapshots/snapshots__github_backend.snap
Comment thread justfiles/anvil/checks/miri.just
Comment thread crates/cargo-anvil/docs/design/checks.md Outdated
Comment thread crates/cargo-anvil/tests/recipe_contracts.rs
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just Outdated

@martin-kolinek martin-kolinek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖: Approved.

Copilot AI review requested due to automatic review settings September 1, 2026 17:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated 2 comments.

Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just
Copilot AI review requested due to automatic review settings September 1, 2026 19:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new _anvil-miri-test runner uses rustc ... --print sysroot output as a scalar, which can break if PowerShell captures multiple lines/arrays; the sysroot parsing should be made robust (comments added in both miri.just copies).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 17/18 changed files
  • Comments generated: 2
  • Review effort level: Lite

Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just Outdated
Comment thread justfiles/anvil/checks/miri.just Outdated
Copilot AI review requested due to automatic review settings September 1, 2026 20:19
Copilot AI review requested due to automatic review settings September 4, 2026 16:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It substantially changes the CI-critical Miri execution model (parallel artifact orchestration, filtering, and log replay) and should receive final human review despite strong test coverage.

Review details
  • Files reviewed: 21/22 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Stanislav Andras (from Dev Box) and others added 15 commits September 7, 2026 10:35
Compile each Miri profile once to preserve workspace feature unification, then execute independent test artifacts concurrently through cargo-miri.

Add configurable concurrency, package-level exclusions, grouped artifact output, memory telemetry, and aggregated failure reporting for GitHub and Azure DevOps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Remove tuning telemetry inherited from oxidizer #706, including the process-wide forced collections around every artifact. Keep the generic concurrency control through ANVIL_MIRI_JOBS and document the private recipe clearly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Use the minimal raw-string delimiter accepted by Rust 1.97 and clear inherited GitHub and ADO markers so each recipe-contract test selects its output backend explicitly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Use the resolved stable toolchain for metadata and expose the pinned nightly directly on Miri invocations, matching cargo-anvil's deterministic toolchain contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Forward the Miri worker cap into containers, handle fully excluded workspaces, and identify artifacts by package and target. Strengthen impact/exclusion contracts and document the corresponding behavior and rationale.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Match deterministic replay group headers instead of parallel progress messages, and canonicalize fixture working directories before comparing Windows path spellings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Use one pinned nightly Cargo for metadata and artifacts, keep package IDs internal, and normalize rustc sysroot output. Clarify executable admission, public recipe behavior, and worker terminology.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Keep profile execution in Just's dependency graph by parameterizing the shared compile-once runner. Centralize impact selection and profile setup, and refresh the executable contracts and generated artifacts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Separate impact filtering from the table-driven profile flag checks so each contract remains focused and below the Clippy line limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Keep compile-once execution and concurrency in the public contract while moving Cargo artifact discovery and the pinned cargo-miri runner protocol into the implementation guide. Document the architecture and verification boundaries, then regenerate derived artifacts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Copilot AI review requested due to automatic review settings September 7, 2026 09:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The Miri exclusion-to---exclude argument construction iterates a HashSet, producing non-deterministic cargo argument ordering that should be stabilized.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 22/23 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread justfiles/anvil/checks/miri.just
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/miri.just Outdated
Use deterministic package exclusion ordering and forward libtest filters to every interpreted test artifact.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 98e52fe3-a435-412e-9154-73a557fef79f
Copilot AI review requested due to automatic review settings September 7, 2026 11:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The implementation, templates, docs, snapshots, and contract/structural tests are consistently updated to the new parallel Miri execution model with no verified correctness issues in the reviewed diffs.

Review details
  • Files reviewed: 22/23 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants