ci: route all Codecov flags through the deferred workflow_run upload

[aglinxinyuan]

What changes were proposed in this PR?

Route every Codecov flag's upload through a follow-up workflow_run workflow instead of uploading inline from the pull_request build, so fork PRs get authenticated (non-rate-limited) uploads and a reliable Codecov PR comment.

Problem. A PR from a fork never receives secrets.CODECOV_TOKEN (the build runs on pull_request, and GitHub withholds secrets from fork runs). The inline codecov-action steps therefore fall back to Codecov's tokenless path, which is rate-limited; under a burst of fork-PR uploads Codecov still processes the report but silently drops the PR comment (reported on #6685).

Fix.

• build.yml: each coverage job stages its reports as a codecov-<flag> artifact via .github/scripts/stage-codecov.sh (coverage/ + results/ + pr/sha/branch identifiers) instead of uploading inline. The script preserves the directory tree so amber's ~8 same-basename module jacoco.xml don't collide. Staging is skipped on backport builds (artifact-name collisions across the target matrix) and pinned to the single producing leg for the dual-OS / multi-python jobs.
• codecov-upload.yml: a per-flag matrix runs on workflow_run after Required Checks, in the base-repo context where CODECOV_TOKEN is available. It downloads each artifact and re-uploads with the token, using directory: scoping so Codecov's search finds every file per flag.
• Failure surfacing: coverage uploads fail hard, and a notify-failure job then comments on the PR (a link to the failed run + the run_id to manually dispatch) — Codecov only self-comments on success, so this covers the failure case. A delayed / no-runner run posts nothing at all, which is itself the signal that the upload is stuck.

Flags covered: frontend, amber, amber-integration (test-results only), pyamber, agent-service, and the 6 platform services.

Why not batch into fewer calls: Codecov requires one upload per flag (flags in a single upload apply to all its files, and carryforward — which codecov.yml relies on — needs the flag set at upload time), with no path-based auto-assignment, so the calls can't be collapsed without dropping the per-flag / carryforward model. The token — not batching — is what removes the tokenless rate-limit pressure.

Any related issues, documentation, discussions?

Reported on #6685.

How was this PR tested?

• workflow_run only triggers from the copy of the file on the default branch, so codecov-upload.yml activates only after merge; it exposes a workflow_dispatch (run_id) entry to validate against a finished Required Checks run immediately after merge.
• .github/scripts/stage-codecov.sh was exercised locally: multi-module glob → no basename collisions, single-file flags, and the empty-PR (push) case.
• YAML validated for both workflows. The staging half runs in this PR's own CI, so the codecov-<flag> artifacts can be inspected on this run.
• Reviewed for coverage parity (every prior inline upload is still staged + re-uploaded — nothing dropped), artifact-name collisions (backport guard + per-leg pinning), and deferred-upload correctness (empty-override omission, missing-artifact handling, matrix.coverage gating).
• First-live-run check after merge: confirm each flag's test-results (Test Analytics) JUnit files are discovered under directory: — best-effort, and separate from the coverage report.

Since the deferred workflow can only activate on main, if anything is off the whole change reverts as a single PR.

Was this PR authored or co-authored using generative AI tooling?

Yes.

Generated-by: Claude Code (Opus 4.8 [1M context])

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• No candidates found from git blame history.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.83%. Comparing base (0467c76) to head (48efa63).
⚠️ Report is 17 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6730      +/-   ##
============================================
+ Coverage     74.65%   74.83%   +0.17%     
- Complexity     3438     3448      +10     
============================================
  Files          1160     1160              
  Lines         45793    45793              
  Branches       5069     5069              
============================================
+ Hits          34189    34267      +78     
+ Misses         9953     9878      -75     
+ Partials       1651     1648       -3     

Flag	Coverage Δ		*Carryforward flag
access-control-service	70.00% <ø> (ø)
agent-service	76.76% <ø> (ø)
amber	67.19% <ø> (+0.38%)	⬆️	Carriedforward from e24cb25
computing-unit-managing-service	20.49% <ø> (+2.49%)	⬆️
config-service	66.66% <ø> (ø)
file-service	66.80% <ø> (ø)
frontend	78.56% <ø> (ø)		Carriedforward from e24cb25
notebook-migration-service	78.94% <ø> (ø)
pyamber	92.08% <ø> (+0.09%)	⬆️
workflow-compiling-service	55.14% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

This PR reroutes frontend Codecov uploads away from the pull_request build (which cannot access CODECOV_TOKEN on fork PRs) to a follow-up workflow_run workflow that runs in the base-repo context, enabling authenticated uploads and more reliable Codecov PR comments for fork PRs.

Changes:

• Add a new codecov-upload.yml workflow that runs on workflow_run (after “Required Checks”) to download a staged artifact and upload coverage + test results to Codecov using CODECOV_TOKEN.
• Remove the direct frontend Codecov upload steps from build.yml and instead stage lcov.info + JUnit XML + PR/sha/branch metadata into a codecov-frontend artifact.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File	Description
.github/workflows/codecov-upload.yml	New workflow that downloads the staged frontend Codecov artifact from the source run and re-uploads it with an authenticated token.
.github/workflows/build.yml	Replaces direct frontend Codecov uploads with artifact staging + upload for deferred processing by codecov-upload.yml.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 2 worse · ⚪ 13 noise (<±5%) · 0 without baseline

Compared against main ef55ad8 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	398	0.243	23,506/32,853/32,853 us	🔴 -11.2% / 🔴 +103.8%
⚪	bs=100 sw=10 sl=64	941	0.574	104,267/130,142/130,142 us	⚪ within ±5% / 🔴 +21.2%
⚪	bs=1000 sw=10 sl=64	1,083	0.661	926,271/993,907/993,907 us	⚪ within ±5% / 🟢 +5.9%

Baseline details

Latest main ef55ad8 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	398 tuples/sec	448 tuples/sec	760.2 tuples/sec	-11.2%	-47.6%
bs=10 sw=10 sl=64	MB/s	0.243 MB/s	0.273 MB/s	0.464 MB/s	-11.0%	-47.6%
bs=10 sw=10 sl=64	p50	23,506 us	22,563 us	12,781 us	+4.2%	+83.9%
bs=10 sw=10 sl=64	p95	32,853 us	32,034 us	16,118 us	+2.6%	+103.8%
bs=10 sw=10 sl=64	p99	32,853 us	32,034 us	19,074 us	+2.6%	+72.2%
bs=100 sw=10 sl=64	throughput	941 tuples/sec	963 tuples/sec	988.68 tuples/sec	-2.3%	-4.8%
bs=100 sw=10 sl=64	MB/s	0.574 MB/s	0.588 MB/s	0.603 MB/s	-2.4%	-4.9%
bs=100 sw=10 sl=64	p50	104,267 us	100,066 us	101,448 us	+4.2%	+2.8%
bs=100 sw=10 sl=64	p95	130,142 us	128,180 us	107,410 us	+1.5%	+21.2%
bs=100 sw=10 sl=64	p99	130,142 us	128,180 us	120,258 us	+1.5%	+8.2%
bs=1000 sw=10 sl=64	throughput	1,083 tuples/sec	1,088 tuples/sec	1,023 tuples/sec	-0.5%	+5.9%
bs=1000 sw=10 sl=64	MB/s	0.661 MB/s	0.664 MB/s	0.624 MB/s	-0.5%	+5.9%
bs=1000 sw=10 sl=64	p50	926,271 us	924,740 us	983,888 us	+0.2%	-5.9%
bs=1000 sw=10 sl=64	p95	993,907 us	967,303 us	1,030,100 us	+2.8%	-3.5%
bs=1000 sw=10 sl=64	p99	993,907 us	967,303 us	1,056,513 us	+2.8%	-5.9%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,502.93,200,128000,398,0.243,23505.91,32853.07,32853.07
1,100,10,64,20,2126.08,2000,1280000,941,0.574,104266.89,130141.51,130141.51
2,1000,10,64,20,18460.04,20000,12800000,1083,0.661,926270.65,993906.98,993906.98

[aglinxinyuan]

Looked into all five Copilot findings — they're non-issues; details below. Pushed one comment-only commit (48efa63) to spell out the first point.

always() on the staging / artifact steps (build.yml). Both steps use if: matrix.os == 'ubuntu-latest' && inputs.backport_target_branch == '' && !cancelled(). Because the expression contains a status-check function (!cancelled()), GitHub does not auto-wrap it with the implicit success() — per the docs, success() is only inserted when the if contains none of the status functions. So staging already runs when the unit or browser tests fail; it only skips a cancelled run. That reproduces the old always()/!cancelled() behavior (partial coverage + JUnit still staged), and staging from a cancelled/timed-out run is undesirable since the reports are incomplete. Added a comment saying exactly this.

Empty override_pr on push runs (codecov-upload.yml). The codecov-action wrapper only emits a flag when its value is non-empty. From dist/codecov.sh at the pinned v7.0.0 SHA:

k_arg() { if [ -n "$(eval echo \$"CC_$1")" ]; then echo "--$(lower "$1")"; fi }
v_arg() { if [ -n "$(eval echo \$"CC_$1")" ]; then echo "$(eval echo \$"CC_$1")"; fi }
...
CC_ARGS+=( $(k_arg PR) $(v_arg PR))
CC_ARGS+=( $(k_arg SHA) $(v_arg SHA))
CC_ARGS+=( $(k_arg BRANCH) $(v_arg BRANCH))

So on a push to main pr-number.txt is empty → CC_PR is empty → --pr is omitted entirely (not passed as --pr ""). override_commit/override_branch behave the same way. No empty-arg failure and no mis-association — the push commit is still keyed correctly from the stashed commit-sha.txt (= github.sha), so a PR/push split isn't needed.

[Yicong-Huang]

Thank for the fix, but why only frontend has this issue? should we defer all coverage report uploading to the same job? also can batching them together reduce the usage on the allowence?

[aglinxinyuan]

Good questions.

Why only frontend? Not frontend-specific — any flag on a fork PR can hit it. Frontend just surfaced it: the campaign opens ~10 frontend fork PRs within minutes, and each fires 3 tokenless uploads (coverage + unit + browser junit) vs 1–2 for other flags, so it trips Codecov's tokenless rate limit fastest.

Defer all flags? Right direction — I scoped the PoC to frontend for a reviewable diff. Not a pure copy-paste, though: amber's coverage is a same-basename glob across ~8 modules (flat-copy would drop 7/8), amber-integration is results-only and uploads from both the ubuntu+macOS legs, and every flag needs the backport_target_branch=='' guard + PR/sha/branch stash. Also, since the deferred job runs on workflow_run (not a required check), a failed upload goes silent. Happy to do it as a careful follow-up.

Batching to save allowance? Not really — no per-upload quota is in play; the pressure is the tokenless rate limit. Per-module reports can't collapse into fewer uploads without cross-tagging, and the artifact hand-off adds a little Actions work. Authentication (the token), not batching, is the fix.

[aglinxinyuan]

Done — pushed. Three things:

Failure notification (your main concern). Added a notify-failure job: if a coverage upload fails, it comments on the PR with a link to the failed run + the run_id to manually dispatch — same idea as auto-backport. Success stays silent since Codecov comments itself. And the delayed/no-runner case is covered by design: if the job is queued and nothing runs, no comment appears at all — which is your signal to go look. (I made the coverage upload fail hard so a real error actually trips this; test-results stay best-effort so a Test-Analytics hiccup doesn't false-alarm.)

All flags in one PR. Every flag now stages a codecov-<flag> artifact in the build and re-uploads through the deferred workflow — frontend, amber, amber-integration, pyamber, agent-service, and the 6 platform services.

Batching to cut Codecov calls. We can't, not without dropping the per-flag model: Codecov requires one upload per flag (flags in a single upload apply to all its files, and carryforward — which our codecov.yml relies on — needs the flag set at upload time), and there's no path-based auto-assignment. The token is what relieves the pressure anyway — the tokenless rate limit was the problem, and authenticated uploads aren't subject to it, so call count isn't the lever. (Components could collapse to one upload but would mean giving up flags + carryforward — a separate, bigger change.)

One thing I'll confirm on the first live run after merge: the test-results (Test Analytics) upload is best-effort, so I want to eyeball that each flag's JUnit files are found — but that's separate from the coverage report.