-
Notifications
You must be signed in to change notification settings - Fork 1
1206 lines (1171 loc) · 83.2 KB
/
Copy pathpr-risk.yml
File metadata and controls
1206 lines (1171 loc) · 83.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
name: PR Risk Grade (reusable)
# Reusable ADVISORY PR risk grader — the shadow-check rung of the PR risk-grading ladder.
# Grades every PR event into a tier R0 (safest) .. R3 (riskiest) and syncs ONE visible label
# (`risk:R0` .. `risk:R3`, or `risk:ungraded` when an input was unreadable). A human can override
# that visible label with `risk-dispute:low` .. `risk-dispute:xhigh`; the computed tier remains in
# the grade record. Nothing is gated, blocked, routed, commented, or merged.
#
# grade = worst(path_floor, provenance, reversibility) — three deterministic axes; the worst
# tier wins, so no axis can move a PR into a safer lane than another axis put it. No LLM, no
# model call anywhere: `gh` + `jq` over the PR's own API record. See
# scripts/pr-risk/grade-pr-risk.sh for the axes and the unknown contract.
#
# The grader and its default risk map load from THIS repo at the pinned `workflows_ref`,
# never from the graded PR's checkout (no PR code is checked out at all) — a PR cannot edit
# the rules that judge it. `workflows_ref` has deliberately no default and its SHAPE is now
# ENFORCED: every job that checks it out fails BEFORE the checkout unless the value is a full
# 40-hex lowercase commit SHA. That rejects everything MUTABLE — a branch, a tag,
# `refs/pull/N/head` — so the grading logic cannot change after the caller was reviewed, and
# "pin it, don't float it" is machine-checked rather than trusted prose in this header.
#
# A SECOND AXIS, ANCESTRY, now proves WHOSE commit it is. Shape alone never could: a fork of this
# PUBLIC repo shares its object store, and GitHub serves a fork PR's head objects from this repo's
# own URL to an unauthenticated client — so a fork-authored commit is a perfectly well-shaped
# 40-hex SHA, and it would have been checked out into a job holding the caller's
# `pull-requests: write` token. Every job that checks the ref out now also fetches it from
# `Comfy-Org/github-workflows` (URL written LITERALLY — no context can name this repo from inside a
# reusable workflow) and requires it to be an ANCESTOR OF UPSTREAM `main`. Merged upstream history,
# and nothing else, may be pinned. There is deliberately NO opt-out input: an opt-out would be set
# by the very pin-bump PR the check exists to distrust. Two operational consequences, both by
# design — pinning a NOT-YET-MERGED SHA is now rejected, so merge first and then bump (which is what
# every consumer has always done in practice); and the axis FAILS CLOSED if this repo ever goes
# private or `main` is force-rewritten past a consumer's pin, in which case consumers re-pin to a
# commit that is on `main` to recover.
#
# WHAT ANCESTRY STILL DOES NOT PROVE is that the pin is the CURRENT one. A stale pin, left behind
# when `uses:` moved, is a merged ancestor too and passes. The check that would close that is
# "`workflows_ref` equals the commit the caller's `uses:` resolved to". That commit IS reachable,
# just not the two ways this comment used to say it was not: `github.workflow_sha` is the CALLER's
# top-level workflow file, and the `job_workflow_sha` OIDC claim would need an `id-token: write`
# grant from every caller plus a token exchange in a job that today holds `permissions: {}` — but
# `job.workflow_sha`, the `job`-context accessor added in runner v2.334.0 (Apr 2026), needs neither
# and is what groom.yml reads (BE-8077). So this is a WIRING gap, not a runner limitation. It is
# deliberately still open: asserting equality here would fail red on every caller whose `uses:` and
# `workflows_ref` have drifted apart, which is a caller-contract change tracked separately. Do not
# assume this pin is proven CURRENT. The other residual is narrower and irreducible: a fork commit
# that edits THIS FILE. Such a commit is not an ancestor of upstream `main`, so the axis rejects it
# on the way in — but a caller pinned at one would be running the fork's own copy of this guard, and
# no check written inside a file can bound a revision of that file chosen by an attacker. Both
# residuals are bounded by REVIEW OF THE CALLER on its base branch, which is also the only thing
# that can see the ref the caller actually wrote: `uses:` must name this repo at a full commit SHA
# and `with: workflows_ref:` must be that same SHA written out LITERALLY, never an expression and
# never a tag. Call this workflow DIRECTLY; a nested `workflow_call` chain through an org wrapper
# is unsupported. See the `workflows_ref` input for the detail.
# A consumer repo sharpens the generic defaults by committing
# `.github/risk.json` (map) / `.github/risk-runbooks.json` (runbook registry), which are read
# from the PR's BASE ref: present-but-invalid fails the run loudly; absent falls back to the
# defaults in scripts/pr-risk/.
#
# CHECKS SETTLE BEFORE THE LABEL DOES: the reversibility axis asks "did tests covering these
# lines actually run", and at event time the rest of the rollup is usually still running (the
# grading job itself is excluded from the rollup it reads — see --self-context in the
# grader). The job re-polls until the other checks settle or `wait_for_checks_minutes` runs
# out, then labels what it has. Pair the caller with a per-PR cancel-in-progress concurrency
# group so a new push supersedes a waiting run instead of stacking behind it.
#
# ON-DEMAND GRADING (`pr_number` / `pr_numbers`): supply a PR number and that PR is graded with
# no `pull_request` event involved — which is how a repo enrolling mid-stream grades the open
# queue it already has, and how a PR is re-graded after a `.github/risk.json` change. Absent, the
# workflow reads the event exactly as before. Three things differ on
# the by-number path, all of them deliberate:
# * FORK PRs ARE GRADED. The `head.repo.full_name == github.repository` clause callers put in
# their `if:` is a TOKEN guard, not a policy one: a fork's `pull_request` run gets a
# read-only GITHUB_TOKEN that the caller's `permissions:` block cannot elevate, so the label
# write — this workflow's entire product — would 403. On a dispatch the token is writable, so
# the reason evaporates. Fork RISK is unaffected either way, because `external` is derived
# from the API's `isCrossRepository`, never from the actor, and forks grade R3 with no
# exceptions. DEPENDABOT-TRIGGERED RUNS need no such hatch and are graded on BOTH paths: the
# caller pattern below deliberately carries no `github.actor != 'dependabot[bot]'` clause,
# because the caller's `permissions:` block elevates Dependabot's read-only token and this
# workflow declares no secrets (see that pattern for the citation, for the one thing the
# clause did buy, and for the callers the reasoning does NOT transfer to). Read that clause
# literally: `github.actor` is who TRIGGERED the run, not who authored the PR, so it never
# keyed on bot authorship — a human pushing to a Dependabot branch made the actor human and
# the PR graded even with the clause in place. Skipping those runs would also bias any
# backfill corpus badly; bots author a substantial minority of merged PRs.
# * THE BASE REF IS RE-READ FROM THE API, per target, because there is no event payload to take
# it from. It is load-bearing rather than cosmetic: it selects which branch's
# `.github/risk.json` judges the PR, live PRs are commonly stacked on feature branches rather
# than the default branch, and an empty ref is NOT an error to the contents API — it silently
# resolves to the default branch. So an unresolvable base ref FAILS that target instead of
# grading it against rules nobody read.
# * A LOW `wait_for_checks_minutes` IS RIGHT HERE, and is not the same trade-off as on the event
# path. The wait exists to outlast the rest of the rollup while our own check sits in it; a
# dispatched run's check is attached to the dispatched ref, not to the PR's head commit, so
# it is not in that rollup at all and a settled PR reads its true state (`SUCCESS`, nothing
# pending) on the first poll. It is still not FREE: `0` breaks out after a single read, ahead
# of both the not-yet-registered grace window and the "require a settled reading to repeat"
# confirmation, so a target someone pushed to minutes ago lands the honest R2 floor. `1` costs
# one 15s backoff and a second read per PR and keeps the confirmation. Prefer `1`, not `0`.
#
# Batch (`pr_numbers`) grades one target at a time and ONE UNREADABLE PR NEVER ABANDONS THE REST:
# each target's outcome is recorded, the whole list is attempted, and the run then reports whether
# any target failed. There is deliberately no `all_open: true` — an explicit list is bounded,
# reviewable and re-runnable, and the list is capped rather than silently truncated. A dispatch
# that reaches this workflow with no target at all FAILS LOUDLY instead of exiting 0: a button
# that silently grades nothing is worse than no button.
#
# A BATCH CANNOT SERIALIZE PER-PR, and no concurrency key can make it: one run covers N pull
# requests and a run belongs to exactly one group, so a batch that overlaps a `pull_request` run for
# one of ITS numbers WILL interleave with it. That is now safe to let happen: the label sync is a
# single atomic `PUT` of the PR's whole label set (see apply-risk-label.sh), so two writers end
# last-writer-wins with still EXACTLY ONE `risk:*` label — possibly the staler tier, which gates
# nothing meanwhile and which the next grade re-syncs (on the LAST push there is no next grade, so
# re-dispatch on `pr_number` if a final grade looks wrong). It can no longer leave two contradictory
# labels on a PR under one `label_map`; remapping `label_map` orphans the old names, which is a
# one-time repo-side cleanup. What the shape does cost is a narrower residual: the PUT is built from
# a snapshot read, so a NON-owned label added by someone else in the read→PUT window is dropped
# (`risk-dispute:*` included) and one removed in it is resurrected. That window opens only on a run
# that actually changes the grade and is roughly one API round-trip — about three on the first
# grade in a repo, where the label pre-create sits inside it. The drop is recorded on the PR
# timeline as an `unlabeled` event, so re-add a dispute that happens to land in that instant.
# Still prefer `pr_number` when you want the
# per-PR concurrency group to serialize a re-grade against event runs (with `cancel-in-progress`
# that is also how a superseded wait gets cut short), and dispatch a backfill when the queue is
# quiet.
#
# SECRETS: none. This workflow declares no `secrets:` inputs and callers pass none — the only
# credential in the job is the automatic `GITHUB_TOKEN` (`github.token`), used for the PR read
# and the one label write. There is no `secrets: inherit` to add and nothing to rotate.
#
# The label is applied with the plain GITHUB_TOKEN on purpose: GITHUB_TOKEN-applied labels
# cannot fire `labeled` triggers, so the shadow check is structurally unable to start a
# workflow cascade. Fork PRs under a plain `pull_request` trigger get a read-only token and
# the label write will fail — enroll public repos with `pull_request_target` instead (safe
# here by construction: this workflow never checks out or executes PR code).
#
# ENROLL THIS AS ITS OWN WORKFLOW, not as one job inside an existing CI workflow. The grading
# job is part of the check rollup it reads, so it excludes its own RUN from that rollup; a job
# sharing a run with the rest of CI therefore excludes its siblings too and lands on the honest
# R2 floor instead of grading off a full rollup. (It can never grade a red PR green either way
# — a FAILING check is never excluded.)
#
# Caller pattern (consumer repo, .github/workflows/ci-pr-risk.yml):
#
# name: CI - PR Risk Grade
# on:
# pull_request:
# types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]
# workflow_dispatch:
# inputs:
# pr_number:
# description: Grade ONE pr by number (enrollment backfill, or a manual re-grade).
# required: false
# pr_numbers:
# description: Grade several, comma-separated (12,15,20). Wins over pr_number.
# required: false
# concurrency:
# # THE KEY MUST CARRY THE RESOLVED TARGET, not just the event PR. On a workflow_dispatch
# # `github.event.pull_request.number` is empty, so an event-only key collapses to ONE
# # constant group for every dispatch — and with cancel-in-progress that means each
# # dispatched PR cancels the one before it, which is precisely the shape a backfill has.
# # `inputs` is empty on a `pull_request` run, so this one expression is correct on both.
# # A `pr_numbers` LIST keys its own group, which serializes identical batches but cannot
# # serialize a batch against a `pull_request` run for one of its members — see "A BATCH
# # CANNOT SERIALIZE PER-PR" above for what that costs and how to avoid it.
# group: ${{ github.workflow }}-${{ inputs.pr_numbers || inputs.pr_number || github.event.pull_request.number }}
# cancel-in-progress: true
# permissions:
# contents: read
# jobs:
# pr-risk:
# # ONE CLAUSE, AND IT STAYS SCOPED TO THE EVENT IT DESCRIBES. The fork clause is a TOKEN
# # guard, not a risk judgement: a fork's `pull_request` run gets a read-only GITHUB_TOKEN,
# # the `permissions:` block below CANNOT elevate it (the Dependabot carve-out below does
# # not extend to forks), so the label write would 403 and the check would go red. Fork
# # RISK is untouched either way — forks grade R3 from the API's own fork flag, never from
# # the actor — so a fork PR is simply ungraded-by-absence here; grade it by dispatching on
# # `pr_number`, or enroll with `pull_request_target` instead, which is safe by construction
# # for this workflow (see the fork paragraph above). Keep the clause BEHIND the event test:
# # on a workflow_dispatch there is no `github.event.pull_request`, so an unscoped
# # `head.repo.full_name == github.repository` is FALSE and the job SKIPS SILENTLY — the
# # dispatch button appears to do nothing, no run, no error, no annotation. That event test
# # is what makes the button work at all, and on a dispatch the token is writable anyway, so
# # there is nothing left for the clause to protect.
# #
# # THERE IS DELIBERATELY NO `github.actor != 'dependabot[bot]'` CLAUSE, AND ADDING ONE BACK
# # IS A REGRESSION. Dependabot's `pull_request` runs do start from a read-only
# # GITHUB_TOKEN, but the `permissions:` block below ELEVATES it — GitHub documents exactly
# # that under "Changing GITHUB_TOKEN permissions" in *Troubleshooting Dependabot on GitHub
# # Actions* / *Automating Dependabot with GitHub Actions* — and this workflow declares NO
# # secrets, so the other half of the Dependabot restriction (a Dependabot run sees
# # Dependabot secrets only, never Actions secrets) costs it nothing. Skipping those runs
# # instead leaves most repos' highest-volume automated PR producer ungraded.
# #
# # ONE THING THE CLAUSE DID BUY, and it does not earn the cost: a Dependabot
# # github-actions PR that bumps THIS caller's own `uses:` pin now runs the newly pinned
# # reusable inside that PR's own run (a `pull_request` run evaluates the HEAD's workflow
# # file) under the elevated token, where the clause used to skip it. The exposure is
# # narrow — Dependabot only proposes SHAs behind this repo's own published tags, the tool
# # checkout is separately gated on `workflows_ref` being an ancestor of `main`, and the
# # fleet's own bump PRs always had this property because the actor clause never covered
# # them. Review that PR before merging it for a different reason: Dependabot rewrites
# # `uses:` only and leaves `workflows_ref` behind, and the equality check that would catch
# # that is a deliberate open gap (see "WHAT ANCESTRY STILL DOES NOT PROVE" above), so an
# # unedited Dependabot bump runs a NEW workflow file against an OLD tool checkout. Fix the
# # input in the same PR.
# #
# # THIS REASONING DOES NOT TRANSFER to a caller that needs an ACTIONS SECRET — an app
# # private key, an API token; `cursor-review-auto-label.yml` in this repo is the local
# # example. Those callers still need the dependabot skip, because no `permissions:` block
# # can hand a Dependabot-triggered run an Actions secret.
# if: >-
# github.event_name != 'pull_request' ||
# (((github.event.action != 'labeled' && github.event.action != 'unlabeled') ||
# startsWith(github.event.label.name, 'risk-dispute:')) &&
# github.event.pull_request.head.repo.full_name == github.repository)
# permissions:
# contents: read
# issues: write # create the risk:* labels repo-side on first use
# pull-requests: write # the label write itself — labeling a PR rides the
# # pull-requests permission, not issues (the labels
# # endpoint is dual-mapped by what the "issue" is)
# checks: write # REQUIRED OF EVERY CALLER, opted into `check_run` or not — see
# # GRANT ALL SEVEN below. The `publish-check` job DECLARES
# # `checks: write`, and GitHub validates every nested job's
# # declaration against this block at STARTUP; a job `if:` cannot
# # save a short grant, because nothing has run yet when the check
# # is made. Only the grade job's own `checks: read` (the rollup
# # the reversibility axis reads) is used unless `check_run: true`.
# actions: read # the rollup query walks CheckRun -> checkSuite ->
# # workflowRun (an Actions resource) for self-exclusion
# statuses: read
# uses: Comfy-Org/github-workflows/.github/workflows/pr-risk.yml@<sha> # v1
# with:
# # ENFORCED, not merely asked for: it must be a full 40-hex LOWERCASE commit SHA or the
# # run fails before the tool checkout — a branch, a tag or `refs/pull/N/head` is rejected.
# # Write out the SAME SHA as the `uses:` above, LITERALLY: that the two agree is what
# # review of this caller checks, and it is not something the workflow can see. See the
# # paragraph about the pinned ref at the top of this header for why.
# workflows_ref: <same sha>
# # OFF BY DEFAULT. Enrolling and switching on are two decisions: land the caller, get
# # it reviewed, then start grading. Either pin it on here, or leave this out and set
# # the repo variable RISK_CONFIG to {"enabled": true} — the variable outranks this
# # input in BOTH directions, so {"enabled": false} is also a kill switch needing no PR.
# enabled: true
# # Both are empty on a `pull_request` run (`inputs` is empty there), which is exactly
# # the no-input event path — so ONE caller shape serves both event and dispatch and
# # there is nothing to keep in sync between two jobs.
# pr_number: ${{ inputs.pr_number }}
# pr_numbers: ${{ inputs.pr_numbers }}
# # For a backfill, dispatch with this lowered — see ON-DEMAND GRADING above for why a
# # low wait is sound on the by-number path and why `0` still is not the right value.
# # wait_for_checks_minutes: 1
#
# A SKIPPED CALLER JOB IS INVISIBLE FROM HERE. This workflow cannot detect, warn about or recover
# from a caller whose `if:` excluded it — no run is created, so nothing of ours executes. The
# block above is the only lever, which is why the `if:` is spelled out rather than left to the
# enroller. What this workflow CAN do, and does, is refuse to be a silent no-op once it is
# actually reached: a run with no resolvable target fails with a message naming both inputs.
#
# GRANT ALL SEVEN OR THE RUN NEVER STARTS. A reusable workflow can only NARROW the caller's
# token, never elevate it, so a caller whose block is short of what ANY of this workflow's jobs
# declares is rejected by GitHub before any step executes ("is requesting 'pull-requests: write',
# but is only allowed 'pull-requests: read'") — the check goes red with no grade, and the PR keeps
# whatever label the previous push left rather than degrading to `risk:ungraded`. Copy the
# block above verbatim when you enroll, and re-check it against this header whenever you move
# the SHA pin.
#
# THE UNION INCLUDES JOBS THAT NEVER RUN. GitHub checks the DECLARATION on every nested job at
# startup, not the set of jobs that actually get scheduled, so `checks: write` is required even
# from a caller leaving `check_run` false and skipping the `publish-check` job entirely. An `if:`
# is a runtime condition and nothing has run yet when the grant is validated. This is the same
# union rule groom.yml documents, learned the same way: a short grant fails the whole run with an
# opaque "workflow file issue" and no job-level detail. IF YOU ARE MOVING AN EXISTING PIN ONTO
# THIS COMMIT, ADD `checks: write` TO THE CALLER IN THE SAME PR — a pin bump alone will fail the
# caller's next run at startup.
on:
workflow_call:
inputs:
pr_number:
description: >-
Grade ONE pull request by number instead of the event's. Leave it EMPTY on a
`pull_request` run: with no number supplied the target, the base ref and every emitted
label are exactly what they were before this input existed. Supplying it is what makes
grading possible without a `pull_request` event — the enrollment backfill of an
already-open queue, and the manual re-grade after a risk-map change.
Bot-authored and fork PRs ARE graded on this path (see the header).
Typed `string` rather than `number` because `workflow_dispatch` inputs arrive as
strings, and because an empty string is what lets the fall-through to the event's own
number stay a single expression.
type: string
required: false
default: ''
pr_numbers:
description: >-
Grade SEVERAL pull requests by number (comma-separated, e.g. `12,15,20`). Takes
precedence over `pr_number` when both are set. Targets are graded one at a time, and
one unreadable PR is REPORTED without abandoning the rest — the whole list is
attempted before the run decides its own outcome. Pair a long list with a low
`wait_for_checks_minutes`: the per-target waits are additive and the run stops starting
new targets once the job's budget is spent, reporting the un-attempted ones by number
rather than being cancelled mid-label. There is deliberately no `all_open: true`.
type: string
required: false
default: ''
fleet_logins:
description: >-
GitHub logins whose PRs are supervised-agent output (comma-separated).
Grades provenance `agent-supervised` alongside the `agent-coded` label.
type: string
required: false
default: mattmillerai
bot_logins:
description: >-
Extra logins treated as bots, on top of any `[bot]`-suffixed login
and any author GitHub itself types as a `Bot` (comma-separated). A
real GitHub App needs no entry here; a machine USER account does. A
bot with no runbook registry entry grades as human — identity alone
never buys trust. LOAD-BEARING, not a hint: a listed login skips the
first-time-contributor test, moving a non-fork NONE PR from `external`
(R3) to `human` (R1), and nothing validates that the login really is a
machine account. List only accounts you control; prune retired ones.
type: string
required: false
default: github-actions,dependabot,renovate,coderabbitai,cursor,comfy-pr-bot,web-flow
label_map:
description: >-
Rename the five grader-owned labels, as `tier=label` pairs
(comma-separated). Default:
`R0=risk:R0,R1=risk:R1,R2=risk:R2,R3=risk:R3,unknown=risk:ungraded`.
Tier KEYS are fixed; only the label text is yours. Missing labels are
created on first use, color-coded green through red.
type: string
required: false
default: ''
wait_for_checks_minutes:
description: >-
How long to wait, PER TARGET, for the REST of the check rollup to settle before
labeling (the grading run itself is excluded from the rollup it
reads). 0 labels immediately — expect R2 floors from still-pending
checks. CLAMPED to what a 30-minute job can actually spend waiting (25),
so an over-large value degrades to a shorter wait instead of a job
cancelled mid-sleep with the label never applied. On the by-number path a dispatched
run's own check is not in the graded PR's rollup at all, so `1` is usually enough
there; see ON-DEMAND GRADING in the header for why `0` still is not.
type: number
required: false
default: 10
repo_map_path:
description: >-
Path of the consumer repo's risk-map override, read from the PR base ref.
Repo-relative: a leading `/` or a `..` segment is refused rather than
resolved, since it would address an endpoint outside the repo's contents.
type: string
required: false
default: .github/risk.json
repo_runbooks_path:
description: >-
Path of the consumer repo's runbook-registry override, read from the PR base
ref. Repo-relative, on the same terms as `repo_map_path`.
type: string
required: false
default: .github/risk-runbooks.json
sticky_comment:
description: >-
Publish ONE sticky PR comment, and keep it SMALL: the visible body is a single line —
the tier, the axis that decided it, that axis's reason, and (only when the path axis
decided) how much of the diff carries it. Everything else — the worst() formula, the
per-axis reasons, the per-file path-axis breakdown, the map versions and the caveats —
is folded into a collapsed `<details>`, with the "this grade is wrong" checkbox below
it. An advisory grade nothing routes on does not get to out-shout the review panel and
CodeRabbit on the same PR; the Check Run keeps the long form for anyone who wants it.
Created once and UPDATED IN PLACE on every re-grade, so N pushes leave one comment.
DEFAULTS TO FALSE. A bot comment on every PR across every enrolled repo is a visible
behaviour change, so it is a per-consumer decision — the same precedent `pr-size.yml`
set with `mode: warn`. A caller that does not opt in behaves byte-identically to
before this input existed: nothing is rendered, nothing is posted, and no dispute
label is touched.
Ticking the checkbox applies `risk-grade-disputed` on the next grade, and a re-grade
preserves the tick rather than resetting it. That label is DISTINCT from the
human-owned `risk-dispute:*` override labels: one is a machine-maintained mirror of a
checkbox, while the others select the visible risk tier. Needs no permission beyond the
`pull-requests: write` the grade job already holds for the label.
type: boolean
required: false
default: false
check_run:
description: >-
Publish a Check Run on the PR's head commit carrying the tier and the reason — the
immutable, timestamped, commit-attached record a label cannot be (a label is mutable
and carries no history). Its conclusion is hardcoded `neutral`, so it cannot fail a PR
even if a repo later marks it required.
DEFAULTS TO FALSE — but the `checks: write` it needs is REQUIRED OF EVERY CALLER
REGARDLESS, because the `publish-check` job declares it and GitHub validates every
nested job's declaration against the caller's block at STARTUP, before any `if:` is
evaluated. Leaving this false skips the job and publishes nothing; it does NOT excuse
the grant. A caller moving its pin onto this version must add `checks: write` in the
same PR or its next run fails before a step executes. The separate job still earns its
keep: it is the only place `checks: write` is ever HELD, and it reads nothing from a
pull request, so no PR-controlled text is ever rendered under the elevated token.
Published whenever `grade` ran — including a manual `workflow_dispatch` on a repo whose
`enabled` resolves false — so the staged-rollout preview shows the Check Run too.
type: boolean
required: false
default: false
check_name:
description: Name of the published Check Run (only used when `check_run` is true).
type: string
required: false
default: PR risk (advisory)
workflows_ref:
description: >-
Ref of Comfy-Org/github-workflows to load the grader + default map
from. REQUIRED, and pin it to the SAME full commit SHA you pin `uses:`
to. There is deliberately no default: a floating default (`main`) let a
caller SHA-pin `uses:` and then download the grader from HEAD of main,
so the two halves of one tool drifted apart and the grading logic
stayed mutable after review. A supply chain with a floating link in it
is not a chain.
ITS SHAPE IS ENFORCED before the tool checkout: exactly 40 characters,
every one of them in `[0-9a-f]`. Branches, tags and `refs/pull/N/head`
are mutable, so they are rejected rather than documented against; a
multi-line value, trailing whitespace and uppercase hex are rejected
with them, since this value is handed to `actions/checkout` verbatim.
ITS ANCESTRY IS ENFORCED TOO, in the same step, before the same
checkout: the commit must be reachable from `main` of
Comfy-Org/github-workflows. Shape alone proves only that the ref is
IMMUTABLE, never WHICH repository authored the commit — a fork of this
PUBLIC repo shares its object store, and GitHub serves a fork PR's head
objects from this repo's own URL to an unauthenticated client, so a
fork-authored commit is as well-shaped as any other and would be
checked out into a job holding the caller's write token. The step
fetches the pinned SHA and `main` from a LITERAL upstream URL (no
`github` context can name this repo from inside a reusable workflow, so
a variable there would be an alias a fork could point at itself) and
requires `merge-base --is-ancestor`. There is deliberately NO opt-out
input: an opt-out would be set by the very pin-bump PR the check exists
to distrust.
TWO OPERATIONAL CONSEQUENCES, BOTH BY DESIGN. Pinning a NOT-YET-MERGED
SHA is now rejected outright — merge to `main` first, then bump the
pin, which is what every consumer has always done in practice. And the
axis FAILS CLOSED: if this repo ever goes private, or `main` is
force-rewritten past a consumer's pin, the fetch or the ancestry test
fails and the run goes red rather than proceeding on an unproven ref.
Recovery is to re-pin to a commit that is on `main`.
A RED RUN HERE IS NOT A NETWORK BLIP. Because this makes two
unauthenticated fetches a hard precondition of every run, each one gets
up to 3 attempts with a linear backoff, bounded by git's low-speed
timeout, so a transient github.com error or a secondary rate limit on
shared Actions egress resolves itself rather than reddening every
enrolled consumer at once. What is never retried is the VERDICT. If the
step still fails, read the annotation: it distinguishes a scratch
workspace that could not be CREATED or INITIALIZED, a pin that could
not be FETCHED, a `main` that could not be READ, an ancestry test that
could not be COMPLETED (all five are infrastructure faults, not a
judgement about the pin), and a pin that is genuinely NOT AN
ANCESTOR — which is the only one of the six that says anything about
the pin itself.
WHAT ANCESTRY DOES NOT PROVE is that the pin is the CURRENT one: a pin
left behind when `uses:` moved is a merged ancestor too, and passes.
The test that would close that is "equal to the commit `uses:` resolved
to for this job". `github.workflow_sha` is the CALLER's top-level
workflow file, not this one, and the `job_workflow_sha` OIDC claim would
need an `id-token: write` grant from every caller plus a token exchange
in a job that today holds `permissions: {}` — but `job.workflow_sha`,
the `job`-context accessor added in runner v2.334.0, is neither of those
and answers it directly (BE-8077; groom.yml reads it). Wiring it in is
still a caller-contract change — every split pin would start failing red
— so it is tracked separately and NOT done: do not assume this pin is
proven CURRENT. The
other residual is narrower and irreducible: a fork commit that edits
`pr-risk.yml` itself is not an ancestor of upstream `main`, so it is
rejected on the way in, but a caller pinned at one would be running the
fork's own copy of this guard, and nothing written inside a file can
bound a revision of that file an attacker chose.
SO REVIEW OF THE CALLER IS WHAT BOUNDS BOTH RESIDUALS, and it is the
only place that CAN: the ref the caller actually wrote in `uses:` is
never sent here, so no check inside this workflow can tell a SHA-pinned
caller from a `@v1` one passing whatever that tag currently points at,
nor a written-out SHA from a context expression that re-resolves every
run. WHEN REVIEWING A CALLER, require `uses:` at a full commit SHA of
this repo (org policy already does — a floating one fails the
pin-validation consumer CI runs) and `with: workflows_ref:` set to that
same SHA LITERALLY, character-for-character. An expression of any kind
here, and the contract is decoration.
CALL IT DIRECTLY — a nested `workflow_call` chain (caller → an org
wrapper workflow → this one) is NOT supported: the wrapper's own pin
and this input name different files, and nothing reconciles them.
Name `pr-risk.yml` in the caller's `uses:`.
type: string
required: true
enabled:
description: >-
Whether to grade AUTOMATICALLY. DEFAULTS TO FALSE: enrolling this
workflow and switching it on are two separate decisions, so a repo can
land the caller, get it reviewed, and start grading later — or stop
grading without reverting anything. When it resolves false a
`pull_request` run does not grade: nothing is read, nothing is
labelled, and no existing `risk:*` label is touched (a disabled run
leaves whatever the last enabled one left, it does not clean up).
A MANUAL `workflow_dispatch` GRADES REGARDLESS (and, when opted in,
posts the sticky comment and the Check Run). Dispatching requires
write access and is an explicit, attributed act, so there is no
unattended behaviour left for this switch to govern — and without that
exemption the staged rollout this input exists for cannot happen: a
repo enrolled OFF could not grade even one PR to see what the grader
would say, so "try it here before turning it on" would require turning
it on. The cost, stated plainly: false is a MUTE on the PR stream, not
a lockout. A repo needing the harder guarantee should remove the
caller — nothing here can bind a maintainer who can edit the workflow
file anyway.
`vars.RISK_CONFIG` on the CALLING repo OVERRIDES this, in both
directions — `{"enabled": true}` switches a caller on with no `with:`
change, and `{"enabled": false}` is a kill switch that needs no PR at
all. That works because the `vars` context inside a reusable resolves
against the caller's repository (the same mechanism groom.yml uses for
`vars.GROOM_CONFIG`). The variable is the OPERATIONAL lever; this input
is the REVIEWED default the variable falls back to when it is absent,
empty, not an object, or carries no boolean `enabled` — a malformed
variable must never be the reason a repo silently stops grading, so it
degrades to this value and says so in an annotation.
SCOPE OF THE KILL SWITCH: it stops the GRADING, not a broken
enrollment. The `workflows_ref` pin contract is enforced BEFORE this
resolves and is not subject to it — the resolver script is itself
loaded from `workflows_ref`, so there is no point in the run at which a
mutable pin could be read as "switched off" without first checking out
the very ref under suspicion. A caller pinned to a branch or a tag
therefore fails red on every PR even with `{"enabled": false}` set. The
fix is the one-line repin (or dropping the caller) — both PRs, but a
caller that cannot say which revision of the tool it runs is not in a
state the variable is meant to cover.
type: boolean
required: false
default: false
permissions:
contents: read
jobs:
# Resolve enablement before anything is read. Its own job rather than a first step of `grade`
# so a disabled repo pays one bare runner instead of booting the grading job to immediately
# no-op, and so the decision — and the reason for it — is visible in the run graph rather than
# buried in a step log. It holds NO token scopes: it reads only its inputs and the caller's
# variable, and must never be the thing that touches a PR.
gate:
name: Resolve enablement
runs-on: ubuntu-latest
# 10, not 5: the ancestry guard below makes two unauthenticated fetches, each up to 3
# attempts at a 40s wall-clock cap plus 15s of backoff (~135s), sequentially — ~270s worst
# case before the tool checkout even starts. 5 minutes left no margin for that plus runner
# setup; a job-timeout kill on top of it would silently swallow the guard's own ::error::
# annotation, the exact cry-wolf the retry/backoff exists to avoid.
timeout-minutes: 10
permissions: {}
outputs:
enabled: ${{ steps.resolve.outputs.enabled }}
steps:
# THE PIN CONTRACT IS ENFORCED, NOT DOCUMENTED. The ref below supplies the code that runs
# in this job, so its SHAPE is checked before any checkout happens: a mutable ref (branch,
# tag, `refs/pull/N/head`) means that code can change after the caller was reviewed, and
# only a full commit SHA is immutable.
#
# SHAPE IS ONE AXIS; ANCESTRY (below) IS THE OTHER — together they are what this file can
# check. Shape alone does not say WHICH commit the ref names: a fork of this PUBLIC repo
# shares its object store, so a fork-authored 40-hex SHA is a perfectly well-shaped ref, and
# a pin left behind when `uses:` moved is too. The ancestry check ~50 lines below closes the
# FIRST case — a fork-authored commit is not, in general, an ancestor of this repo's own
# `main` — leaving only the stale-but-merged-pin case, which needs "equal to the commit
# `uses:` resolved to for this job." `github.workflow_sha` is the CALLER's top-level workflow
# file; `job_workflow_sha` is an OIDC token CLAIM, not a `github` context property (which is
# why `actionlint` rejects `github.job_workflow_sha`), and reading the claim needs
# `id-token: write` from every caller plus an exchange in a job that holds `permissions: {}`
# today. The value IS available as `job.workflow_sha` — the `job`-context accessor added in
# runner v2.334.0 (Apr 2026), which groom.yml reads and which needs no extra grant (BE-8077).
# Asserting equality against it is deliberately NOT done here: it would fail red on every
# caller whose `uses:` and `workflows_ref` have drifted, so it is a caller-contract change
# tracked separately. Until then, reviewing the caller is what bounds that residual.
#
# Enforced in the WORKFLOW, not in a script: the scripts are what the ref loads, so a
# script-side check would sit inside the blast radius it is meant to bound. EVERY job that
# checks out `workflows_ref` re-asserts this itself — `grade` does not inherit its safety
# from this job's step, so neither one can be made unsafe by a later `if:` or a re-ordering
# of the graph. This is also the FIRST checkout in the run, so a bad pin is reported before
# any of this repo's code has executed anywhere.
#
# The value arrives via `env:` and is never interpolated into the script body — inline
# `${{ }}` of the very input being validated is a shell-injection vector. The two copies of
# this step are byte-identical on purpose, and scripts/pr-risk/tests/test_pin_contract.sh
# pins this step's executable body verbatim, fails the build if the copies drift apart, and
# fails it if a job grows a `workflows_ref` checkout without one.
- name: Enforce workflows_ref pin contract
env:
WORKFLOWS_REF: ${{ inputs.workflows_ref }}
run: |
set -euo pipefail
# Never interpolate a raw value into a `::error::`. A multi-line value — exactly what the
# length/character test below exists to catch — would end the annotation at the first
# newline and leave the remainder to be re-parsed by the runner as workflow commands
# (`::add-mask::`, `::stop-commands::`, a forged `::notice::`) in a PUBLIC log. Anything
# outside the ref alphabet becomes `?` and the result is truncated, so what is echoed is
# always one bounded line.
safe_ref=$(printf '%s' "$WORKFLOWS_REF" | tr -c 'A-Za-z0-9._/-' '?')
safe_ref=${safe_ref:0:64}
# LENGTH + CHARACTER CLASS, not `grep -Eq '^[0-9a-f]{40}$'` and not `[[ =~ ]]`: both raise
# an anchoring question this form does not have. `grep` anchors `^…$` per LINE, so a
# multi-line value carrying one SHA-shaped line passes it and is then handed to checkout
# in full; `[[ =~ ]]` anchors the whole string, but whether `$` may ALSO match just before
# a trailing newline is a libc-dependent detail no trust boundary should rest on. A length
# test plus "contains a character outside [0-9a-f]" has no anchors at all — a newline, a
# space or an uppercase letter is simply a character outside the class.
#
# LOWERCASE ONLY, deliberately. The value is handed to `actions/checkout` VERBATIM, so
# accepting a spelling the fetch may not resolve would trade this step's clear message for
# an obscure failure one step later — the opposite of the point. Everything that emits a
# SHA (`git rev-parse`, `gh`, the REST API, the UI) emits lowercase, so a false reject
# here is loud and fixed by lowercasing.
if [[ ${#WORKFLOWS_REF} -ne 40 || "$WORKFLOWS_REF" == *[!0-9a-f]* ]]; then
echo "::error::workflows_ref must be the FULL 40-hex lowercase commit SHA of Comfy-Org/github-workflows (got '${safe_ref}'). Pin it to the SAME SHA you pin uses: to — see the pr-risk.yml header. Branches, tags and refs/pull/N/head are mutable (and PR-head refs resolve fork-authored code), so they are rejected before the tool checkout."
exit 1
fi
# SECOND AXIS — ANCESTRY. Shape proves the value is IMMUTABLE; it does not prove WHICH
# REPOSITORY authored the commit it names. A fork of this PUBLIC repo shares the upstream
# object store, and GitHub serves a fork PR's head objects from THIS repo's URL to an
# UNAUTHENTICATED client — so a fork-authored 40-hex SHA passes the shape test above and is
# then handed to `actions/checkout` and run inside a job holding the caller's write token.
# Assert instead that the commit is REACHABLE FROM UPSTREAM `main`: merged upstream history,
# and nothing else, may be pinned. There is deliberately NO opt-out input for this — an
# opt-out would be set by the very pin-bump PR the check exists to distrust.
#
# THE URL IS A LITERAL ON PURPOSE. Inside a reusable workflow `github.repository` names the
# CALLER, and `github.job_workflow_ref` only repeats the `uses:` string the caller wrote, so
# no context can supply this repo's identity. A variable here would be an alias a fork could
# point back at itself — the exact class of bug this axis closes. Unauthenticated is correct
# (the repo is public), and every failure mode fails CLOSED: a fetch that errors, a `main`
# that cannot be read, or a network outage all leave the step non-zero.
#
# TWO FETCHES, NOT ONE, AND BOTH ARE BOUNDED. The depth-1 fetch is what proves the SHA
# exists in this repo's object network at all; the second brings `main` with an explicit
# refspec (rather than relying on remote-tracking config, which a bare-URL remote has none
# of) to walk down to the pin. `--filter=blob:none` holds both to a commit/tree graph — no
# file CONTENT is ever downloaded, because `merge-base` reads none. That bound matters most
# on the REJECTION path: when the pin is not an ancestor there is no shallow boundary
# anywhere in `main`'s ancestry, so the hostile pin is the case that fetches the MOST.
#
# THE SHALLOW BOUNDARY IS REAL, AND HARMLESS. The depth-1 fetch registers a boundary at the
# pinned commit and the server honours it on the second fetch, so `main` arrives TRUNCATED
# there — it is not a complete history and must not be reasoned about as one. What saves it
# is the direction of the walk: `merge-base --is-ancestor` runs from `main` DOWN to the
# pinned commit and stops on reaching it, so it never needs that commit's parents. Do not
# reorder these two fetches, or drop `--depth=1`, on the strength of a completeness the
# second fetch does not actually deliver.
#
# THE FETCH RETRIES; THE VERDICT DOES NOT. Two unauthenticated round-trips are now a hard
# precondition for every run in every enrolled consumer, and there is no opt-out — so a
# transient github.com error or a secondary rate limit on shared Actions egress would
# otherwise turn one bad minute into a fleet-wide red. Three attempts with a linear backoff
# absorb that, and the low-speed knobs turn a stalled connection into a fast failure rather
# than a hang to the job timeout. What does NOT get retried is the ANSWER: a fetch that
# keeps failing, and a commit that is simply not an ancestor, both still fail the step.
#
# `git` ECHOES THE REF ITSELF on a failed fetch (`upload-pack: not our ref <sha>`), which
# looks like it contradicts the never-emit-the-raw-value rule above. It does not: the shape
# test has already exited on anything that is not exactly 40 characters of `[0-9a-f]`, and
# such a value can carry neither a newline nor a `::`, so it cannot forge a workflow command
# in this PUBLIC log. That ordering is load-bearing — do not move this axis above the shape
# test.
#
# WHAT THIS STILL DOES NOT PROVE is that the pin is the CURRENT one: a stale-but-merged SHA
# is an ancestor too, and lock-step against the commit `uses:` resolved to remains
# unavailable from in here (see the header). Reviewing the caller still bounds that.
#
# THE SCRATCH CLONE IS CLEANED UP on every path, success or rejection. Hosted runners are
# ephemeral and would not care; self-hosted ones would accumulate one clone of `main` per
# run, per job.
if ! scratch=$(mktemp -d); then
echo "::error::A scratch directory for the workflows_ref (${safe_ref}) ancestry check could not be created, so the pin's provenance cannot be established. This is an infrastructure fault, not evidence about the pin. Failing closed."
exit 1
fi
trap 'rm -rf "$scratch"' EXIT
if ! git init -q "$scratch"; then
echo "::error::A scratch git repository for the workflows_ref (${safe_ref}) ancestry check could not be initialized, so the pin's provenance cannot be established. This is an infrastructure fault, not evidence about the pin. Failing closed."
exit 1
fi
# The upstream URL is passed IN by each call site rather than baked in here, so that both
# fetches keep naming the repo literally, in the line a reviewer reads, while sharing one
# retry and transfer-bound policy. See the literal-URL note above for why it is not a var.
fetch_upstream() {
for attempt in 1 2 3; do
if timeout 40 git -C "$scratch" -c http.lowSpeedLimit=1000 -c http.lowSpeedTime=30 fetch --quiet --no-tags --filter=blob:none "$@"; then
return 0
fi
if [ "$attempt" -lt 3 ]; then
sleep $((attempt * 5))
fi
done
return 1
}
if ! fetch_upstream --depth=1 https://github.com/Comfy-Org/github-workflows "$WORKFLOWS_REF"; then
echo "::error::workflows_ref (${safe_ref}) could not be fetched from Comfy-Org/github-workflows after 3 attempts, so its provenance cannot be established. Failing closed."
exit 1
fi
if ! fetch_upstream https://github.com/Comfy-Org/github-workflows +refs/heads/main:refs/heads/upstream-main; then
echo "::error::main of Comfy-Org/github-workflows could not be read after 3 attempts, so workflows_ref (${safe_ref}) cannot be shown to be merged upstream. Failing closed."
exit 1
fi
# `--is-ancestor` HAS THREE OUTCOMES, NOT TWO: 0 is yes, 1 is no, and anything above 1 is
# git failing to answer at all (an unresolvable `upstream-main`, a missing or corrupt
# object, a fetched ref that is not a commit). Collapsing those into the `no` branch would
# report an infrastructure fault to the consumer as "treat the pin-bump that introduced
# this as hostile" — the same cry-wolf the retry above exists to avoid. Both branches still
# fail closed; only the message differs.
ancestry_rc=0
GIT_NO_LAZY_FETCH=1 git -C "$scratch" merge-base --is-ancestor "$WORKFLOWS_REF" upstream-main || ancestry_rc=$?
if [ "$ancestry_rc" -gt 1 ]; then
echo "::error::The ancestry test for workflows_ref (${safe_ref}) could not be COMPLETED — git merge-base failed rather than returning a verdict (see its output above). This is an infrastructure fault, not evidence the pin is hostile. Failing closed."
exit 1
fi
if [ "$ancestry_rc" -ne 0 ]; then
echo "::error::workflows_ref (${safe_ref}) is not an ancestor of Comfy-Org/github-workflows main. A well-shaped SHA can still be a FORK-AUTHORED commit served out of this repo's own object network; only merged upstream commits may be pinned. If this is a genuine just-merged pin, re-run; otherwise treat the pin-bump that introduced it as hostile."
exit 1
fi
# The tool checkout is the same pinned-ref load the grade job does: the resolver is this
# repo's code at `workflows_ref`, never the graded PR's.
- name: Load pr-risk tool
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: Comfy-Org/github-workflows
ref: ${{ inputs.workflows_ref }}
path: _pr_risk_tool
persist-credentials: false
- name: Resolve enabled
id: resolve
env:
INPUT_ENABLED: ${{ inputs.enabled }}
# Empty when the caller repo has no such variable — which is the common case and is
# NOT an error; it just means the reviewed input decides.
RISK_CONFIG: ${{ vars.RISK_CONFIG }}
run: bash _pr_risk_tool/scripts/pr-risk/resolve-enabled.sh
grade:
needs: gate
# `enabled` governs AUTOMATIC grading — the `pull_request` stream. A manual dispatch is
# never blocked by it: running a workflow_dispatch requires write access on the repo and is
# an explicit, attributed act, so there is no unattended behaviour left for the switch to
# govern. Without this clause the staged rollout the switch exists for does not work — a
# repo that enrols OFF cannot grade even a single PR to see what the grader would say, so
# "try it here before turning it on" requires switching the whole repo on first, which is
# the thing being evaluated.
#
# What this costs, stated rather than buried: `{"enabled": false}` stops the PR-stream
# noise, but it does NOT stop a human running the grader by hand. It is a mute, not a
# lockout. If a repo ever needs the harder guarantee, remove the caller — nothing here can
# bind a maintainer who can edit the workflow file anyway.
if: >-
needs.gate.outputs.enabled == 'true' ||
github.event_name == 'workflow_dispatch'
name: Grade PR risk
runs-on: ubuntu-latest
timeout-minutes: 30
outputs:
# The RENDERED Check Run payloads, one per target. Rendering happens HERE — in the job with
# the narrow token — and the separate publish job only ever POSTs strings this job already
# escaped, so the job holding `checks: write` never renders anything it read from a PR.
surfaces: ${{ steps.grade.outputs.surfaces }}
permissions:
contents: read # read the consumer's .github/risk.json override from the BASE ref
issues: write # create the risk:* labels repo-side on first use (repo label CRUD
# maps to Issues; enrolling a repo needs no manual label setup)
pull-requests: write # Sync the one risk label — the labels endpoint is dual-mapped:
# labeling an ISSUE rides issues:write, labeling a PR rides
# pull-requests:write, so `issues: write` alone 403s here
# ("Resource not accessible by integration"). There is no narrower
# grant that can label a PR; this is minimum-necessary, not tidy.
# SCOPE HONESTLY: `pull-requests: write` is a TOKEN-level grant, so
# it also permits commenting, reviewing, editing and closing — the
# label-only guarantee is CODE-level (apply-risk-label.sh is the
# only writer, and the only state it INTENDS to change is the
# mapped label names — its atomic PUT re-sends every unowned label
# as the snapshot read saw it, with one residual an auditor should
# know: a label added or removed by someone else inside that
# read->PUT window is dropped or resurrected respectively; see the
# script's RESIDUAL note), backed by the fact that the job checks
# out no PR code: the only
# code holding this token is this repo's, loaded from the pinned
# workflows_ref. Public repos enroll via `pull_request_target`, so
# that token IS present on fork-authored events — which is exactly
# why workflows_ref must be a full SHA of THIS repo and never a
# mutable ref. A GITHUB_TOKEN-applied label also cannot fire
# `labeled` triggers, so the write cannot start a cascade.
checks: read # the check rollup the reversibility axis reads (CheckRun contexts)
actions: read # the same rollup's CheckRun -> checkSuite -> workflowRun traversal
# (self-exclusion keys on workflowRun.databaseId, and workflow runs
# are an Actions resource — without this the WHOLE GraphQL read
# errors and every PR grades unknown/ungraded)
statuses: read # the same rollup's legacy commit-status contexts
env:
REPO: ${{ github.repository }}
# The target list. `pr_numbers` wins over `pr_number`, and with neither supplied this is
# the event's own PR — so the event path resolves to the same single number it always did.
PR_NUMBERS: ${{ inputs.pr_numbers || inputs.pr_number || github.event.pull_request.number }}
# The event PR's base ref, passed through ONLY when no number was supplied. When one WAS,
# the target may be a different PR than the event's (or there may be no event at all), and
# the event's base ref would then point the override read at the wrong branch — so the
# by-number path re-reads it from the API per target instead. The expression reads oddly
# because GitHub's `||` yields the first TRUTHY operand: the `&& ... || ''` tail is how a
# conditional passthrough is spelled, and it yields '' both when a number was supplied and
# when there is no pull_request payload to read.
BASE_REF: ${{ (inputs.pr_number || inputs.pr_numbers) == '' && github.event.pull_request.base.ref || '' }}
GH_TOKEN: ${{ github.token }}
steps:
# The same guard the `gate` job applies, restated here rather than inherited from it. This
# job is where the stakes are: the checked-out code runs holding the job's `GH_TOKEN`, which
# carries `pull-requests: write`, and public repos enroll via `pull_request_target` so that
# token is present on fork-authored events. A job must not depend on ANOTHER job's step for
# its own trust boundary — see the `gate` job above for the full rationale.
- name: Enforce workflows_ref pin contract
env:
WORKFLOWS_REF: ${{ inputs.workflows_ref }}
run: |
set -euo pipefail
# Never interpolate a raw value into a `::error::`. A multi-line value — exactly what the
# length/character test below exists to catch — would end the annotation at the first
# newline and leave the remainder to be re-parsed by the runner as workflow commands
# (`::add-mask::`, `::stop-commands::`, a forged `::notice::`) in a PUBLIC log. Anything
# outside the ref alphabet becomes `?` and the result is truncated, so what is echoed is
# always one bounded line.
safe_ref=$(printf '%s' "$WORKFLOWS_REF" | tr -c 'A-Za-z0-9._/-' '?')
safe_ref=${safe_ref:0:64}
# LENGTH + CHARACTER CLASS, not `grep -Eq '^[0-9a-f]{40}$'` and not `[[ =~ ]]`: both raise
# an anchoring question this form does not have. `grep` anchors `^…$` per LINE, so a
# multi-line value carrying one SHA-shaped line passes it and is then handed to checkout
# in full; `[[ =~ ]]` anchors the whole string, but whether `$` may ALSO match just before
# a trailing newline is a libc-dependent detail no trust boundary should rest on. A length
# test plus "contains a character outside [0-9a-f]" has no anchors at all — a newline, a
# space or an uppercase letter is simply a character outside the class.
#
# LOWERCASE ONLY, deliberately. The value is handed to `actions/checkout` VERBATIM, so
# accepting a spelling the fetch may not resolve would trade this step's clear message for
# an obscure failure one step later — the opposite of the point. Everything that emits a
# SHA (`git rev-parse`, `gh`, the REST API, the UI) emits lowercase, so a false reject
# here is loud and fixed by lowercasing.
if [[ ${#WORKFLOWS_REF} -ne 40 || "$WORKFLOWS_REF" == *[!0-9a-f]* ]]; then
echo "::error::workflows_ref must be the FULL 40-hex lowercase commit SHA of Comfy-Org/github-workflows (got '${safe_ref}'). Pin it to the SAME SHA you pin uses: to — see the pr-risk.yml header. Branches, tags and refs/pull/N/head are mutable (and PR-head refs resolve fork-authored code), so they are rejected before the tool checkout."
exit 1
fi
# SECOND AXIS — ANCESTRY. Shape proves the value is IMMUTABLE; it does not prove WHICH
# REPOSITORY authored the commit it names. A fork of this PUBLIC repo shares the upstream
# object store, and GitHub serves a fork PR's head objects from THIS repo's URL to an
# UNAUTHENTICATED client — so a fork-authored 40-hex SHA passes the shape test above and is
# then handed to `actions/checkout` and run inside a job holding the caller's write token.
# Assert instead that the commit is REACHABLE FROM UPSTREAM `main`: merged upstream history,
# and nothing else, may be pinned. There is deliberately NO opt-out input for this — an
# opt-out would be set by the very pin-bump PR the check exists to distrust.
#
# THE URL IS A LITERAL ON PURPOSE. Inside a reusable workflow `github.repository` names the
# CALLER, and `github.job_workflow_ref` only repeats the `uses:` string the caller wrote, so
# no context can supply this repo's identity. A variable here would be an alias a fork could
# point back at itself — the exact class of bug this axis closes. Unauthenticated is correct
# (the repo is public), and every failure mode fails CLOSED: a fetch that errors, a `main`
# that cannot be read, or a network outage all leave the step non-zero.
#
# TWO FETCHES, NOT ONE, AND BOTH ARE BOUNDED. The depth-1 fetch is what proves the SHA
# exists in this repo's object network at all; the second brings `main` with an explicit
# refspec (rather than relying on remote-tracking config, which a bare-URL remote has none
# of) to walk down to the pin. `--filter=blob:none` holds both to a commit/tree graph — no
# file CONTENT is ever downloaded, because `merge-base` reads none. That bound matters most
# on the REJECTION path: when the pin is not an ancestor there is no shallow boundary
# anywhere in `main`'s ancestry, so the hostile pin is the case that fetches the MOST.
#
# THE SHALLOW BOUNDARY IS REAL, AND HARMLESS. The depth-1 fetch registers a boundary at the
# pinned commit and the server honours it on the second fetch, so `main` arrives TRUNCATED
# there — it is not a complete history and must not be reasoned about as one. What saves it
# is the direction of the walk: `merge-base --is-ancestor` runs from `main` DOWN to the
# pinned commit and stops on reaching it, so it never needs that commit's parents. Do not
# reorder these two fetches, or drop `--depth=1`, on the strength of a completeness the
# second fetch does not actually deliver.
#
# THE FETCH RETRIES; THE VERDICT DOES NOT. Two unauthenticated round-trips are now a hard
# precondition for every run in every enrolled consumer, and there is no opt-out — so a
# transient github.com error or a secondary rate limit on shared Actions egress would
# otherwise turn one bad minute into a fleet-wide red. Three attempts with a linear backoff
# absorb that, and the low-speed knobs turn a stalled connection into a fast failure rather
# than a hang to the job timeout. What does NOT get retried is the ANSWER: a fetch that
# keeps failing, and a commit that is simply not an ancestor, both still fail the step.
#
# `git` ECHOES THE REF ITSELF on a failed fetch (`upload-pack: not our ref <sha>`), which
# looks like it contradicts the never-emit-the-raw-value rule above. It does not: the shape
# test has already exited on anything that is not exactly 40 characters of `[0-9a-f]`, and
# such a value can carry neither a newline nor a `::`, so it cannot forge a workflow command
# in this PUBLIC log. That ordering is load-bearing — do not move this axis above the shape
# test.
#
# WHAT THIS STILL DOES NOT PROVE is that the pin is the CURRENT one: a stale-but-merged SHA
# is an ancestor too, and lock-step against the commit `uses:` resolved to remains
# unavailable from in here (see the header). Reviewing the caller still bounds that.
#
# THE SCRATCH CLONE IS CLEANED UP on every path, success or rejection. Hosted runners are
# ephemeral and would not care; self-hosted ones would accumulate one clone of `main` per
# run, per job.
if ! scratch=$(mktemp -d); then
echo "::error::A scratch directory for the workflows_ref (${safe_ref}) ancestry check could not be created, so the pin's provenance cannot be established. This is an infrastructure fault, not evidence about the pin. Failing closed."
exit 1
fi
trap 'rm -rf "$scratch"' EXIT
if ! git init -q "$scratch"; then
echo "::error::A scratch git repository for the workflows_ref (${safe_ref}) ancestry check could not be initialized, so the pin's provenance cannot be established. This is an infrastructure fault, not evidence about the pin. Failing closed."
exit 1
fi
# The upstream URL is passed IN by each call site rather than baked in here, so that both
# fetches keep naming the repo literally, in the line a reviewer reads, while sharing one
# retry and transfer-bound policy. See the literal-URL note above for why it is not a var.
fetch_upstream() {
for attempt in 1 2 3; do
if timeout 40 git -C "$scratch" -c http.lowSpeedLimit=1000 -c http.lowSpeedTime=30 fetch --quiet --no-tags --filter=blob:none "$@"; then
return 0
fi
if [ "$attempt" -lt 3 ]; then
sleep $((attempt * 5))
fi
done
return 1
}
if ! fetch_upstream --depth=1 https://github.com/Comfy-Org/github-workflows "$WORKFLOWS_REF"; then
echo "::error::workflows_ref (${safe_ref}) could not be fetched from Comfy-Org/github-workflows after 3 attempts, so its provenance cannot be established. Failing closed."
exit 1
fi
if ! fetch_upstream https://github.com/Comfy-Org/github-workflows +refs/heads/main:refs/heads/upstream-main; then
echo "::error::main of Comfy-Org/github-workflows could not be read after 3 attempts, so workflows_ref (${safe_ref}) cannot be shown to be merged upstream. Failing closed."
exit 1
fi
# `--is-ancestor` HAS THREE OUTCOMES, NOT TWO: 0 is yes, 1 is no, and anything above 1 is
# git failing to answer at all (an unresolvable `upstream-main`, a missing or corrupt
# object, a fetched ref that is not a commit). Collapsing those into the `no` branch would
# report an infrastructure fault to the consumer as "treat the pin-bump that introduced
# this as hostile" — the same cry-wolf the retry above exists to avoid. Both branches still
# fail closed; only the message differs.
ancestry_rc=0
GIT_NO_LAZY_FETCH=1 git -C "$scratch" merge-base --is-ancestor "$WORKFLOWS_REF" upstream-main || ancestry_rc=$?
if [ "$ancestry_rc" -gt 1 ]; then
echo "::error::The ancestry test for workflows_ref (${safe_ref}) could not be COMPLETED — git merge-base failed rather than returning a verdict (see its output above). This is an infrastructure fault, not evidence the pin is hostile. Failing closed."
exit 1
fi
if [ "$ancestry_rc" -ne 0 ]; then
echo "::error::workflows_ref (${safe_ref}) is not an ancestor of Comfy-Org/github-workflows main. A well-shaped SHA can still be a FORK-AUTHORED commit served out of this repo's own object network; only merged upstream commits may be pinned. If this is a genuine just-merged pin, re-run; otherwise treat the pin-bump that introduced it as hostile."
exit 1
fi
- name: Load pr-risk tool
# The grader + default map come from THIS workflow's repo (public, pinned
# via workflows_ref) — never from the graded PR. No PR code is checked
# out anywhere in this job.
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: Comfy-Org/github-workflows
ref: ${{ inputs.workflows_ref }}
path: _pr_risk_tool
persist-credentials: false
- name: Grade the target PR(s) and sync the label
id: grade
# The per-target sequence — resolve the base ref, fetch that ref's override files, poll
# the grader until the rest of the rollup settles, sync the one label — lives in
# scripts/pr-risk/grade-targets.sh rather than inline here, so the event path and the
# by-number path cannot drift into two copies of it, and so it is unit-testable at all
# (tests/test_grade_targets.sh drives every branch with a stubbed `gh`). It grades ONE
# target or fifty through the same code; a batch's per-target failures are recorded and
# the remaining targets still graded.
env:
MAP_PATH: ${{ inputs.repo_map_path }}
RB_PATH: ${{ inputs.repo_runbooks_path }}
FLEET_LOGINS: ${{ inputs.fleet_logins }}
BOT_LOGINS: ${{ inputs.bot_logins }}