Repository navigation
Conversation
A ruleset rejection can name a required workflow and a required status check at once. The renderer returned on the first group it found, so the status checks were unreachable whenever a workflow was named too, and the summary listed conditions that pass while omitting the one that blocks. Observed on python-workflows#84: all three reported workflows had passed, and the sole blocker was the status context "pre-commit.ci - pr", which never appeared. Each kind now carries its own verb. A single verb is read from the outcome wording nearest the quoted names, so on a combined message the workflows borrowed the status check's "failing" and were reported as failed when they had merely not finished. Those two call for opposite responses: one needs a human, the other only needs waiting. Bullets are labelled only when both kinds appear, so every existing single-kind report renders exactly as before. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
A merge rejection states what was unsatisfied at the instant the merge was attempted, which routinely names required checks that had merely not finished. Nothing re-read that before the summary printed, so a pull request that went green moments later was still reported as failed, under a cause that had stopped being true. Two of the four failures sampled from a 157-PR run were fully mergeable by the time the run printed them: test-python-project#396 and docker-workflows#70 were both clean, and the second had been judged before the rebase it was sent had completed. The confirmation step already re-reads every about-to-be-reported failure, and the payload it fetches carries mergeable_state, so settling this costs no further request. A PR GitHub would merge now is recorded as UNSETTLED and counted apart from FAILED: one needs another run, the other needs a human, and reporting both the same way sent operators looking for causes that did not exist. An unknown mergeable_state stays a failure. GitHub computes mergeability in the background, so unknown is an absence of evidence rather than evidence of a block, but it is equally not evidence the PR would merge, and only that would justify withdrawing a failure. UNSETTLED is deliberately neither PENDING, the initial non-terminal state every result starts in, nor AUTO_MERGE_PENDING, where GitHub finishes the merge server-side without another run. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
A rejection message is preferred over state-based inference because it usually carries the actionable cause. On a ruleset rejection it frequently does not: it lists the rules GitHub evaluated, including ones that had merely not finished, and it can omit the condition actually holding the merge. python-workflows#84 was reported against three required workflows. All three had passed. The sole blocker was the status context "pre-commit.ci - pr", which the message never mentioned. A blocked PR is now re-read and the blocking set derived from live state. Two sources are needed, because GitHub reports through two mechanisms that do not overlap: Actions workflows report as check runs, while pre-commit.ci, DCO and similar integrations report as commit status contexts. A view built from check runs alone cannot see that context at all, which is how the wrong answer was reached. Status contexts are filtered to those the base branch requires, so an advisory integration cannot be presented as the reason a merge was refused. Check runs are deliberately not filtered that way: a required workflow declared by a ruleset never appears among the required status contexts, so filtering on that list would discard the names a ruleset rejection is about. Only blocked PRs are re-examined. A conflicted, behind or draft PR is already described accurately by its state, so reading its checks would spend requests without adding anything. An empty reading is not an all-clear. When nothing can be established the recorded reason stands, so a token that cannot read rulesets leaves the operator no worse off than before. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
Two gaps left a stalled pre-commit.ci in place, and the context is a required status check, so the PR stayed blocked either way. The stuck test accepted only pending. Any other state read as a reported result, which lumped error in with success and failure. The commit status API separates "the check failed" from "the checker failed", and pre-commit.ci uses that separation: failure is the hooks reporting a genuine problem with the change, error is a run that never completed -- an upload that 5xx'd, a container that died. The first reports the same result however often it is re-run. The second routinely clears on the next attempt, and until it does nobody has reached a verdict at all. Age is not consulted for an error. A terminal state does not become more stuck with time, and there is nothing left to await. A genuine failure is still left alone, so re-running one cannot cost a comment and a five-minute wait to be told the same thing. The repair gate ran only for blocked. That state does cover pending required checks, so the gap was narrower than it first appears: it was unknown that was skipped, which is what GitHub reports while it works mergeability out in the background -- the window a freshly pushed PR sits in, and exactly the PRs a stalled check repair exists for. Acting there is safe because the repair's own preconditions never consult the mergeable state. Per-PR timing needed no change: the pending age already comes from the status's own updated_at, not from a run-wide clock. The existing parametrised test asserted that error posts no comment, and now asserts the opposite, with the distinction it turns on named in the class docstring. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
The run now reports an outcome the README did not describe, and the right response to it does not follow from the word alone: unsettled means re-run, where every neighbouring category means either nothing or investigate. Lists all seven together rather than documenting the new one in isolation, since the value lies in the contrast. Records why unsettled exists at all, so a reader meeting a non-zero count knows it is not a softened failure, and notes that a failure now names the condition blocking the merge. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
b80e167 to
315b1d6
Compare
315b1d6 to
2164796
Compare
2164796 to
3795777
Compare
1dfd112 to
365fb16
Compare
365fb16 to
4606bd4
Compare
Fifteen review rounds; the substantive corrections are grouped by what
they change rather than by which round found them. Several were defects
in code this branch added while fixing an earlier finding.
Reading the wrong commit. The probes took head_sha and base_branch from
the snapshot captured before the merge attempt. A dependabot rebase
moves the head and this tool requests those rebases, so the PR most
likely to reach the diagnosis with a stale head is one the run itself
moved -- docker-workflows#70, an issue fixture, among them. Both now
come from the refreshed payload.
Reconciling too early. The per-PR confirmation fires as each task
completes, which in an owner-wide run is long before anything prints, so
a pull request refused at minute two and clean by minute ten was still
reported failed. That is the issue's own acceptance criterion. Every
failure now gets one last look after all tasks finish, costing one
request per PR still reported as failed, with the tracker tally
corrected alongside so the live counts and the summary agree.
Withdrawing failures that were never verdicts. The run reports FAILED
for its own troubles too: an unhandled exception, a rebase that did not
complete, a 502, a missing token scope. None says anything about
mergeability, so a later clean read must not rewrite them as unsettled
and bury the message explaining what to fix. The classification travels
with the reason, decided from the HTTP status by allowlist -- 405 and
409 -- so an unrecognised status, or an exception carrying none, keeps
its message. Withdrawal also requires affirmative mergeability, matching
_state_is_waitable, and a closed PR never qualifies.
Absence inferred from partial reads. Check runs asked for one page;
the combined-status endpoint defaults to thirty and was asked for no
page size at all. Each feeds a message that replaces the recorded
reason, and each reasons about absence, so a short page read as an
all-clear for whatever sat on page two. Both paginate, as do the issue
comments the duplicate-nudge check reads. A failed probe was likewise
indistinguishable from an empty answer -- the 503-PR audit records
GET /orgs/{org}/rulesets returning 403 -- so each probe now reports
whether it answered, and the required-check lookup exposes the
reliability signal it was previously using only to decide caching. A
proven blocker still stands on its own evidence; an unproven one
requires every probe to have answered.
Blaming the wrong thing. Names are matched case-sensitively, as GitHub
matches them. Required workflows are matched against Actions workflow
*run* names rather than check-run names, which are two namespaces:
codeql.yml declares the workflow CodeQL and the job Audit Repository.
Workflow runs collapse to the latest per name, so a re-run does not
leave a superseded failure to be blamed. FAILING_CONCLUSIONS gains
action_required and startup_failure, both terminal and both blocking;
stale stays out, since GitHub applies it to a run a later push made
irrelevant. Everything failing is now named exactly once, either as a
blocker or as also-failing.
Retriggering that did not work. An error status sits on the commit
until pre-commit.ci replaces it, so the first poll after a nudge
returned the very failure being retried and ended the wait. The reading
that prompted the trigger is carried into polling. A missing status is
stuck only once GitHub has settled the PR, since on an unsettled one it
cannot be told from one that has not propagated. Suppression is scoped
to the incident by timestamp.
Reporting. get_results_summary exposes the new outcome, in both its
populated and empty shapes, so the counts add up for a programmatic
consumer. The note holding GitHub's rejection is written only when
nothing is noted yet, since the end-of-run pass would otherwise replace
it with our own live reading.
Deliberately not included. Several findings concerned inputs that need
a rule name containing our delimiters, a workflow name containing a
comma, two workflow files sharing a name, or an app-pinned requirement.
Each was implemented and then removed: the hardening cost more in
machinery and tests than the failures it prevented are worth, and none
of them reaches the reported cause, which a blocked PR now reads from
live API state rather than from the rejection string. rule_violations
is left close to its original shape for the same reason. Also declined:
rulesets are not enumerated to resolve required workflow names, since a
ruleset names a workflow by file path while a run carries its name:,
and that endpoint returned 403 throughout the audited run.
Splitting the pre-commit.ci status readers and the poll into their own
modules keeps the file under the size gate. Split rather than
suppressed, along a seam the code already had.
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
4606bd4 to
e2543f0
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
Live blocker attribution can be wrong for app-pinned checks and can degrade during final reconciliation.
Review details
Suppressed comments (2)
src/dependamerge/merge_manager/_not_mergeable.py:246
- The end-of-run pass no longer has the original rejection in
result.error: the first diagnosis replaced it withblocked by required workflow: …and preserved the rejection inwarning. Parsing the rewritten text yields no required workflow names, so a still-failing workflow is downgraded on the second pass to merelyfailing checks. Reuse the preserved rejection when present so required-workflow attribution remains stable across reconciliation.
rejection = result.error or ""
src/dependamerge/merge_manager/_live_blockers.py:190
- This drops
integration_id, so an app-pinned requirement is treated as required by context name alone. A same-named failing status from another GitHub App does not satisfy or block that requirement, but this code reports it as the proven blocker. Since the failing-status probe carries no source identity, exclude app-pinned entries from promotion (as the PR description requires) or propagate app identity through the probe.
return {
str(entry.get("context", "")).strip()
for entry in required
if isinstance(entry, dict) and entry.get("context")
}, True
- Files reviewed: 30/30 changed files
- Comments generated: 0 new
- Review effort level: Balanced
Review follow-ups capturedFifteen review rounds produced 36 findings. The substantive ones are fixed on this branch; everything declined or deliberately removed now has an issue, so nothing rests only in a resolved thread.
The removed work in #495 is archived with its tests at Declined outright, not deferredThree findings are decisions rather than debts, and are recorded in the commit body so they are not re-litigated by a future reviewer:
A note for whoever reviews nextIf Copilot is asked for another pass, it will re-raise most of #495 — those findings were valid, and we judged the machinery not worth it rather than judging them wrong. Worth reading as a recorded trade rather than as regression. |
Closes #482.
A 157-PR owner-wide run reported 17 failures. Sampling four showed the stated cause was wrong in every case, and two were fully mergeable by the time the run printed them as failed.
A merge rejection is a snapshot. GitHub states what was unsatisfied at the instant the merge was attempted, which routinely names required checks that had merely not finished — and nothing re-read that before the summary printed.
Semantic Pull Request 🛠️clean— nothing wrong with itSemantic Pull Request 🛠️clean— judged before its rebase landedpre-commit.ci - prwas the blockerAI Slop Scan 🧹,Zizmor Scan 🌈All four are fixtures in
tests/test_merge_failure_diagnosis.py.What changes
1. A status check no longer hides behind a workflow
_format_failure_reasonreturned on the first group it found, so a rejection naming both showed only the workflows. Each kind is now listed, with its own verb read from its own clause — workflows that have not finished routinely accompany a context that has already failed.2. Unfinished is not failed
A PR GitHub would merge now is recorded as
UNSETTLEDand counted apart fromFAILED: one needs another run, the other needs a human. The confirmation step already re-read every reported failure, and the payload it fetches carriesmergeable_state, so the reconciliation itself costs no extra request.Three guards, each earned during review:
FAILEDfor its own troubles — an exception, a rebase that did not complete, a 502, a missing token scope. Decided from the HTTP status by allowlist (405,409), so an unrecognised status, or an exception carrying none, keeps its message rather than losing it._state_is_waitable. A closed PR never qualifies.3. Name what is blocking, without over-claiming
Read against the live head — a dependabot rebase moves it, and this tool requests those rebases. Three sources, weighted by what they prove:
name:codeql.ymldeclares the workflowCodeQLand the jobAudit Repository, so matching a rejection against check-run names compares two vocabularies that need not agree.All three paginate; each reports whether it answered; a proven blocker stands on its own evidence, an unproven one requires every probe to have answered. Everything failing is named exactly once.
4. Retry a run that never reached a verdict
erroris pre-commit.ci failing to complete,failureis the hooks reporting a real problem — only the first is retried. The repair gate coversunknownas well asblocked, though a missing status still needs a settled PR. Suppression is scoped to the incident by timestamp, and the poll ignores the pre-trigger reading — otherwise the first poll returns the very error being retried.Corrections to the issue
Recorded in-thread:
blockedcovers pending required checks;unknownwas the gap._pending_agealready uses the status's own timestamp.MergeStatus.PENDINGis the initial non-terminal state.Scope
Fifteen Copilot rounds produced 36 findings. Many were genuine defects in code this branch added while fixing an earlier one — the stale head, the over-broad refusal classification, the unpaginated status read (that endpoint defaults to 30), and the end-of-run reconciulation. Those are all here.
A cluster of hardening was implemented and then deliberately removed, about 680 lines. It addressed inputs that need a rule name containing our delimiters, a workflow name containing a comma, two workflow files sharing a
name:, an app-pinned requirement, or more than a thousand status contexts. The machinery and its tests cost more than the failures they prevent are worth, and none of it reaches the reported cause — which a blocked PR now reads from live API state rather than from the rejection string.rule_violations.pyis consequently left close to its original shape (+59 lines, against +171 at the peak).Also declined, with reasons: rulesets are not enumerated to resolve required workflow names (a ruleset names a workflow by file path while a run carries its
name:, andGET /orgs/{org}/rulesetsreturned 403 throughout the 503-PR run indocs/BULK_RUN_PERFORMANCE_AUDIT.md);staleis not treated as failing (GitHub applies it to a run a later push made irrelevant).One review finding is correct but pre-existing and out of scope — required checks from rulesets and branch protection are cumulative, and only one is consulted. Filed as #494 rather than folded in: it touches four callers, costs a request per repo/branch, and tightens
reliablefor all of them.Validation
pytestbasedpyright --warnings src testsmypy src testsruff check/format --checkreuse lintaislopmainprekEvery fix was reverted in isolation and its tests shown to fail.
_precommit_ci.pycrossed the 400-line gate during review and was split rather than suppressed, following #475's precedent:_precommit_status.py(what pre-commit.ci said) and_precommit_wait.py(reading it, and waiting).One existing test asserted that a pre-commit.ci
errorposts no comment, and now asserts the opposite, with the distinction it turns on named in the class docstring.Also here
#429closed as completed — verified that shorthand targets and local-checkout defaulting both landed with #475.