[Benkapner]

Closes #1800.

Problem

When REVIEW_FINDING_SEVERITY_THRESHOLD filters away every finding, the posted review reads only "Looks good to me". To a PR author that is indistinguishable from a clean pass, even though the agent reported findings that the configured threshold withheld.

Change

post-review already filters the structured findings array as defense-in-depth, and already computes original_count and filtered_count. This appends a disclosure to the review body when, and only when, that filter actually dropped something:

• fully filtered: No findings at or above `high` severity; 13 lower-severity finding(s) were not posted.
• partly filtered: 2 finding(s) below the `high` severity threshold were not posted.

Both counts are derived from the same filter that drops the findings, so the disclosed number cannot disagree with what was posted. The body-append uses the existing jq '.body = (.body + $notice)' idiom already used for the label and footer notices.

Why the post-script and not the template

The "Looks good to me" body comes from the agent, via the template in skills/pr-review/SKILL.md. The agent cannot know what the post-script will filter, because filtering runs afterwards. Putting the disclosure in the post-script is what makes the count consistent with what was actually posted, which is the issue's third acceptance criterion. It also keeps this change out of SKILL.md, which #1813 is scheduled to rewrite.

Acceptance criteria

• With threshold high and only medium/low/info findings, the body states the threshold and the count of omitted findings.
• With no findings at all, the body is unchanged (a plain pass).
• The threshold-hidden count is consistent with the post-script's defense-in-depth filtering of the structured findings array.

Tests

Three cases added to scripts/post-review-test.sh, using the existing run_body_count_test helper, which asserts against the body actually handed to the forge:

• severity-threshold-discloses-fully-filtered
• severity-threshold-discloses-partially-filtered
• severity-threshold-silent-when-nothing-filtered

run_body_count_test gained an optional 5th parameter for the threshold, defaulting to low so every existing caller is unaffected.

bash scripts/post-review-test.sh passes (444 tests). make check-bundle and make lint are clean, and pre-commit run passes on the changed files. The generated post-review.sh was produced by make script-build, not edited directly.

Notes for review

• The optional collapsed <details> list of omitted findings mentioned in the issue is deliberately left out to keep the change minimal. Happy to add it as a follow-up if wanted.
• The issue also asks that the disclosure survive a re-review sticky being reduced to an "unchanged" message (Sticky re-review comments should surface the specific facts re-verified, not just "unchanged" #1353). The disclosure is attached to the body that post-review hands to the forge, so it follows that body, but I did not find a dedicated "unchanged" reduction path in post-review.src.sh to test against directly. Pointers welcome if there is one I missed.

[qodo-code-review]

PR Summary by Qodo

Disclose findings omitted by the severity threshold

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

AI Description

• Disclose the threshold and omitted-finding count when severity filtering removes findings.
• Distinguish fully filtered reviews from clean passes without changing reviews where nothing was
 filtered.
• Test full, partial, and absent filtering against the body sent to the forge.

Diagram

graph TD
  A["Agent result"] --> B["Severity filter"] --> C{"Findings dropped?"}
  C -->|no| F["Forge review"]
  C -->|yes| D{"Any retained?"}
  D -->|no| E["Downgrade and disclose"] --> F
  D -->|yes| G["Disclose omissions"] --> F

Loading

High-Level Assessment

Appending the notice in post-review is appropriate: it uses the counts from the filter that determines what is posted. An agent-template notice was considered but cannot know the post-filter outcome.

Files changed (3) +54 / -1

Bug fix (2) +32 / -0

post-review.shInclude threshold disclosures in the generated post-review script +16/-0

Include threshold disclosures in the generated post-review script

• Mirrors the source-script change in the generated executable, appending a notice only when filtering drops findings.

scripts/post-review.sh

post-review.src.shDisclose omitted findings after severity filtering +16/-0

Disclose omitted findings after severity filtering

• Uses the existing before-and-after finding counts to append a threshold-specific notice to the review body. Distinguishes fully filtered reviews from partially filtered ones and leaves unchanged results alone.

scripts/post-review.src.sh

Tests (1) +22 / -1

post-review-test.shTest disclosures for filtered and clean reviews +22/-1

Test disclosures for filtered and clean reviews

• Allows body-count tests to specify a severity threshold while preserving the existing default. Adds cases for fully filtered, partly filtered, and genuinely clean reviews using the body captured at the forge boundary.

scripts/post-review-test.sh

[fullsend-ai-review]

🤖 Finished Review · ⏭️ Skipped (PR #1826 is already closed) · Started 1:27 PM UTC · Completed 1:27 PM UTC

Commit: e6bf2d1 · View workflow run →

Effort: high

[fullsend-ai-review]

Review skipped — this PR is already closed.

The /fs-review command only reviews open PRs/MRs.

_{Posted by fullsend pre-review check}

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Threshold tests can miss wrong findings 🐞 Bug ⚙ Maintainability

Description

run_body_count_test only counts the expected text in .body, so the two new threshold cases never
inspect the posted action or findings payload. A regression that leaves a blocking
request-changes action or drops or retains the wrong findings would still pass these tests as long
as the disclosure sentence remains.

Code

scripts/post-review-test.sh[R1718-1720]

+run_body_count_test "severity-threshold-discloses-fully-filtered" \
+  "${THRESHOLD_ALL_HIDDEN}" \
+  'No findings at or above `high` severity; 2 lower-severity finding(s) were not posted.' "1" "high"

Relevance

●●● Strong

Recent precedents accept strengthening tests to verify actual payloads and filtered actions, not
merely generic output text.

PR-#1710
PR-#1085

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The helper captures and checks only .body, exit status, and result-file existence, while the new
cases invoke it for both filtered scenarios. The production path also changes the action and
structured findings when filtering removes all findings, so those behaviors are not exercised by the
new integration tests.

scripts/post-review-test.sh[1635-1664]
scripts/post-review-test.sh[1718-1725]
scripts/post-review.sh[1308-1351]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The new severity-threshold integration tests validate only that the disclosure text appears in the posted body. They do not verify the structured result that the post script sends, so incorrect action downgrades or finding filtering could pass unnoticed.

### Fix Focus Areas
- scripts/post-review-test.sh[1609-1664]
- scripts/post-review-test.sh[1718-1725]

### Recommended Fix
Extend the helper or add a dedicated assertion helper that reads the captured posted JSON and checks the fully filtered case has action `comment` with no `findings` key, while the partially filtered case retains only the high-severity finding. Keep the existing body assertions so the disclosure wording and structured payload are both covered.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

---

Context sources

✅ Tickets: 🎫 Review output should say how many findings were hidden by REVIEW_FINDING_SEVERITY_THRESHOLD instead of a bare "Looks good to me" 🎫 Sticky re-review comments should surface the specific facts re-verified, not just "unchanged" 🎫 review: render the review body and hidden SHA comment with render-review-body.sh

✅ Compliance rules (platform): 57 rules

✅ Skills: 4 invoked

  code-review

  code-implementation

  pr-review

  docs-review

✅ Cross-repo context — repo relationships

  Explored: repo: fullsend-ai/autonomy-analysis (sha: 46717359) — View relationship

  Explored: repo: fullsend-ai/fullsend (sha: e6820106) — View relationship

Review mode: Auto: 🚀 Fast: Localized shell-script disclosure change with focused tests and limited behavioral scope.

Tip of the day

💡 Did you know, you can turn on the rule miner and Qodo learns your standards from review history

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[qodo-code-review]

1. Threshold tests can miss wrong findings 🐞 Bug ⚙ Maintainability

run_body_count_test only counts the expected text in .body, so the two new threshold cases never
inspect the posted action or findings payload. A regression that leaves a blocking
request-changes action or drops or retains the wrong findings would still pass these tests as long
as the disclosure sentence remains.

Agent Prompt

### Issue description
The new severity-threshold integration tests validate only that the disclosure text appears in the posted body. They do not verify the structured result that the post script sends, so incorrect action downgrades or finding filtering could pass unnoticed.

### Fix Focus Areas
- scripts/post-review-test.sh[1609-1664]
- scripts/post-review-test.sh[1718-1725]

### Recommended Fix
Extend the helper or add a dedicated assertion helper that reads the captured posted JSON and checks the fully filtered case has action `comment` with no `findings` key, while the partially filtered case retains only the high-severity finding. Keep the existing body assertions so the disclosure wording and structured payload are both covered.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗