Add PR self-merge rubric by hahn-kev-bot · Pull Request #2471 · sillsdev/languageforge-lexbox

hahn-kev-bot · 2026-07-23T08:24:46Z

Adds a repo-specific rubric that lets internal authors decide, on their own, whether a PR can be merged without a second human review — aimed at keeping genuinely low-stakes changes from burdening reviewers, given the growing volume of agent-generated PRs.

The rubric is default-closed (uncertain → get a human) and self-sufficient about our danger zones — it does not assume the advisory AI review caught them.

Preconditions: CI green + an advisory AI review has run (Devin / CodeRabbit / your agent). The rubric only governs whether a second human is still required.
Green lane: a PR is self-mergeable iff every changed file is green, judged by its (path, change-type) pair.
- Universal green (docs, formatting, new tests, i18n) — safe on any path.
- Conditional green (everything else) — green only outside the 🔴 always-human paths, subject to a size ceiling (≤150 net non-green lines + coherence) and a test if it changes behavior.
🔴 Always-human zones mirror our AGENTS.md CRITICAL flags and sentinel agents: FwLite sync core, FwHeadless, migrations, Harmony bumps, auth/secrets, breaking GraphQL schema, deployment, etc.
Attestation on every PR: the reviewer-of-record honesty check (read / explain / blast-radius / no-mystery-code), anchored in our "Integrity is non-negotiable" ethos.

Interactive version (tick-through artifact): https://claude.ai/code/artifact/7792cc42-991f-4d2e-8990-7a582b29162c — private until shared from the artifact's share menu.

Test plan

Docs-only change; no code paths affected. Renders as GitHub-flavored Markdown.

Discussion points

Where it should live — docs/pr-self-merge-rubric.md for now; open to linking it from AGENTS.md if we adopt it.
The 150-line ceiling is a provisional anchor to tune during a trial.
A path/size helper tool was deliberately left out of scope; the rubric is written tool-able (globs + computable line count) so it can be built later.

🤖 Generated with Claude Code

Scaffold the per-repo config the engineering skills expect: a local markdown issue tracker under .scratch/ (git-ignored), the five canonical triage-role strings, and multi-context domain-doc consumer rules. The pointer block lives in AGENTS.md so all agents pick it up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

A repo-specific, default-closed rubric internal authors apply to decide whether a PR can merge without a second human review (given green CI and an advisory AI review). Encodes the always-human danger zones from our AGENTS.md CRITICAL flags and sentinel agents, and defines the green lane by change-type and size. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

coderabbitai · 2026-07-23T09:09:03Z

Warning

Review limit reached

@hahn-kev-bot, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 46 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f486e364-3ce5-4b85-9944-92426ca09f30

📥 Commits

Reviewing files that changed from the base of the PR and between 17d693e and 42b4bcc.

📒 Files selected for processing (6)

.gitignore
AGENTS.md
docs/agents/domain.md
docs/agents/issue-tracker.md
docs/agents/triage-labels.md
docs/pr-self-merge-rubric.md

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch claude/pr-self-review-rubric-6f5c2e

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

myieye · 2026-07-23T09:46:47Z

Did you mean to add these 3 agents files? (and the matching gitignore and agents.md changes)

myieye

The various usages of "green" confused me, so I tried to decrease it.

myieye · 2026-07-23T09:49:36Z

+2. **Its path is not 🔴** *and* its change-type is **conditional-green**
+   (everything else) *and* it clears the gates in §2 that apply to it.


"(everything else)" means the current wording is saying redundant i.e.:

not red and not red

Suggested change

2. **Its path is not 🔴** *and* its change-type is **conditional-green**

(everything else) *and* it clears the gates in §2 that apply to it.

2. **Its path is not 🔴** *and* it clears the gates in §2.

myieye · 2026-07-23T09:50:31Z

+
+---
+
+## 2. Gates — apply to every *conditional-green* file


Suggested change

## 2. Gates — apply to every *conditional-green* file

## 2. Gates — apply to files that are not 🔴

myieye · 2026-07-23T09:54:25Z

+- **Size (always).** ≤ **150 net changed lines** of *reviewable, non-green* code
+  — excluding generated files, universal-green files, lockfiles, and snapshots —
+  **AND** it's *one coherent change you can hold in your head* (not 40 lines
+  smeared across 15 unrelated files).


Suggested change

- **Size (always).** ≤ **150 net changed lines** of *reviewable, non-green* code

— excluding generated files, universal-green files, lockfiles, and snapshots —

**AND** it's *one coherent change you can hold in your head* (not 40 lines

smeared across 15 unrelated files).

- **Size (always).** ≤ **150 net changed lines** of *reviewable* code

— excluding generated files, universal-green files, lockfiles, and snapshots —

**AND** it's *one coherent change you can hold in your head* (not 40 lines

smeared across 15 unrelated files).

myieye · 2026-07-23T10:02:16Z

+even a 🔴 path. **Conditional** is green only outside 🔴, and is subject to the §2
+gates.
+
+| Change-type | Reach | Test needed? / Notes |


"Test needed" should be its own column for readability/scanability.

myieye · 2026-07-23T10:04:21Z

+- **Generated files** (`frontend/viewer/src/lib/dotnet-types/generated-types/**`,
+  `.po` catalogs) are *transparent*: classify the PR by the source edit; they're
+  excluded from the size count.


Already mostly stated higher up (except for .po, which is only partially generated, because you can add translator comments, but I'd still classify it as generated)

Suggested change

- **Generated files** (`frontend/viewer/src/lib/dotnet-types/generated-types/**`,

`.po` catalogs) are *transparent*: classify the PR by the source edit; they're

excluded from the size count.

myieye · 2026-07-23T10:04:36Z

+- **On the two tiers.** An earlier draft split conditional-green into
+  "zone-limited" and "gated." They had identical *reach* (green only outside 🔴,
+  same size + attestation) and differed only by the test requirement — so
+  they're merged, and "behavior change → test" is just a rule in §2.


I don't think recording this is helpful.

Suggested change

- **On the two tiers.** An earlier draft split conditional-green into

"zone-limited" and "gated." They had identical *reach* (green only outside 🔴,

same size + attestation) and differed only by the test requirement — so

they're merged, and "behavior change → test" is just a rule in §2.

hahn-kev and others added 2 commits July 17, 2026 16:29

myieye reviewed Jul 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add PR self-merge rubric - #2471

Add PR self-merge rubric#2471
hahn-kev-bot wants to merge 2 commits into
developfrom
claude/pr-self-review-rubric-6f5c2e

hahn-kev-bot commented Jul 23, 2026

Uh oh!

coderabbitai Bot commented Jul 23, 2026

Review limit reached

Uh oh!

myieye Jul 23, 2026 •

edited

Loading

Uh oh!

myieye left a comment

Uh oh!

myieye Jul 23, 2026

Uh oh!

myieye Jul 23, 2026

Uh oh!

myieye Jul 23, 2026

Uh oh!

myieye Jul 23, 2026

Uh oh!

myieye Jul 23, 2026

Uh oh!

myieye Jul 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		2. Its path is not 🔴 and its change-type is conditional-green
		(everything else) and it clears the gates in §2 that apply to it.

	2. Its path is not 🔴 and its change-type is conditional-green
	(everything else) and it clears the gates in §2 that apply to it.
	2. Its path is not 🔴 and it clears the gates in §2.

	## 2. Gates — apply to every conditional-green file
	## 2. Gates — apply to files that are not 🔴

	- Generated files (`frontend/viewer/src/lib/dotnet-types/generated-types/**`,
	`.po` catalogs) are transparent: classify the PR by the source edit; they're
	excluded from the size count.

Uh oh!

Uh oh!

Conversation

hahn-kev-bot commented Jul 23, 2026

Test plan

Discussion points

Uh oh!

coderabbitai Bot commented Jul 23, 2026

Review limit reached

Uh oh!

myieye Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

myieye left a comment

Choose a reason for hiding this comment

Uh oh!

myieye Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

myieye Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

myieye Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

myieye Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

myieye Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

myieye Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

myieye Jul 23, 2026 •

edited

Loading