feat(ci): add opt-in review gate to GitHub Action and GitLab CI - #1461
Open
Qiyuanqiii wants to merge 5 commits into
Open
Qiyuanqiii wants to merge 5 commits into
Qiyuanqiii wants to merge 5 commits into
Conversation
Contributor
|
🔍 OpenCodeReview found 2 issue(s) in this PR.
|
Qiyuanqiii
force-pushed
the
codex/result-gate-ci
branch
from
September 19, 2026 14:41
1abb5e8 to
fd79de9
Compare
Qiyuanqiii
marked this pull request as ready for review
September 19, 2026 14:45
7 tasks
Contributor
Author
|
I forgot to add the relevant md docs; I’ll complete them at noon. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Summary
ocr gateover the original review JSON.Dependency and Review Scope
Depends on #1457; merge the shared CLI gate before this integration. This PR targets upstream
main, so its aggregate diff currently includes the prerequisite gate implementation as well as the CI integration.8d612afb0975955d96c3ecfddc171c5f9b2f4b58, reviewed separately in feat: add opt-in gate for saved review results #1457.f9dd1a058669f12c0f106997527a1b9afb07c776andfd79de92ebc4d913348c29103f3538a42dd9815e. Documentation completion:b49c13079ca4a7e583b618eca6eec4e2ced59fa3. Together they cover 14 integration, test, and documentation files relative to the prerequisite.After #1457 is merged, this branch can be rebased onto the resulting upstream history so the prerequisite disappears from the diff, including if the prerequisite is squash-merged.
Motivation and Scope
A successful review process is not sufficient evidence that a CI review requirement has been met. A budget-limited run may leave selected files unreviewed, a complete run may report blocking findings, and a publishing failure may leave contributors without the resulting feedback. Separate interpretations of these outcomes in each CI integration can produce different merge decisions for the same review artifact.
#1457 supplies the common offline policy evaluator. This PR connects the GitHub Action and GitLab example to it while retaining each platform's responsibility for process execution, publication, and artifact retention. For example, a partial budget-limited review can still publish useful findings, but its gate remains inconclusive and the job fails after diagnostics have been saved. Conversely, a gate pass cannot turn a failed review or publishing step into a successful job.
The optional gate also rejects budget-only incomplete coverage through the saved manifest. This complements #1030's operational partial-failure exit policy and works independently of that PR.
Usage and Defaults
Add these inputs to an existing GitHub Action step:
For the GitLab example:
gate/ GitLabOCR_GATEfalsefail_on_severitycritical,high,medium, orlowthreshold; requiresgate: 'true'.OCR_FAIL_ON_SEVERITYwith the gate enabledOCR_FAIL_ON_SEVERITYwith the gate disabledEnabled severity thresholds ignore case and surrounding whitespace. Invalid policy values fail before installation/model calls. Empty severity leaves the severity check disabled while retaining coverage, delivery, revision, and publication requirements.
Opting in requires an OCR build containing the
gatecommand. Select it through the GitHubocr_versioninput or GitLabOCR_VERSIONvariable. Both integrations probeocr gate --helpbefore calling the model and report an upgrade instruction if the pinned CLI lacks support.GitHub Action Implementation
gate_exit_code, which is empty when disabled or not reached. This describes only the gate command, not overall job success.GitLab CI and Publication Implementation
post_review.py --require-publicationwith its legacy severity setting cleared; the CLI evaluates severity over all original findings.OCR_SUMMARY_PUBLISHEDto the dotenv statistics.artifacts: when: always. Gate files remain empty when disabled or not reached.Outcome Rules
The table assumes other required stages succeed unless stated otherwise:
code_commentfailure or an expected revision mismatchCompatibility and Boundaries
Type of Change
How Has This Been Tested?
Validation Environment
1.26.5, Node.js24.14.0, Python3.14, and Git Bash.8d612afb0975955d96c3ecfddc171c5f9b2f4b58.fd79de92ebc4d913348c29103f3538a42dd9815e.Initial Implementation Validation
make license-addand projectmake checkthroughmingw32-makewith Git Bash: license headers, English-only source check, module tidying, formatting, andgo vet.make test: complete Go suite with the race detector.node scripts/github-actions/action-contract.test.js: 55 tests passed, including the seven new gate contract cases and their scenario matrices.python -m unittest discover -s examples/gitlab_ci -p '*_test.py': 156 tests passed across publication and executable pipeline contracts.action.yml, the GitLab pipeline, and the modified workflow using the repository's YAML dependency.git diff --check, and LF-only verification of all nine changed files. Rangit add --renormalize .before committing.The full
npm run test:github-actionscommand is not claimed as passing locally: after the passing publication, translation, and Action suites, its existing plugin-contract suite fails to create a symbolic link with WindowsEPERM. The same failure was reproduced at the unchanged prerequisite baseline. Linux CI must confirm that suite and the newly added GitLab contract job.Documentation Completion
Commit
b49c13079ca4a7e583b618eca6eec4e2ced59fa3updates the five website CI guides and the GitHub Actions example README. The guides cover the opt-in settings, CLI requirement, three gate decisions, full-range review cost, publication failures, and review/gate artifacts. GitLab setup now includespost_review.py, the version variable, and project-relative.ocr/paths. GitHub troubleshooting explains the effect of failed inline comments when the gate is enabled.Documentation checks passed: UTF-8/LF, Markdown fences, Python snippet syntax, five-language configuration consistency, translation sync, repository links, and
git diff --check. The 55 Action contract tests passed after the documentation update.Regression and Integration Coverage
Executed 10 additional offline integration scenarios through the actual YAML shell blocks using the built
ocr gatebinary: complete review, high-severity finding, budget-only partial review, mismatched head, and non-zero review exit with an otherwise passing artifact, for both platforms. Review and hosting operations were test doubles; the gate command was real. No live model or hosting API was used in these integration scenarios.Pre-commit Review
Completed the repository-required
ocr review --audience agent --background ...with exit0: 4 selected production files reviewed, 0 skipped; no critical, high, or medium findings. The test and documentation diffs were also inspected separately.OCR reported two low-severity suggestions, both checked against the implementation before committing:
safe.directory '*'setup is unnecessary for the new GitLab contract job: the harness places its Git doubles first inside Bash and runs the pipeline in temporary fixtures, rather than operating on the checkout with real Git.The checkpoint condition now consistently consumes the normalized validation output. A follow-up pre-commit review of that change completed with 0 findings; all 55 Action contract tests and
make checkpassed. The synchronized prerequisite also rejects zero-count tool failure entries: its gate/CLI race tests andmake checkpassed, with 100% gate-package coverage. The initial model review recovered two transient request errors and completed the selected scope.Known Limitations and Follow-up
Checklist
AI/LLM Disclosure
Codex (GPT-6) was used to implement inspect the changes, run validation and prepare this PR description. The repository-required model review used open-code-review CLI v1.11.9 with
deepseek-v4-flash.Related Issues and Pull Requests
Depends on #1457. Tracked in #1489 alongside #1030. Related operational failure report: #1027.