feat(workflow): one review gate, three checkers — CodeRabbit, Codex, local attestation (PP-w6u1) - #2117
feat(workflow): one review gate, three checkers — CodeRabbit, Codex, local attestation (PP-w6u1)#2117timothyfroehlich wants to merge 4 commits into
Conversation
Trial of CodeRabbit as a replacement for the Codex review gate, which is weekly-quota-metered and currently blocking merges when exhausted. Config points at REVIEW.md and docs/NON_NEGOTIABLES.md as canonical rather than restating them (AGENTS.md §8), with the three rules REVIEW.md flags as "each of these has shipped a real bug" inlined as a safety net. request_changes_workflow is the substantive setting: CodeRabbit requests changes while findings are open and approves once they are resolved and the latest commit is reviewed. Its approval attests that automated review is complete; it does not authorize merging, which stays Tim's call (PP-wi85). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FnQa5Wn2UDaDebFTCR4tX3
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reachedNext included review available in 31 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (18)
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe PR adds ChangesCodeRabbit Review Configuration
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~5 minutes Change: Other Merge Risk: ⚪ Minimal · up to The configuration retains repository-wide coverage for the cited safety rules, so no actionable merge risk remains. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
timothyfroehlich
added
the
Claude
Claude
|
@coderabbitai review Triggering manually while this stays a draft — PinPoint promotes a PR out of draft only after current-head CI succeeds, and this PR exists to observe review behaviour, so a manual trigger is the cheaper path than changing the draft convention. Generated by Claude Code |
|
|
|
@coderabbitai review Generated by Claude Code |
|
|
|
@coderabbitai review |
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
…te (PP-w6u1) CodeRabbit is an extra, Tim-initiated review for large PRs. Its native APPROVED review pinned to the exact head is now a third accepted record in Gate 3, alongside Codex evidence and the manual attestation. Nothing else CodeRabbit posts changes the Codex-derived state. pr-watch.py and pr-dashboard.py apply the same rule so wait and dashboard verdicts match. .coderabbit.yaml turns auto_review off: the plan's budget is one included review per hour, so agents request it only when Tim says to (gh pr comment <PR> --body "@coderabbitai review"). Also fixes check_ci for a head carrying two CI Gate runs (draft promotion re-runs the workflow on the same SHA): the unfiltered jq returned two objects, status read as "COMPLETED\nCOMPLETED", and the gate sat in WAIT forever. It now picks the authoritative run the way merge-pr.sh's poller and pr-watch.py already do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
timothyfroehlich
changed the title
Review fixes from /code-review medium on 82d55a6: - _compute_review_state read the three-field verdict with `read -r`, which collapses whitespace runs; a review whose commit_id GitHub returned as null has an empty sha field, so the reviewer login shifted into RS_REVIEW_SHA and the gate printed "Codex approved chatgpt". The verdict is now TSV and split with cut. Regression test with a null commit_id. - CI Gate authoritative-run jq is one shared CI_GATE_SELECT_JQ used by check_ci and merge-pr.sh's re-poll; the comment no longer claims pr-watch.py ranks the same way (it does not). - scripts/workflow/AGENTS.md: three remaining "Codex only" rows updated. - .coderabbit.yaml header says what the path instructions actually restate. - Tests for a CodeRabbit non-approval with no Codex record (bash + pr-watch). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…w6u1) Replace the twelve-state review machine with `_review_summary`: one fetch of the PR's reviews and comments, run through three independent checkers (CodeRabbit approval, Codex evidence, local attestation), any of which covers head. The label is one of approved / changes requested / stale review / not reviewed. pr-watch.py and pr-dashboard.py now shell out to the bash gate instead of mirroring it; their Python state machines and parity tests are gone. merge-handoff.sh and request-codex-review.sh read the same summary. Gate fixes surfaced by the rewrite: - a failed reviews/comments/threads fetch fails the summary instead of reading as "no evidence" (`|| return 1`, `pipefail` in the shell-outs) - a DISMISSED/PENDING Codex review on head is "none", not "stale" Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
timothyfroehlich
changed the title
|
@coderabbitai review |
|
|
@codex review |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
2 participants
Summary
Closes PP-w6u1.
Experiment result (commit 1,
afd9e4f): CodeRabbit's approval lands as a real GitHubAPPROVEDreview fromcoderabbitai[bot]pinned to the head SHA, and Tim confirmed it counts toward a required-approvals rule. The plan's budget is one included review per hour, so it is not replacing Codex — it is an extra review for large PRs, requested only when Tim says so.The review gate, rewritten (commit 4,
44275b5). The first cut bolted CodeRabbit onto a twelve-state precedence machine that three scripts each mirrored in their own language. It is now one function with three independent checkers:_review_summary <PR>inscripts/workflow/_pr-gates.shfetches the PR's reviews and comments once and asks three questions — does CodeRabbit approve head, does Codex cover head (native approval, exact-head finding review, trusted clean comment, or reaction witness), does a local attestation pin head. Any yes passes Gate 3. Checkers never consult each other, so a CodeRabbitCHANGES_REQUESTEDcannot mask a Codex approval and a stale marker cannot hide a current finding review.approved/changes requested/stale review/not reviewed. The FAIL block prints one line per checker so you can see what each reviewer's newest evidence names.pr-watch.pyandpr-dashboard.pyshell out to the bash gate instead of re-implementing it. Their Python state machines and the bot-login / state-vocabulary parity tests are deleted. The dashboard's Review column is the label, or?when the gate could not answer.merge-handoff.shandrequest-codex-review.shread the same JSON.Two gate bugs surfaced by the rewrite, both fixed: a failed reviews/comments/threads fetch read as "no evidence" (the
gh … | jqpipe hid the exit code; fetches now fail the summary), and aDISMISSED/PENDINGCodex review on head reported asstalerather thannone..coderabbit.yaml—auto_review.enabled: false. Manual-only; auto-reviewing every draft promotion would spend the hourly budget on small PRs. Agents never request CodeRabbit on their own; the trigger isgh pr comment <PR> --body "@coderabbitai review"when Tim says so. Docs: AGENTS.md §5, REVIEW.md,scripts/workflow/AGENTS.md(checker table replaces the state table),pinpoint-pr-workflowPhase 3.Also fixed (commit 2) —
check_cion a head with twoCI Gateruns. Draft promotion re-runs the workflow on the same SHA; this PR carried twoCOMPLETED/SUCCESSentries and the unfiltered jq returned both, so the gate sat inWAITforever. It now picks the authoritative run (live over cancelled, then newest), shared withmerge-pr.sh's poller.Test Plan
pnpm run checkgreen;pnpm run check:python588 passed.test_pr_gates.py(integration-marked, 69),test_pr_dashboard.py(26, drives the real gate through a fakegh),test_pr_watch.py(85),test_merge_handoff.py,test_request_codex_review.py,test_merge_pr_automerge.py— all green, run together twice.44275b5: gate,pr-watch --check-ready,merge-handoff.sh, andpr-dashboard.shall agree —stale review, CodeRabbit newest evidence namesafd9e4f./code-review+mark-review.sh; Codex quota is out) — the CodeRabbit approval onafd9e4fis stale by design.—Claude
🤖 Generated with Claude Code