[1f6a06a2] PR-review gate: verify ACs literally and require green CI before pr_pass (#428)

* [a1bde3b9] Add CI-status guard to pr_pass + update pr_reviewer prompt (#417) (#420)

* [a1bde3b9] feat(gateway): CI-status guard on pr_pass + reviewer prompt update

* [a1bde3b9] docs(pr-gate-review, worksession-git): document CI-status guard on pr_pass

Updated two architecture documentation files to reflect the new CI-status guard:

**pr-gate-review.md:**
- Documented _ci_status_guard method: blocks pr_pass on failing/pending/unscheduled/error CI with reviewer-aware pr_fail remediation
- Documented _resolve_ci_status: best-effort GitHub check-runs lookup with fail-open behavior
- Updated _pr_pass_blocked description: now returns (rejection_envelope, ci_note) tuple
- Updated _record_gate_verdict_for/verdict to note ci_status field stamping on pr_pass
- Added ci_note parameter documentation for evidence tracking when no CI is configured
- Updated Logical Tree to show new methods
- Added Config Flags note: CI guard is always armed, fails open on config gaps
- Added two regression risks: check-runs-only limitation, fail-open design

**worksession-git.md:**
- Documented GitService.get_pr_ci_status(project_slug, pr_number): CI status lookup with state classification
- Documented supporting methods: _ci_status_prereqs, _fetch_check_runs, _classify_check_runs, _classify_zero_check_runs
- Each method notes its fail-open behavior and configuration gap handling

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>
Co-authored-by: Backend Documenter <be-doc@roboco.tech>

* [e8f275d7] test(gateway): lock the 7-AC-to-test map + assert pr_reviewer prompt content (#425) (#426)

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>

* [24b4237e] Fix reflow-check, CI-status classification, and noqa suppression (#440) (#443)

* [24b4237e] fix(gateway): classify unreachable/nonexistent CI-status repo as no_ci_configured, remove test noqa, reflow pr_reviewer.md

Split GitService.get_pr_ci_status's PR-head-sha lookup into a dedicated
helper so a config gap (missing project/git_url/token) or an unreachable/
nonexistent repo/PR (network error or 404) classifies as no_ci_configured
(pr_pass passes through and stamps the evidence note) while a genuine
GitHub API failure on a real, reachable repo (any other non-2xx, or an
unparseable body) stays the fail-closed error state. Replaced the
`# noqa: PLR2004` in test_git_pr_ci_status.py with a named HTTP-status
range constant, updated the config-gap tests to assert the new
classification, and added tests for the unreachable-repo and real-repo-
API-error branches. Reflowed agents/prompts/roles/pr_reviewer.md's one
hard-wrapped continuation line so it passes make reflow-check.

* [24b4237e] docs(gateway): update pr-gate-review.md for CI-status classification refactor

Updated the internal architectural map to reflect the new CI-status classification
scheme introduced in PR #440. Configuration gaps (missing project/git_url/token) and
unreachable/nonexistent repos (404 or network error) now explicitly classify as
no_ci_configured and pass through with evidence stamps. Genuine GitHub API failures
on reachable repos classify as error and stay fail-closed (retryable).

- Clarified _ci_status_guard behavior: config gaps/unreachable repos pass through
  with distinct classification; only real API failures stay fail-closed
- Updated Config Flags section to describe the new three-way classification
- Updated Regression Risks section to document the new explicit classification scheme
- Noted that _resolve_ci_status now wraps git.get_pr_ci_status and interprets its result dict

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>
Co-authored-by: Backend Documenter <be-doc@roboco.tech>

* [1f6a06a2] round-3 fixes: pr_gate back to xenon rank A; 404 means no CI, not error

Eight extracted helpers bring the module average from B(5.05) to A(4.04)
with every external contract untouched (170 gate tests byte-identical).
The CI-status guard now classifies a 404 on the check-runs or workflows
endpoints as no_ci_configured (pass-through with evidence note) —
a repo without Actions is not a transport failure — reserving the
fail-closed error state for network/5xx/auth failures, with pinning
tests for all four shapes. The e2e fake-GitHub router gains check-runs
and workflows routes so the scripted lifecycle exercises the guard's
green-CI success branch end to end.

* [1f6a06a2] merge master; align gate-diff-base tests with the tuple contract

The merged tree is the first integration of the CI-status guard with the
preferred-parent diff-base guard: _pr_pass_blocked now returns
(rejection, ci_note), so the diff-base tests unpack it instead of
asserting on a bare result. Both guards verified live in the merged
pr_gate (preferred_parent threading and _ci_status_guard present).

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>
Co-authored-by: Backend Documenter <be-doc@roboco.tech>
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
Renzo F
2026-07-11 07:38:06 +02:00
committed by GitHub
co-authored by Backend Developer 1 Backend Documenter Renn F
parent 15a3e87a2f
commit 4d52f6ff59
10 changed files with 1270 additions and 117 deletions
+13 -5
View File
@@ -17,12 +17,14 @@ The two choreographer mixins that implement the PR-reviewer's two distinct surfa
| PRGateMixin.pr_pass | method | roboco/services/gateway/choreographer/pr_gate.py:115 | Pass the assembled PR: awaiting_pr_review → awaiting_pm_review; delegates to _gate_decision |
| PRGateMixin.pr_fail | method | roboco/services/gateway/choreographer/pr_gate.py:123 | Fail the assembled PR with concrete issues → needs_revision; rejects empty issues list, formats issues into notes, delegates to _gate_decision |
| PRGateMixin._gate_preflight | method | roboco/services/gateway/choreographer/pr_gate.py:148 | Ownership + role + spec gate (with self_review_block via actor_slug/original_developer_slug) + free-text soup guard for pr_pass/pr_fail; returns rejection Envelope or (t,agent,role_str,briefing,spec_ctx) |
| PRGateMixin._record_gate_verdict_for | method | roboco/services/gateway/choreographer/pr_gate.py:229 | Author the canonical pr_review verdict note before the transition; on pr_fail also capture the assembled PR head SHA for the unchanged-PR gate |
| PRGateMixin._record_gate_verdict_for | method | roboco/services/gateway/choreographer/pr_gate.py:229 | Author the canonical pr_review verdict note before the transition; on pr_fail also capture the assembled PR head SHA for the unchanged-PR gate; on pr_pass with ci_note, stamp the ci_status field into the verdict with evidence the CI guard ran |
| PRGateMixin._post_gate_review | method | roboco/services/gateway/choreographer/pr_gate.py:245 | Post the gate verdict to the PR itself (best-effort, after the DB transition); resolves reviewer slug |
| PRGateMixin._deliver_pr_fail_to_owner | method | roboco/services/gateway/choreographer/pr_gate.py:253 | a2a the pr_fail change-requests to the owning PM (best-effort) with a Main-PM-root steer to re-delegate not re-submit; closes the blind re-submit loop |
| PRGateMixin._gate_decision | method | roboco/services/gateway/choreographer/pr_gate.py:292 | Shared body for pr_pass/pr_fail: preflight + tracing + pr_pass blocked guards + record verdict + run_intent + None-guard for concurrent transition + post-PR + a2a on fail |
| PRGateMixin._pr_pass_blocked | method | roboco/services/gateway/choreographer/pr_gate.py:373 | Refuse pr_pass on a broken toolchain or block-level convention violation; pr_fail stays available; both guards inert when their flag is off |
| PRGateMixin._record_gate_verdict | method | roboco/services/gateway/choreographer/pr_gate.py:403 | Persist the gate verdict as the canonical pr_review structured note (passed/failed), with issues slot for pr_fail and head_sha stamp; best-effort (ContentValidationError logged not raised) |
| PRGateMixin._pr_pass_blocked | method | roboco/services/gateway/choreographer/pr_gate.py:373 | Refuse pr_pass on a broken toolchain, block-level convention violation, or non-green CI on the assembled PR's head commit; returns (rejection_envelope, ci_note). Both toolchain and conventions guards inert when their flags are off; CI guard fails open on configuration gaps |
| PRGateMixin._ci_status_guard | method | roboco/services/gateway/choreographer/pr_gate.py:520 | Refuse pr_pass unless CI on the assembled PR's head commit is green. Failing/pending/unscheduled CI blocks with reviewer-aware remediation pointing at pr_fail; configuration gaps, unreachable/nonexistent repos, or real API failures on reachable repos each pass through with distinct classifications (no_ci_configured vs error) via git.get_pr_ci_status |
| PRGateMixin._resolve_ci_status | method | roboco/services/gateway/choreographer/pr_gate.py:480 | Thin wrapper: calls git.get_pr_ci_status, interprets the returned dict (no_ci_configured/pending/failure/error/success), and composes a rejection Envelope if CI must block pr_pass |
| PRGateMixin._record_gate_verdict | method | roboco/services/gateway/choreographer/pr_gate.py:403 | Persist the gate verdict as the canonical pr_review structured note (passed/failed), with issues slot for pr_fail, head_sha stamp for pr_fail, and ci_status evidence on pr_pass; best-effort (ContentValidationError logged not raised) |
| PRGateMixin._capture_pr_head_sha | method | roboco/services/gateway/choreographer/pr_gate.py:468 | Best-effort capture of the assembled PR head SHA at pr_fail time via _project_slug_for + git.get_pr_head_sha; returns None on any failure (fail-open) |
| PRGateMixin._post_gate_review_to_pr | method | roboco/services/gateway/choreographer/pr_gate.py:502 | Post APPROVE/REQUEST_CHANGES on cell→root PRs; always COMMENT on root→master (only CEO merges master); best-effort |
| PRGateMixin._gate_role_or_rejection | method | roboco/services/gateway/choreographer/pr_gate.py:545 | Parse the role enum from role_str or return a not_authorized rejection Envelope |
@@ -85,8 +87,10 @@ pr-gate-review slice
│ ├── _gate_decision — shared body (preflight→tracing→blocked→record→run_intent→None-guard→post→a2a)
│ ├── _gate_preflight — ownership/role/spec-gate (self_review_block) + soup guard
│ ├── _gate_tracing — journal:learning + pr_reviewer_notes min chars
│ ├── _pr_pass_blocked — toolchain-broken + conventions block guards
│ ├── _record_gate_verdict_for / _record_gate_verdict — structured pr_review note (+ issues + head_sha)
│ ├── _pr_pass_blocked — toolchain-broken + conventions block + CI-status guards (returns rejection, ci_note)
│ ├── _ci_status_guard — refuse pr_pass on failing/pending/unscheduled CI; config gaps + unreachable repos pass through with evidence stamp; real API failures stay fail-closed
│ ├── _resolve_ci_status — thin wrapper calling git.get_pr_ci_status, interprets result dict, returns rejection Envelope if CI must block
│ ├── _record_gate_verdict_for / _record_gate_verdict — structured pr_review note (+ issues + head_sha + ci_status)
│ ├── _re_stamp_pr_fail_head_sha_if_advanced — re-capture head SHA post-transition and re-stamp verdict note if advanced (#189)
│ ├── _capture_pr_head_sha — best-effort PR head SHA for unchanged-PR gate
│ ├── _post_gate_review / _post_gate_review_to_pr — PR review post (COMMENT on root→master or MegaTask root-subtask)
@@ -125,6 +129,7 @@ pr-gate-review slice
- ROBOCO_TOOLCHAIN_MATCH_ENABLED (gates _toolchain_broken_guard in _pr_pass_blocked — inert when off)
- ROBOCO_CONVENTIONS_ENABLED (gates _conventions_guard in _pr_pass_blocked — inert when off)
- ROBOCO_PR_REVIEWER_NOTES_MIN_CHARS / settings.pr_reviewer_notes_min_chars (tracing gate substantive-note threshold for pr_pass/pr_fail/post_pr_review)
- CI-status guard is always armed when the toolchain can reach get_pr_ci_status via git service. Configuration gaps (missing project/git_url/token) and unreachable/nonexistent repos (404 or network error) classify as no_ci_configured and pass through with evidence stamp. Genuine API failures on reachable repos classify as error and stay fail-closed (retryable). A project with no CI configured at all also passes through cleanly (no_ci_configured). The guard never blocks pr_pass on a misconfigured project.
## Gotchas
@@ -168,6 +173,9 @@ pr-gate-review slice
| Title | File:Line | Claim | Severity |
|---|---|---|---|
| CI-status guard reads GitHub check-runs only, not legacy commit-status API | roboco/services/gateway/choreographer/pr_gate.py:520 | _ci_status_guard and get_pr_ci_status read only the check-runs API endpoint. A repo whose only CI signal is the legacy commit-status API would show zero check-runs and be classified as no_ci_configured (passes through). Noted as ponytail-comment in git.py with the upgrade path if a project ever needs it. | low |
| CI-status classification: config gaps and unreachable repos now explicitly no_ci_configured | roboco/services/git.py:_resolve_ci_head_sha | Missing project/git_url/git-token, or unreachable/nonexistent repo (404 on PR head lookup or network error), all classify as no_ci_configured → pr_pass passes through with evidence stamp. Only genuine GitHub API failures on reachable repos classify as error → pr_pass stays fail-closed (retryable). By design: configuration gaps should not block the gate, but real API failures should fail-closed to avoid false-green verdicts. A deliberately misconfigured project's CI is silently not enforced, with clear evidence in the verdict note. | low |
|---|---|---|---|
| self_review_block could fire if a reviewer is also the original developer | roboco/services/gateway/choreographer/pr_gate.py:202 | actor_slug=str(reviewer_agent_id) + original_developer_slug=markers.get_original_developer(t). The comment asserts dormancy because the marker is never set on assembled coordination tasks. If a future change sets the marker on an assembled task (or a reviewer UUID coincides with the recorded dev UUID), pr_pass/pr_fail would be refused as self-review with no remediate path. The defense is correctly wired but unguarded by a test asserting dormancy. | low |
| ~~resolve_task_project_slug cell_projects branch can raise AttributeError on malformed mapping~~ **FIXED 536bbb64 #82** | roboco/services/gateway/choreographer/pr_review.py:594 | ~~sorted(cell_map, key=lambda m: m.team.value) assumes every mapping has a non-None team with .value. _capture_pr_head_sha wraps the slug call in try/except (fail-open), but _post_gate_review_to_pr calls self._project_slug_for(t) WITHOUT a try/except — a malformed cell_map mapping would raise and abort the verdict PR post (best-effort but the exception escapes the helper, caught only by the outer try in _post_gate_review_to_pr's git.post_pr_review call, NOT the slug resolution).~~ _post_gate_review_to_pr now wraps the slug-resolve call in its own try/except (mirrors _capture_pr_head_sha) — a malformed mapping logs and returns, no longer 500s the reviewer after the committed gate transition. The underlying AttributeError possibility in resolve_task_project_slug remains but is contained. | medium |
| ~~_is_hand_formatted_verdict false-positive on summaries quoting PR-added headers~~ **FIXED 536bbb64 #188** | roboco/services/gateway/choreographer/pr_review.py:174 | ~~Substring match on '## summary'/'## issues'/'## verdict'/'## findings' in lowercased body. A reviewer summarizing a PR that itself adds a '## Summary' section (quoting it in the body) with findings=[] would be falsely refused.~~ Regex now anchored to line-start (^[ \t]*## ..., re.MULTILINE) — quoted headers (> ## Summary) and mid-prose mentions no longer trip the guard. | low |
+5
View File
@@ -64,6 +64,11 @@ This slice is the git substrate every delivery agent works on. `GitService` runs
| `GitService.update_pr_for_task` | method | git.py:2506 | Patch PR title/body; 404→typed GitError |
| `GitService.get_pr_head_sha` | method | git.py:2451 | PR head SHA for pr_fail re-submit loop guard (fail-open) |
| `GitService.get_latest_ci_conclusion` | method | git.py:1929 | Per-project CI signal (unknown never false-green) |
| `GitService.get_pr_ci_status` | method | git.py:2662 | CI status of a PR's current head commit for the in-path pr_pass gate; returns {state, failing_checks?, head_sha} or None on config gaps (fail-open) |
| `GitService._ci_status_prereqs` | method | git.py:2698 | Resolve (owner, repo, auth headers, head_sha) for CI-status lookup or None on any gap |
| `GitService._fetch_check_runs` | method | git.py:2730 | GET check-runs for head_sha; None on any API failure |
| `GitService._classify_check_runs` | method | git.py:2770 | State classification: success/failure/pending from check-run conclusions list |
| `GitService._classify_zero_check_runs` | method | git.py:2790 | State classification when zero check-runs exist: pending_not_scheduled or no_ci_configured (depends on workflow count) |
| `GitService.list_open_prs` | method | git.py:1844 | Normalized open-PR list |
| `GitService.post_pr_review` | method | git.py:2329 | Post reviewer comments via GitHub API |
| `GitService.merge_pull_request` | method | git.py:2914 | GitHub merge API + method fallback + already-merged disambiguation |