mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
feat(lifecycle): revision findings ledger — structured failure feedback, persisted and delivered down the chain (#486)
* feat(lifecycle): revision findings ledger — structured QA/PR/PM/CEO failure feedback, persisted and delivered down the chain Every bounce used to survive only as flattened prose: rounds overwrote each other in notes_structured, request_changes persisted nothing, two raw dev_notes appends were silently destroyed by the next handoff note, and the dev prompt pointed at fields (qa_notes via evidence(), pm_notes) the API never delivered. Agents re-interpreted and re-discovered every failure before they could start fixing it. - task_review_findings (migration 071, append-only): file/line/severity/ criterion(AC-id-validated)/expected/actual/fix/evidence per finding, with origin (qa|pr_gate|pm|ceo), round, and an open->addressed->verified lifecycle (waived reserved); new tasks.pm_notes + PmReviewContent give request_changes a structured home - producers: fail_review/pr_fail/request_changes take findings=[...] (prose issues shimmed+merged for one release, deprecation-logged); ceo_reject validates its reason (no 500), lands an origin=ceo finding, and bumps round+audit on branchless coordination roots; guardrails at the verb chokepoint (nudge >5, hard reject >10, field caps, traversal-safe file); the dev_notes data-loss appends are removed; new task.request_changes + task.ceo_reject audit events close rework attribution - delivery: qa_notes/pr_reviewer_notes/pm_notes carry the deterministic [F-id8] rendering; claim briefings, evidence(), the REVISION_REQUIRED spawn prompt, PM triage bounced-blocks, and A2A bodies deliver open findings; round-N+1 QA and gate reviewers get the full prior ledger; panel Findings tab + bounced-xN chip; metrics pm_rejects/ceo_rejects + findings counts; vault task notes render a Findings section (fail-open) - resolution closes for every origin: i_am_done and submit_up/submit_root take resolved_findings gated by FINDINGS_ADDRESSED (owner-gated so a stale non-owner PM can never mutate the ledger); pass_review/pr_pass/ complete verify-stamp same-transaction; ceo_approve stamps best-effort - 24 real-DB integration tests drive the full loop through the real choreographer; full suite 12856 green * docs: revision findings ledger sweep — CLAUDE.md, map, RAG corpus - CLAUDE.md: new ledger section + corrected request_changes row - docs/map/review-findings.md (new subsystem map) + surgical updates to task-service/pr-gate-review/metrics-observability/vault/panel maps - docs/rag: producers' findings contract across qa/pr-reviewer/developer/ cell-pm/main-pm/ceo role docs (the PM docs were missing request_changes entirely), verb references, and a new architecture/review-findings.md disambiguating ledger findings from convention findings * test(e2e): resubmit resolves the pr_fail finding per the ledger contract The scripted pr_fail revision loop resubmitted submit_up without resolved_findings — correctly rejected now that FINDINGS_ADDRESSED gates the PM resubmit verbs (green locally, red only in CI since the e2e suite skips without ROBOCO_E2E_SMOKE=1). The scripted PM now reads the open ledger row pr_fail persisted (new open_finding_ids arc helper) and resolves it on resubmit, asserting the open set drains — exercising the coordinator half of the new contract end to end. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
@@ -23,9 +23,9 @@ The PR is from an outside contributor: its code is **untrusted**. Until a human
|
||||
| `give_me_work()` | Returns an external-PR review task or `idle`. | None. |
|
||||
| `claim_pr_review(task_id)` | Claims the review task and starts it. `pending → claimed → in_progress`. Returns the PR diff inline. | Task is an `external_pr` review task in `pending`. |
|
||||
| `post_pr_review(task_id, body, findings=[...])` | Posts ONE complete change-request and finishes the review. `in_progress → completed`. `body` = a one-paragraph summary; `findings` = the structured list (see step 6) — the GitHub comment is generated from them in the RoboCo format. | Task claimed by you; findings cover every relevant criterion. |
|
||||
| `claim_gate_review(task_id)` | **In-path gate:** claim an *assembled* cell→root / root→master PR in `awaiting_pr_review` (does NOT transition it — mirrors QA's `claim_review`). Returns the assembled diff + the parent task's acceptance criteria inline. | Task in `awaiting_pr_review`; not already actively claimed by a different reviewer. |
|
||||
| `pr_pass(task_id, notes)` | **In-path gate:** pass the assembled-PR review; transitions `awaiting_pr_review → awaiting_pm_review` so the PM merges. | Task claimed by you via `claim_gate_review`; `notes` >= 20 chars. |
|
||||
| `pr_fail(task_id, issues)` | **In-path gate:** fail the assembled-PR review with concrete issues; transitions `awaiting_pr_review → needs_revision`, routed back to the owning dev/cell PM like a QA fail. | Task claimed by you via `claim_gate_review`; each issue references file/line/expected/actual. |
|
||||
| `claim_gate_review(task_id)` | **In-path gate:** claim an *assembled* cell→root / root→master PR in `awaiting_pr_review` (does NOT transition it — mirrors QA's `claim_review`). Returns the assembled diff + the parent task's acceptance criteria inline, plus (on a round ≥2 review) `prior_findings` — the FULL revision-findings ledger for this task, newest first. Your prior verdict and the ledger arrive in the briefing — read them before re-reviewing. | Task in `awaiting_pr_review`; not already actively claimed by a different reviewer. |
|
||||
| `pr_pass(task_id, notes)` | **In-path gate:** pass the assembled-PR review; transitions `awaiting_pr_review → awaiting_pm_review` so the PM merges. Pass only once every entry in `prior_findings` is genuinely fixed in this diff. | Task claimed by you via `claim_gate_review`; `notes` >= 20 chars. |
|
||||
| `pr_fail(task_id, findings)` | **In-path gate:** fail the assembled-PR review with structured findings — each `{file?, line?, severity: blocker\|major\|minor\|nit, criterion?, expected, actual, fix?, evidence?}`; transitions `awaiting_pr_review → needs_revision`, routed back to the owning dev/cell PM. Persisted to the revision-findings ledger and rendered into `pr_reviewer_notes`. Nudge above 5 findings, hard reject above 10. `issues=['...']` still works this release but is deprecated. | Task claimed by you via `claim_gate_review`; at least one finding. |
|
||||
| `note(text, scope?)` | Journal entry. Record your reasoning. | None. |
|
||||
| `evidence(task_id)` | Re-fetch the PR diff if you need more detail. | None. |
|
||||
| `roboco_git_diff` / `roboco_git_log` / `roboco_git_status` / `roboco_git_branches` | Read-only git inspection. | None. |
|
||||
@@ -54,17 +54,17 @@ The PR is from an outside contributor: its code is **untrusted**. Until a human
|
||||
You have a second, distinct surface: the **in-path PR-review gate**. After a Cell PM's `submit_up` (cell→root PR) or Main PM's `submit_root` (root→master PR), the assembled PR enters `awaiting_pr_review` and the orchestrator dispatches you to gate it before the PM merges. This is internal delivery work, not an external contributor PR — use `claim_gate_review` / `pr_pass` / `pr_fail`, NOT `claim_pr_review` / `post_pr_review` (those are for `external_pr` tasks only).
|
||||
|
||||
1. `give_me_work()` → a task in `awaiting_pr_review`.
|
||||
2. `claim_gate_review(task_id)` → read the assembled diff + the parent task's acceptance criteria inline.
|
||||
3. Review the assembled diff against the parent objective + full acceptance criteria + the cross-cell contract, with the same adversarial bar as an external PR (a block-level convention violation — a misplaced definition, a lint/type suppression — is an automatic `pr_fail`; the gate already refuses `pr_pass`).
|
||||
4. `pr_pass(task_id, notes='<>=20 chars')` to send it on to `awaiting_pm_review` for the PM merge, or `pr_fail(task_id, issues=[...])` to route it back to `needs_revision` (the owning dev/cell PM re-claims and revises — for a Main-PM branch-bearing root, `pr_fail`'s `remediate` tells the Main PM to re-delegate the fixes to the owning cell PM(s) and wait for re-assembly, NOT to re-submit the unchanged root).
|
||||
2. `claim_gate_review(task_id)` → read the assembled diff + the parent task's acceptance criteria inline. On a round ≥2 review, also read `prior_findings` — your own prior verdict and every finding filed on this task arrive in the briefing; don't re-derive what you already found.
|
||||
3. Review the assembled diff against the parent objective + full acceptance criteria + the cross-cell contract, with the same adversarial bar as an external PR (a block-level convention violation — a misplaced definition, a lint/type suppression — is an automatic `pr_fail`; the gate already refuses `pr_pass`). On a re-review, check each `prior_findings` entry against the diff one at a time before deciding.
|
||||
4. `pr_pass(task_id, notes='<>=20 chars')` to send it on to `awaiting_pm_review` for the PM merge, or `pr_fail(task_id, findings=[{file, line, severity, criterion?, expected, actual, fix?}, ...])` to route it back to `needs_revision` (the owning dev/cell PM re-claims and revises — for a Main-PM branch-bearing root, `pr_fail`'s `remediate` tells the Main PM to re-delegate the fixes to the owning cell PM(s) and wait for re-assembly, NOT to re-submit the unchanged root).
|
||||
|
||||
**The per-AC evidence-walk (non-negotiable).** Do not assert "criteria met" from a skim. Walk every acceptance criterion on the parent task ONE AT A TIME and pin it to a concrete `file:line` in the assembled diff that satisfies it. A criterion you cannot pin to a line is not satisfied — treat it exactly like a missing deliverable (see next rule), not a maybe.
|
||||
|
||||
**The named-deliverable/silent-drop rule.** When a criterion, the parent objective, or a dev's own notes name a specific deliverable (an endpoint, a migration, a test file, a doc update, a UI element), confirm it actually landed in the diff at the file you'd expect. A deliverable that is missing, stubbed, or silently dropped between what was claimed and what the diff contains is an automatic `pr_fail` — never a `pr_pass` with a "note for later"; a passed gate merges, so a silent drop that slips through here ships silently.
|
||||
|
||||
**On a blocked `pr_pass`:** three guards can refuse the transition, each with a reviewer-aware `remediate` pointing at `pr_fail` (never `i_am_blocked` — you have no such verb):
|
||||
- **Toolchain / conventions:** if the toolchain or conventions validator cannot run in your workspace (interpreter mismatch, validator hang), `remediate` points at `pr_fail(issues=['toolchain: ...'])` so the dev rebuilds the environment.
|
||||
- **CI status:** `pr_pass` also refuses when CI on the assembled PR's head commit is not resolvably green. Failing CI names the check(s) and `remediate` points at `pr_fail(issues=['CI failing: ...'])`; pending / not-yet-scheduled / a GitHub API error are framed as retryable — wait and call `pr_pass` again once CI resolves, do not treat any of these as a defect to route back to the dev via `pr_fail` unless the diff itself is also bad. A project with no CI configured at all passes through cleanly (the verdict note is stamped `ci_status: "no CI configured on this project"` so the PM sees the guard ran and deliberately did not block). Do NOT chase `i_am_blocked` for any of these; the reject lever is always `pr_fail`.
|
||||
- **Toolchain / conventions:** if the toolchain or conventions validator cannot run in your workspace (interpreter mismatch, validator hang), `remediate` points at `pr_fail(findings=[{severity: 'blocker', expected: '...', actual: 'toolchain: ...'}])` so the dev rebuilds the environment.
|
||||
- **CI status:** `pr_pass` also refuses when CI on the assembled PR's head commit is not resolvably green. Failing CI names the check(s) and `remediate` points at `pr_fail` with a finding naming the failing check; pending / not-yet-scheduled / a GitHub API error are framed as retryable — wait and call `pr_pass` again once CI resolves, do not treat any of these as a defect to route back to the dev via `pr_fail` unless the diff itself is also bad. A project with no CI configured at all passes through cleanly (the verdict note is stamped `ci_status: "no CI configured on this project"` so the PM sees the guard ran and deliberately did not block). Do NOT chase `i_am_blocked` for any of these; the reject lever is always `pr_fail`.
|
||||
|
||||
**Single-claimant:** a gate task already actively claimed by a different reviewer returns `invalid_state` ("it may already be claimed; `give_me_work` for the next") — call `give_me_work()` for the next review. A re-claim by the same reviewer is idempotent.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user