mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
feat(lifecycle): revision findings ledger — structured failure feedback, persisted and delivered down the chain (#486)
* feat(lifecycle): revision findings ledger — structured QA/PR/PM/CEO failure feedback, persisted and delivered down the chain Every bounce used to survive only as flattened prose: rounds overwrote each other in notes_structured, request_changes persisted nothing, two raw dev_notes appends were silently destroyed by the next handoff note, and the dev prompt pointed at fields (qa_notes via evidence(), pm_notes) the API never delivered. Agents re-interpreted and re-discovered every failure before they could start fixing it. - task_review_findings (migration 071, append-only): file/line/severity/ criterion(AC-id-validated)/expected/actual/fix/evidence per finding, with origin (qa|pr_gate|pm|ceo), round, and an open->addressed->verified lifecycle (waived reserved); new tasks.pm_notes + PmReviewContent give request_changes a structured home - producers: fail_review/pr_fail/request_changes take findings=[...] (prose issues shimmed+merged for one release, deprecation-logged); ceo_reject validates its reason (no 500), lands an origin=ceo finding, and bumps round+audit on branchless coordination roots; guardrails at the verb chokepoint (nudge >5, hard reject >10, field caps, traversal-safe file); the dev_notes data-loss appends are removed; new task.request_changes + task.ceo_reject audit events close rework attribution - delivery: qa_notes/pr_reviewer_notes/pm_notes carry the deterministic [F-id8] rendering; claim briefings, evidence(), the REVISION_REQUIRED spawn prompt, PM triage bounced-blocks, and A2A bodies deliver open findings; round-N+1 QA and gate reviewers get the full prior ledger; panel Findings tab + bounced-xN chip; metrics pm_rejects/ceo_rejects + findings counts; vault task notes render a Findings section (fail-open) - resolution closes for every origin: i_am_done and submit_up/submit_root take resolved_findings gated by FINDINGS_ADDRESSED (owner-gated so a stale non-owner PM can never mutate the ledger); pass_review/pr_pass/ complete verify-stamp same-transaction; ceo_approve stamps best-effort - 24 real-DB integration tests drive the full loop through the real choreographer; full suite 12856 green * docs: revision findings ledger sweep — CLAUDE.md, map, RAG corpus - CLAUDE.md: new ledger section + corrected request_changes row - docs/map/review-findings.md (new subsystem map) + surgical updates to task-service/pr-gate-review/metrics-observability/vault/panel maps - docs/rag: producers' findings contract across qa/pr-reviewer/developer/ cell-pm/main-pm/ceo role docs (the PM docs were missing request_changes entirely), verb references, and a new architecture/review-findings.md disambiguating ledger findings from convention findings * test(e2e): resubmit resolves the pr_fail finding per the ledger contract The scripted pr_fail revision loop resubmitted submit_up without resolved_findings — correctly rejected now that FINDINGS_ADDRESSED gates the PM resubmit verbs (green locally, red only in CI since the e2e suite skips without ROBOCO_E2E_SMOKE=1). The scripted PM now reads the open ledger row pr_fail persisted (new open_finding_ids arc helper) and resolves it on resubmit, asserting the open set drains — exercising the coordinator half of the new contract end to end. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
@@ -0,0 +1,85 @@
|
||||
# Revision Findings Ledger
|
||||
|
||||
Structured, persistent code-level feedback for every QA/PR-gate/PM/CEO bounce back to `needs_revision` — the replacement for a prose `issues` list that got flattened into free text and forgotten by the next round. If a task ever comes back to you as `needs_revision`, this is the mechanism that carries WHAT was wrong, not just THAT something was wrong. Always on — there is no feature flag.
|
||||
|
||||
## The four producers
|
||||
|
||||
| Verb | Caller | Origin | Note it renders into |
|
||||
|---|---|---|---|
|
||||
| `fail` (`fail_review`) | QA | `qa` | `qa_notes` |
|
||||
| `pr_fail` | PR reviewer (in-path gate) | `pr_gate` | `pr_reviewer_notes` |
|
||||
| `request_changes` | Cell PM / Main PM | `pm` | `pm_notes` |
|
||||
| `ceo_reject` | CEO (panel-only, not a gateway verb) | `ceo` | — (a single finding derived from your rejection reason) |
|
||||
|
||||
Each takes `findings: list[dict]` — a list of structured findings. The legacy `issues: list[str]` (plain strings) still works this release as a deprecated shim: each string becomes a file-less `severity=major` finding. Sending both `findings` and `issues` in the same call merges them rather than dropping one.
|
||||
|
||||
## The `Finding` shape
|
||||
|
||||
```python
|
||||
{
|
||||
"file": "roboco/api/routes/rate_limit.py", # optional; repo-relative, no ".."
|
||||
"line": 88, # optional; >= 1
|
||||
"severity": "blocker", # required: blocker | major | minor | nit
|
||||
"criterion": "<acceptance-criterion id or exact text>", # optional
|
||||
"expected": "429 on the 101st request", # required, <=300 chars
|
||||
"actual": "the 100th request also 429s", # required, <=300 chars
|
||||
"fix": "use > not >= on the window limit", # optional, <=500 chars — describe the change, never a literal patch
|
||||
"evidence": "<failing test output / CI lines / diff hunk>", # optional, <=2000 chars
|
||||
}
|
||||
```
|
||||
|
||||
- `severity`: `blocker` (must fix before merge/pass) → `major` (significant defect) → `minor` (small defect, fix advised) → `nit` (cosmetic).
|
||||
- `criterion`, if supplied, must match one of the task's acceptance-criterion ids or exact text — a criterion that matches neither is rejected outright, so a typo doesn't silently detach the finding from what it's actually about.
|
||||
- **Count guard**: a soft nudge appears (non-blocking) above 5 findings in one call; more than 10 is a hard reject — split across calls or prioritize the blocking ones first. An oversized findings list is as unreviewable as an oversized task.
|
||||
|
||||
## What happens when you file one
|
||||
|
||||
Every finding is validated, then inserted as one append-only row on the task's `task_review_findings` ledger (`origin`, `round` = the revision count this bounce belongs to, `status=open`), then rendered into a deterministic line — `[F-xxxxxxxx] file:line (severity) — expected → actual → fix` — that becomes both the structured note's `summary` (so `qa_notes`/`pr_reviewer_notes`/`pm_notes` show it directly) and the A2A message body. Nothing here is a snapshot that a later round can overwrite — every round's findings stay on the ledger alongside every earlier round's.
|
||||
|
||||
## Resolving findings (the bounced side)
|
||||
|
||||
`i_am_done`, `submit_up`, and `submit_root` all accept `resolved_findings`:
|
||||
|
||||
```python
|
||||
i_am_done(
|
||||
task_id="<task>",
|
||||
notes="...",
|
||||
resolved_findings=[
|
||||
{"finding_id": "a1b2c3d4", "commit": "<sha>", "note": "fixed the off-by-one"},
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
`finding_id` is the 8-char id shown in the `[F-xxxxxxxx]` rendering (an unambiguous longer prefix, or the full id, also matches). Every entry on the ledger still `open` for this task must be named or the call is refused, listing exactly which ids are still open — you don't have to guess or re-derive them. This is a gate, not a suggestion: an unresolved finding blocks resubmission.
|
||||
|
||||
## Where findings arrive
|
||||
|
||||
- **`evidence(task_id)`** carries `revision_findings` — the OPEN findings on the task, for any role.
|
||||
- **`claim_review`** (QA) and **`claim_gate_review`** (PR reviewer) additionally carry `prior_findings` — the FULL ledger (every round, every status), so a round-2+ reviewer checks each prior finding against the current diff instead of re-deriving what was already found.
|
||||
- Your **respawn prompt**, if the task bounced, renders the open findings inline (id, file:line, expected → actual → fix) — you don't have to call anything extra to see them.
|
||||
- A PM re-spawned onto a bounced root sees a "bounced" block in its triage prompt with the same rendering.
|
||||
- The A2A message a producer sends alongside the bounce carries the identical rendering.
|
||||
- The panel's task-detail **Findings tab** shows the full ledger per round with status badges; the task header shows a `bounced xN` chip.
|
||||
|
||||
## Verification (the reviewer's side)
|
||||
|
||||
When the SAME origin's review passes on a later round, every `addressed` finding of that origin is bulk-promoted to `verified` in the same transaction — `pass` (QA) verifies `qa`-origin findings, `pr_pass` verifies `pr_gate`-origin, `complete` (PM) verifies `pm`-origin. `ceo_approve` does the same for `ceo`-origin findings, best-effort. A finding can also be `waived` (the repository supports it) but no verb currently calls that path — an unaddressed finding cannot yet be dismissed without actually resolving it.
|
||||
|
||||
## `ceo_reject` specifically
|
||||
|
||||
The CEO acts through the panel, not a gateway verb — there is no agent-facing `ceo_reject` call. But the CEO's rejection reason is no longer just a status flip: it becomes one `origin=ceo`, `severity=blocker` finding on the ledger (`expected="CEO sign-off on this task"`, `actual=<your reason>`), visible in the Findings tab and delivered to whoever reworks it exactly like a QA or PR-gate bounce. An empty or placeholder reason ("", "wip", "n/a") is rejected cleanly rather than causing a server error.
|
||||
|
||||
## Don't confuse this with `convention_findings`
|
||||
|
||||
`convention_findings` (surfaced in QA's `claim_review` evidence when the architectural-conventions standard is enabled) is a completely different concept: diff-time lint findings from the architecture validator (misplaced definitions, lint suppressions), keyed to a different table (`project_convention_findings`), replaced per-task rather than append-only. A findings-driven `claim_review` evidence payload can carry BOTH `convention_findings` and `revision_findings`/`prior_findings` at once — don't conflate the two words.
|
||||
|
||||
## See also
|
||||
|
||||
- `docs/rag/roles/qa.md` — `fail`/`pass` in practice
|
||||
- `docs/rag/roles/pr-reviewer.md` — `pr_fail`/`pr_pass` in practice
|
||||
- `docs/rag/roles/cell-pm.md` / `docs/rag/roles/main-pm.md` — `request_changes` in practice
|
||||
- `docs/rag/roles/developer.md` — resolving a bounce with `resolved_findings`
|
||||
- `docs/rag/roles/ceo.md` — `ceo_reject`
|
||||
- `docs/rag/lifecycle/intent-verbs.md` — the canonical verb reference
|
||||
- `docs/rag/standards/conventions.md` — the unrelated `convention_findings` concept
|
||||
- `docs/map/review-findings.md` — the implementation map (code-facing, not agent-facing)
|
||||
Reference in New Issue
Block a user