feat(lifecycle): revision findings ledger — structured failure feedback, persisted and delivered down the chain (#486)

* feat(lifecycle): revision findings ledger — structured QA/PR/PM/CEO failure feedback, persisted and delivered down the chain Every bounce used to survive only as flattened prose: rounds overwrote each other in notes_structured, request_changes persisted nothing, two raw dev_notes appends were silently destroyed by the next handoff note, and the dev prompt pointed at fields (qa_notes via evidence(), pm_notes) the API never delivered. Agents re-interpreted and re-discovered every failure before they could start fixing it. - task_review_findings (migration 071, append-only): file/line/severity/ criterion(AC-id-validated)/expected/actual/fix/evidence per finding, with origin (qa|pr_gate|pm|ceo), round, and an open->addressed->verified lifecycle (waived reserved); new tasks.pm_notes + PmReviewContent give request_changes a structured home - producers: fail_review/pr_fail/request_changes take findings=[...] (prose issues shimmed+merged for one release, deprecation-logged); ceo_reject validates its reason (no 500), lands an origin=ceo finding, and bumps round+audit on branchless coordination roots; guardrails at the verb chokepoint (nudge >5, hard reject >10, field caps, traversal-safe file); the dev_notes data-loss appends are removed; new task.request_changes + task.ceo_reject audit events close rework attribution - delivery: qa_notes/pr_reviewer_notes/pm_notes carry the deterministic [F-id8] rendering; claim briefings, evidence(), the REVISION_REQUIRED spawn prompt, PM triage bounced-blocks, and A2A bodies deliver open findings; round-N+1 QA and gate reviewers get the full prior ledger; panel Findings tab + bounced-xN chip; metrics pm_rejects/ceo_rejects + findings counts; vault task notes render a Findings section (fail-open) - resolution closes for every origin: i_am_done and submit_up/submit_root take resolved_findings gated by FINDINGS_ADDRESSED (owner-gated so a stale non-owner PM can never mutate the ledger); pass_review/pr_pass/ complete verify-stamp same-transaction; ceo_approve stamps best-effort - 24 real-DB integration tests drive the full loop through the real choreographer; full suite 12856 green * docs: revision findings ledger sweep — CLAUDE.md, map, RAG corpus - CLAUDE.md: new ledger section + corrected request_changes row - docs/map/review-findings.md (new subsystem map) + surgical updates to task-service/pr-gate-review/metrics-observability/vault/panel maps - docs/rag: producers' findings contract across qa/pr-reviewer/developer/ cell-pm/main-pm/ceo role docs (the PM docs were missing request_changes entirely), verb references, and a new architecture/review-findings.md disambiguating ledger findings from convention findings * test(e2e): resubmit resolves the pr_fail finding per the ledger contract The scripted pr_fail revision loop resubmitted submit_up without resolved_findings — correctly rejected now that FINDINGS_ADDRESSED gates the PM resubmit verbs (green locally, red only in CI since the e2e suite skips without ROBOCO_E2E_SMOKE=1). The scripted PM now reads the open ledger row pr_fail persisted (new open_finding_ids arc helper) and resolves it on resubmit, asserting the open set drains — exercising the coordinator half of the new contract end to end. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-08-03 07:23:24 +02:00 · 2026-07-11 22:54:42 +02:00
parent d03181ab48
commit cea3e56628
103 changed files with 7283 additions and 399 deletions
@@ -3,18 +3,23 @@
 from __future__ import annotations

 import pytest
+from pydantic import ValidationError
 from roboco.foundation.policy.content import (
    AuditorNote,
    ContentValidationError,
    DeveloperNote,
    DocNote,
+    Finding,
+    PmReviewContent,
    PrReviewContent,
    QaNote,
    ResumptionNote,
    TaskDescription,
    validate_content,
+    validate_findings,
 )
 from roboco.foundation.policy.content.enums import Severity, Verdict
+from roboco.foundation.policy.content.models import CONTENT_MODELS

 # --------------------------------------------------------------------------- #
 # Valid construction
@@ -277,3 +282,182 @@ def test_developer_and_doc_and_auditor_models() -> None:
        ),
        AuditorNote,
    )
+
+
+# --------------------------------------------------------------------------- #
+# Finding — revision-findings ledger caps (fix / evidence / file-relative)
+# --------------------------------------------------------------------------- #
+
+_FINDING_TEXT_CAP = 300
+
+
+def _finding(**overrides: object) -> dict[str, object]:
+    base: dict[str, object] = {
+        "file": "roboco/services/task.py",
+        "line": 42,
+        "severity": "major",
+        "expected": "raises on invalid input",
+        "actual": "swallows the error silently",
+    }
+    base.update(overrides)
+    return base
+
+
+def test_finding_accepts_fix_and_evidence() -> None:
+    f = Finding.model_validate(
+        _finding(
+            fix="raise ValueError instead", evidence="Traceback: ...\nAssertionError"
+        )
+    )
+    assert f.fix == "raise ValueError instead"
+    assert f.evidence is not None
+    assert f.evidence.startswith("Traceback")
+
+
+def test_finding_file_is_optional() -> None:
+    f = Finding.model_validate(_finding(file=None))
+    assert f.file is None
+
+
+def test_finding_rejects_absolute_unix_path() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(file="/etc/passwd"))
+
+
+def test_finding_rejects_absolute_windows_path() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(file="C:\\Windows\\system32"))
+
+
+def test_finding_rejects_dotdot_traversal_path() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(file="a/../../etc/passwd"))
+
+
+def test_finding_rejects_dotdot_leading_segment() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(file="../roboco/services/task.py"))
+
+
+def test_finding_accepts_dot_segment_and_double_dot_substring() -> None:
+    # A literal ".." SEGMENT is rejected, but a filename merely containing
+    # dots (not a traversal component) must not false-positive.
+    ok = Finding.model_validate(_finding(file="./roboco/services/foo..bar.py"))
+    assert ok.file == "./roboco/services/foo..bar.py"
+
+
+def test_finding_rejects_non_positive_line() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(line=0))
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(line=-1))
+
+
+def test_finding_expected_actual_cap_at_300() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(expected="x" * (_FINDING_TEXT_CAP + 1)))
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(actual="x" * (_FINDING_TEXT_CAP + 1)))
+    # exactly at the cap is fine
+    ok = Finding.model_validate(_finding(expected="x" * _FINDING_TEXT_CAP))
+    assert len(ok.expected) == _FINDING_TEXT_CAP
+
+
+def test_finding_fix_cap_at_500() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(fix="x" * 501))
+
+
+def test_finding_evidence_cap_at_2000() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(evidence="x" * 2001))
+
+
+def test_finding_file_cap_at_300() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(file="a/" * 200 + "f.py"))
+
+
+def test_finding_rejects_placeholder_fix() -> None:
+    with pytest.raises(ValidationError):
+        Finding.model_validate(_finding(fix="tbd"))
+
+
+# --------------------------------------------------------------------------- #
+# validate_findings — the ledger's raw list[dict] -> list[Finding] entry point
+# --------------------------------------------------------------------------- #
+
+_EXPECTED_TWO_FINDINGS = 2
+
+
+def test_validate_findings_returns_typed_list() -> None:
+    findings = validate_findings([_finding(), _finding(file=None, line=None)])
+    assert len(findings) == _EXPECTED_TWO_FINDINGS
+    assert all(isinstance(f, Finding) for f in findings)
+    assert findings[1].file is None
+
+
+def test_validate_findings_passes_through_existing_finding_instances() -> None:
+    f = Finding.model_validate(_finding())
+    assert validate_findings([f]) == [f]
+
+
+def test_validate_findings_raises_content_validation_error() -> None:
+    with pytest.raises(ContentValidationError):
+        validate_findings([_finding(severity="catastrophic")])
+
+
+# --------------------------------------------------------------------------- #
+# QaNote.findings — parity with PrReviewContent.findings
+# --------------------------------------------------------------------------- #
+
+
+def test_qa_note_accepts_findings() -> None:
+    c = QaNote.model_validate(
+        {
+            "summary": "[F-abc12345] task.py:42 (major) — expected → actual",
+            "findings": [_finding()],
+            "verdict": "failed",
+        }
+    )
+    assert len(c.findings) == 1
+    assert c.findings[0].severity is Severity.MAJOR
+    # QaNote deliberately does not re-render findings into their own section
+    # (summary already carries the deterministic per-finding rendering) —
+    # avoids the double-rendering the PR-gate summary explicitly avoids.
+    rendered = c.render_markdown()
+    assert "## Findings" not in rendered
+
+
+def test_qa_note_findings_default_empty() -> None:
+    c = QaNote.model_validate(
+        {"summary": "Everything checked out fine.", "verdict": "passed"}
+    )
+    assert c.findings == []
+
+
+# --------------------------------------------------------------------------- #
+# PmReviewContent — the request_changes note (no verdict field)
+# --------------------------------------------------------------------------- #
+
+
+def test_pm_review_content_valid() -> None:
+    c = validate_content(
+        "pm_review",
+        {
+            "summary": "[F-abc12345] file.py:10 (major) — expected → actual",
+            "findings": [_finding()],
+        },
+    )
+    assert isinstance(c, PmReviewContent)
+    assert len(c.findings) == 1
+    assert c.render_markdown() == "## Summary\n" + c.summary
+
+
+def test_pm_review_content_rejects_trivial_summary() -> None:
+    with pytest.raises(ContentValidationError):
+        validate_content("pm_review", {"summary": "wip"})
+
+
+def test_pm_review_content_registered_in_content_models() -> None:
+    assert CONTENT_MODELS["pm_review"] is PmReviewContent
@@ -0,0 +1,54 @@
+"""The FINDINGS_ADDRESSED tracing requirement — i_am_done's ledger resolution gate.
+
+Pure unit tests against ``foundation.policy.tracing`` (no DB, no choreographer):
+the checker itself, its registration in VERB_REQUIREMENTS["i_am_done"], and the
+"empty ledger passes untouched" contract.
+"""
+
+from __future__ import annotations
+
+from roboco.foundation.policy import tracing as tr
+
+
+def test_findings_addressed_is_required_by_i_am_done() -> None:
+    assert tr.Requirement.FINDINGS_ADDRESSED in tr.VERB_REQUIREMENTS["i_am_done"]
+
+
+def test_no_open_findings_passes_trivially() -> None:
+    result = tr.check_requirements(
+        task=object(),
+        requirements=[tr.Requirement.FINDINGS_ADDRESSED],
+        ctx=tr.GateContext(open_finding_ids=()),
+    )
+    assert result.passed
+    assert result.missing == []
+
+
+def test_open_findings_block_and_name_each_id() -> None:
+    result = tr.check_requirements(
+        task=object(),
+        requirements=[tr.Requirement.FINDINGS_ADDRESSED],
+        ctx=tr.GateContext(open_finding_ids=("abc12345", "def67890")),
+    )
+    assert not result.passed
+    assert result.missing == ["finding:abc12345", "finding:def67890"]
+
+
+def test_default_gate_context_has_no_open_findings() -> None:
+    assert tr.GateContext().open_finding_ids == ()
+
+
+def test_i_am_done_requirements_include_the_pre_existing_set_too() -> None:
+    """Adding FINDINGS_ADDRESSED must not have dropped any prior requirement."""
+    required = tr.VERB_REQUIREMENTS["i_am_done"]
+    for expected in (
+        tr.Requirement.COMMITS_AT_LEAST_ONE,
+        tr.Requirement.PR_OPEN,
+        tr.Requirement.PROGRESS_AT_LEAST_ONE,
+        tr.Requirement.SELF_VERIFIED,
+        tr.Requirement.JOURNAL_REFLECT,
+        tr.Requirement.JOURNAL_DURING_WORK_AT_LEAST_ONE,
+        tr.Requirement.ACCEPTANCE_CRITERIA_ADDRESSED,
+        tr.Requirement.DEV_NOTES_MIN_CHARS,
+    ):
+        assert expected in required