feat(cockpit): expose first_pass_yield and a real escaped-defects metric (#709)

The Company Scorecard renders three charter objectives but the cockpit
summary only ever carried one of the metrics, so two cards read "No data
yet" permanently.

first_pass_yield is a pass-through — MetricsService.get_org_scorecard()
already computes it on the same 30d/org scope the rest of the delivery block
uses, and CockpitService.summary simply never forwarded it.

escaped_defects is new. The obvious definition — a blocker finding opened on
a task that already reached a terminal state — is unimplementable: every
producer of a task_review_findings row fires as part of a bounce whose
transition requires a non-terminal task, so it would read zero forever, and a
permanently-green card is the same fabrication the panel change removes.

What it counts instead: a blocker still at 'addressed', never 'verified', on
a task that has since completed. That is reachable because
stamp_addressed_verified only bulk-verifies rows matching its OWN origin, so
a blocker raised by one origin and never re-confirmed by that origin survives
to completion on the developer's word alone.

docs/map/metrics-observability.md documents what a zero actually means: the
one reachable trigger is a PM-origin blocker on a task escalated to the CEO
rather than completed by the PM, since escalate_to_ceo carries no
findings-resolved precondition and ceo_approve verifies only ceo-origin rows.
It also records that the count is per-finding over a rolling 30-day window,
which is not the same unit as the charter's "per release".

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
Renzo F
2026-07-26 19:11:27 +02:00
committed by GitHub
co-authored by Renn F
parent 0fe21b1f97
commit 80ccf415cb
7 changed files with 268 additions and 10 deletions
+24
View File
@@ -25,6 +25,8 @@ _BUDGET = 100.0
_SPEND_30D = 150.0
_COMPLETED_30D = 5
_MEDIAN_LEAD_TIME = 12.5
_FIRST_PASS_YIELD = 0.92
_ESCAPED_DEFECTS = 2
def _agent(role: AgentRole) -> AgentContext:
@@ -60,6 +62,24 @@ def _patch(monkeypatch: pytest.MonkeyPatch) -> None:
get_delivery_stats_30d=AsyncMock(return_value=delivery_stats),
),
)
monkeypatch.setattr(
cm,
"get_metrics_service",
lambda _s: MagicMock(
get_org_scorecard=AsyncMock(
return_value=MagicMock(first_pass_yield=_FIRST_PASS_YIELD)
)
),
)
monkeypatch.setattr(
cm,
"ReviewFindingsRepository",
lambda _s: MagicMock(
escaped_defects_since=AsyncMock(
return_value=[(uuid4(), "qa"), (uuid4(), "pr_gate")]
)
),
)
usage = MagicMock(
get_summary=AsyncMock(return_value={"total_cost_usd": _SPEND_30D}),
get_projection=AsyncMock(return_value={"projected_monthly_cost_usd": 200.0}),
@@ -95,6 +115,8 @@ async def test_summary_aggregates(monkeypatch: pytest.MonkeyPatch) -> None:
assert out["delivery"]["blocked"] == _BLOCKED
assert out["delivery"]["completed_30d"] == _COMPLETED_30D
assert out["delivery"]["median_lead_time_hours"] == _MEDIAN_LEAD_TIME
assert out["delivery"]["first_pass_yield"] == _FIRST_PASS_YIELD
assert out["delivery"]["escaped_defects"] == _ESCAPED_DEFECTS
assert out["spend"]["spend_30d_usd"] == _SPEND_30D
assert out["spend"]["over_budget"] is True
assert out["pending_pitches"] == 1
@@ -121,6 +143,8 @@ async def test_route_ok_for_ceo(monkeypatch: pytest.MonkeyPatch) -> None:
"awaiting_ceo": 0,
"completed_30d": 0,
"median_lead_time_hours": None,
"first_pass_yield": None,
"escaped_defects": 0,
},
"spend": {
"spend_30d_usd": 0.0,