mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
* feat(metrics): capture per-session turns + tool_calls (phase 1)
Persist LLM iterations (turns) and tool invocations per agent spawn session,
the raw signal the granular per-member performance metrics build on (real
effort/iterations vs wall-clock).
- sum_transcript_usage returns a 5-tuple adding turns = unique assistant
message-id count; _usage_from_transcript + _resolve_active_tokens updated to
the 5-tuple (active-tokens keeps its 4-tuple contract by slicing).
- SDK: _SessionState.turns, set by /usage/sync; /usage/status (TokenUsageStatus)
now carries turns + tool_calls (= total_calls).
- orchestrator: new _resolve_final_turns_tools (SDK primary, transcript fallback
for turns only; Grok -> 0/0) wired into _finalize_spawn_session, which writes
turns + tool_calls to agent_spawn_sessions.
- migration 055 adds turns + tool_calls (BigInteger DEFAULT 0 -> historical/Grok
rows read 0, surfaced as n/a). Verified real alembic upgrade/downgrade.
Part of metrics-granularity (v0.15.0); recon-adjusted plan on disk.
* feat(metrics): pure compute_stage_effort helper (phase 2, part 1)
Foundation-layer overlap math (no DB): split each task status window into
active (merged wall-clock overlap of spawn stints — concurrent stints counted
once, so active <= window) vs wait (queue/review idle). Distinct from summed
effort. The per-task metrics service will feed it audit-log windows + spawn
stints. 9 unit tests (disjoint/nested/partial/merged/clamped/zero/multi-window).
* feat(metrics): per-task live metrics + GET /metrics/task/{id} (phase 2)
TaskMetrics dataclass + MetricsService.get_task_metrics: summed spawn effort
(vs wall-clock), turns/tool_calls/tokens/cost, per-stage active-vs-wait
(compute_stage_effort over audit windows x spawn stints), and who-caused-rework
(revision_count + named qa/pr fail events). Open stints and the open final
stage window close at completed_at for a terminal task (else now), so stages
don't grow past completion. Exposed at GET /dashboard/metrics/task/{task_id}
(404 if absent). Real-PG tests (compose/none/in-flight) + route tests (200/404).
* feat(metrics): CEO-as-member scorecard + ceo_reject audit regression (phase 3)
The human CEO is a measured member, read purely from audit_log (agent_role='ceo'
serializes from the CEO StrEnum): approval dwell (awaiting_ceo_approval -> a CEO
decision, incl. the coordination-root reject that lands in pending), unblock
dwell (blocked -> a CEO revive), and god-mode action count (every CEO-attributed
transition). CeoScorecard + MetricsService.get_ceo_scorecard (p50/p90 via
PERCENTILE_CONT, expanding IN for the decision sets) + GET
/dashboard/metrics/member/ceo (declared before any future member/{id} route).
The ceo_reject coordination-root audit gap the plan meant to close was already
closed by the gap-sweep (routes through admin_set_status -> agent_role='ceo'
audit); locked with a regression assertion in the existing coordination-reject
test. Real-PG tests: approval/unblock/godmode, non-ceo exclusion, empty->zeros.
* feat(metrics): audit instrumentation for escalations/blocked-others/idle (phase 4a)
The three extra per-member metrics that had no data source get durable,
in-session audit events (additive; never gate the underlying action):
- apply_escalation -> task.escalated (details.escalator_slug) on both the
normal block path and the pool-divert path -> escalations count.
- _unblock_dependents -> task.unblocked_dependents (details.count) on the
completed BLOCKER task, captured before the dependency edges are pruned ->
blocked-others count (sweeper attributes to the blocker's owner).
- mark_agent_idle -> agent.idle (details.agent_slug) -> idle/utilization (the
sweeper pairs an idle mark to the member's next spawn for idle duration).
(QA pass-rate needs no new event — reuses task.awaiting_documentation[qa] +
task.qa_fail.) Real-PG tests for each; 111 transition tests still green.
* feat(metrics): member_performance_daily rollup table + migration 056 (phase 4b)
The per-member scorecard rollup: one row per (date, member_kind, agent_slug),
CEO as a first-class member_kind='ceo' row (agent_slug='' NOT NULL so the
NULL-distinct UNIQUE keeps it unique). Full column set + the four CEO-approved
extras (qa_reviews_total/passed, escalations, blocked_others, idle_seconds) plus
blocked_seconds. Overwrite-upsert on (date, member_kind, agent_slug) for an
idempotent sweep. Migration 056 verified real up/down (24 cols, 4 indexes).
* feat(metrics): _sweep_member_performance rollup sweeper (phase 4c)
The daily per-member rollup sweep (mirrors _sweep_daily_rollup): a trailing
7-day, idempotent overwrite-upsert wired into _run_sweep. One focused query per
metric merges into a (date, agent_slug) accumulator — spawn effort/turns/tokens/
cost, completed/first-pass/revisions-received, revisions-caused (qa/pr fails),
QA pass-rate (passed + total), escalations (by escalator_slug), blocked-others
(unblocked_dependents by blocker owner), idle_seconds (idle mark -> next spawn),
blocked_seconds (blocked dwell) — plus one CEO row/day (approval/unblock dwell +
god-mode). Real-PG test asserts every facet + idempotency (a 2nd sweep
overwrites, never doubles); spawn-day != completion-day split is by-design.
* feat(metrics): member/org rollup scorecards + endpoints + live overlay (phase 5)
MemberScorecard + OrgScorecard with derived rates (FPY, effort-throughput,
turns/tool-calls per task, QA pass-rate, utilization) — all division-guarded to
None. get_member_scorecard reads member_performance_daily by slug and overlays
the member's live in-flight (non-terminal) tasks' effort via get_task_metrics
(disjoint by status: completion counts stay rollup-only, overlay only enriches
effort/turns/cost; includes_live_inflight flags it). get_org_scorecard
aggregates the cell (?team=) or whole org. Routes: GET /metrics/member/{agent_id}
(404 if absent, after the ceo literal route) + GET /metrics/org?team=. Real-PG
tests (derived rates, overlay no double-count, guards, org) + route tests.
* feat(metrics): granular CEO completion notification (phase 6)
There was no CEO completion notification at all (EventType.TASK_COMPLETED was
defined but never emitted). Add notify_ceo_of_completion in
NotificationDeliveryService — a granular body (real effort vs wall-clock +
stints/turns/tool-calls/revisions[QA/PR]/cost from get_task_metrics; degrades to
wall-clock-only, turns 'n/a', when there are no spawn sessions). Reuses the
existing ALERT type (no enum migration; the notificationtype PG enum is fixed at
001). ceo_approve now emits TASK_COMPLETED + fires the notification (best-effort
via _notify_completion — never blocks completion); complete() emits
TASK_COMPLETED too (closes the dead-code gap; the WS bridge can forward it).
Pure formatter tests + real-PG notification test.
* [metrics-granularity] Phase 7: panel Scorecards tab + dashboard overview
Add the CEO-facing metrics surfaces for the granularity feature:
- New "Scorecards" tab on the Metrics page: org rollup headline, the
CEO-as-member card (approval/unblock dwell + god-mode count), and a
per-member table (completed, first-pass yield, active effort, turns/task,
QA pass-rate, escalations, blocked-others, utilization). Each member row
self-fetches its rollup scorecard; live in-flight rows carry a "live" badge.
- New dashboard overview card (ScorecardOverviewPanel): org-wide 30-day
headline (completed, FPY, throughput/hr, active effort, cost) deep-linking
into the Scorecards tab.
- Plumbing: TaskMetrics/MemberScorecard/OrgScorecard/CeoScorecard types,
observability API client methods + empty fallbacks, and the four
useCeoScorecard/useMemberScorecard/useOrgScorecard/useTaskMetrics hooks.
Panel gate green: tsc, eslint, prettier, vitest (175 tests, +6 new).
* [metrics-granularity] test: make completion-notification robust to shared-DB CEO
test_notify_ceo_of_completion_creates_alert errored in the full suite (passed
in isolation): the session-scoped test DB is shared across the run, and the
sibling real-DB board-gate test commits a role=CEO agent (slug="ceo") without
cleanup — so my env fixture's hardcoded slug="ceo" insert hit a unique-constraint
violation, and a second role=CEO row would also make _get_ceo_agent()'s
scalar_one_or_none() raise. Reuse an existing CEO when present (the singleton the
production system actually has), else create one with a unique slug. Order-
independent. Also reflow test_metrics_instrumentation.py to ruff format.
* chore(release): 0.15.0
Metrics granularity: per-member/per-task/org + CEO-as-member scorecards,
turn/tool-call capture (migration 055), member_performance_daily rollup
(migration 056) with QA pass-rate / escalations / blocked-others / utilization,
per-task active-vs-wait metrics, granular completion notification, panel
Scorecards tab + dashboard Performance card, and the ceo_reject audit fix.
Version bump across the canonical set + CHANGELOG.
* [metrics-granularity] fix pre-tag audit findings (overlay double-count + panel error states)
Adversarial review before the v0.15.0 tag surfaced two real logical gaps:
- MAJOR (backend): the live in-flight overlay re-summed ALL sessions of every
non-terminal task via get_task_metrics, but _msweep_spawn already rolls up
every CLOSED session regardless of task status — so a closed session on a
still-open task was counted twice (rollup + overlay), permanently inflating a
member's effort/turns/tokens/cost on the common reap/respawn path. The overlay
now sums only OPEN sessions (ended_at IS NULL), which the closed-only rollup
can never contain — disjoint by construction. A just-closed session lands in
the rollup on the next ~60s sweep (no gap of note). Aggregated in SQL to mirror
_msweep_spawn. Regression test reproduces the double-count (turns 10→5).
- MAJOR (panel): the four new scorecard surfaces used `isLoading || !data` with
no isError branch, so a failed query span forever on a skeleton. They now
surface a load error. Tests added.
Also: OrgSummary active-effort formatting no longer round-trips hours→seconds→
hours; dashboard grid uses xl:grid-cols-4 (was 2xl) so 4 panels show at 1280px;
corrected the inaccurate "NULL distinct" CEO-row uniqueness comment (agent_slug
is NOT NULL; the '' tuple is simply distinct from agent rows).
make quality GREEN (cov 95.31%); panel GREEN (vitest 178).
* [metrics-granularity] fix: decode bytes stream message-id before XCLAIM
StreamEventBus._recover_stream passed the pending message id to XCLAIM via
str() on the raw bytes the client returns (redis client has no
decode_responses), producing "b'1782066556728-0'". Redis rejects that with
"Unrecognized XCLAIM option", so pending-message recovery threw on every
reclaim tick and unacked messages from crashed/slow consumers were never
reclaimed (leaking in the PEL on every stream, spamming the error log). Decode
via the existing _to_str helper — the fix the sibling claim path already uses.
Pre-existing in v0.14.0 (unrelated to metrics granularity); folded into this
release per CEO. TDD regression test + CHANGELOG entry. make quality GREEN.
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
526 lines
16 KiB
Python
526 lines
16 KiB
Python
"""Dashboard API route coverage."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import uuid
|
|
from http import HTTPStatus
|
|
from typing import TYPE_CHECKING, cast
|
|
from uuid import uuid4
|
|
|
|
import pytest
|
|
import pytest_asyncio
|
|
from fastapi import FastAPI
|
|
from httpx import ASGITransport, AsyncClient
|
|
from roboco.api.deps import get_agent_context, get_db
|
|
from roboco.api.routes.dashboard import get_main_pm_kanban
|
|
from roboco.api.routes.dashboard import router as dashboard_router
|
|
from roboco.db.tables import AgentTable, ProjectTable, TaskTable
|
|
from roboco.models import AgentRole, AgentStatus
|
|
from roboco.models.base import (
|
|
Complexity,
|
|
TaskNature,
|
|
TaskStatus,
|
|
TaskType,
|
|
Team,
|
|
)
|
|
from roboco.models.permissions import AgentContext
|
|
from roboco.services.dashboard import reset_storage
|
|
from sqlalchemy import select
|
|
|
|
if TYPE_CHECKING:
|
|
from collections.abc import AsyncGenerator, AsyncIterator
|
|
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
|
|
|
|
@pytest_asyncio.fixture
|
|
async def dashboard_client(
|
|
db_session: AsyncSession,
|
|
) -> AsyncIterator[AsyncClient]:
|
|
reset_storage()
|
|
agent = AgentTable(
|
|
id=uuid4(),
|
|
name="CEO",
|
|
slug=f"ceo-{uuid4().hex[:8]}",
|
|
role=AgentRole.CEO,
|
|
team=None,
|
|
status=AgentStatus.ACTIVE,
|
|
model_config={},
|
|
system_prompt="ceo",
|
|
capabilities=[],
|
|
permissions={},
|
|
metrics={},
|
|
)
|
|
db_session.add(agent)
|
|
await db_session.flush()
|
|
|
|
app = FastAPI()
|
|
app.include_router(dashboard_router, prefix="/api/dashboard")
|
|
|
|
async def _override_db() -> AsyncGenerator[AsyncSession]:
|
|
yield db_session
|
|
|
|
async def _override_agent() -> AgentContext:
|
|
return AgentContext(
|
|
agent_id=cast("uuid.UUID", agent.id), role=AgentRole.CEO, team=None
|
|
)
|
|
|
|
app.dependency_overrides[get_db] = _override_db
|
|
app.dependency_overrides[get_agent_context] = _override_agent
|
|
|
|
transport = ASGITransport(app=app)
|
|
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
|
yield client
|
|
app.dependency_overrides.clear()
|
|
|
|
|
|
_HDR = {"X-Agent-ID": str(uuid4()), "X-Agent-Role": "ceo"}
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_create_auditor_flag(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.post(
|
|
"/api/dashboard/auditor/flags",
|
|
json={
|
|
"severity": "urgent",
|
|
"category": "quality",
|
|
"title": "Bug found",
|
|
"description": "Critical issue",
|
|
},
|
|
headers=_HDR,
|
|
)
|
|
assert response.status_code == HTTPStatus.CREATED
|
|
body = response.json()
|
|
assert body["severity"] == "urgent"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_auditor_flags(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get("/api/dashboard/auditor/flags", headers=_HDR)
|
|
assert response.status_code == HTTPStatus.OK
|
|
assert isinstance(response.json(), list)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Observability endpoints (0.10.0)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_cycle_time_endpoint(dashboard_client: AsyncClient) -> None:
|
|
resp = await dashboard_client.get(
|
|
"/api/dashboard/metrics/cycle-time?days=30", headers=_HDR
|
|
)
|
|
assert resp.status_code == HTTPStatus.OK
|
|
assert isinstance(resp.json(), list)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_bottlenecks_endpoint(dashboard_client: AsyncClient) -> None:
|
|
resp = await dashboard_client.get(
|
|
"/api/dashboard/metrics/bottlenecks", headers=_HDR
|
|
)
|
|
assert resp.status_code == HTTPStatus.OK
|
|
body = resp.json()
|
|
assert "by_stage" in body and "worst_stage" in body and "active_blockers" in body
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_rework_endpoint(dashboard_client: AsyncClient) -> None:
|
|
resp = await dashboard_client.get("/api/dashboard/metrics/rework", headers=_HDR)
|
|
assert resp.status_code == HTTPStatus.OK
|
|
body = resp.json()
|
|
assert "rate" in body and "by_team" in body and "by_agent" in body
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_agent_scorecard_404_when_absent(dashboard_client: AsyncClient) -> None:
|
|
resp = await dashboard_client.get(
|
|
f"/api/dashboard/metrics/scorecard/agent/{uuid4()}", headers=_HDR
|
|
)
|
|
assert resp.status_code == HTTPStatus.NOT_FOUND
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_team_scorecard_endpoint(dashboard_client: AsyncClient) -> None:
|
|
resp = await dashboard_client.get(
|
|
"/api/dashboard/metrics/scorecard/team/backend", headers=_HDR
|
|
)
|
|
assert resp.status_code == HTTPStatus.OK
|
|
assert resp.json()["scope"] == "cell"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_resolve_auditor_flag(dashboard_client: AsyncClient) -> None:
|
|
create = await dashboard_client.post(
|
|
"/api/dashboard/auditor/flags",
|
|
json={
|
|
"severity": "warning",
|
|
"category": "quality",
|
|
"title": "Warning",
|
|
"description": "x",
|
|
},
|
|
headers=_HDR,
|
|
)
|
|
flag_id = create.json()["id"]
|
|
response = await dashboard_client.put(
|
|
f"/api/dashboard/auditor/flags/{flag_id}/resolve",
|
|
params={"notes": "fixed"},
|
|
headers=_HDR,
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_resolve_unknown_flag_returns_404(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
response = await dashboard_client.put(
|
|
f"/api/dashboard/auditor/flags/{uuid4()}/resolve", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.NOT_FOUND
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_create_auditor_report(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.post(
|
|
"/api/dashboard/auditor/reports",
|
|
json={
|
|
"report_type": "weekly",
|
|
"title": "Q1 Report",
|
|
"summary": "Strong week",
|
|
"sections": [],
|
|
},
|
|
headers=_HDR,
|
|
)
|
|
assert response.status_code == HTTPStatus.CREATED
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_auditor_reports(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get(
|
|
"/api/dashboard/auditor/reports", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_kanban_for_team_known_bug(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
"""Pre-existing bug — board.team is already a string (not enum) at line 334.
|
|
|
|
The route does `team.value` on a value already coerced to a string,
|
|
raising AttributeError. We assert the bug exists so a fix flips the test.
|
|
"""
|
|
with pytest.raises(AttributeError, match="'str' object has no attribute 'value'"):
|
|
await dashboard_client.get("/api/dashboard/kanban/backend", headers=_HDR)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_all_agent_status(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get("/api/dashboard/agents/status", headers=_HDR)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_recent_activity(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get(
|
|
"/api/dashboard/activity/recent",
|
|
headers=_HDR,
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_auditor_dashboard(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get("/api/dashboard/auditor", headers=_HDR)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_send_auditor_report_not_found(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
response = await dashboard_client.post(
|
|
f"/api/dashboard/auditor/reports/{uuid4()}/send", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.NOT_FOUND
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_send_auditor_report_success(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
create = await dashboard_client.post(
|
|
"/api/dashboard/auditor/reports",
|
|
json={
|
|
"report_type": "weekly",
|
|
"title": "T",
|
|
"summary": "s",
|
|
"sections": [],
|
|
},
|
|
headers=_HDR,
|
|
)
|
|
rid = create.json()["id"]
|
|
response = await dashboard_client.post(
|
|
f"/api/dashboard/auditor/reports/{rid}/send", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_ceo_overview(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get("/api/dashboard/ceo", headers=_HDR)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_ceo_team_details(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get("/api/dashboard/ceo/teams", headers=_HDR)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_ceo_blocker_details(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get("/api/dashboard/ceo/blockers", headers=_HDR)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_ceo_velocity(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get(
|
|
"/api/dashboard/ceo/velocity?days=14", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_main_pm_kanban_via_http(dashboard_client: AsyncClient) -> None:
|
|
"""`/kanban/main-pm` is now declared before `/kanban/{team}`, so it routes
|
|
correctly to `get_main_pm_kanban` instead of being matched as
|
|
`team=main-pm` (which would 422)."""
|
|
response = await dashboard_client.get("/api/dashboard/kanban/main-pm", headers=_HDR)
|
|
assert response.status_code == HTTPStatus.OK
|
|
body = response.json()
|
|
# main_pm board has columns; shape is from KanbanBoard.model_dump().
|
|
assert "columns" in body
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_velocity_metrics(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get(
|
|
"/api/dashboard/metrics/velocity?days=7", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_blocker_metrics(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get(
|
|
"/api/dashboard/metrics/blockers", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_team_metrics(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get(
|
|
"/api/dashboard/metrics/team/backend", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_communication_metrics(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
response = await dashboard_client.get(
|
|
"/api/dashboard/metrics/communication", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_health_metrics(dashboard_client: AsyncClient) -> None:
|
|
response = await dashboard_client.get("/api/dashboard/metrics/health", headers=_HDR)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_agent_metrics_not_found(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
response = await dashboard_client.get(
|
|
f"/api/dashboard/metrics/agent/{uuid4()}", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.NOT_FOUND
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_auditor_flags_filter_severity(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
response = await dashboard_client.get(
|
|
"/api/dashboard/auditor/flags?severity=warning", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_auditor_reports_with_filter(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
response = await dashboard_client.get(
|
|
"/api/dashboard/auditor/reports?report_type=weekly", headers=_HDR
|
|
)
|
|
assert response.status_code == HTTPStatus.OK
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_agent_metrics_existing_agent(
|
|
dashboard_client: AsyncClient,
|
|
db_session: AsyncSession,
|
|
) -> None:
|
|
"""Existing agent → exercise route happy path (line 494)."""
|
|
agent = AgentTable(
|
|
id=uuid4(),
|
|
name="Probe",
|
|
slug=f"probe-{uuid4().hex[:8]}",
|
|
role=AgentRole.DEVELOPER,
|
|
team=None,
|
|
status=AgentStatus.ACTIVE,
|
|
model_config={},
|
|
system_prompt="x",
|
|
capabilities=[],
|
|
permissions={},
|
|
metrics={},
|
|
)
|
|
db_session.add(agent)
|
|
await db_session.flush()
|
|
response = await dashboard_client.get(
|
|
f"/api/dashboard/metrics/agent/{agent.id}", headers=_HDR
|
|
)
|
|
# MetricsService may return None for an empty agent → 404; or a metrics
|
|
# object if there's enough data. Either way the route is exercised.
|
|
assert response.status_code in (HTTPStatus.OK, HTTPStatus.NOT_FOUND)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_get_main_pm_kanban_function_directly(
|
|
db_session: AsyncSession,
|
|
) -> None:
|
|
"""Route /kanban/main-pm is unreachable via HTTP (intercepted by /kanban/{team}).
|
|
|
|
Call the route function directly to cover lines 367-369.
|
|
"""
|
|
result = await get_main_pm_kanban(db_session)
|
|
assert isinstance(result, dict)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_ceo_scorecard_endpoint(dashboard_client: AsyncClient) -> None:
|
|
resp = await dashboard_client.get(
|
|
"/api/dashboard/metrics/member/ceo?days=30", headers=_HDR
|
|
)
|
|
assert resp.status_code == HTTPStatus.OK
|
|
body = resp.json()
|
|
assert body["member_kind"] == "ceo"
|
|
assert set(body) >= {
|
|
"approval_p50_seconds",
|
|
"approval_count",
|
|
"unblock_p50_seconds",
|
|
"unblock_count",
|
|
"godmode_actions",
|
|
}
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_member_scorecard_404_when_absent(dashboard_client: AsyncClient) -> None:
|
|
resp = await dashboard_client.get(
|
|
f"/api/dashboard/metrics/member/{uuid4()}", headers=_HDR
|
|
)
|
|
assert resp.status_code == HTTPStatus.NOT_FOUND
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_ceo_route_wins_over_member_uuid_route(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
# The literal "ceo" must resolve to the CEO route, not the {agent_id} route.
|
|
resp = await dashboard_client.get("/api/dashboard/metrics/member/ceo", headers=_HDR)
|
|
assert resp.status_code == HTTPStatus.OK
|
|
assert resp.json()["member_kind"] == "ceo"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_org_scorecard_endpoint(dashboard_client: AsyncClient) -> None:
|
|
resp = await dashboard_client.get("/api/dashboard/metrics/org", headers=_HDR)
|
|
assert resp.status_code == HTTPStatus.OK
|
|
body = resp.json()
|
|
assert body["scope"] == "org"
|
|
assert set(body) >= {"member_count", "tasks_completed", "first_pass_yield"}
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_task_metrics_404_for_missing_task(
|
|
dashboard_client: AsyncClient,
|
|
) -> None:
|
|
resp = await dashboard_client.get(
|
|
f"/api/dashboard/metrics/task/{uuid4()}", headers=_HDR
|
|
)
|
|
assert resp.status_code == HTTPStatus.NOT_FOUND
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_task_metrics_returns_shape_for_existing_task(
|
|
db_session: AsyncSession, dashboard_client: AsyncClient
|
|
) -> None:
|
|
creator = (await db_session.execute(select(AgentTable).limit(1))).scalar_one()
|
|
project = ProjectTable(
|
|
id=uuid4(),
|
|
name="P",
|
|
slug=f"p-{uuid4().hex[:6]}",
|
|
git_url="https://example.com/r.git",
|
|
assigned_cell=Team.BACKEND,
|
|
created_by=creator.id,
|
|
)
|
|
db_session.add(project)
|
|
await db_session.flush()
|
|
task = TaskTable(
|
|
id=uuid4(),
|
|
title="t",
|
|
description="d",
|
|
acceptance_criteria=["ac"],
|
|
task_type=TaskType.CODE,
|
|
nature=TaskNature.TECHNICAL,
|
|
status=TaskStatus.IN_PROGRESS,
|
|
team=Team.BACKEND,
|
|
project_id=project.id,
|
|
created_by=creator.id,
|
|
estimated_complexity=Complexity.MEDIUM,
|
|
)
|
|
db_session.add(task)
|
|
await db_session.flush()
|
|
|
|
resp = await dashboard_client.get(
|
|
f"/api/dashboard/metrics/task/{task.id}", headers=_HDR
|
|
)
|
|
assert resp.status_code == HTTPStatus.OK
|
|
body = resp.json()
|
|
assert body["task_id"] == str(task.id)
|
|
assert set(body) >= {
|
|
"active_runtime_seconds",
|
|
"wall_clock_seconds",
|
|
"turns",
|
|
"tool_calls",
|
|
"tokens",
|
|
"cost_usd",
|
|
"revision_count",
|
|
"qa_fails",
|
|
"pr_fails",
|
|
"stints",
|
|
"stages",
|
|
}
|
|
assert isinstance(body["stages"], list)
|