[4865ff8b] Add WebSocket support to the usage dashboard (#115)

* [e7349d84] feat(dashboard): WS usage store, hook extension, status badge, and smooth animations (#111) (#113)

- Add src/store/usage-store.ts with typed UsageData interface, useUsageStore
  Zustand store, setUsageData, clearUsageData, and setWsState actions
- Export useUsageStore and UsageData from store/index.ts
- Extend use-rate-limit-websocket.ts: rename msg type to SystemWsMessage,
  add key_metrics field; add useEffect syncing wsState into useUsageStore;
  add USAGE_UPDATE/USAGE_SNAPSHOT handler dispatching to useUsageStore
  (RATE_LIMIT_HIT/LIFTED handling and onReconnect unchanged)
- Update CommandCenter to read key_metrics from useUsageStore when
  wsState === 'connected' and usageData non-null; falls back to
  useCeoOverview() (refetchInterval: 60000) when WS disconnected
- Update KeyMetricsPanel: add wsState prop, render connection status Badge
  matching AgentStreamViewer pattern (bg-green-500+Wifi / bg-yellow-500+
  Loader2 spin / bg-gray-500+WifiOff); add transition-all duration-300
  ease-in-out to metric value spans for smooth animated updates

Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev>

* [c9745ee8] feat(events): add USAGE_UPDATE/SNAPSHOT event types, throttled publisher, /ws/system usage bridge (#112) (#114)

- Add EventType.USAGE_UPDATE='usage.update' and EventType.USAGE_SNAPSHOT='usage.snapshot'
  to the EventType StrEnum in roboco/models/events.py

- Create roboco/services/usage_events.py with _UsageThrottle class (5-second per-agent
  window using time.monotonic()) and publish_usage_update() / publish_usage_snapshot()
  helpers; lazy imports prevent circular dependency with roboco.events

- Extend orchestrator._sweep_token_snapshots() to publish USAGE_UPDATE per active agent
  (throttled) and a USAGE_SNAPSHOT aggregate after each sweep cycle; wrapped in
  contextlib.suppress so event errors never abort DB snapshot operations

- Add _handle_usage_event() to websocket_bridge.py following _handle_rate_limit_event
  pattern; register USAGE_UPDATE and USAGE_SNAPSHOT subscriptions in
  register_websocket_bridge_handlers() forwarding both to /ws/system via broadcast_system()

- Add unit tests: test_usage_events.py (throttle suppression, publish helpers) and
  test_websocket_bridge.py extended with _handle_usage_event coverage and updated
  registration assertion to include USAGE_UPDATE/USAGE_SNAPSHOT

Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev>

* fix(usage-ws): reconcile the realtime token/cost contract end-to-end

The backend and frontend halves shipped mismatched contracts, so the usage
dashboard never received live data:

- The bridge forwarded the dotted event value ("usage.update") while the panel
  switched on "USAGE_UPDATE"; map both to the UPPER_SNAKE type string the same
  way the rate-limit handler does.
- The backend emitted token/cost telemetry but the frontend read a key_metrics
  field and fed the org-metrics panel. Rewire the frontend to consume the
  USAGE_SNAPSHOT token/cost payload into the "Token Usage & Cost" panel —
  WS-first with polling fallback and a connection-status badge — and revert the
  unrelated KeyMetricsPanel / CommandCenter wiring.

Backend cleanups in the same path:

- Replace the multi-argument publish helpers with typed UsageUpdate /
  UsageSnapshot payloads, removing the too-many-arguments lint suppressions.
- Extract _fetch_agent_tokens and _persist_token_snapshot from the token sweep,
  removing the too-many-statements suppression; label the live snapshot "live".

Hardening uncovered while fixing the above:

- _finalize_spawn_session pulled the full RAG stack into the
  session-finalization path through a transcript-parse import; move the pure
  parser into a dependency-light roboco.agent_sdk.transcript_usage module so
  finalization never imports the agent SDK server.
- Reduce _finalize_spawn_session complexity by extracting
  _resolve_final_token_usage, and widen the transcript-fallback guard so a read
  error can never abort finalization.

Also align KeyMetricsPanel with the metrics /dashboard/ceo actually returns: it
read velocity_24h / avg_time_to_done / active_agents, none of which
get_key_metrics() emits, so four of five rows rendered "—". Render
velocity_weekly, completion_rate, documentation_coverage and active_blockers.

* docs: note live usage push over /ws/system on the usage dashboard

* fix(usage): finalize on self-exit and de-duplicate transcript token counts

Two bugs left token capture broken even after the transcript-read fallback
landed — surfaced by a live agent run:

- Agents that self-exit (the normal i_am_idle -> container shutdown, exit 0)
  were never finalized. _finalize_spawn_session is only called from
  stop_agent(), but a graceful self-exit goes through _handle_stopped_container,
  which set the instance OFFLINE and returned without finalizing — leaving the
  spawn-session row open with zero tokens. Finalize there for both graceful
  (exit_reason="completed") and crash (exit_reason="crashed") exits.

- sum_transcript_usage double-counted. Claude Code logs one assistant message
  as several JSONL lines (one per content block — thinking / text / tool_use),
  each repeating the same message.usage, so summing every line roughly doubled
  the totals. De-duplicate by message.id.

Verified against a live agent transcript: the raw sum (12, 1068, 62502, 115828)
vs the de-duped (6, 516, 62502, 63336), which matches the session's
authoritative result.usage exactly.

* feat(usage): fall back to the transcript in the live token sweep

The 60s token sweep read only the agent SDK's /usage/status, which races
container teardown and reports zero mid-run — so live usage (and the
USAGE_SNAPSHOT pushed to /ws/system) stayed at zero for active agents.
Extract _resolve_active_tokens: try the SDK, then fall back to the durable
transcript (the same source finalize uses) so running agents report live.

* feat(usage): add GET /usage/sessions for the dashboard's Recent Sessions

The panel's Recent Sessions table was mock-only — the backend had no sessions
endpoint, so production always showed 'No sessions recorded yet'. Add
UsageService.get_recent_sessions + a /usage/sessions route returning the most
recent spawn-session rows (token totals + cost), and point the panel client
at it.

---------

Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev>
Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev>
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
Renzo F
2026-06-11 23:19:50 +02:00
committed by GitHub
co-authored by Frontend Developer 1 Backend Developer 1 Renn F
parent 1d1ec88aad
commit 547fe444f2
22 changed files with 1300 additions and 187 deletions
+71 -2
View File
@@ -17,6 +17,7 @@ from roboco.api.websocket_bridge import (
_handle_notification_sent,
_handle_rate_limit_event,
_handle_session_event,
_handle_usage_event,
register_websocket_bridge_handlers,
start_websocket_bridge,
)
@@ -291,13 +292,78 @@ async def test_handle_rate_limit_ignores_unrelated_event() -> None:
mgr.broadcast_system.assert_not_called()
# ---------------------------------------------------------------------------
# _handle_usage_event
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_handle_usage_update_broadcasts_to_system() -> None:
"""USAGE_UPDATE event → broadcast_system tagged USAGE_UPDATE + data fields."""
expected_input = 100
expected_output = 50
event = _evt(
EventType.USAGE_UPDATE,
{
"agent_id": "be-dev-1",
"task_id": "task-abc",
"input_tokens": expected_input,
"output_tokens": expected_output,
"model": "claude-sonnet-4-6",
"timestamp": "2026-06-11T00:00:00+00:00",
},
)
with patch("roboco.api.websocket_bridge.manager") as mgr:
mgr.broadcast_system = AsyncMock()
await _handle_usage_event(event)
mgr.broadcast_system.assert_awaited_once()
msg = mgr.broadcast_system.await_args.args[0]
assert msg["type"] == "USAGE_UPDATE"
assert msg["agent_id"] == "be-dev-1"
assert msg["input_tokens"] == expected_input
assert msg["output_tokens"] == expected_output
@pytest.mark.asyncio
async def test_handle_usage_snapshot_broadcasts_to_system() -> None:
"""USAGE_SNAPSHOT event → broadcast_system tagged USAGE_SNAPSHOT + aggregate."""
expected_input = 500
event = _evt(
EventType.USAGE_SNAPSHOT,
{
"period": "live",
"totals": {"input_tokens": expected_input, "output_tokens": 200},
"cost_estimate": 0.0025,
"by_agent": [
{
"agent_id": "be-dev-1",
"input_tokens": 500,
"output_tokens": 200,
"model": "sonnet",
"cost_estimate": 0.0025,
}
],
"timestamp": "2026-06-11T00:01:00+00:00",
},
)
with patch("roboco.api.websocket_bridge.manager") as mgr:
mgr.broadcast_system = AsyncMock()
await _handle_usage_event(event)
mgr.broadcast_system.assert_awaited_once()
msg = mgr.broadcast_system.await_args.args[0]
assert msg["type"] == "USAGE_SNAPSHOT"
assert msg["period"] == "live"
assert msg["totals"]["input_tokens"] == expected_input
assert len(msg["by_agent"]) == 1
# ---------------------------------------------------------------------------
# Registration + start
# ---------------------------------------------------------------------------
def test_register_websocket_bridge_handlers_subscribes_all_event_types() -> None:
"""Registration wires up notification + session + agent event handlers."""
"""Registration wires up all handler categories, including usage events."""
class _FakeBus:
def __init__(self) -> None:
@@ -310,7 +376,7 @@ def test_register_websocket_bridge_handlers_subscribes_all_event_types() -> None
with patch("roboco.api.websocket_bridge.get_event_bus", return_value=fake):
register_websocket_bridge_handlers()
types = [t for t, _ in fake.subscribed]
# All 10 expected event types appear at least once.
# All 14 expected event types appear at least once.
assert EventType.NOTIFICATION_SENT in types
assert EventType.NOTIFICATION_ACKED in types
assert EventType.SESSION_CREATED in types
@@ -323,6 +389,9 @@ def test_register_websocket_bridge_handlers_subscribes_all_event_types() -> None
assert EventType.AGENT_ERROR in types
assert EventType.RATE_LIMIT_HIT in types
assert EventType.RATE_LIMIT_LIFTED in types
# Usage events forwarded to /ws/system.
assert EventType.USAGE_UPDATE in types
assert EventType.USAGE_SNAPSHOT in types
@pytest.mark.asyncio