mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
* [e7349d84] feat(dashboard): WS usage store, hook extension, status badge, and smooth animations (#111) (#113) - Add src/store/usage-store.ts with typed UsageData interface, useUsageStore Zustand store, setUsageData, clearUsageData, and setWsState actions - Export useUsageStore and UsageData from store/index.ts - Extend use-rate-limit-websocket.ts: rename msg type to SystemWsMessage, add key_metrics field; add useEffect syncing wsState into useUsageStore; add USAGE_UPDATE/USAGE_SNAPSHOT handler dispatching to useUsageStore (RATE_LIMIT_HIT/LIFTED handling and onReconnect unchanged) - Update CommandCenter to read key_metrics from useUsageStore when wsState === 'connected' and usageData non-null; falls back to useCeoOverview() (refetchInterval: 60000) when WS disconnected - Update KeyMetricsPanel: add wsState prop, render connection status Badge matching AgentStreamViewer pattern (bg-green-500+Wifi / bg-yellow-500+ Loader2 spin / bg-gray-500+WifiOff); add transition-all duration-300 ease-in-out to metric value spans for smooth animated updates Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> * [c9745ee8] feat(events): add USAGE_UPDATE/SNAPSHOT event types, throttled publisher, /ws/system usage bridge (#112) (#114) - Add EventType.USAGE_UPDATE='usage.update' and EventType.USAGE_SNAPSHOT='usage.snapshot' to the EventType StrEnum in roboco/models/events.py - Create roboco/services/usage_events.py with _UsageThrottle class (5-second per-agent window using time.monotonic()) and publish_usage_update() / publish_usage_snapshot() helpers; lazy imports prevent circular dependency with roboco.events - Extend orchestrator._sweep_token_snapshots() to publish USAGE_UPDATE per active agent (throttled) and a USAGE_SNAPSHOT aggregate after each sweep cycle; wrapped in contextlib.suppress so event errors never abort DB snapshot operations - Add _handle_usage_event() to websocket_bridge.py following _handle_rate_limit_event pattern; register USAGE_UPDATE and USAGE_SNAPSHOT subscriptions in register_websocket_bridge_handlers() forwarding both to /ws/system via broadcast_system() - Add unit tests: test_usage_events.py (throttle suppression, publish helpers) and test_websocket_bridge.py extended with _handle_usage_event coverage and updated registration assertion to include USAGE_UPDATE/USAGE_SNAPSHOT Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * fix(usage-ws): reconcile the realtime token/cost contract end-to-end The backend and frontend halves shipped mismatched contracts, so the usage dashboard never received live data: - The bridge forwarded the dotted event value ("usage.update") while the panel switched on "USAGE_UPDATE"; map both to the UPPER_SNAKE type string the same way the rate-limit handler does. - The backend emitted token/cost telemetry but the frontend read a key_metrics field and fed the org-metrics panel. Rewire the frontend to consume the USAGE_SNAPSHOT token/cost payload into the "Token Usage & Cost" panel — WS-first with polling fallback and a connection-status badge — and revert the unrelated KeyMetricsPanel / CommandCenter wiring. Backend cleanups in the same path: - Replace the multi-argument publish helpers with typed UsageUpdate / UsageSnapshot payloads, removing the too-many-arguments lint suppressions. - Extract _fetch_agent_tokens and _persist_token_snapshot from the token sweep, removing the too-many-statements suppression; label the live snapshot "live". Hardening uncovered while fixing the above: - _finalize_spawn_session pulled the full RAG stack into the session-finalization path through a transcript-parse import; move the pure parser into a dependency-light roboco.agent_sdk.transcript_usage module so finalization never imports the agent SDK server. - Reduce _finalize_spawn_session complexity by extracting _resolve_final_token_usage, and widen the transcript-fallback guard so a read error can never abort finalization. Also align KeyMetricsPanel with the metrics /dashboard/ceo actually returns: it read velocity_24h / avg_time_to_done / active_agents, none of which get_key_metrics() emits, so four of five rows rendered "—". Render velocity_weekly, completion_rate, documentation_coverage and active_blockers. * docs: note live usage push over /ws/system on the usage dashboard * fix(usage): finalize on self-exit and de-duplicate transcript token counts Two bugs left token capture broken even after the transcript-read fallback landed — surfaced by a live agent run: - Agents that self-exit (the normal i_am_idle -> container shutdown, exit 0) were never finalized. _finalize_spawn_session is only called from stop_agent(), but a graceful self-exit goes through _handle_stopped_container, which set the instance OFFLINE and returned without finalizing — leaving the spawn-session row open with zero tokens. Finalize there for both graceful (exit_reason="completed") and crash (exit_reason="crashed") exits. - sum_transcript_usage double-counted. Claude Code logs one assistant message as several JSONL lines (one per content block — thinking / text / tool_use), each repeating the same message.usage, so summing every line roughly doubled the totals. De-duplicate by message.id. Verified against a live agent transcript: the raw sum (12, 1068, 62502, 115828) vs the de-duped (6, 516, 62502, 63336), which matches the session's authoritative result.usage exactly. * feat(usage): fall back to the transcript in the live token sweep The 60s token sweep read only the agent SDK's /usage/status, which races container teardown and reports zero mid-run — so live usage (and the USAGE_SNAPSHOT pushed to /ws/system) stayed at zero for active agents. Extract _resolve_active_tokens: try the SDK, then fall back to the durable transcript (the same source finalize uses) so running agents report live. * feat(usage): add GET /usage/sessions for the dashboard's Recent Sessions The panel's Recent Sessions table was mock-only — the backend had no sessions endpoint, so production always showed 'No sessions recorded yet'. Add UsageService.get_recent_sessions + a /usage/sessions route returning the most recent spawn-session rows (token totals + cost), and point the panel client at it. --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> Co-authored-by: Renn F <rennf93@users.noreply.github.com>
403 lines
15 KiB
Python
403 lines
15 KiB
Python
"""websocket_bridge coverage — event handlers + bridge starter.
|
|
|
|
The handlers fan events from the Redis-stream bus to per-recipient WebSocket
|
|
connections. We don't need real Redis or sockets; we patch `manager` and the
|
|
`broadcast_*` helpers so each handler exercises its branches against
|
|
in-memory state.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from unittest.mock import AsyncMock, patch
|
|
from uuid import uuid4
|
|
|
|
import pytest
|
|
from roboco.api.websocket_bridge import (
|
|
_handle_agent_event,
|
|
_handle_notification_sent,
|
|
_handle_rate_limit_event,
|
|
_handle_session_event,
|
|
_handle_usage_event,
|
|
register_websocket_bridge_handlers,
|
|
start_websocket_bridge,
|
|
)
|
|
from roboco.models.events import Event, EventType
|
|
|
|
|
|
def _evt(event_type: EventType, data: dict, source_agent: str | None = None) -> Event:
|
|
return Event(type=event_type, data=data, source_agent=source_agent)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# _handle_notification_sent
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_notification_sent_skips_when_missing_ids() -> None:
|
|
"""Incomplete event (missing recipient/notification IDs) → log + return."""
|
|
event = _evt(EventType.NOTIFICATION_SENT, {}) # No notification_id/recipient_id
|
|
with patch("roboco.api.websocket_bridge.broadcast_notification") as bcast:
|
|
await _handle_notification_sent(event)
|
|
bcast.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_notification_sent_skips_invalid_uuid() -> None:
|
|
"""Invalid UUID strings → log error + return without broadcasting."""
|
|
event = _evt(
|
|
EventType.NOTIFICATION_SENT,
|
|
{"notification_id": "not-a-uuid", "recipient_id": str(uuid4())},
|
|
)
|
|
with patch("roboco.api.websocket_bridge.broadcast_notification") as bcast:
|
|
await _handle_notification_sent(event)
|
|
bcast.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_notification_sent_skips_when_no_connections() -> None:
|
|
"""Recipient has no WS connections → no broadcast."""
|
|
nid = uuid4()
|
|
rid = uuid4()
|
|
event = _evt(
|
|
EventType.NOTIFICATION_SENT,
|
|
{
|
|
"notification_id": str(nid),
|
|
"recipient_id": str(rid),
|
|
"type": "blocker",
|
|
"subject": "x",
|
|
"priority": "high",
|
|
},
|
|
)
|
|
with (
|
|
patch("roboco.api.websocket_bridge.broadcast_notification") as bcast,
|
|
patch("roboco.api.websocket_bridge.manager") as mgr,
|
|
):
|
|
mgr.notification_connections = {} # No connections for any agent.
|
|
await _handle_notification_sent(event)
|
|
bcast.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_notification_sent_broadcasts_when_connected() -> None:
|
|
"""Recipient has WS connection → broadcast_notification called."""
|
|
nid = uuid4()
|
|
rid = uuid4()
|
|
event = _evt(
|
|
EventType.NOTIFICATION_SENT,
|
|
{
|
|
"notification_id": str(nid),
|
|
"recipient_id": str(rid),
|
|
"type": "qa_ready",
|
|
"subject": "Task ready",
|
|
"priority": "normal",
|
|
},
|
|
)
|
|
bcast = AsyncMock()
|
|
with (
|
|
patch("roboco.api.websocket_bridge.broadcast_notification", bcast),
|
|
patch("roboco.api.websocket_bridge.manager") as mgr,
|
|
):
|
|
mgr.notification_connections = {rid: {"socket-1"}} # Has a connection.
|
|
await _handle_notification_sent(event)
|
|
bcast.assert_awaited_once()
|
|
call_kwargs = bcast.await_args.kwargs
|
|
assert call_kwargs["notification_id"] == nid
|
|
assert call_kwargs["agent_ids"] == [rid]
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_notification_acked_broadcasts_using_agent_id() -> None:
|
|
"""ACKED events carry `agent_id`, not `recipient_id`; the shared handler
|
|
must still forward (to the acking agent) rather than log 'Incomplete
|
|
notification event' on every acknowledgement."""
|
|
nid = uuid4()
|
|
aid = uuid4()
|
|
event = _evt(
|
|
EventType.NOTIFICATION_ACKED,
|
|
{"notification_id": str(nid), "agent_id": str(aid), "ack_type": "read"},
|
|
)
|
|
bcast = AsyncMock()
|
|
with (
|
|
patch("roboco.api.websocket_bridge.broadcast_notification", bcast),
|
|
patch("roboco.api.websocket_bridge.manager") as mgr,
|
|
):
|
|
mgr.notification_connections = {aid: {"socket-1"}}
|
|
await _handle_notification_sent(event)
|
|
bcast.assert_awaited_once()
|
|
call_kwargs = bcast.await_args.kwargs
|
|
assert call_kwargs["notification_id"] == nid
|
|
assert call_kwargs["agent_ids"] == [aid]
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# _handle_session_event
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_session_event_skips_missing_session_id() -> None:
|
|
event = _evt(EventType.SESSION_CREATED, {})
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.broadcast_to_session = AsyncMock()
|
|
await _handle_session_event(event)
|
|
mgr.broadcast_to_session.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_session_event_skips_invalid_uuid() -> None:
|
|
event = _evt(EventType.SESSION_CREATED, {"session_id": "bad-uuid"})
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.broadcast_to_session = AsyncMock()
|
|
await _handle_session_event(event)
|
|
mgr.broadcast_to_session.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_session_event_skips_when_no_connections() -> None:
|
|
sid = uuid4()
|
|
event = _evt(EventType.SESSION_CREATED, {"session_id": str(sid)})
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.session_connections = {}
|
|
mgr.broadcast_to_session = AsyncMock()
|
|
await _handle_session_event(event)
|
|
mgr.broadcast_to_session.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_session_event_broadcasts() -> None:
|
|
sid = uuid4()
|
|
event = _evt(EventType.SESSION_CLOSED, {"session_id": str(sid)})
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.session_connections = {sid: {"sock-1"}}
|
|
mgr.broadcast_to_session = AsyncMock()
|
|
await _handle_session_event(event)
|
|
mgr.broadcast_to_session.assert_awaited_once()
|
|
# Payload includes the trailing piece of the event-type ('closed').
|
|
call_args = mgr.broadcast_to_session.await_args
|
|
assert call_args.args[0] == sid
|
|
assert call_args.args[1]["type"] == "session.closed"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# _handle_agent_event
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_agent_event_skips_when_no_agent_id() -> None:
|
|
event = _evt(EventType.AGENT_SPAWNED, {})
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.broadcast_to_agent_watchers = AsyncMock()
|
|
await _handle_agent_event(event)
|
|
mgr.broadcast_to_agent_watchers.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_agent_event_skips_invalid_uuid() -> None:
|
|
event = _evt(EventType.AGENT_SPAWNED, {"agent_id": "bad"})
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.broadcast_to_agent_watchers = AsyncMock()
|
|
await _handle_agent_event(event)
|
|
mgr.broadcast_to_agent_watchers.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_agent_event_uses_source_agent_fallback() -> None:
|
|
"""When data has no agent_id, falls back to event.source_agent."""
|
|
aid = uuid4()
|
|
event = _evt(EventType.AGENT_STOPPED, {}, source_agent=str(aid))
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.agent_connections = {aid: {"sock"}}
|
|
mgr.broadcast_to_agent_watchers = AsyncMock()
|
|
await _handle_agent_event(event)
|
|
mgr.broadcast_to_agent_watchers.assert_awaited_once()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_agent_event_skips_when_no_connections() -> None:
|
|
aid = uuid4()
|
|
event = _evt(EventType.AGENT_SPAWNED, {"agent_id": str(aid)})
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.agent_connections = {}
|
|
mgr.broadcast_to_agent_watchers = AsyncMock()
|
|
await _handle_agent_event(event)
|
|
mgr.broadcast_to_agent_watchers.assert_not_called()
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_agent_event_broadcasts() -> None:
|
|
aid = uuid4()
|
|
event = _evt(EventType.AGENT_RESUMED, {"agent_id": str(aid)})
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.agent_connections = {aid: {"sock"}}
|
|
mgr.broadcast_to_agent_watchers = AsyncMock()
|
|
await _handle_agent_event(event)
|
|
mgr.broadcast_to_agent_watchers.assert_awaited_once()
|
|
call_args = mgr.broadcast_to_agent_watchers.await_args
|
|
assert call_args.args[0] == aid
|
|
assert call_args.args[1]["type"] == "agent.resumed"
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# _handle_rate_limit_event
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_rate_limit_hit_broadcasts_to_system() -> None:
|
|
"""RATE_LIMIT_HIT → broadcast_system tagged with the type, payload intact."""
|
|
retry_after = 60.0
|
|
event = _evt(
|
|
EventType.RATE_LIMIT_HIT,
|
|
{
|
|
"provider": "anthropic",
|
|
"affectedAgents": ["be-dev-1"],
|
|
"retryAfterSeconds": retry_after,
|
|
"timestamp": "2026-06-11T00:00:00+00:00",
|
|
},
|
|
)
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.broadcast_system = AsyncMock()
|
|
await _handle_rate_limit_event(event)
|
|
mgr.broadcast_system.assert_awaited_once()
|
|
msg = mgr.broadcast_system.await_args.args[0]
|
|
assert msg["type"] == "RATE_LIMIT_HIT"
|
|
assert msg["provider"] == "anthropic"
|
|
assert msg["affectedAgents"] == ["be-dev-1"]
|
|
assert msg["retryAfterSeconds"] == retry_after
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_rate_limit_lifted_broadcasts_to_system() -> None:
|
|
event = _evt(
|
|
EventType.RATE_LIMIT_LIFTED,
|
|
{"provider": "anthropic", "timestamp": "2026-06-11T00:01:00+00:00"},
|
|
)
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.broadcast_system = AsyncMock()
|
|
await _handle_rate_limit_event(event)
|
|
msg = mgr.broadcast_system.await_args.args[0]
|
|
assert msg["type"] == "RATE_LIMIT_LIFTED"
|
|
assert msg["provider"] == "anthropic"
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_rate_limit_ignores_unrelated_event() -> None:
|
|
"""A non-rate-limit event type is a no-op (defensive guard)."""
|
|
event = _evt(EventType.AGENT_SPAWNED, {"provider": "anthropic"})
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.broadcast_system = AsyncMock()
|
|
await _handle_rate_limit_event(event)
|
|
mgr.broadcast_system.assert_not_called()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# _handle_usage_event
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_usage_update_broadcasts_to_system() -> None:
|
|
"""USAGE_UPDATE event → broadcast_system tagged USAGE_UPDATE + data fields."""
|
|
expected_input = 100
|
|
expected_output = 50
|
|
event = _evt(
|
|
EventType.USAGE_UPDATE,
|
|
{
|
|
"agent_id": "be-dev-1",
|
|
"task_id": "task-abc",
|
|
"input_tokens": expected_input,
|
|
"output_tokens": expected_output,
|
|
"model": "claude-sonnet-4-6",
|
|
"timestamp": "2026-06-11T00:00:00+00:00",
|
|
},
|
|
)
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.broadcast_system = AsyncMock()
|
|
await _handle_usage_event(event)
|
|
mgr.broadcast_system.assert_awaited_once()
|
|
msg = mgr.broadcast_system.await_args.args[0]
|
|
assert msg["type"] == "USAGE_UPDATE"
|
|
assert msg["agent_id"] == "be-dev-1"
|
|
assert msg["input_tokens"] == expected_input
|
|
assert msg["output_tokens"] == expected_output
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_handle_usage_snapshot_broadcasts_to_system() -> None:
|
|
"""USAGE_SNAPSHOT event → broadcast_system tagged USAGE_SNAPSHOT + aggregate."""
|
|
expected_input = 500
|
|
event = _evt(
|
|
EventType.USAGE_SNAPSHOT,
|
|
{
|
|
"period": "live",
|
|
"totals": {"input_tokens": expected_input, "output_tokens": 200},
|
|
"cost_estimate": 0.0025,
|
|
"by_agent": [
|
|
{
|
|
"agent_id": "be-dev-1",
|
|
"input_tokens": 500,
|
|
"output_tokens": 200,
|
|
"model": "sonnet",
|
|
"cost_estimate": 0.0025,
|
|
}
|
|
],
|
|
"timestamp": "2026-06-11T00:01:00+00:00",
|
|
},
|
|
)
|
|
with patch("roboco.api.websocket_bridge.manager") as mgr:
|
|
mgr.broadcast_system = AsyncMock()
|
|
await _handle_usage_event(event)
|
|
mgr.broadcast_system.assert_awaited_once()
|
|
msg = mgr.broadcast_system.await_args.args[0]
|
|
assert msg["type"] == "USAGE_SNAPSHOT"
|
|
assert msg["period"] == "live"
|
|
assert msg["totals"]["input_tokens"] == expected_input
|
|
assert len(msg["by_agent"]) == 1
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Registration + start
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def test_register_websocket_bridge_handlers_subscribes_all_event_types() -> None:
|
|
"""Registration wires up all handler categories, including usage events."""
|
|
|
|
class _FakeBus:
|
|
def __init__(self) -> None:
|
|
self.subscribed: list[tuple[EventType, object]] = []
|
|
|
|
def subscribe(self, event_type: EventType, handler: object) -> None:
|
|
self.subscribed.append((event_type, handler))
|
|
|
|
fake = _FakeBus()
|
|
with patch("roboco.api.websocket_bridge.get_event_bus", return_value=fake):
|
|
register_websocket_bridge_handlers()
|
|
types = [t for t, _ in fake.subscribed]
|
|
# All 14 expected event types appear at least once.
|
|
assert EventType.NOTIFICATION_SENT in types
|
|
assert EventType.NOTIFICATION_ACKED in types
|
|
assert EventType.SESSION_CREATED in types
|
|
assert EventType.SESSION_CLOSED in types
|
|
assert EventType.SESSION_TIMEOUT in types
|
|
assert EventType.AGENT_SPAWNED in types
|
|
assert EventType.AGENT_STOPPED in types
|
|
assert EventType.AGENT_WAITING in types
|
|
assert EventType.AGENT_RESUMED in types
|
|
assert EventType.AGENT_ERROR in types
|
|
assert EventType.RATE_LIMIT_HIT in types
|
|
assert EventType.RATE_LIMIT_LIFTED in types
|
|
# Usage events forwarded to /ws/system.
|
|
assert EventType.USAGE_UPDATE in types
|
|
assert EventType.USAGE_SNAPSHOT in types
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_start_websocket_bridge_registers_handlers() -> None:
|
|
"""start_websocket_bridge() calls register_websocket_bridge_handlers."""
|
|
with patch("roboco.api.websocket_bridge.register_websocket_bridge_handlers") as reg:
|
|
await start_websocket_bridge()
|
|
reg.assert_called_once()
|