From b3057628b03232d730524d4ddca407e89c9dee03 Mon Sep 17 00:00:00 2001 From: Renzo F <45401804+rennf93@users.noreply.github.com> Date: Wed, 10 Jun 2026 14:38:44 +0200 Subject: [PATCH] =?UTF-8?q?[499f9eb1]=20Token=20Usage=20&=20Cost=20Analyti?= =?UTF-8?q?cs=20=E2=80=94=20Full-Stack=20Instrumentation,=20Persistence,?= =?UTF-8?q?=20and=20Visualization=20(#90)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [cd2bf666] feat(usage): add token usage types, API client, hooks, and UI components (#87) (#88) - Append 5 TypeScript interfaces to src/types/index.ts: TokenUsageSnapshot, AgentUsageRow, UsageSession, UsageTimePoint, ModelUsageSlice - Create src/lib/api/usage.ts: Axios singleton + isMockMode guards for getUsageSnapshot, getUsageTimeSeries, getAgentUsage, getUsageSessions, getModelUsage - Create src/hooks/use-usage.ts: usageKeys factory + useUsageSnapshot, useUsageTimeSeries, useAgentUsage, useUsageSessions, useModelUsage hooks - Create UsageOverviewPanel (dashboard/usage-overview-panel.tsx): 6 metric rows with Skeleton loading state; week-over-week trend arrow for cost - Update CommandCenter: Metrics+Alerts row expanded from 2-col to 3-col grid adding UsageOverviewPanel - Create src/components/metrics/ folder: UsageTimeSeriesChart (recharts stacked AreaChart with var(--chart-1/2/3)), ModelUsageDonut (PieChart), AgentUsageChart and TeamUsageChart (BarChart), SessionsTable (sortable columns + 10-row Prev/Next pagination) - Update Metrics page: Token Usage & Costs section with 5 rows (summary cards, time series+donut, agent+team bar charts, projection+cache efficiency, sessions table) - Add usage mini-bar to AgentCard: token count + cost + progress bar; AgentGrid and Agents page pass agentUsageMap through - Install recharts 3.8.1 - Export all new symbols through their barrel index.ts files Co-authored-by: Frontend Developer 1 * [10372f0f] Implement full token usage instrumentation: DB migration, SDK endpoints, orchestrator hooks, analytics API, WebSocket events, dashboard integration (#86) (#89) * [10372f0f] feat(token-usage): add Alembic migration 026 for token usage tables Create agent_spawn_sessions, token_usage_snapshots, and daily_usage_rollups tables with correct BIGINT columns, indexes, and unique constraint. Chain: 025_agentrole_prompter → 026_token_usage_tables. * [10372f0f] feat(token-usage): add ORM table classes for token usage instrumentation Add AgentSpawnSessionTable, TokenUsageSnapshotTable, DailyUsageRollupTable to db/tables.py. Import BigInteger and Date from SQLAlchemy. All columns match the migration schema with BIGINT token counts and proper indexes. * [10372f0f] feat(billing): add pricing module with calculate_cost() function Create roboco/billing/__init__.py and roboco/billing/pricing.py with calculate_cost() supporting Claude opus/sonnet/haiku models with input/output/cache pricing. Unknown models return 0.0 without raising. * [10372f0f] feat(sdk): add POST /usage/report and GET /usage/status endpoints to agent SDK Extend _SessionState with token counters. Add TokenReportRequest and TokenUsageStatus models. POST /usage/report additively accumulates token counts; GET /usage/status returns current session totals for sweeper polling. * [10372f0f] feat(orchestrator): add token usage instrumentation hooks - _launch_spawn() calls _record_spawn_session() after successful container spawn - stop_agent() calls _finalize_spawn_session() before container removal - _run_sweep() calls _sweep_token_snapshots() and _sweep_daily_rollup() each tick - New methods: _record_spawn_session, _finalize_spawn_session, _sweep_token_snapshots, _sweep_daily_rollup in TOKEN USAGE section * [10372f0f] feat(api): add token usage analytics API with 7 endpoints Create roboco/services/usage.py (UsageService) and roboco/api/routes/usage.py. Endpoints: GET /api/usage/summary, /time-series, /by-agent, /by-team, /by-model, /projection, /cache-efficiency. Register in app.py. * [10372f0f] feat(dashboard): add usage_summary field to CEO dashboard Add UsageSummary schema (tokens_today, cost_today_usd) to dashboard schemas. Add usage_summary: UsageSummary | None to CEOOverview. Update get_ceo_overview() to populate usage_summary from daily_usage_rollups. * [10372f0f] fix(billing/tests): remove dead except block in _sweep_daily_rollup, add unit tests for pricing.py and services/usage.py - Remove unreachable `except Exception as e` block in orchestrator.py _sweep_daily_rollup() (lines 3376-3381) which referenced undefined `agent_id` and was copy-pasted from _sweep_token_snapshots by mistake - Add tests/unit/billing/test_pricing.py: 31 tests covering opus/sonnet/ haiku tiers with all 4 token types, unknown model → 0.0, empty string → 0.0, and substring-match priority (longer fragment wins) - Add tests/unit/services/test_usage.py: 25 tests covering get_summary trend_pct edge cases (prev=0, both=0, prev>0), get_by_agent/team/model pct_of_total summing to 100%, get_projection formula (avg_daily×30), and get_cache_efficiency hit-rate and cost_saved arithmetic - pricing.py: 100% coverage; services/usage.py: 83% coverage (>80% target) * [10372f0f] fix(usage): include cache tokens in time-series total_tokens to fix AC9 consistency violation get_time_series() previously computed total_tokens as tokens_input + tokens_output only. get_summary() includes all 4 token types (input + output + cache_read + cache_write). AC9 requires both endpoints to agree on their totals for the same period. Fix: add tokens_cache_read and tokens_cache_write to the SELECT query in get_time_series() and include them in the total_tokens calculation. Also adds 4 new unit tests in TestGetTimeSeries covering: - total_tokens includes cache_read and cache_write (the AC9 guard) - zero cache tokens still produces correct total - empty result returns empty list - required fields are present in each point * [10372f0f] fix(usage): remove unused imports and include cache tokens in breakdown totals (AC10) - Remove import math (F401 — never used) - Remove text from sqlalchemy import (F401 — never used) - Remove unused local calculate_cost import inside get_cache_efficiency (F401) - Add tokens_cache_read and tokens_cache_write to SELECT in get_by_agent, get_by_team, and get_by_model; update grand_total and per-item total to include all 4 token types so totals match get_summary() (AC10 fix) - Update test mock rows to include explicit tokens_cache_read=0 and tokens_cache_write=0 so they work with the fixed code - Add new test cases: test_cache_tokens_included_in_total_tokens and test_pct_of_total_sums_to_100_with_cache_tokens for each breakdown class --------- Co-authored-by: Backend Developer 1 * [44b9eb1f] feat(usage): align frontend API client, TS types, and chart components to real backend contract (#92) (#94) Update all usage-related frontend code to match the actual FastAPI backend response shapes and endpoint paths: - panel/src/lib/api/usage.ts: rewrite all 7 API functions to use correct endpoint paths (/usage/summary, /usage/by-agent, /usage/by-model, /usage/by-team, /usage/time-series, /usage/projection, /usage/cache-efficiency); send period query param (24h/7d/30d not hours); mock generators produce data matching real backend shapes exactly; getUsageSessions returns [] in prod (no /usage/sessions endpoint exists) - panel/src/types/index.ts: replace TokenUsageSnapshot with UsageSummary (tokens_input/tokens_output/total_cost_usd/trend_pct); update AgentUsageRow to use agent_slug/total_tokens/cost_usd/pct_of_total; add TeamUsageRow, UsageProjection, CacheEfficiencyResponse; update UsageTimePoint to use bucket field; update UsageSession to use agent_slug - panel/src/hooks/use-usage.ts: rewrite all hooks to match new API and types; add useTeamUsage, useUsageProjection, useCacheEfficiency hooks - panel/src/components/metrics/usage-time-series-chart.tsx: use bucket field (not timestamp) for axis labels - panel/src/components/metrics/agent-usage-chart.tsx: use agent_slug and total_tokens (not agent_name/tokens_today) - panel/src/components/metrics/team-usage-chart.tsx: rewrite to accept TeamUsageRow[] from API directly - panel/src/components/metrics/model-usage-donut.tsx: use total_tokens, cost_usd, pct_of_total (not tokens/cost/percentage) - panel/src/components/metrics/sessions-table.tsx: use agent_slug, sort keys updated - panel/src/components/dashboard/usage-overview-panel.tsx: use useUsageSummary with tokens_input/tokens_output/total_cost_usd/trend_pct - panel/src/app/(dashboard)/metrics/page.tsx: wire all new hooks, add TeamUsageChart, ProjectionCard, CacheEfficiencyCard with correct types - panel/src/app/(dashboard)/agents/page.tsx: key agentUsageMap by agent_slug - panel/src/components/agents/agent-card.tsx: use total_tokens and cost_usd Co-authored-by: Frontend Developer 1 * [2161b832] fix: SDK_PORT constant, stop_agent lock refactor, usage_session_id binding, rollup 7-day window (#93) (#95) - Add SDK_PORT = 9000 module-level constant to orchestrator.py; replace hardcoded 9000 in _sweep_budget_exceeded URL with SDK_PORT - Add UUID to TYPE_CHECKING imports to satisfy ruff F821 - Refactor stop_agent: call _finalize_spawn_session BEFORE acquiring self._lock so the SDK HTTP round-trip does not hold the lock - Add usage_session_id: UUID | None field to AgentInstance dataclass - Change _record_spawn_session to return UUID | None; wire return value back to instance.usage_session_id in _launch_spawn - Update _finalize_spawn_session to use WHERE id=usage_session_id for direct session row lookup when usage_session_id is not None - Add started_at >= (now_utc - 7 days) filter to _sweep_daily_rollup aggregate query to avoid re-aggregating all-time history each sweep Co-authored-by: Backend Developer 1 * [2e0759e1] fix: pricing accuracy, import ordering, session-id binding, rollup cleanup, write-hook tests (#97) (#98) - pricing.py: correct claude-opus-4 prices (5/25/0.50/6.25 not 15/75/1.5/3.75) and haiku family prices (1/5/0.10/1.25 not 0.8/4/0.08/0.20); add Ollama zero-cost early-return; add structlog warning for unmatched model names - app.py: move usage_router import before routes.v1 block (ruff isort fix) - orchestrator.py _sweep_daily_rollup: remove unused calculate_cost import; add blank line between stdlib (uuid4) and third-party (sqlalchemy) imports - orchestrator.py _sweep_token_snapshots: prefer direct lookup by instance.usage_session_id; fall back to agent_slug heuristic only when None - tests: add test_sweep_daily_rollup_inserts_new_row and test_stop_agent_finalizes_before_lock to test_orchestrator_write_hooks.py - usage.py, routes/usage.py, stream_bus.py, test files: ruff format/lint fixes Co-authored-by: Backend Developer 1 * Mypy compliance * fix(migrations,tests): linearize forked migration chain + correct ceo_reject coordination-root expectation The master merge brought in 026_completed_dependency_ids alongside the rework's 026_token_usage_tables — both off 025, forking the alembic head and breaking the enum-parity test. Rebase token-usage onto 026_completed_dependency_ids (linear chain, single head). Also: test_ceo_reject_routes_coordination_task_to_main_pm asserted the old NEEDS_REVISION behavior; the lifecycle fix correctly routes a coordination root to PENDING (Main PM's claim source). Update the assertion. --------- Co-authored-by: Frontend Developer 1 Co-authored-by: Backend Developer 1 Co-authored-by: Renn F --- .gitignore | 1 + alembic/versions/026_token_usage_tables.py | 170 ++++ panel/package.json | 1 + panel/pnpm-lock.yaml | 345 +++++++- panel/src/app/(dashboard)/agents/page.tsx | 18 +- panel/src/app/(dashboard)/metrics/page.tsx | 231 ++++++ panel/src/components/agents/agent-card.tsx | 28 +- panel/src/components/agents/agent-grid.tsx | 7 +- .../components/dashboard/command-center.tsx | 6 +- panel/src/components/dashboard/index.ts | 1 + .../dashboard/usage-overview-panel.tsx | 105 +++ .../components/metrics/agent-usage-chart.tsx | 79 ++ panel/src/components/metrics/index.ts | 5 + .../components/metrics/model-usage-donut.tsx | 78 ++ .../src/components/metrics/sessions-table.tsx | 191 +++++ .../components/metrics/team-usage-chart.tsx | 76 ++ .../metrics/usage-time-series-chart.tsx | 112 +++ panel/src/hooks/index.ts | 1 + panel/src/hooks/use-usage.ts | 112 +++ panel/src/lib/api/index.ts | 1 + panel/src/lib/api/usage.ts | 254 ++++++ panel/src/types/index.ts | 89 ++ roboco/agent_sdk/models.py | 39 + roboco/agent_sdk/server.py | 55 ++ roboco/api/app.py | 8 + roboco/api/routes/dashboard.py | 14 + roboco/api/routes/usage.py | 147 ++++ roboco/api/schemas/dashboard.py | 15 + roboco/billing/__init__.py | 8 + roboco/billing/pricing.py | 106 +++ roboco/db/tables.py | 144 ++++ roboco/events/stream_bus.py | 4 +- roboco/models/runtime.py | 4 + roboco/runtime/orchestrator.py | 440 +++++++++- roboco/services/usage.py | 554 +++++++++++++ .../test_task_service_transitions.py | 6 +- tests/unit/billing/__init__.py | 0 tests/unit/billing/test_pricing.py | 299 +++++++ .../runtime/test_orchestrator_write_hooks.py | 543 +++++++++++++ tests/unit/services/test_usage.py | 761 ++++++++++++++++++ 40 files changed, 5039 insertions(+), 19 deletions(-) create mode 100644 alembic/versions/026_token_usage_tables.py create mode 100644 panel/src/components/dashboard/usage-overview-panel.tsx create mode 100644 panel/src/components/metrics/agent-usage-chart.tsx create mode 100644 panel/src/components/metrics/index.ts create mode 100644 panel/src/components/metrics/model-usage-donut.tsx create mode 100644 panel/src/components/metrics/sessions-table.tsx create mode 100644 panel/src/components/metrics/team-usage-chart.tsx create mode 100644 panel/src/components/metrics/usage-time-series-chart.tsx create mode 100644 panel/src/hooks/use-usage.ts create mode 100644 panel/src/lib/api/usage.ts create mode 100644 roboco/api/routes/usage.py create mode 100644 roboco/billing/__init__.py create mode 100644 roboco/billing/pricing.py create mode 100644 roboco/services/usage.py create mode 100644 tests/unit/billing/__init__.py create mode 100644 tests/unit/billing/test_pricing.py create mode 100644 tests/unit/runtime/test_orchestrator_write_hooks.py create mode 100644 tests/unit/services/test_usage.py diff --git a/.gitignore b/.gitignore index 935f3e1e..0bcd4c32 100644 --- a/.gitignore +++ b/.gitignore @@ -102,3 +102,4 @@ panel/.env.local panel/.env.*.local # Internal-only: strategy/scratch/reference dumps — never publish docs/internal/ +.pnpm-store/ diff --git a/alembic/versions/026_token_usage_tables.py b/alembic/versions/026_token_usage_tables.py new file mode 100644 index 00000000..1385b98c --- /dev/null +++ b/alembic/versions/026_token_usage_tables.py @@ -0,0 +1,170 @@ +"""026_token_usage_tables + +Create token usage instrumentation tables: +- agent_spawn_sessions: tracks each agent container spawn with token totals +- token_usage_snapshots: periodic snapshots of token usage per session +- daily_usage_rollups: aggregated daily usage per agent/team/model + +Revision ID: 026_token_usage_tables +Revises: 025_agentrole_prompter +Create Date: 2026-06-09 +""" + +from __future__ import annotations + +import sqlalchemy as sa +from alembic import op +from sqlalchemy.dialects.postgresql import UUID + +revision = "026_token_usage_tables" +# Rebased onto 026_completed_dependency_ids so the chain stays linear: the +# master merge brought in a second migration off 025_agentrole_prompter, which +# forked the head. Chain is now 025 -> 026_completed_dependency_ids -> this. +down_revision = "026_completed_dependency_ids" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + # ------------------------------------------------------------------ + # agent_spawn_sessions + # One row per container spawn. Opened on spawn, closed on stop. + # ------------------------------------------------------------------ + op.create_table( + "agent_spawn_sessions", + sa.Column("id", UUID(as_uuid=True), primary_key=True), + sa.Column("agent_slug", sa.String(100), nullable=False), + sa.Column("team", sa.String(50), nullable=False), + sa.Column("role", sa.String(50), nullable=False), + sa.Column("model", sa.String(100), nullable=False), + sa.Column("task_id", sa.String(36), nullable=True), + sa.Column( + "started_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column("ended_at", sa.DateTime(timezone=True), nullable=True), + # BIGINT for token counts — they can exceed INT32 for long sessions + sa.Column("tokens_input", sa.BigInteger, nullable=False, server_default="0"), + sa.Column("tokens_output", sa.BigInteger, nullable=False, server_default="0"), + sa.Column( + "tokens_cache_read", sa.BigInteger, nullable=False, server_default="0" + ), + sa.Column( + "tokens_cache_write", sa.BigInteger, nullable=False, server_default="0" + ), + sa.Column("exit_reason", sa.String(100), nullable=True), + sa.Column("estimated_cost_usd", sa.Float, nullable=True), + ) + + # Indexes for common query patterns + op.create_index( + "ix_agent_spawn_sessions_agent_slug", + "agent_spawn_sessions", + ["agent_slug"], + ) + op.create_index( + "ix_agent_spawn_sessions_started_at", + "agent_spawn_sessions", + ["started_at"], + ) + op.create_index( + "ix_agent_spawn_sessions_ended_at", + "agent_spawn_sessions", + ["ended_at"], + ) + op.create_index( + "ix_agent_spawn_sessions_team", + "agent_spawn_sessions", + ["team"], + ) + + # ------------------------------------------------------------------ + # token_usage_snapshots + # Periodic snapshots (every ~60s) of token counts for active sessions. + # ------------------------------------------------------------------ + op.create_table( + "token_usage_snapshots", + sa.Column("id", UUID(as_uuid=True), primary_key=True), + sa.Column( + "agent_spawn_session_id", + UUID(as_uuid=True), + sa.ForeignKey( + "agent_spawn_sessions.id", ondelete="CASCADE", name="fk_snapshot_session" + ), + nullable=False, + ), + sa.Column( + "snapshotted_at", + sa.DateTime(timezone=True), + nullable=False, + server_default=sa.func.now(), + ), + sa.Column("tokens_input", sa.BigInteger, nullable=False, server_default="0"), + sa.Column("tokens_output", sa.BigInteger, nullable=False, server_default="0"), + sa.Column( + "tokens_cache_read", sa.BigInteger, nullable=False, server_default="0" + ), + sa.Column( + "tokens_cache_write", sa.BigInteger, nullable=False, server_default="0" + ), + ) + + op.create_index( + "ix_token_usage_snapshots_session_id", + "token_usage_snapshots", + ["agent_spawn_session_id"], + ) + op.create_index( + "ix_token_usage_snapshots_snapshotted_at", + "token_usage_snapshots", + ["snapshotted_at"], + ) + + # ------------------------------------------------------------------ + # daily_usage_rollups + # Pre-aggregated daily totals per (date, agent_slug, team, model). + # Populated by the sweeper; upserted on each sweep so re-runs are safe. + # ------------------------------------------------------------------ + op.create_table( + "daily_usage_rollups", + sa.Column("id", UUID(as_uuid=True), primary_key=True), + sa.Column("date", sa.Date, nullable=False), + sa.Column("agent_slug", sa.String(100), nullable=False), + sa.Column("team", sa.String(50), nullable=False), + sa.Column("model", sa.String(100), nullable=False), + sa.Column("tokens_input", sa.BigInteger, nullable=False, server_default="0"), + sa.Column("tokens_output", sa.BigInteger, nullable=False, server_default="0"), + sa.Column( + "tokens_cache_read", sa.BigInteger, nullable=False, server_default="0" + ), + sa.Column( + "tokens_cache_write", sa.BigInteger, nullable=False, server_default="0" + ), + sa.Column("total_cost_usd", sa.Float, nullable=False, server_default="0"), + sa.Column("session_count", sa.Integer, nullable=False, server_default="0"), + ) + + # Unique constraint enables ON CONFLICT upsert in the sweeper + op.create_unique_constraint( + "uq_daily_rollup_date_agent_team_model", + "daily_usage_rollups", + ["date", "agent_slug", "team", "model"], + ) + op.create_index( + "ix_daily_rollups_date", + "daily_usage_rollups", + ["date"], + ) + op.create_index( + "ix_daily_rollups_agent_slug", + "daily_usage_rollups", + ["agent_slug"], + ) + + +def downgrade() -> None: + op.drop_table("daily_usage_rollups") + op.drop_table("token_usage_snapshots") + op.drop_table("agent_spawn_sessions") diff --git a/panel/package.json b/panel/package.json index 64a18e9b..fc1c318a 100644 --- a/panel/package.json +++ b/panel/package.json @@ -42,6 +42,7 @@ "react-dom": "19.2.3", "react-hook-form": "^7.71.0", "react-markdown": "^10.1.0", + "recharts": "^3.8.1", "remark-gfm": "^4.0.1", "sonner": "^2.0.7", "tailwind-merge": "^3.4.0", diff --git a/panel/pnpm-lock.yaml b/panel/pnpm-lock.yaml index e92165e0..6bd2a253 100644 --- a/panel/pnpm-lock.yaml +++ b/panel/pnpm-lock.yaml @@ -107,6 +107,9 @@ importers: react-markdown: specifier: ^10.1.0 version: 10.1.0(@types/react@19.2.8)(react@19.2.3) + recharts: + specifier: ^3.8.1 + version: 3.8.1(@types/react@19.2.8)(react-dom@19.2.3(react@19.2.3))(react-is@16.13.1)(react@19.2.3)(redux@5.0.1) remark-gfm: specifier: ^4.0.1 version: 4.0.1 @@ -121,7 +124,7 @@ importers: version: 4.3.5 zustand: specifier: ^5.0.10 - version: 5.0.10(@types/react@19.2.8)(react@19.2.3)(use-sync-external-store@1.6.0(react@19.2.3)) + version: 5.0.10(@types/react@19.2.8)(immer@11.1.8)(react@19.2.3)(use-sync-external-store@1.6.0(react@19.2.3)) devDependencies: '@tailwindcss/postcss': specifier: ^4 @@ -362,89 +365,105 @@ packages: resolution: {integrity: sha512-excjX8DfsIcJ10x1Kzr4RcWe1edC9PquDRRPx3YVCvQv+U5p7Yin2s32ftzikXojb1PIFc/9Mt28/y+iRklkrw==} cpu: [arm64] os: [linux] + libc: [glibc] '@img/sharp-libvips-linux-arm@1.2.4': resolution: {integrity: sha512-bFI7xcKFELdiNCVov8e44Ia4u2byA+l3XtsAj+Q8tfCwO6BQ8iDojYdvoPMqsKDkuoOo+X6HZA0s0q11ANMQ8A==} cpu: [arm] os: [linux] + libc: [glibc] '@img/sharp-libvips-linux-ppc64@1.2.4': resolution: {integrity: sha512-FMuvGijLDYG6lW+b/UvyilUWu5Ayu+3r2d1S8notiGCIyYU/76eig1UfMmkZ7vwgOrzKzlQbFSuQfgm7GYUPpA==} cpu: [ppc64] os: [linux] + libc: [glibc] '@img/sharp-libvips-linux-riscv64@1.2.4': resolution: {integrity: sha512-oVDbcR4zUC0ce82teubSm+x6ETixtKZBh/qbREIOcI3cULzDyb18Sr/Wcyx7NRQeQzOiHTNbZFF1UwPS2scyGA==} cpu: [riscv64] os: [linux] + libc: [glibc] '@img/sharp-libvips-linux-s390x@1.2.4': resolution: {integrity: sha512-qmp9VrzgPgMoGZyPvrQHqk02uyjA0/QrTO26Tqk6l4ZV0MPWIW6LTkqOIov+J1yEu7MbFQaDpwdwJKhbJvuRxQ==} cpu: [s390x] os: [linux] + libc: [glibc] '@img/sharp-libvips-linux-x64@1.2.4': resolution: {integrity: sha512-tJxiiLsmHc9Ax1bz3oaOYBURTXGIRDODBqhveVHonrHJ9/+k89qbLl0bcJns+e4t4rvaNBxaEZsFtSfAdquPrw==} cpu: [x64] os: [linux] + libc: [glibc] '@img/sharp-libvips-linuxmusl-arm64@1.2.4': resolution: {integrity: sha512-FVQHuwx1IIuNow9QAbYUzJ+En8KcVm9Lk5+uGUQJHaZmMECZmOlix9HnH7n1TRkXMS0pGxIJokIVB9SuqZGGXw==} cpu: [arm64] os: [linux] + libc: [musl] '@img/sharp-libvips-linuxmusl-x64@1.2.4': resolution: {integrity: sha512-+LpyBk7L44ZIXwz/VYfglaX/okxezESc6UxDSoyo2Ks6Jxc4Y7sGjpgU9s4PMgqgjj1gZCylTieNamqA1MF7Dg==} cpu: [x64] os: [linux] + libc: [musl] '@img/sharp-linux-arm64@0.34.5': resolution: {integrity: sha512-bKQzaJRY/bkPOXyKx5EVup7qkaojECG6NLYswgktOZjaXecSAeCWiZwwiFf3/Y+O1HrauiE3FVsGxFg8c24rZg==} engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0} cpu: [arm64] os: [linux] + libc: [glibc] '@img/sharp-linux-arm@0.34.5': resolution: {integrity: sha512-9dLqsvwtg1uuXBGZKsxem9595+ujv0sJ6Vi8wcTANSFpwV/GONat5eCkzQo/1O6zRIkh0m/8+5BjrRr7jDUSZw==} engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0} cpu: [arm] os: [linux] + libc: [glibc] '@img/sharp-linux-ppc64@0.34.5': resolution: {integrity: sha512-7zznwNaqW6YtsfrGGDA6BRkISKAAE1Jo0QdpNYXNMHu2+0dTrPflTLNkpc8l7MUP5M16ZJcUvysVWWrMefZquA==} engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0} cpu: [ppc64] os: [linux] + libc: [glibc] '@img/sharp-linux-riscv64@0.34.5': resolution: {integrity: sha512-51gJuLPTKa7piYPaVs8GmByo7/U7/7TZOq+cnXJIHZKavIRHAP77e3N2HEl3dgiqdD/w0yUfiJnII77PuDDFdw==} engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0} cpu: [riscv64] os: [linux] + libc: [glibc] '@img/sharp-linux-s390x@0.34.5': resolution: {integrity: sha512-nQtCk0PdKfho3eC5MrbQoigJ2gd1CgddUMkabUj+rBevs8tZ2cULOx46E7oyX+04WGfABgIwmMC0VqieTiR4jg==} engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0} cpu: [s390x] os: [linux] + libc: [glibc] '@img/sharp-linux-x64@0.34.5': resolution: {integrity: sha512-MEzd8HPKxVxVenwAa+JRPwEC7QFjoPWuS5NZnBt6B3pu7EG2Ge0id1oLHZpPJdn3OQK+BQDiw9zStiHBTJQQQQ==} engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0} cpu: [x64] os: [linux] + libc: [glibc] '@img/sharp-linuxmusl-arm64@0.34.5': resolution: {integrity: sha512-fprJR6GtRsMt6Kyfq44IsChVZeGN97gTD331weR1ex1c1rypDEABN6Tm2xa1wE6lYb5DdEnk03NZPqA7Id21yg==} engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0} cpu: [arm64] os: [linux] + libc: [musl] '@img/sharp-linuxmusl-x64@0.34.5': resolution: {integrity: sha512-Jg8wNT1MUzIvhBFxViqrEhWDGzqymo3sV7z7ZsaWbZNDLXRJZoRGrjulp60YYtV4wfY8VIKcWidjojlLcWrd8Q==} engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0} cpu: [x64] os: [linux] + libc: [musl] '@img/sharp-wasm32@0.34.5': resolution: {integrity: sha512-OdWTEiVkY2PHwqkbBI8frFxQQFekHaSSkUIJkwzclWZe64O1X4UlUjqqqLaPbUpMOQk6FBu/HtlGXNblIs0huw==} @@ -511,24 +530,28 @@ packages: engines: {node: '>= 10'} cpu: [arm64] os: [linux] + libc: [glibc] '@next/swc-linux-arm64-musl@16.1.1': resolution: {integrity: sha512-MFHrgL4TXNQbBPzkKKur4Fb5ICEJa87HM7fczFs2+HWblM7mMLdco3dvyTI+QmLBU9xgns/EeeINSZD6Ar+oLg==} engines: {node: '>= 10'} cpu: [arm64] os: [linux] + libc: [musl] '@next/swc-linux-x64-gnu@16.1.1': resolution: {integrity: sha512-20bYDfgOQAPUkkKBnyP9PTuHiJGM7HzNBbuqmD0jiFVZ0aOldz+VnJhbxzjcSabYsnNjMPsE0cyzEudpYxsrUQ==} engines: {node: '>= 10'} cpu: [x64] os: [linux] + libc: [glibc] '@next/swc-linux-x64-musl@16.1.1': resolution: {integrity: sha512-9pRbK3M4asAHQRkwaXwu601oPZHghuSC8IXNENgbBSyImHv/zY4K5udBusgdHkvJ/Tcr96jJwQYOll0qU8+fPA==} engines: {node: '>= 10'} cpu: [x64] os: [linux] + libc: [musl] '@next/swc-win32-arm64-msvc@16.1.1': resolution: {integrity: sha512-bdfQkggaLgnmYrFkSQfsHfOhk/mCYmjnrbRCGgkMcoOBZ4n+TRRSLmT/CU5SATzlBJ9TpioUyBW/vWFXTqQRiA==} @@ -1071,9 +1094,23 @@ packages: '@radix-ui/rect@1.1.1': resolution: {integrity: sha512-HPwpGIzkl28mWyZqG52jiqDJ12waP11Pa1lGoiyUkIEuMLBP0oeK/C89esbXrxsky5we7dfd8U58nm0SgAWpVw==} + '@reduxjs/toolkit@2.12.0': + resolution: {integrity: sha512-KiT+RzZbp6mQET+Mg+h2c97+9j1sNflUxQkIHI7Yuzf6Peu+OYpmkn6nbHWmLLWj+1ZODUJFwGZ7gx3L9R9EOw==} + peerDependencies: + react: ^16.9.0 || ^17.0.0 || ^18 || ^19 + react-redux: ^7.2.1 || ^8.1.3 || ^9.0.0 + peerDependenciesMeta: + react: + optional: true + react-redux: + optional: true + '@rtsao/scc@1.1.0': resolution: {integrity: sha512-zt6OdqaDoOnJ1ZYsCYGt9YmWzDXl4vQdKTyJev62gFhRGKdx7mcT54V9KIjg+d2wi9EXsPvAPKe7i7WjfVWB8g==} + '@standard-schema/spec@1.1.0': + resolution: {integrity: sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==} + '@standard-schema/utils@0.3.0': resolution: {integrity: sha512-e7Mew686owMaPJVNNLs55PUvgz371nKgwsc4vxE49zsODpJEnxgxRo2y/OKrqueavXgZNMDVj3DdHFlaSAeU8g==} @@ -1118,24 +1155,28 @@ packages: engines: {node: '>= 10'} cpu: [arm64] os: [linux] + libc: [glibc] '@tailwindcss/oxide-linux-arm64-musl@4.1.18': resolution: {integrity: sha512-1px92582HkPQlaaCkdRcio71p8bc8i/ap5807tPRDK/uw953cauQBT8c5tVGkOwrHMfc2Yh6UuxaH4vtTjGvHg==} engines: {node: '>= 10'} cpu: [arm64] os: [linux] + libc: [musl] '@tailwindcss/oxide-linux-x64-gnu@4.1.18': resolution: {integrity: sha512-v3gyT0ivkfBLoZGF9LyHmts0Isc8jHZyVcbzio6Wpzifg/+5ZJpDiRiUhDLkcr7f/r38SWNe7ucxmGW3j3Kb/g==} engines: {node: '>= 10'} cpu: [x64] os: [linux] + libc: [glibc] '@tailwindcss/oxide-linux-x64-musl@4.1.18': resolution: {integrity: sha512-bhJ2y2OQNlcRwwgOAGMY0xTFStt4/wyU6pvI6LSuZpRgKQwxTec0/3Scu91O8ir7qCR3AuepQKLU/kX99FouqQ==} engines: {node: '>= 10'} cpu: [x64] os: [linux] + libc: [musl] '@tailwindcss/oxide-wasm32-wasi@4.1.18': resolution: {integrity: sha512-LffYTvPjODiP6PT16oNeUQJzNVyJl1cjIebq/rWWBF+3eDst5JGEFSc5cWxyRCJ0Mxl+KyIkqRxk1XPEs9x8TA==} @@ -1193,6 +1234,33 @@ packages: '@tybys/wasm-util@0.10.1': resolution: {integrity: sha512-9tTaPJLSiejZKx+Bmog4uSubteqTvFrVrURwkmHixBo0G4seD0zUxp98E1DzUBJxLQ3NPwXrGKDiVjwx/DpPsg==} + '@types/d3-array@3.2.2': + resolution: {integrity: sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw==} + + '@types/d3-color@3.1.3': + resolution: {integrity: sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==} + + '@types/d3-ease@3.0.2': + resolution: {integrity: sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==} + + '@types/d3-interpolate@3.0.4': + resolution: {integrity: sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==} + + '@types/d3-path@3.1.1': + resolution: {integrity: sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==} + + '@types/d3-scale@4.0.9': + resolution: {integrity: sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==} + + '@types/d3-shape@3.1.8': + resolution: {integrity: sha512-lae0iWfcDeR7qt7rA88BNiqdvPS5pFVPpo5OfjElwNaT2yyekbM0C9vK+yqBqEmHr6lDkRnYNoTBYlAgJa7a4w==} + + '@types/d3-time@3.0.4': + resolution: {integrity: sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==} + + '@types/d3-timer@3.0.2': + resolution: {integrity: sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==} + '@types/debug@4.1.12': resolution: {integrity: sha512-vIChWdVG3LG1SMxEvI/AK+FWJthlrqlTu7fbrlywTkkaONwk/UAGaULXRlf8vkzFBLVm0zkMdCquhL5aOjhXPQ==} @@ -1234,6 +1302,9 @@ packages: '@types/unist@3.0.3': resolution: {integrity: sha512-ko/gIFJRv177XgZsZcBwnqJN5x/Gien8qNOn0D5bQU/zAzVf9Zt3BlcUiLqhV9y4ARk0GbT3tnUiPNgnTXzc/Q==} + '@types/use-sync-external-store@0.0.6': + resolution: {integrity: sha512-zFDAD+tlpf2r4asuHEj0XH6pY6i0g5NeAHPn+15wk3BV6JA69eERFXC1gyGThDkVa1zCyKr5jox1+2LbV/AMLg==} + '@typescript-eslint/eslint-plugin@8.49.0': resolution: {integrity: sha512-JXij0vzIaTtCwu6SxTh8qBc66kmf1xs7pI4UOiMDFVct6q86G0Zs7KRcEoJgY3Cav3x5Tq0MF5jwgpgLqgKG3A==} engines: {node: ^18.18.0 || ^20.9.0 || >=21.1.0} @@ -1295,6 +1366,7 @@ packages: '@ungap/structured-clone@1.3.0': resolution: {integrity: sha512-WmoN8qaIAo7WTYWbAZuG8PYEhn5fkz7dZrqTBZ7dtt//lL2Gwms1IcnQ5yHqjDfX8Ft5j4YzDM23f87zBfDe9g==} + deprecated: Potential CWE-502 - Update to 1.3.1 or higher '@unrs/resolver-binding-android-arm-eabi@1.11.1': resolution: {integrity: sha512-ppLRUgHVaGRWUx0R0Ut06Mjo9gBaBkg3v/8AxusGLhsIotbBLuRk51rAzqLC8gq6NyyAojEXglNjzf6R948DNw==} @@ -1335,41 +1407,49 @@ packages: resolution: {integrity: sha512-34gw7PjDGB9JgePJEmhEqBhWvCiiWCuXsL9hYphDF7crW7UgI05gyBAi6MF58uGcMOiOqSJ2ybEeCvHcq0BCmQ==} cpu: [arm64] os: [linux] + libc: [glibc] '@unrs/resolver-binding-linux-arm64-musl@1.11.1': resolution: {integrity: sha512-RyMIx6Uf53hhOtJDIamSbTskA99sPHS96wxVE/bJtePJJtpdKGXO1wY90oRdXuYOGOTuqjT8ACccMc4K6QmT3w==} cpu: [arm64] os: [linux] + libc: [musl] '@unrs/resolver-binding-linux-ppc64-gnu@1.11.1': resolution: {integrity: sha512-D8Vae74A4/a+mZH0FbOkFJL9DSK2R6TFPC9M+jCWYia/q2einCubX10pecpDiTmkJVUH+y8K3BZClycD8nCShA==} cpu: [ppc64] os: [linux] + libc: [glibc] '@unrs/resolver-binding-linux-riscv64-gnu@1.11.1': resolution: {integrity: sha512-frxL4OrzOWVVsOc96+V3aqTIQl1O2TjgExV4EKgRY09AJ9leZpEg8Ak9phadbuX0BA4k8U5qtvMSQQGGmaJqcQ==} cpu: [riscv64] os: [linux] + libc: [glibc] '@unrs/resolver-binding-linux-riscv64-musl@1.11.1': resolution: {integrity: sha512-mJ5vuDaIZ+l/acv01sHoXfpnyrNKOk/3aDoEdLO/Xtn9HuZlDD6jKxHlkN8ZhWyLJsRBxfv9GYM2utQ1SChKew==} cpu: [riscv64] os: [linux] + libc: [musl] '@unrs/resolver-binding-linux-s390x-gnu@1.11.1': resolution: {integrity: sha512-kELo8ebBVtb9sA7rMe1Cph4QHreByhaZ2QEADd9NzIQsYNQpt9UkM9iqr2lhGr5afh885d/cB5QeTXSbZHTYPg==} cpu: [s390x] os: [linux] + libc: [glibc] '@unrs/resolver-binding-linux-x64-gnu@1.11.1': resolution: {integrity: sha512-C3ZAHugKgovV5YvAMsxhq0gtXuwESUKc5MhEtjBpLoHPLYM+iuwSj3lflFwK3DPm68660rZ7G8BMcwSro7hD5w==} cpu: [x64] os: [linux] + libc: [glibc] '@unrs/resolver-binding-linux-x64-musl@1.11.1': resolution: {integrity: sha512-rV0YSoyhK2nZ4vEswT/QwqzqQXw5I6CjoaYMOX0TqBlWhojUf8P94mvI7nuJTeaCkkds3QE4+zS8Ko+GdXuZtA==} cpu: [x64] os: [linux] + libc: [musl] '@unrs/resolver-binding-wasm32-wasi@1.11.1': resolution: {integrity: sha512-5u4RkfxJm+Ng7IWgkzi3qrFOvLvQYnPBmjmZQ8+szTK/b31fQCnleNl1GgEt7nIsZRIf5PLhPwT0WM+q45x/UQ==} @@ -1581,6 +1661,50 @@ packages: csstype@3.2.3: resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==} + d3-array@3.2.4: + resolution: {integrity: sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==} + engines: {node: '>=12'} + + d3-color@3.1.0: + resolution: {integrity: sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==} + engines: {node: '>=12'} + + d3-ease@3.0.1: + resolution: {integrity: sha512-wR/XK3D3XcLIZwpbvQwQ5fK+8Ykds1ip7A2Txe0yxncXSdq1L9skcG7blcedkOX+ZcgxGAmLX1FrRGbADwzi0w==} + engines: {node: '>=12'} + + d3-format@3.1.2: + resolution: {integrity: sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==} + engines: {node: '>=12'} + + d3-interpolate@3.0.1: + resolution: {integrity: sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==} + engines: {node: '>=12'} + + d3-path@3.1.0: + resolution: {integrity: sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==} + engines: {node: '>=12'} + + d3-scale@4.0.2: + resolution: {integrity: sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==} + engines: {node: '>=12'} + + d3-shape@3.2.0: + resolution: {integrity: sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==} + engines: {node: '>=12'} + + d3-time-format@4.1.0: + resolution: {integrity: sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==} + engines: {node: '>=12'} + + d3-time@3.1.0: + resolution: {integrity: sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==} + engines: {node: '>=12'} + + d3-timer@3.0.1: + resolution: {integrity: sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==} + engines: {node: '>=12'} + damerau-levenshtein@1.0.8: resolution: {integrity: sha512-sdQSFB7+llfUcQHUQO3+B8ERRj0Oa4w9POWMI/puGtuf7gFywGmkaLCElnudfTiKZV+NvHqL0ifzdrI8Ro7ESA==} @@ -1616,6 +1740,9 @@ packages: supports-color: optional: true + decimal.js-light@2.5.1: + resolution: {integrity: sha512-qIMFpTMZmny+MMIitAB6D7iVPEorVw6YQRWkvarTkT4tBeSLLiHzcwj6q0MmYSFCiVpiqPJTJEYIrpcPzVEIvg==} + decode-named-character-reference@1.2.0: resolution: {integrity: sha512-c6fcElNV6ShtZXmsgNgFFV5tVX2PaV4g+MOAkb8eXHvn6sryJBrZa9r0zV6+dtTyoCKxtDy5tyQ5ZwQuidtd+Q==} @@ -1698,6 +1825,9 @@ packages: resolution: {integrity: sha512-w+5mJ3GuFL+NjVtJlvydShqE1eN3h3PbI7/5LAsYJP/2qtuMXjfL2LpHSRqo4b4eSF5K/DH1JXKUAHSB2UW50g==} engines: {node: '>= 0.4'} + es-toolkit@1.47.0: + resolution: {integrity: sha512-n1GuoD0WEQZMBk5tttoZSqwgyLx01oqa5XsBmCHwPyNe1S9jPBEmtR2pSgp2kJuWE3ciFZ6yRHmY4pM4C3OOkw==} + escalade@3.2.0: resolution: {integrity: sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==} engines: {node: '>=6'} @@ -1829,6 +1959,9 @@ packages: resolution: {integrity: sha512-kVscqXk4OCp68SZ0dkgEKVi6/8ij300KBWTJq32P/dYeWTSwK41WyTxalN1eRmA5Z9UU/LX9D7FWSmV9SAYx6g==} engines: {node: '>=0.10.0'} + eventemitter3@5.0.4: + resolution: {integrity: sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==} + extend@3.0.2: resolution: {integrity: sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==} @@ -2007,6 +2140,12 @@ packages: resolution: {integrity: sha512-Hs59xBNfUIunMFgWAbGX5cq6893IbWg4KnrjbYwX3tx0ztorVgTDA6B2sxf8ejHJ4wz8BqGUMYlnzNBer5NvGg==} engines: {node: '>= 4'} + immer@10.2.0: + resolution: {integrity: sha512-d/+XTN3zfODyjr89gM3mPq1WNX2B8pYsu7eORitdwyA2sBubnTl3laYlBk4sXY5FUa5qTZGBDPJICVbvqzjlbw==} + + immer@11.1.8: + resolution: {integrity: sha512-/tbkHMW7y10Lx6i1crLjD4/OhNkRG+Fo7byZHtah0547nIeXYcpIXaUh0IAQY6gO5459qpGGYapcEOHtFXkIuA==} + import-fresh@3.3.1: resolution: {integrity: sha512-TR3KfrTZTYLPB6jUjfx6MF9WcWrHL9su5TObK4ZkYgBdWKPOFoSoQIdEuTuR82pmtxH2spWG9h6etwfr1pLBqQ==} engines: {node: '>=6'} @@ -2022,6 +2161,10 @@ packages: resolution: {integrity: sha512-4gd7VpWNQNB4UKKCFFVcp1AVv+FMOgs9NKzjHKusc8jTMhd5eL1NqQqOpE0KzMds804/yHlglp3uxgluOqAPLw==} engines: {node: '>= 0.4'} + internmap@2.0.3: + resolution: {integrity: sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==} + engines: {node: '>=12'} + is-alphabetical@2.0.1: resolution: {integrity: sha512-FWyyY60MeTNyeSRpkM2Iry0G9hpr7/9kD40mD/cGQEuilcZYS4okz8SN2Q6rLCJ8gbCt6fN+rC+6tMGS99LaxQ==} @@ -2238,24 +2381,28 @@ packages: engines: {node: '>= 12.0.0'} cpu: [arm64] os: [linux] + libc: [glibc] lightningcss-linux-arm64-musl@1.30.2: resolution: {integrity: sha512-5Vh9dGeblpTxWHpOx8iauV02popZDsCYMPIgiuw97OJ5uaDsL86cnqSFs5LZkG3ghHoX5isLgWzMs+eD1YzrnA==} engines: {node: '>= 12.0.0'} cpu: [arm64] os: [linux] + libc: [musl] lightningcss-linux-x64-gnu@1.30.2: resolution: {integrity: sha512-Cfd46gdmj1vQ+lR6VRTTadNHu6ALuw2pKR9lYq4FnhvgBc4zWY1EtZcAc6EffShbb1MFrIPfLDXD6Xprbnni4w==} engines: {node: '>= 12.0.0'} cpu: [x64] os: [linux] + libc: [glibc] lightningcss-linux-x64-musl@1.30.2: resolution: {integrity: sha512-XJaLUUFXb6/QG2lGIW6aIk6jKdtjtcffUT0NKvIqhSBY3hh9Ch+1LCeH80dR9q9LBjG3ewbDjnumefsLsP6aiA==} engines: {node: '>= 12.0.0'} cpu: [x64] os: [linux] + libc: [musl] lightningcss-win32-arm64-msvc@1.30.2: resolution: {integrity: sha512-FZn+vaj7zLv//D/192WFFVA0RgHawIcHqLX9xuWiQt7P0PtdFEVaxgF9rjM/IRYHQXNnk61/H/gb2Ei+kUQ4xQ==} @@ -2639,6 +2786,18 @@ packages: '@types/react': '>=18' react: '>=18' + react-redux@9.3.0: + resolution: {integrity: sha512-KQopgqFo/p/fgmAs5qz6p5RWaNAzq40WAu7fJIXnQpYxFPbJYtsJPWvGeF2rOBaY/kEuV77AVsX8TsQzKm+A/g==} + peerDependencies: + '@types/react': ^18.2.25 || ^19 + react: ^18.0 || ^19 + redux: ^5.0.0 + peerDependenciesMeta: + '@types/react': + optional: true + redux: + optional: true + react-remove-scroll-bar@2.3.8: resolution: {integrity: sha512-9r+yi9+mgU33AKcj6IbT9oRCO78WriSj6t/cF8DWBZJ9aOGPOTEDvdUDz1FwKim7QXWwmHqtdHnRJfhAxEG46Q==} engines: {node: '>=10'} @@ -2673,6 +2832,22 @@ packages: resolution: {integrity: sha512-Ku/hhYbVjOQnXDZFv2+RibmLFGwFdeeKHFcOTlrt7xplBnya5OGn/hIRDsqDiSUcfORsDC7MPxwork8jBwsIWA==} engines: {node: '>=0.10.0'} + recharts@3.8.1: + resolution: {integrity: sha512-mwzmO1s9sFL0TduUpwndxCUNoXsBw3u3E/0+A+cLcrSfQitSG62L32N69GhqUrrT5qKcAE3pCGVINC6pqkBBQg==} + engines: {node: '>=18'} + peerDependencies: + react: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 + react-dom: ^16.0.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 + react-is: ^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0 + + redux-thunk@3.1.0: + resolution: {integrity: sha512-NW2r5T6ksUKXCabzhL9z+h206HQw/NJkcLm1GPImRQ8IzfXwRGqjVhKJGauHirT0DAuyy6hjdnMZaRoAcy0Klw==} + peerDependencies: + redux: ^5.0.0 + + redux@5.0.1: + resolution: {integrity: sha512-M9/ELqF6fy8FwmkpnF0S3YKOqMyoWJ4+CS5Efg2ct3oY9daQvd/Pc71FpGZsVsbl3Cpb+IIcjBDUnnyBdQbq4w==} + reflect.getprototypeof@1.0.10: resolution: {integrity: sha512-00o4I+DVrefhv+nX0ulyi3biSHCPDe+yLv5o/p6d/UVlirijB8E16FtfwSAi4g3tcqrQ4lRAqQSoFEZJehYEcw==} engines: {node: '>= 0.4'} @@ -2693,6 +2868,9 @@ packages: remark-stringify@11.0.0: resolution: {integrity: sha512-1OSmLd3awB/t8qdoEOMazZkNsfVTeY4fTsgzcQFdXNq8ToTN4ZGwrMnlda4K6smTFKD+GRV6O48i6Z4iKgPPpw==} + reselect@5.1.1: + resolution: {integrity: sha512-K/BG6eIky/SBpzfHZv/dd+9JBFiS4SWV7FIujVyJRux6e45+73RaUHXLmIR1f7WOMaQ0U1km6qwklRQxpJJY0w==} + resolve-from@4.0.0: resolution: {integrity: sha512-pb/MYmXstAkysRFx8piNI1tGFNQIFA3vkE3Gq4EuA1dF6gHp/+vgZqsCGJapvy8N3Q+4o7FwvquPJcnZ7RYy4g==} engines: {node: '>=4'} @@ -2871,6 +3049,9 @@ packages: resolution: {integrity: sha512-g9ljZiwki/LfxmQADO3dEY1CbpmXT5Hm2fJ+QaGKwSXUylMybePR7/67YW7jOrrvjEgL1Fmz5kzyAjWVWLlucg==} engines: {node: '>=6'} + tiny-invariant@1.3.3: + resolution: {integrity: sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==} + tinyglobby@0.2.15: resolution: {integrity: sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==} engines: {node: '>=12.0.0'} @@ -3003,6 +3184,9 @@ packages: vfile@6.0.3: resolution: {integrity: sha512-KzIbH/9tXat2u30jf+smMwFCsno4wHVdNmzFyL+T/L3UGqqk6JKfVqOFOZEpZSHADH1k40ab6NUIXZq422ov3Q==} + victory-vendor@37.3.6: + resolution: {integrity: sha512-SbPDPdDBYp+5MJHhBCAyI7wKM3d5ivekigc2Dk2s7pgbZ9wIgIBYGVw4zGHBml/qTFbexrofXW6Gu4noGxrOwQ==} + which-boxed-primitive@1.1.1: resolution: {integrity: sha512-TbX3mj8n0odCBFVlY8AxkqcHASw3L60jIuF8jFP78az3C2YhmGvqbHBpAjTRH2/xqYunrJ9g1jSyjCjpoWzIAA==} engines: {node: '>= 0.4'} @@ -3971,8 +4155,22 @@ snapshots: '@radix-ui/rect@1.1.1': {} + '@reduxjs/toolkit@2.12.0(react-redux@9.3.0(@types/react@19.2.8)(react@19.2.3)(redux@5.0.1))(react@19.2.3)': + dependencies: + '@standard-schema/spec': 1.1.0 + '@standard-schema/utils': 0.3.0 + immer: 11.1.8 + redux: 5.0.1 + redux-thunk: 3.1.0(redux@5.0.1) + reselect: 5.1.1 + optionalDependencies: + react: 19.2.3 + react-redux: 9.3.0(@types/react@19.2.8)(react@19.2.3)(redux@5.0.1) + '@rtsao/scc@1.1.0': {} + '@standard-schema/spec@1.1.0': {} + '@standard-schema/utils@0.3.0': {} '@swc/helpers@0.5.15': @@ -4073,6 +4271,30 @@ snapshots: tslib: 2.8.1 optional: true + '@types/d3-array@3.2.2': {} + + '@types/d3-color@3.1.3': {} + + '@types/d3-ease@3.0.2': {} + + '@types/d3-interpolate@3.0.4': + dependencies: + '@types/d3-color': 3.1.3 + + '@types/d3-path@3.1.1': {} + + '@types/d3-scale@4.0.9': + dependencies: + '@types/d3-time': 3.0.4 + + '@types/d3-shape@3.1.8': + dependencies: + '@types/d3-path': 3.1.1 + + '@types/d3-time@3.0.4': {} + + '@types/d3-timer@3.0.2': {} + '@types/debug@4.1.12': dependencies: '@types/ms': 2.1.0 @@ -4113,6 +4335,8 @@ snapshots: '@types/unist@3.0.3': {} + '@types/use-sync-external-store@0.0.6': {} + '@typescript-eslint/eslint-plugin@8.49.0(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3)': dependencies: '@eslint-community/regexpp': 4.12.2 @@ -4476,6 +4700,44 @@ snapshots: csstype@3.2.3: {} + d3-array@3.2.4: + dependencies: + internmap: 2.0.3 + + d3-color@3.1.0: {} + + d3-ease@3.0.1: {} + + d3-format@3.1.2: {} + + d3-interpolate@3.0.1: + dependencies: + d3-color: 3.1.0 + + d3-path@3.1.0: {} + + d3-scale@4.0.2: + dependencies: + d3-array: 3.2.4 + d3-format: 3.1.2 + d3-interpolate: 3.0.1 + d3-time: 3.1.0 + d3-time-format: 4.1.0 + + d3-shape@3.2.0: + dependencies: + d3-path: 3.1.0 + + d3-time-format@4.1.0: + dependencies: + d3-time: 3.1.0 + + d3-time@3.1.0: + dependencies: + d3-array: 3.2.4 + + d3-timer@3.0.1: {} + damerau-levenshtein@1.0.8: {} data-view-buffer@1.0.2: @@ -4506,6 +4768,8 @@ snapshots: dependencies: ms: 2.1.3 + decimal.js-light@2.5.1: {} + decode-named-character-reference@1.2.0: dependencies: character-entities: 2.0.2 @@ -4656,6 +4920,8 @@ snapshots: is-date-object: 1.1.0 is-symbol: 1.1.1 + es-toolkit@1.47.0: {} + escalade@3.2.0: {} escape-string-regexp@4.0.0: {} @@ -4667,7 +4933,7 @@ snapshots: '@next/eslint-plugin-next': 16.1.1 eslint: 9.39.2(jiti@2.6.1) eslint-import-resolver-node: 0.3.9 - eslint-import-resolver-typescript: 3.10.1(eslint-plugin-import@2.32.0)(eslint@9.39.2(jiti@2.6.1)) + eslint-import-resolver-typescript: 3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.2(jiti@2.6.1)))(eslint@9.39.2(jiti@2.6.1)) eslint-plugin-import: 2.32.0(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-typescript@3.10.1)(eslint@9.39.2(jiti@2.6.1)) eslint-plugin-jsx-a11y: 6.10.2(eslint@9.39.2(jiti@2.6.1)) eslint-plugin-react: 7.37.5(eslint@9.39.2(jiti@2.6.1)) @@ -4690,7 +4956,7 @@ snapshots: transitivePeerDependencies: - supports-color - eslint-import-resolver-typescript@3.10.1(eslint-plugin-import@2.32.0)(eslint@9.39.2(jiti@2.6.1)): + eslint-import-resolver-typescript@3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.2(jiti@2.6.1)))(eslint@9.39.2(jiti@2.6.1)): dependencies: '@nolyfill/is-core-module': 1.0.39 debug: 4.4.3 @@ -4705,14 +4971,14 @@ snapshots: transitivePeerDependencies: - supports-color - eslint-module-utils@2.12.1(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.10.1)(eslint@9.39.2(jiti@2.6.1)): + eslint-module-utils@2.12.1(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.2(jiti@2.6.1)))(eslint@9.39.2(jiti@2.6.1)))(eslint@9.39.2(jiti@2.6.1)): dependencies: debug: 3.2.7 optionalDependencies: '@typescript-eslint/parser': 8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3) eslint: 9.39.2(jiti@2.6.1) eslint-import-resolver-node: 0.3.9 - eslint-import-resolver-typescript: 3.10.1(eslint-plugin-import@2.32.0)(eslint@9.39.2(jiti@2.6.1)) + eslint-import-resolver-typescript: 3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.2(jiti@2.6.1)))(eslint@9.39.2(jiti@2.6.1)) transitivePeerDependencies: - supports-color @@ -4727,7 +4993,7 @@ snapshots: doctrine: 2.1.0 eslint: 9.39.2(jiti@2.6.1) eslint-import-resolver-node: 0.3.9 - eslint-module-utils: 2.12.1(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.10.1)(eslint@9.39.2(jiti@2.6.1)) + eslint-module-utils: 2.12.1(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint-import-resolver-node@0.3.9)(eslint-import-resolver-typescript@3.10.1(eslint-plugin-import@2.32.0(@typescript-eslint/parser@8.49.0(eslint@9.39.2(jiti@2.6.1))(typescript@5.9.3))(eslint@9.39.2(jiti@2.6.1)))(eslint@9.39.2(jiti@2.6.1)))(eslint@9.39.2(jiti@2.6.1)) hasown: 2.0.2 is-core-module: 2.16.1 is-glob: 4.0.3 @@ -4867,6 +5133,8 @@ snapshots: esutils@2.0.3: {} + eventemitter3@5.0.4: {} + extend@3.0.2: {} fast-deep-equal@3.1.3: {} @@ -5051,6 +5319,10 @@ snapshots: ignore@7.0.5: {} + immer@10.2.0: {} + + immer@11.1.8: {} + import-fresh@3.3.1: dependencies: parent-module: 1.0.1 @@ -5066,6 +5338,8 @@ snapshots: hasown: 2.0.2 side-channel: 1.1.0 + internmap@2.0.3: {} + is-alphabetical@2.0.1: {} is-alphanumerical@2.0.1: @@ -5886,6 +6160,15 @@ snapshots: transitivePeerDependencies: - supports-color + react-redux@9.3.0(@types/react@19.2.8)(react@19.2.3)(redux@5.0.1): + dependencies: + '@types/use-sync-external-store': 0.0.6 + react: 19.2.3 + use-sync-external-store: 1.6.0(react@19.2.3) + optionalDependencies: + '@types/react': 19.2.8 + redux: 5.0.1 + react-remove-scroll-bar@2.3.8(@types/react@19.2.8)(react@19.2.3): dependencies: react: 19.2.3 @@ -5915,6 +6198,32 @@ snapshots: react@19.2.3: {} + recharts@3.8.1(@types/react@19.2.8)(react-dom@19.2.3(react@19.2.3))(react-is@16.13.1)(react@19.2.3)(redux@5.0.1): + dependencies: + '@reduxjs/toolkit': 2.12.0(react-redux@9.3.0(@types/react@19.2.8)(react@19.2.3)(redux@5.0.1))(react@19.2.3) + clsx: 2.1.1 + decimal.js-light: 2.5.1 + es-toolkit: 1.47.0 + eventemitter3: 5.0.4 + immer: 10.2.0 + react: 19.2.3 + react-dom: 19.2.3(react@19.2.3) + react-is: 16.13.1 + react-redux: 9.3.0(@types/react@19.2.8)(react@19.2.3)(redux@5.0.1) + reselect: 5.1.1 + tiny-invariant: 1.3.3 + use-sync-external-store: 1.6.0(react@19.2.3) + victory-vendor: 37.3.6 + transitivePeerDependencies: + - '@types/react' + - redux + + redux-thunk@3.1.0(redux@5.0.1): + dependencies: + redux: 5.0.1 + + redux@5.0.1: {} + reflect.getprototypeof@1.0.10: dependencies: call-bind: 1.0.8 @@ -5969,6 +6278,8 @@ snapshots: mdast-util-to-markdown: 2.1.2 unified: 11.0.5 + reselect@5.1.1: {} + resolve-from@4.0.0: {} resolve-pkg-maps@1.0.0: {} @@ -6206,6 +6517,8 @@ snapshots: tapable@2.3.0: {} + tiny-invariant@1.3.3: {} + tinyglobby@0.2.15: dependencies: fdir: 6.5.0(picomatch@4.0.3) @@ -6391,6 +6704,23 @@ snapshots: '@types/unist': 3.0.3 vfile-message: 4.0.3 + victory-vendor@37.3.6: + dependencies: + '@types/d3-array': 3.2.2 + '@types/d3-ease': 3.0.2 + '@types/d3-interpolate': 3.0.4 + '@types/d3-scale': 4.0.9 + '@types/d3-shape': 3.1.8 + '@types/d3-time': 3.0.4 + '@types/d3-timer': 3.0.2 + d3-array: 3.2.4 + d3-ease: 3.0.1 + d3-interpolate: 3.0.1 + d3-scale: 4.0.2 + d3-shape: 3.2.0 + d3-time: 3.1.0 + d3-timer: 3.0.1 + which-boxed-primitive@1.1.1: dependencies: is-bigint: 1.1.0 @@ -6448,9 +6778,10 @@ snapshots: zod@4.3.5: {} - zustand@5.0.10(@types/react@19.2.8)(react@19.2.3)(use-sync-external-store@1.6.0(react@19.2.3)): + zustand@5.0.10(@types/react@19.2.8)(immer@11.1.8)(react@19.2.3)(use-sync-external-store@1.6.0(react@19.2.3)): optionalDependencies: '@types/react': 19.2.8 + immer: 11.1.8 react: 19.2.3 use-sync-external-store: 1.6.0(react@19.2.3) diff --git a/panel/src/app/(dashboard)/agents/page.tsx b/panel/src/app/(dashboard)/agents/page.tsx index d8a72d53..9aff9dc9 100644 --- a/panel/src/app/(dashboard)/agents/page.tsx +++ b/panel/src/app/(dashboard)/agents/page.tsx @@ -6,7 +6,8 @@ import { useWaitingAgents, useAgentDefinitions, } from "@/hooks/use-agents"; -import { AgentStatusResponse } from "@/types"; +import { useAgentUsage } from "@/hooks/use-usage"; +import { AgentStatusResponse, AgentUsageRow } from "@/types"; import { Button } from "@/components/ui/button"; import { RefreshCw } from "lucide-react"; import { OfflineState } from "@/components/ui/offline-state"; @@ -27,6 +28,7 @@ export default function AgentsPage() { const { data: agents = [], isLoading: agentsLoading } = useAgentDefinitions(); const { data: status, isLoading, error, refetch } = useOrchestratorStatus(); const { data: waitingAgents } = useWaitingAgents(); + const { data: usageRows } = useAgentUsage(); // Check if it's a connection error (backend not running) const isOffline = error && ( @@ -46,6 +48,15 @@ export default function AgentsPage() { return result; }, [status]); + // Convert usage rows to a record keyed by agent_slug + const agentUsageMap = useMemo(() => { + const result: Record = {}; + for (const row of usageRows ?? []) { + result[row.agent_slug] = row; + } + return result; + }, [usageRows]); + return (
{/* Header */} @@ -83,6 +94,7 @@ export default function AgentsPage() { title="Board" agents={getBoardAgents(agents)} agentStatuses={agentStatuses} + agentUsage={agentUsageMap} isLoading={(isLoading || agentsLoading) && !isOffline} columns={4} /> @@ -91,6 +103,7 @@ export default function AgentsPage() { title="Main PM" agents={getMainPm(agents)} agentStatuses={agentStatuses} + agentUsage={agentUsageMap} isLoading={(isLoading || agentsLoading) && !isOffline} columns={4} /> @@ -99,6 +112,7 @@ export default function AgentsPage() { title="Backend Cell" agents={getBackendAgents(agents)} agentStatuses={agentStatuses} + agentUsage={agentUsageMap} isLoading={(isLoading || agentsLoading) && !isOffline} columns={5} /> @@ -107,6 +121,7 @@ export default function AgentsPage() { title="Frontend Cell" agents={getFrontendAgents(agents)} agentStatuses={agentStatuses} + agentUsage={agentUsageMap} isLoading={(isLoading || agentsLoading) && !isOffline} columns={5} /> @@ -115,6 +130,7 @@ export default function AgentsPage() { title="UX/UI Cell" agents={getUxAgents(agents)} agentStatuses={agentStatuses} + agentUsage={agentUsageMap} isLoading={(isLoading || agentsLoading) && !isOffline} columns={4} /> diff --git a/panel/src/app/(dashboard)/metrics/page.tsx b/panel/src/app/(dashboard)/metrics/page.tsx index cf42c7a4..6f4be174 100644 --- a/panel/src/app/(dashboard)/metrics/page.tsx +++ b/panel/src/app/(dashboard)/metrics/page.tsx @@ -2,14 +2,33 @@ import { useOrchestratorStatus } from "@/hooks/use-agents"; import { useTasks } from "@/hooks/use-tasks"; +import { + useUsageSummary, + useUsageTimeSeries, + useAgentUsage, + useTeamUsage, + useModelUsage, + useUsageProjection, + useCacheEfficiency, + useUsageSessions, +} from "@/hooks/use-usage"; import { TaskStatus, Team } from "@/types"; import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; import { Button } from "@/components/ui/button"; import { Progress } from "@/components/ui/progress"; import { OfflineState } from "@/components/ui/offline-state"; +import { Skeleton } from "@/components/ui/skeleton"; +import { + UsageTimeSeriesChart, + ModelUsageDonut, + AgentUsageChart, + TeamUsageChart, + SessionsTable, +} from "@/components/metrics"; import { Activity, TrendingUp, + TrendingDown, Clock, AlertTriangle, Users, @@ -18,6 +37,8 @@ import { RefreshCw, Zap, Timer, + Coins, + Sparkles, } from "lucide-react"; interface MetricCardProps { @@ -295,8 +316,218 @@ export default function MetricsPage() { ))}
+ + {/* ─── Token Usage & Costs ─────────────────────────────────── */} + )} ); } + +// ============================================================================= +// TOKEN USAGE & COSTS SECTION +// ============================================================================= + +function TokenUsageCostsSection() { + const { data: summary, isLoading: loadingSnap } = useUsageSummary("24h"); + const { data: timeSeries, isLoading: loadingTS } = useUsageTimeSeries("24h"); + const { data: agentUsage, isLoading: loadingAgents } = useAgentUsage("24h"); + const { data: teamUsage, isLoading: loadingTeams } = useTeamUsage("24h"); + const { data: sessions, isLoading: loadingSessions } = useUsageSessions(100); + const { data: modelUsage, isLoading: loadingModels } = useModelUsage("24h"); + const { data: projection, isLoading: loadingProj } = useUsageProjection(); + const { data: cacheStats, isLoading: loadingCache } = useCacheEfficiency("24h"); + + const trendUp = (summary?.trend_pct ?? 0) >= 0; + + return ( +
+

Token Usage & Costs

+ + {/* Row 1 — Summary cards */} +
+ } + isLoading={loadingSnap} + /> + } + isLoading={loadingSnap} + /> + } + isLoading={loadingSnap} + /> + + ) : ( + + ) + } + isLoading={loadingSnap} + /> + } + isLoading={loadingSnap} + /> + } + isLoading={loadingCache} + /> +
+ + {/* Row 2 — Time series + model donut */} +
+
+ +
+ +
+ + {/* Row 3 — Agent bar + team bar */} +
+ + +
+ + {/* Row 4 — Projection + cache efficiency */} +
+ + +
+ + {/* Row 5 — Sessions table (mock-mode only; empty in production) */} + +
+ ); +} + +// ─── Helper sub-components ──────────────────────────────────────────────────── + +function fmtTokens(n: number): string { + if (n >= 1_000_000) return (n / 1_000_000).toFixed(2) + "M"; + if (n >= 1_000) return (n / 1_000).toFixed(1) + "K"; + return String(n); +} + +interface SummaryCardProps { + title: string; + value: string | undefined; + icon: React.ReactNode; + trend?: { dir: "up" | "down"; label: string }; + isLoading: boolean; +} + +function SummaryCard({ title, value, icon, trend, isLoading }: SummaryCardProps) { + return ( + + + {title} + {icon} + + + {isLoading ? ( + + ) : ( + <> +
{value ?? "—"}
+ {trend && ( +

+ {trend.label} +

+ )} + + )} + + + ); +} + +import type { UsageProjection as UP, CacheEfficiencyResponse as CER } from "@/types"; + +interface ProjectionCardProps { + projection: UP | undefined; + isLoading: boolean; +} + +function ProjectionCard({ projection, isLoading }: ProjectionCardProps) { + return ( + + + + + Monthly Projection + + + + {isLoading ? ( + + ) : ( +
+
+ {projection != null ? "$" + projection.projected_monthly_cost_usd.toFixed(2) : "—"} +
+

+ Based on {projection?.basis_days ?? 7}-day rolling average ($ + {projection?.avg_daily_cost_usd.toFixed(4) ?? "—"}/day) +

+
+ )} + + + ); +} + +interface CacheEfficiencyCardProps { + cacheStats: CER | undefined; + isLoading: boolean; +} + +function CacheEfficiencyCard({ cacheStats, isLoading }: CacheEfficiencyCardProps) { + const pct = cacheStats ? cacheStats.cache_hit_rate * 100 : 0; + + return ( + + + + + Cache Efficiency + + + + {isLoading ? ( + + ) : ( +
+
{pct.toFixed(1)}%
+

+ {cacheStats ? fmtTokens(cacheStats.tokens_cache_read) : "—"} cache reads · + saved ${cacheStats?.cost_saved_by_cache_usd.toFixed(4) ?? "—"} +

+ +
+ )} + + + ); +} diff --git a/panel/src/components/agents/agent-card.tsx b/panel/src/components/agents/agent-card.tsx index 200b53c8..2932a0af 100644 --- a/panel/src/components/agents/agent-card.tsx +++ b/panel/src/components/agents/agent-card.tsx @@ -17,13 +17,15 @@ import { MoreHorizontal, Activity, Square } from "lucide-react"; import { toast } from "sonner"; import { AgentStateBadge } from "./agent-state-badge"; import { SpawnAgentDialog } from "./spawn-agent-dialog"; +import type { AgentUsageRow } from "@/types"; interface AgentCardProps { agent: AgentDefinition; agentStatus: AgentStatusResponse | null; + usageRow?: AgentUsageRow | null; } -export function AgentCard({ agent, agentStatus }: AgentCardProps) { +export function AgentCard({ agent, agentStatus, usageRow }: AgentCardProps) { const stopAgent = useStopAgent(); const state = agentStatus?.state || "stopped"; const isActive = ["running", "ready", "starting", "waiting_long"].includes(state); @@ -100,6 +102,30 @@ export function AgentCard({ agent, agentStatus }: AgentCardProps) { Errors: {agentStatus.error_count}

)} + {usageRow && ( +
+
+ + {usageRow.total_tokens >= 1_000 + ? (usageRow.total_tokens / 1_000).toFixed(1) + "K" + : String(usageRow.total_tokens)}{" "} + tokens + + + ${usageRow.cost_usd.toFixed(4)} + +
+
+
+
+
+ )} ); diff --git a/panel/src/components/agents/agent-grid.tsx b/panel/src/components/agents/agent-grid.tsx index f3a0e2c3..a1b6074e 100644 --- a/panel/src/components/agents/agent-grid.tsx +++ b/panel/src/components/agents/agent-grid.tsx @@ -1,4 +1,4 @@ -import { AgentStatusResponse } from "@/types"; +import { AgentStatusResponse, AgentUsageRow } from "@/types"; import { AgentDefinition } from "@/lib/agent-definitions"; import { Card, CardHeader } from "@/components/ui/card"; import { Skeleton } from "@/components/ui/skeleton"; @@ -8,6 +8,7 @@ interface AgentGridProps { title: string; agents: AgentDefinition[]; agentStatuses: Record; + agentUsage?: Record; isLoading: boolean; columns?: number; } @@ -16,8 +17,9 @@ export function AgentGrid({ title, agents, agentStatuses, + agentUsage, isLoading, - columns = 4 + columns = 4, }: AgentGridProps) { const gridCols = { 3: "md:grid-cols-3", @@ -44,6 +46,7 @@ export function AgentGrid({ key={agent.id} agent={agent} agentStatus={agentStatuses[agent.id] || null} + usageRow={agentUsage?.[agent.id] ?? null} /> )) )} diff --git a/panel/src/components/dashboard/command-center.tsx b/panel/src/components/dashboard/command-center.tsx index 4c56e94f..c60629e6 100644 --- a/panel/src/components/dashboard/command-center.tsx +++ b/panel/src/components/dashboard/command-center.tsx @@ -11,6 +11,7 @@ import { QuickActionsBar } from "./quick-actions-bar"; import { CeoApprovalQueue } from "./ceo-approval-queue"; import type { Activity } from "./activity-item"; import { Button } from "@/components/ui/button"; +import { UsageOverviewPanel } from "./usage-overview-panel"; import { RefreshCw, Settings } from "lucide-react"; import Link from "next/link"; @@ -61,13 +62,14 @@ export function CommandCenter() { - {/* Metrics and Alerts Row */} -
+ {/* Metrics, Alerts, and Usage Row */} +
+
{/* Blockers and Activity Row */} diff --git a/panel/src/components/dashboard/index.ts b/panel/src/components/dashboard/index.ts index a333cd3e..2352d619 100644 --- a/panel/src/components/dashboard/index.ts +++ b/panel/src/components/dashboard/index.ts @@ -9,3 +9,4 @@ export { ActivityItem } from "./activity-item"; export { QuickActionsBar } from "./quick-actions-bar"; export { HealthIndicator } from "./health-indicator"; export { CeoApprovalQueue } from "./ceo-approval-queue"; +export { UsageOverviewPanel } from "./usage-overview-panel"; diff --git a/panel/src/components/dashboard/usage-overview-panel.tsx b/panel/src/components/dashboard/usage-overview-panel.tsx new file mode 100644 index 00000000..876180a2 --- /dev/null +++ b/panel/src/components/dashboard/usage-overview-panel.tsx @@ -0,0 +1,105 @@ +"use client"; + +import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; +import { Skeleton } from "@/components/ui/skeleton"; +import { useUsageSummary } from "@/hooks/use-usage"; +import { Coins, TrendingUp, TrendingDown, Zap, Activity } from "lucide-react"; + +function fmt(n: number, decimals = 0): string { + if (n >= 1_000_000) return (n / 1_000_000).toFixed(1) + "M"; + if (n >= 1_000) return (n / 1_000).toFixed(1) + "K"; + return n.toFixed(decimals); +} + +function fmtCost(n: number): string { + return "$" + n.toFixed(2); +} + +interface MetricRowProps { + icon: React.ReactNode; + label: string; + value: string; + sub?: React.ReactNode; +} + +function MetricRow({ icon, label, value, sub }: MetricRowProps) { + return ( +
+
+ {icon} + {label} +
+
+ {value} + {sub} +
+
+ ); +} + +export function UsageOverviewPanel() { + const { data: summary, isLoading } = useUsageSummary("24h"); + + const trendUp = (summary?.trend_pct ?? 0) >= 0; + + return ( + + + + + Token Usage & Cost + + + + {isLoading ? ( +
+ {Array.from({ length: 5 }).map((_, i) => ( + + ))} +
+ ) : ( +
+ } + label="Tokens (input)" + value={summary ? fmt(summary.tokens_input) : "—"} + /> + } + label="Tokens (output)" + value={summary ? fmt(summary.tokens_output) : "—"} + /> + } + label="Total cost" + value={summary ? fmtCost(summary.total_cost_usd) : "—"} + /> + + ) : ( + + ) + } + label="Trend vs prior period" + value={summary ? (trendUp ? "+" : "") + summary.trend_pct.toFixed(1) + "%" : "—"} + sub={ + summary ? ( + + {trendUp ? "▲" : "▼"} + + ) : undefined + } + /> + } + label="Period" + value={summary?.period ?? "—"} + /> +
+ )} + + + ); +} diff --git a/panel/src/components/metrics/agent-usage-chart.tsx b/panel/src/components/metrics/agent-usage-chart.tsx new file mode 100644 index 00000000..bafdfa12 --- /dev/null +++ b/panel/src/components/metrics/agent-usage-chart.tsx @@ -0,0 +1,79 @@ +"use client"; + +import { + BarChart, + Bar, + XAxis, + YAxis, + CartesianGrid, + Tooltip, + ResponsiveContainer, +} from "recharts"; +import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; +import { Skeleton } from "@/components/ui/skeleton"; +import type { AgentUsageRow } from "@/types"; + +interface AgentUsageChartProps { + data: AgentUsageRow[] | undefined; + isLoading: boolean; +} + +function fmtK(n: number): string { + if (n >= 1_000) return (n / 1_000).toFixed(0) + "k"; + return String(n); +} + +export function AgentUsageChart({ data, isLoading }: AgentUsageChartProps) { + const chartData = [...(data ?? [])] + .sort((a, b) => b.total_tokens - a.total_tokens) + .slice(0, 10) + .map((row) => ({ + name: row.agent_slug, + Tokens: row.total_tokens, + })); + + return ( + + + Agent Tokens Today + + + {isLoading ? ( + + ) : ( + + + + + + [ + fmtK(typeof value === "number" ? value : 0), + "Tokens", + ]} + contentStyle={{ fontSize: 12 }} + /> + + + + )} + + + ); +} diff --git a/panel/src/components/metrics/index.ts b/panel/src/components/metrics/index.ts new file mode 100644 index 00000000..92bece6d --- /dev/null +++ b/panel/src/components/metrics/index.ts @@ -0,0 +1,5 @@ +export { UsageTimeSeriesChart } from "./usage-time-series-chart"; +export { ModelUsageDonut } from "./model-usage-donut"; +export { AgentUsageChart } from "./agent-usage-chart"; +export { TeamUsageChart } from "./team-usage-chart"; +export { SessionsTable } from "./sessions-table"; diff --git a/panel/src/components/metrics/model-usage-donut.tsx b/panel/src/components/metrics/model-usage-donut.tsx new file mode 100644 index 00000000..a333eae7 --- /dev/null +++ b/panel/src/components/metrics/model-usage-donut.tsx @@ -0,0 +1,78 @@ +"use client"; + +import { + PieChart, + Pie, + Cell, + Tooltip, + ResponsiveContainer, + Legend, +} from "recharts"; +import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; +import { Skeleton } from "@/components/ui/skeleton"; +import type { ModelUsageSlice } from "@/types"; + +const CHART_COLORS = [ + "var(--chart-1)", + "var(--chart-2)", + "var(--chart-3)", + "var(--chart-4)", + "var(--chart-5)", +]; + +interface ModelUsageDonutProps { + data: ModelUsageSlice[] | undefined; + isLoading: boolean; +} + +export function ModelUsageDonut({ data, isLoading }: ModelUsageDonutProps) { + const chartData = (data ?? []).map((s) => ({ + name: s.model, + value: s.total_tokens, + cost: s.cost_usd, + pct: s.pct_of_total, + })); + + return ( + + + By Model + + + {isLoading ? ( + + ) : ( + + + + {chartData.map((_, idx) => ( + + ))} + + [ + (typeof value === "number" ? value : 0).toLocaleString() + + " tokens", + name, + ]} + contentStyle={{ fontSize: 12 }} + /> + + + + )} + + + ); +} diff --git a/panel/src/components/metrics/sessions-table.tsx b/panel/src/components/metrics/sessions-table.tsx new file mode 100644 index 00000000..81068fdb --- /dev/null +++ b/panel/src/components/metrics/sessions-table.tsx @@ -0,0 +1,191 @@ +"use client"; + +import { useState, useMemo } from "react"; +import { + Table, + TableBody, + TableCell, + TableHead, + TableHeader, + TableRow, +} from "@/components/ui/table"; +import { Button } from "@/components/ui/button"; +import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; +import { Skeleton } from "@/components/ui/skeleton"; +import { ChevronUp, ChevronDown } from "lucide-react"; +import type { UsageSession } from "@/types"; + +const PAGE_SIZE = 10; + +type SortKey = keyof Pick< + UsageSession, + | "agent_slug" + | "started_at" + | "total_tokens" + | "tokens_input" + | "tokens_output" + | "tokens_cache" + | "cost" + | "model" +>; + +type SortDir = "asc" | "desc"; + +interface Column { + key: SortKey; + label: string; +} + +const COLUMNS: Column[] = [ + { key: "agent_slug", label: "Agent" }, + { key: "model", label: "Model" }, + { key: "started_at", label: "Started" }, + { key: "total_tokens", label: "Total" }, + { key: "tokens_input", label: "Input" }, + { key: "tokens_output", label: "Output" }, + { key: "tokens_cache", label: "Cache" }, + { key: "cost", label: "Cost" }, +]; + +function formatTime(ts: string): string { + return new Date(ts).toLocaleTimeString([], { hour: "2-digit", minute: "2-digit" }); +} + +function fmtK(n: number): string { + if (n >= 1_000) return (n / 1_000).toFixed(1) + "k"; + return String(n); +} + +interface SessionsTableProps { + data: UsageSession[] | undefined; + isLoading: boolean; +} + +export function SessionsTable({ data, isLoading }: SessionsTableProps) { + const [sortKey, setSortKey] = useState("started_at"); + const [sortDir, setSortDir] = useState("desc"); + const [page, setPage] = useState(0); + + const sorted = useMemo(() => { + const rows = [...(data ?? [])]; + rows.sort((a, b) => { + const av = a[sortKey]; + const bv = b[sortKey]; + const cmp = + typeof av === "number" && typeof bv === "number" + ? av - bv + : String(av).localeCompare(String(bv)); + return sortDir === "asc" ? cmp : -cmp; + }); + return rows; + }, [data, sortKey, sortDir]); + + const totalPages = Math.max(1, Math.ceil(sorted.length / PAGE_SIZE)); + const visible = sorted.slice(page * PAGE_SIZE, (page + 1) * PAGE_SIZE); + + function toggleSort(key: SortKey) { + if (sortKey === key) { + setSortDir((d) => (d === "asc" ? "desc" : "asc")); + } else { + setSortKey(key); + setSortDir("desc"); + } + setPage(0); + } + + function SortIcon({ col }: { col: SortKey }) { + if (sortKey !== col) return ; + return sortDir === "asc" ? ( + + ) : ( + + ); + } + + return ( + + + Recent Sessions + + + {isLoading ? ( +
+ {Array.from({ length: PAGE_SIZE }).map((_, i) => ( + + ))} +
+ ) : ( + <> +
+ + + + {COLUMNS.map((col) => ( + toggleSort(col.key)} + > + {col.label} + + + ))} + + + + {visible.length === 0 ? ( + + + No sessions recorded yet + + + ) : ( + visible.map((s) => ( + + {s.agent_slug} + {s.model} + {formatTime(s.started_at)} + {fmtK(s.total_tokens)} + {fmtK(s.tokens_input)} + {fmtK(s.tokens_output)} + {fmtK(s.tokens_cache)} + ${s.cost.toFixed(4)} + + )) + )} + +
+
+ + {/* Pagination */} +
+ + {sorted.length === 0 + ? "No sessions" + : `${page * PAGE_SIZE + 1}–${Math.min((page + 1) * PAGE_SIZE, sorted.length)} of ${sorted.length}`} + +
+ + +
+
+ + )} + + + ); +} diff --git a/panel/src/components/metrics/team-usage-chart.tsx b/panel/src/components/metrics/team-usage-chart.tsx new file mode 100644 index 00000000..e9a2d3d3 --- /dev/null +++ b/panel/src/components/metrics/team-usage-chart.tsx @@ -0,0 +1,76 @@ +"use client"; + +import { + BarChart, + Bar, + XAxis, + YAxis, + CartesianGrid, + Tooltip, + ResponsiveContainer, +} from "recharts"; +import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; +import { Skeleton } from "@/components/ui/skeleton"; +import type { TeamUsageRow } from "@/types"; + +interface TeamUsageChartProps { + data: TeamUsageRow[] | undefined; + isLoading: boolean; +} + +function fmtK(n: number): string { + if (n >= 1_000) return (n / 1_000).toFixed(0) + "k"; + return String(n); +} + +export function TeamUsageChart({ data, isLoading }: TeamUsageChartProps) { + const chartData = [...(data ?? [])] + .sort((a, b) => b.total_tokens - a.total_tokens) + .map((row) => ({ + name: row.team.replace(/_/g, " "), + Tokens: row.total_tokens, + })); + + return ( + + + Team Tokens + + + {isLoading ? ( + + ) : ( + + + + + + [ + fmtK(typeof value === "number" ? value : 0), + "Tokens", + ]} + contentStyle={{ fontSize: 12 }} + /> + + + + )} + + + ); +} diff --git a/panel/src/components/metrics/usage-time-series-chart.tsx b/panel/src/components/metrics/usage-time-series-chart.tsx new file mode 100644 index 00000000..fea4f995 --- /dev/null +++ b/panel/src/components/metrics/usage-time-series-chart.tsx @@ -0,0 +1,112 @@ +"use client"; + +import { + AreaChart, + Area, + XAxis, + YAxis, + CartesianGrid, + Tooltip, + Legend, + ResponsiveContainer, +} from "recharts"; +import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; +import { Skeleton } from "@/components/ui/skeleton"; +import type { UsageTimePoint } from "@/types"; + +interface UsageTimeSeriesChartProps { + data: UsageTimePoint[] | undefined; + isLoading: boolean; +} + +function formatBucket(bucket: string): string { + const d = new Date(bucket); + // If the bucket has a non-zero time component it is an hourly bucket → show HH:00. + // Otherwise it is a daily bucket → show MM/DD. + const isHourly = d.getMinutes() === 0 && (d.getHours() !== 0 || bucket.includes("T")); + if (isHourly && d.getSeconds() === 0 && !bucket.endsWith("T00:00:00.000Z")) { + return d.getHours().toString().padStart(2, "0") + ":00"; + } + return (d.getMonth() + 1) + "/" + d.getDate(); +} + +function fmtK(n: number): string { + if (n >= 1_000) return (n / 1_000).toFixed(0) + "k"; + return String(n); +} + +export function UsageTimeSeriesChart({ data, isLoading }: UsageTimeSeriesChartProps) { + const chartData = (data ?? []).map((p) => ({ + hour: formatBucket(p.bucket), + Input: p.tokens_input, + Output: p.tokens_output, + })); + + return ( + + + Token Usage Over Time + + + {isLoading ? ( + + ) : ( + + + + + + + + + + + + + + + + [ + fmtK(typeof value === "number" ? value : 0), + name, + ]} + contentStyle={{ fontSize: 12 }} + /> + + + + + + )} + + + ); +} diff --git a/panel/src/hooks/index.ts b/panel/src/hooks/index.ts index 73bb8b4a..57212f8b 100644 --- a/panel/src/hooks/index.ts +++ b/panel/src/hooks/index.ts @@ -33,3 +33,4 @@ export * from "./use-websocket"; export * from "./use-journals"; export * from "./use-projects"; export * from "./use-work-sessions"; +export * from "./use-usage"; diff --git a/panel/src/hooks/use-usage.ts b/panel/src/hooks/use-usage.ts new file mode 100644 index 00000000..c0e85fac --- /dev/null +++ b/panel/src/hooks/use-usage.ts @@ -0,0 +1,112 @@ +"use client"; + +import { useQuery } from "@tanstack/react-query"; +import { usageApi } from "@/lib/api/usage"; +import type { UsagePeriod } from "@/lib/api/usage"; +import type { + UsageSummary, + AgentUsageRow, + TeamUsageRow, + ModelUsageSlice, + UsageTimePoint, + UsageProjection, + CacheEfficiencyResponse, + UsageSession, +} from "@/types"; + +// ============================================================================= +// QUERY KEYS +// ============================================================================= + +export const usageKeys = { + all: ["usage"] as const, + summary: (period: UsagePeriod) => [...usageKeys.all, "summary", period] as const, + timeSeries: (period: UsagePeriod) => [...usageKeys.all, "time-series", period] as const, + agentUsage: (period: UsagePeriod) => [...usageKeys.all, "by-agent", period] as const, + teamUsage: (period: UsagePeriod) => [...usageKeys.all, "by-team", period] as const, + modelUsage: (period: UsagePeriod) => [...usageKeys.all, "by-model", period] as const, + projection: () => [...usageKeys.all, "projection"] as const, + cacheEfficiency: (period: UsagePeriod) => [...usageKeys.all, "cache-efficiency", period] as const, + sessions: (limit: number) => [...usageKeys.all, "sessions", limit] as const, +}; + +// ============================================================================= +// HOOKS +// ============================================================================= + +/** Aggregated usage summary (tokens_input, tokens_output, total_cost_usd, …) */ +export function useUsageSummary(period: UsagePeriod = "24h") { + return useQuery({ + queryKey: usageKeys.summary(period), + queryFn: () => usageApi.getUsageSummary(period), + refetchInterval: 60_000, + }); +} + +/** Bucketed time-series data for the stacked area chart */ +export function useUsageTimeSeries(period: UsagePeriod = "24h") { + return useQuery({ + queryKey: usageKeys.timeSeries(period), + queryFn: () => usageApi.getUsageTimeSeries(period), + refetchInterval: 120_000, + }); +} + +/** Per-agent usage rows for bar chart and agent card mini-bars */ +export function useAgentUsage(period: UsagePeriod = "24h") { + return useQuery({ + queryKey: usageKeys.agentUsage(period), + queryFn: () => usageApi.getAgentUsage(period), + refetchInterval: 60_000, + }); +} + +/** Per-team usage rows from the dedicated by-team endpoint */ +export function useTeamUsage(period: UsagePeriod = "24h") { + return useQuery({ + queryKey: usageKeys.teamUsage(period), + queryFn: () => usageApi.getTeamUsage(period), + refetchInterval: 60_000, + }); +} + +/** Per-model slices for the donut chart */ +export function useModelUsage(period: UsagePeriod = "24h") { + return useQuery({ + queryKey: usageKeys.modelUsage(period), + queryFn: () => usageApi.getModelUsage(period), + refetchInterval: 120_000, + }); +} + +/** Monthly cost projection based on 7-day rolling average */ +export function useUsageProjection() { + return useQuery({ + queryKey: usageKeys.projection(), + queryFn: () => usageApi.getUsageProjection(), + refetchInterval: 300_000, + }); +} + +/** Cache efficiency stats */ +export function useCacheEfficiency(period: UsagePeriod = "24h") { + return useQuery({ + queryKey: usageKeys.cacheEfficiency(period), + queryFn: () => usageApi.getCacheEfficiency(period), + refetchInterval: 120_000, + }); +} + +/** + * Recent inference sessions — mock-mode only. + * + * Returns an empty array in production (no real backend endpoint for sessions). + * The SessionsTable will display "No sessions recorded yet" gracefully. + */ +export function useUsageSessions(limit: number = 100) { + return useQuery({ + queryKey: usageKeys.sessions(limit), + queryFn: () => usageApi.getUsageSessions(limit), + refetchInterval: 30_000, + }); +} diff --git a/panel/src/lib/api/index.ts b/panel/src/lib/api/index.ts index a1ce645e..6e110f5c 100644 --- a/panel/src/lib/api/index.ts +++ b/panel/src/lib/api/index.ts @@ -1,4 +1,5 @@ export { api, API_URL } from "./client"; +export { usageApi } from "./usage"; export { tasksApi } from "./tasks"; export { orchestratorApi } from "./orchestrator"; export { channelsApi } from "./channels"; diff --git a/panel/src/lib/api/usage.ts b/panel/src/lib/api/usage.ts new file mode 100644 index 00000000..6359d8f0 --- /dev/null +++ b/panel/src/lib/api/usage.ts @@ -0,0 +1,254 @@ +import api from "./client"; +import { isMockMode } from "@/lib/mock-data"; +import type { + UsageSummary, + AgentUsageRow, + TeamUsageRow, + ModelUsageSlice, + UsageTimePoint, + UsageProjection, + CacheEfficiencyResponse, + UsageSession, +} from "@/types"; + +export type UsagePeriod = "24h" | "7d" | "30d"; + +// ============================================================================= +// MOCK DATA — shapes must exactly match the real backend response schemas +// ============================================================================= + +function mockSummary(period: UsagePeriod = "24h"): UsageSummary { + const scale = period === "30d" ? 30 : period === "7d" ? 7 : 1; + const base = 124_800 * scale; + return { + tokens_input: Math.round(base * 0.55), + tokens_output: Math.round(base * 0.35), + total_tokens: base, + total_cost_usd: parseFloat((base * 0.000030).toFixed(6)), + trend_pct: 12.5, + period, + }; +} + +function mockTimeSeries(period: UsagePeriod = "24h"): UsageTimePoint[] { + const now = new Date(); + const points = period === "24h" ? 24 : period === "7d" ? 7 : 30; + const step = period === "24h" ? "hour" : "day"; + return Array.from({ length: points }, (_, i) => { + const ts = new Date(now); + if (step === "hour") { + ts.setHours(now.getHours() - (points - 1 - i), 0, 0, 0); + } else { + ts.setDate(now.getDate() - (points - 1 - i)); + ts.setHours(0, 0, 0, 0); + } + const base = 3_000 + Math.round(Math.random() * 4_000); + const tokens_input = Math.round(base * 0.55); + const tokens_output = Math.round(base * 0.35); + const total_tokens = base; + return { + bucket: ts.toISOString(), + tokens_input, + tokens_output, + total_tokens, + cost_usd: parseFloat((total_tokens * 0.000030).toFixed(6)), + }; + }); +} + +function mockAgentUsage(period: UsagePeriod = "24h"): AgentUsageRow[] { + const scale = period === "30d" ? 30 : period === "7d" ? 7 : 1; + const agents = [ + { agent_slug: "be-dev-1" }, + { agent_slug: "be-dev-2" }, + { agent_slug: "fe-dev-1" }, + { agent_slug: "fe-dev-2" }, + { agent_slug: "ux-dev-1" }, + { agent_slug: "be-qa" }, + { agent_slug: "fe-qa" }, + { agent_slug: "main-pm" }, + ]; + const grand = agents.length * 15_000 * scale; + return agents.map((a) => { + const ti = Math.round((5_000 + Math.random() * 20_000) * scale); + const to_ = Math.round(ti * 0.65); + const total = ti + to_; + return { + agent_slug: a.agent_slug, + tokens_input: ti, + tokens_output: to_, + total_tokens: total, + cost_usd: parseFloat((total * 0.000030).toFixed(6)), + pct_of_total: parseFloat(((total / grand) * 100).toFixed(2)), + }; + }); +} + +function mockTeamUsage(period: UsagePeriod = "24h"): TeamUsageRow[] { + const scale = period === "30d" ? 30 : period === "7d" ? 7 : 1; + const teams = ["backend", "frontend", "ux_ui", "main_pm"]; + const grand = teams.length * 50_000 * scale; + return teams.map((team) => { + const ti = Math.round((30_000 + Math.random() * 40_000) * scale); + const to_ = Math.round(ti * 0.65); + const total = ti + to_; + return { + team, + tokens_input: ti, + tokens_output: to_, + total_tokens: total, + cost_usd: parseFloat((total * 0.000030).toFixed(6)), + pct_of_total: parseFloat(((total / grand) * 100).toFixed(2)), + }; + }); +} + +function mockModelUsage(period: UsagePeriod = "24h"): ModelUsageSlice[] { + const scale = period === "30d" ? 30 : period === "7d" ? 7 : 1; + const models = [ + { model: "claude-opus-4", share: 0.548 }, + { model: "claude-sonnet-4", share: 0.346 }, + { model: "claude-haiku-4", share: 0.106 }, + ]; + const base = 124_800 * scale; + return models.map((m) => { + const ti = Math.round(base * m.share * 0.55); + const to_ = Math.round(base * m.share * 0.35); + const total = Math.round(base * m.share); + return { + model: m.model, + tokens_input: ti, + tokens_output: to_, + total_tokens: total, + cost_usd: parseFloat((total * 0.000030).toFixed(6)), + pct_of_total: parseFloat((m.share * 100).toFixed(1)), + }; + }); +} + +function mockProjection(): UsageProjection { + const total_cost_7d = parseFloat((124_800 * 7 * 0.000030).toFixed(6)); + return { + total_cost_7d, + avg_daily_cost_usd: parseFloat((total_cost_7d / 7).toFixed(6)), + projected_monthly_cost_usd: parseFloat((total_cost_7d / 7 * 30).toFixed(4)), + basis_days: 7, + }; +} + +function mockCacheEfficiency(period: UsagePeriod = "24h"): CacheEfficiencyResponse { + return { + cache_hit_rate: 0.3142, + tokens_cache_read: 39_168, + tokens_cache_write: 12_480, + tokens_input: 85_632, + cost_saved_by_cache_usd: parseFloat((39_168 * (3.00 - 0.30) / 1_000_000).toFixed(6)), + period, + }; +} + +function mockSessions(): UsageSession[] { + const models = ["claude-opus-4", "claude-sonnet-4", "claude-haiku-4"]; + const agentSlugs = ["be-dev-1", "be-dev-2", "fe-dev-1", "fe-qa", "main-pm"]; + return Array.from({ length: 35 }, (_, i) => { + const agent_slug = agentSlugs[i % agentSlugs.length]; + const model = models[i % models.length]; + const input = Math.round(2_000 + Math.random() * 8_000); + const output = Math.round(500 + Math.random() * 3_000); + const cache = Math.round(100 + Math.random() * 1_000); + const started = new Date(Date.now() - (i + 1) * 12 * 60_000); + const ended = i < 3 ? null : new Date(started.getTime() + Math.round(5 + Math.random() * 55) * 60_000); + return { + id: `session-mock-${i + 1}`, + agent_slug, + started_at: started.toISOString(), + ended_at: ended ? ended.toISOString() : null, + tokens_input: input, + tokens_output: output, + tokens_cache: cache, + total_tokens: input + output + cache, + cost: parseFloat(((input + output) * 0.00003 + cache * 0.000003).toFixed(4)), + model, + }; + }); +} + +// ============================================================================= +// API OBJECT +// ============================================================================= + +export const usageApi = { + /** Aggregated token usage summary — GET /usage/summary?period= */ + getUsageSummary: async (period: UsagePeriod = "24h"): Promise => { + if (isMockMode()) return mockSummary(period); + const { data } = await api.get("/usage/summary", { + params: { period }, + }); + return data; + }, + + /** Bucketed time-series — GET /usage/time-series?period= */ + getUsageTimeSeries: async (period: UsagePeriod = "24h"): Promise => { + if (isMockMode()) return mockTimeSeries(period); + const { data } = await api.get("/usage/time-series", { + params: { period }, + }); + return data; + }, + + /** Per-agent usage rows — GET /usage/by-agent?period= */ + getAgentUsage: async (period: UsagePeriod = "24h"): Promise => { + if (isMockMode()) return mockAgentUsage(period); + const { data } = await api.get("/usage/by-agent", { + params: { period }, + }); + return data; + }, + + /** Per-team usage rows — GET /usage/by-team?period= */ + getTeamUsage: async (period: UsagePeriod = "24h"): Promise => { + if (isMockMode()) return mockTeamUsage(period); + const { data } = await api.get("/usage/by-team", { + params: { period }, + }); + return data; + }, + + /** Per-model usage slices — GET /usage/by-model?period= */ + getModelUsage: async (period: UsagePeriod = "24h"): Promise => { + if (isMockMode()) return mockModelUsage(period); + const { data } = await api.get("/usage/by-model", { + params: { period }, + }); + return data; + }, + + /** Monthly cost projection — GET /usage/projection */ + getUsageProjection: async (): Promise => { + if (isMockMode()) return mockProjection(); + const { data } = await api.get("/usage/projection"); + return data; + }, + + /** Cache efficiency stats — GET /usage/cache-efficiency?period= */ + getCacheEfficiency: async (period: UsagePeriod = "24h"): Promise => { + if (isMockMode()) return mockCacheEfficiency(period); + const { data } = await api.get("/usage/cache-efficiency", { + params: { period }, + }); + return data; + }, + + /** + * Recent inference sessions — mock-mode only. + * + * The backend has no /usage/sessions endpoint. In production this + * returns an empty array so SessionsTable shows a graceful "no data" + * state instead of throwing a 404. + */ + // eslint-disable-next-line @typescript-eslint/no-unused-vars + getUsageSessions: async (_limit: number = 100): Promise => { + if (isMockMode()) return mockSessions(); + return []; + }, +}; diff --git a/panel/src/types/index.ts b/panel/src/types/index.ts index 3be98492..0e420a7e 100644 --- a/panel/src/types/index.ts +++ b/panel/src/types/index.ts @@ -1253,3 +1253,92 @@ export interface CEOApprovalRequest { export interface CEORejectRequest { notes: string; // Required for rejection } + +// ============================================================================= +// TOKEN USAGE TYPES (aligned to real backend: GET /api/usage/*) +// ============================================================================= + +/** Aggregated token and cost totals — GET /usage/summary?period=24h|7d|30d */ +export interface UsageSummary { + tokens_input: number; + tokens_output: number; + total_tokens: number; + total_cost_usd: number; + trend_pct: number; + period: string; +} + +/** Per-agent usage row — GET /usage/by-agent?period=24h|7d|30d */ +export interface AgentUsageRow { + agent_slug: string; + tokens_input: number; + tokens_output: number; + total_tokens: number; + cost_usd: number; + pct_of_total: number; +} + +/** Per-team usage row — GET /usage/by-team?period=24h|7d|30d */ +export interface TeamUsageRow { + team: string; + tokens_input: number; + tokens_output: number; + total_tokens: number; + cost_usd: number; + pct_of_total: number; +} + +/** Per-model usage slice — GET /usage/by-model?period=24h|7d|30d */ +export interface ModelUsageSlice { + model: string; + tokens_input: number; + tokens_output: number; + total_tokens: number; + cost_usd: number; + pct_of_total: number; +} + +/** One data point in a token-usage time series — GET /usage/time-series?period=24h|7d|30d + * + * - 24h → hourly buckets; 7d / 30d → daily buckets + * - bucket is an ISO datetime string (from PostgreSQL date_trunc) + */ +export interface UsageTimePoint { + bucket: string; + tokens_input: number; + tokens_output: number; + total_tokens: number; + cost_usd: number; +} + +/** Monthly cost projection — GET /usage/projection */ +export interface UsageProjection { + total_cost_7d: number; + avg_daily_cost_usd: number; + projected_monthly_cost_usd: number; + basis_days: number; +} + +/** Cache efficiency stats — GET /usage/cache-efficiency?period=24h|7d|30d */ +export interface CacheEfficiencyResponse { + cache_hit_rate: number; + tokens_cache_read: number; + tokens_cache_write: number; + tokens_input: number; + cost_saved_by_cache_usd: number; + period: string; +} + +/** Individual inference session for the sessions table (mock-mode only — no real backend endpoint) */ +export interface UsageSession { + id: string; + agent_slug: string; + started_at: string; + ended_at: string | null; + tokens_input: number; + tokens_output: number; + tokens_cache: number; + total_tokens: number; + cost: number; + model: string; +} diff --git a/roboco/agent_sdk/models.py b/roboco/agent_sdk/models.py index 160de1a3..c3fa87f3 100644 --- a/roboco/agent_sdk/models.py +++ b/roboco/agent_sdk/models.py @@ -202,3 +202,42 @@ class VerbCircuitStatus(BaseModel): "next gateway call." ), ) + + +# ============================================================================= +# TOKEN USAGE +# ============================================================================= + + +class TokenReportRequest(BaseModel): + """Payload for POST /usage/report — reports token usage from a model call. + + Counts are *additive*: the SDK accumulates them per session so multiple + report calls sum up correctly across tool invocations. + """ + + tokens_input: int = Field(default=0, description="Input / prompt tokens consumed") + tokens_output: int = Field( + default=0, description="Output / completion tokens generated" + ) + tokens_cache_read: int = Field( + default=0, description="Prompt-cache read tokens (charged at reduced rate)" + ) + tokens_cache_write: int = Field(default=0, description="Prompt-cache write tokens") + + +class TokenUsageStatus(BaseModel): + """Current cumulative token usage for this session (GET /usage/status).""" + + tokens_input: int = Field( + default=0, description="Total input tokens accumulated this session" + ) + tokens_output: int = Field( + default=0, description="Total output tokens accumulated this session" + ) + tokens_cache_read: int = Field( + default=0, description="Total cache-read tokens this session" + ) + tokens_cache_write: int = Field( + default=0, description="Total cache-write tokens this session" + ) diff --git a/roboco/agent_sdk/server.py b/roboco/agent_sdk/server.py index 256add1d..cce08c0e 100644 --- a/roboco/agent_sdk/server.py +++ b/roboco/agent_sdk/server.py @@ -35,6 +35,8 @@ from roboco.agent_sdk.models import ( SendResponse, TerminalStatus, TerminalToolRecordRequest, + TokenReportRequest, + TokenUsageStatus, VerbAttemptRequest, VerbCircuitStatus, ) @@ -420,6 +422,11 @@ class _SessionState: self.verb_attempts: dict[tuple[str, str | None], deque[float]] = defaultdict( deque ) + # Cumulative token usage for this session (reported via /usage/report) + self.tokens_input: int = 0 + self.tokens_output: int = 0 + self.tokens_cache_read: int = 0 + self.tokens_cache_write: int = 0 def reset(self) -> None: self._init_fields() @@ -680,6 +687,54 @@ def _terminal_snapshot() -> TerminalStatus: ) +# ============================================================================= +# TOKEN USAGE REPORTING +# ============================================================================= + + +@app.post("/usage/report", response_model=TokenUsageStatus) +async def usage_report(req: TokenReportRequest) -> TokenUsageStatus: + """Accumulate token usage counts for the current session. + + Called by Claude Code hooks (e.g. PostToolUse) after each API call + to report the tokens consumed by that invocation. Counts are additive + — multiple calls sum up correctly across the session lifetime. + """ + _state.tokens_input += req.tokens_input + _state.tokens_output += req.tokens_output + _state.tokens_cache_read += req.tokens_cache_read + _state.tokens_cache_write += req.tokens_cache_write + + logger.debug( + "Token usage reported", + delta_input=req.tokens_input, + delta_output=req.tokens_output, + total_input=_state.tokens_input, + total_output=_state.tokens_output, + ) + + return _token_usage_snapshot() + + +@app.get("/usage/status", response_model=TokenUsageStatus) +async def usage_status() -> TokenUsageStatus: + """Return cumulative token usage totals for the current session. + + The orchestrator sweeper calls this endpoint every ~60 s to record + snapshots and to finalize session rows when the container stops. + """ + return _token_usage_snapshot() + + +def _token_usage_snapshot() -> TokenUsageStatus: + return TokenUsageStatus( + tokens_input=_state.tokens_input, + tokens_output=_state.tokens_output, + tokens_cache_read=_state.tokens_cache_read, + tokens_cache_write=_state.tokens_cache_write, + ) + + @app.post("/journal/post_mortem") async def journal_post_mortem(req: PostMortemRequest) -> dict[str, str]: """SessionEnd hook submits a post-mortem; we log it and flush to the main API.""" diff --git a/roboco/api/app.py b/roboco/api/app.py index 8c33c560..34d68821 100644 --- a/roboco/api/app.py +++ b/roboco/api/app.py @@ -36,6 +36,7 @@ from roboco.api.routes.provider import router as provider_router from roboco.api.routes.sessions import router as sessions_router from roboco.api.routes.stream import router as stream_router from roboco.api.routes.tasks import router as tasks_router +from roboco.api.routes.usage import router as usage_router from roboco.api.routes.v1 import do as do_module from roboco.api.routes.v1 import flow_auditor as flow_auditor_module from roboco.api.routes.v1 import flow_board as flow_board_module @@ -340,6 +341,13 @@ def create_app() -> FastAPI: tags=["Documentation"], ) + # Token Usage Analytics + app.include_router( + usage_router, + prefix=f"{api_prefix}/usage", + tags=["Usage Analytics"], + ) + # API v1 — intent-verb flow endpoints app.include_router(flow_dev_module.router) diff --git a/roboco/api/routes/dashboard.py b/roboco/api/routes/dashboard.py index bb09bb27..bc642a44 100644 --- a/roboco/api/routes/dashboard.py +++ b/roboco/api/routes/dashboard.py @@ -21,12 +21,14 @@ from roboco.api.schemas.dashboard import ( CreateReportRequest, FlagSeverity, TeamHealth, + UsageSummary, ) from roboco.models.base import Team from roboco.models.dashboard import CreateFlagParams from roboco.services.dashboard import get_dashboard_service from roboco.services.kanban import get_kanban_service from roboco.services.metrics import get_metrics_service +from roboco.services.usage import get_usage_service router = APIRouter() @@ -277,6 +279,7 @@ async def get_ceo_overview( - Roadmap progress """ service = get_dashboard_service(db) + usage_svc = get_usage_service(db) health_list = await service.get_team_health_list() health_status = [ @@ -291,11 +294,22 @@ async def get_ceo_overview( for h in health_list ] + # Populate usage_summary from daily_usage_rollups for today + try: + today_usage = await usage_svc.get_today_summary() + usage_summary = UsageSummary( + tokens_today=today_usage["tokens_today"], + cost_today_usd=today_usage["cost_today_usd"], + ) + except Exception: + usage_summary = UsageSummary(tokens_today=0, cost_today_usd=0.0) + return CEOOverview( health_status=health_status, key_metrics=await service.get_key_metrics(), auditor_alerts=service.get_auditor_alerts(), roadmap_progress=await service.get_roadmap_progress(), + usage_summary=usage_summary, ) diff --git a/roboco/api/routes/usage.py b/roboco/api/routes/usage.py new file mode 100644 index 00000000..ff93bbad --- /dev/null +++ b/roboco/api/routes/usage.py @@ -0,0 +1,147 @@ +""" +Token Usage Analytics API + +Provides endpoints for querying token usage metrics across agents, +teams, and models. Supports period-based queries (24h, 7d, 30d). +""" + +from typing import Annotated, Any, Literal + +from fastapi import APIRouter, Query + +from roboco.api.deps import DbSession +from roboco.services.usage import get_usage_service + +router = APIRouter() + +_PeriodType = Literal["24h", "7d", "30d"] + +_PeriodQuery = Annotated[ + _PeriodType, + Query(description="Time period: 24h, 7d, 30d"), +] + + +# ============================================================================= +# SUMMARY +# ============================================================================= + + +@router.get("/summary") +async def get_usage_summary( + db: DbSession, + period: _PeriodQuery = "24h", +) -> dict[str, Any]: + """Return aggregated token usage and cost for the given period. + + Response includes: + - tokens_input: total prompt tokens consumed + - tokens_output: total completion tokens generated + - total_tokens: sum of all token types + - total_cost_usd: estimated USD cost + - trend_pct: percent change vs. previous equivalent period + """ + svc = get_usage_service(db) + return await svc.get_summary(period) + + +# ============================================================================= +# TIME SERIES +# ============================================================================= + + +@router.get("/time-series") +async def get_usage_time_series( + db: DbSession, + period: _PeriodQuery = "24h", +) -> list[dict[str, Any]]: + """Return bucketed time-series data points. + + - 24h → hourly buckets + - 7d / 30d → daily buckets + + Each point has: bucket (ISO timestamp), tokens_input, tokens_output, + total_tokens, cost_usd. + """ + svc = get_usage_service(db) + return await svc.get_time_series(period) + + +# ============================================================================= +# BREAKDOWN ENDPOINTS +# ============================================================================= + + +@router.get("/by-agent") +async def get_usage_by_agent( + db: DbSession, + period: _PeriodQuery = "24h", +) -> list[dict[str, Any]]: + """Return per-agent token usage with pct_of_total. + + pct_of_total fields sum to approximately 100%. + """ + svc = get_usage_service(db) + return await svc.get_by_agent(period) + + +@router.get("/by-team") +async def get_usage_by_team( + db: DbSession, + period: _PeriodQuery = "24h", +) -> list[dict[str, Any]]: + """Return per-team token usage with pct_of_total. + + pct_of_total fields sum to approximately 100%. + """ + svc = get_usage_service(db) + return await svc.get_by_team(period) + + +@router.get("/by-model") +async def get_usage_by_model( + db: DbSession, + period: _PeriodQuery = "24h", +) -> list[dict[str, Any]]: + """Return per-model token usage with pct_of_total. + + pct_of_total fields sum to approximately 100%. + """ + svc = get_usage_service(db) + return await svc.get_by_model(period) + + +# ============================================================================= +# PROJECTION +# ============================================================================= + + +@router.get("/projection") +async def get_usage_projection( + db: DbSession, +) -> dict[str, Any]: + """Return projected monthly cost based on 7-day rolling average. + + projected_monthly_cost_usd is computed from avg_daily_cost * 30. + """ + svc = get_usage_service(db) + return await svc.get_projection() + + +# ============================================================================= +# CACHE EFFICIENCY +# ============================================================================= + + +@router.get("/cache-efficiency") +async def get_cache_efficiency( + db: DbSession, + period: _PeriodQuery = "24h", +) -> dict[str, Any]: + """Return cache hit rate and estimated savings from prompt caching. + + - cache_hit_rate: fraction of input-like tokens served from cache + - cost_saved_by_cache_usd: estimated savings vs. full input pricing + """ + svc = get_usage_service(db) + return await svc.get_cache_efficiency(period) diff --git a/roboco/api/schemas/dashboard.py b/roboco/api/schemas/dashboard.py index 48efe774..d1b527f7 100644 --- a/roboco/api/schemas/dashboard.py +++ b/roboco/api/schemas/dashboard.py @@ -78,6 +78,17 @@ class TeamHealth(BaseModel): completed_this_week: int +class UsageSummary(BaseModel): + """Today's token usage summary for the CEO dashboard.""" + + tokens_today: int = Field( + default=0, description="Total tokens (input + output + cache) used today" + ) + cost_today_usd: float = Field( + default=0.0, description="Estimated USD cost for today" + ) + + class CEOOverview(BaseModel): """Complete CEO overview data.""" @@ -85,6 +96,10 @@ class CEOOverview(BaseModel): key_metrics: dict[str, Any] auditor_alerts: dict[str, Any] roadmap_progress: dict[str, Any] + usage_summary: UsageSummary | None = Field( + default=None, + description="Today's token usage and cost from daily_usage_rollups", + ) class CreateFlagRequest(BaseModel): diff --git a/roboco/billing/__init__.py b/roboco/billing/__init__.py new file mode 100644 index 00000000..75efe269 --- /dev/null +++ b/roboco/billing/__init__.py @@ -0,0 +1,8 @@ +"""Billing utilities for RoboCo. + +Provides token-cost calculation for Claude API models. +""" + +from roboco.billing.pricing import calculate_cost + +__all__ = ["calculate_cost"] diff --git a/roboco/billing/pricing.py b/roboco/billing/pricing.py new file mode 100644 index 00000000..e17480f9 --- /dev/null +++ b/roboco/billing/pricing.py @@ -0,0 +1,106 @@ +""" +Token pricing for Claude API models. + +Implements per-model USD cost calculation based on Anthropic's published +pricing. All prices are in USD per 1 million tokens. + +Unknown model names return 0.0 without raising so callers don't need to +guard against missing pricing data. Self-hosted Ollama models always +return 0.0 (no API cost) — matched by the ``ollama/`` prefix convention. +""" + +from __future__ import annotations + +import structlog + +logger = structlog.get_logger(__name__) + +# --------------------------------------------------------------------------- +# Per-model pricing table +# Format: model_name_fragment → (input_usd_per_1m, output_usd_per_1m, +# cache_read_usd_per_1m, cache_write_usd_per_1m) +# +# Cache read is charged at ~10 % of the input price. +# Cache write is charged at ~25 % of the input price. +# +# Match on *substring* of model name so "claude-opus-4-6" and "opus" both +# resolve to the same tier. +# --------------------------------------------------------------------------- +_PRICING: list[tuple[str, float, float, float, float]] = [ + # (fragment, input/1M, output/1M, cache_read/1M, cache_write/1M) + # Opus 4 family + ("claude-opus-4", 5.00, 25.00, 0.50, 6.25), + # Sonnet 4 / 3.7 / 3.5 family + ("claude-sonnet-4", 3.00, 15.00, 0.30, 0.75), + ("claude-3-7-sonnet", 3.00, 15.00, 0.30, 0.75), + ("claude-3-5-sonnet", 3.00, 15.00, 0.30, 0.75), + # Haiku family + ("claude-haiku-4", 1.00, 5.00, 0.10, 1.25), + ("claude-haiku-3-5", 1.00, 5.00, 0.10, 1.25), + ("claude-3-5-haiku", 1.00, 5.00, 0.10, 1.25), + ("claude-haiku-3", 0.25, 1.25, 0.025, 0.0625), + # Short aliases used in ROLE_MODEL_MAP / MODEL_MAP + ("opus", 5.00, 25.00, 0.50, 6.25), + ("sonnet", 3.00, 15.00, 0.30, 0.75), + ("haiku", 1.00, 5.00, 0.10, 1.25), +] + +_MILLION = 1_000_000.0 + + +def calculate_cost( + model: str, + tokens_input: int, + tokens_output: int, + tokens_cache_read: int = 0, + tokens_cache_write: int = 0, +) -> float: + """Calculate the estimated USD cost for a model invocation. + + Matches the model name against the known pricing table using substring + search (longest match wins). Unknown models return 0.0 without raising. + Self-hosted Ollama models (``ollama/`` prefix) always return 0.0. + + Args: + model: Model name or short alias (e.g. ``"claude-sonnet-4-6"``, + ``"sonnet"``, ``"opus"``). + tokens_input: Number of input tokens (prompt / context). + tokens_output: Number of output tokens (completion). + tokens_cache_read: Prompt-cache read tokens (charged at reduced rate). + tokens_cache_write: Prompt-cache write tokens (charged at reduced rate). + + Returns: + Estimated cost in USD as a float. Returns 0.0 for unknown models + rather than raising. + """ + if not model: + return 0.0 + + lower = model.lower() + + # Self-hosted Ollama models have no API cost. + if lower.startswith("ollama/"): + return 0.0 + + # Find the best (longest fragment) match + best_fragment_len = 0 + best_prices: tuple[float, float, float, float] | None = None + + for fragment, inp_price, out_price, cr_price, cw_price in _PRICING: + if fragment in lower and len(fragment) > best_fragment_len: + best_fragment_len = len(fragment) + best_prices = (inp_price, out_price, cr_price, cw_price) + + if best_prices is None: + logger.warning("No pricing data found for model", model=model) + return 0.0 + + inp_price, out_price, cr_price, cw_price = best_prices + + cost = ( + tokens_input * inp_price / _MILLION + + tokens_output * out_price / _MILLION + + tokens_cache_read * cr_price / _MILLION + + tokens_cache_write * cw_price / _MILLION + ) + return round(cost, 8) diff --git a/roboco/db/tables.py b/roboco/db/tables.py index 441a110e..393b019e 100644 --- a/roboco/db/tables.py +++ b/roboco/db/tables.py @@ -11,7 +11,9 @@ from uuid import uuid4 from sqlalchemy import ( JSON, + BigInteger, Boolean, + Date, DateTime, Enum, Float, @@ -1823,6 +1825,148 @@ class GatewayTriggerTable(Base): ) +# ============================================================================= +# TOKEN USAGE TABLES +# ============================================================================= + + +class AgentSpawnSessionTable(Base): + """Records each agent container spawn lifecycle. + + Opened when the orchestrator successfully starts a container; closed + (ended_at set) when stop_agent() finishes. Final token counts are + accumulated from the agent SDK's /usage/status endpoint. + """ + + __tablename__ = "agent_spawn_sessions" + + id: Mapped[UUID] = mapped_column( + UUID(as_uuid=True), primary_key=True, default=uuid4 + ) + agent_slug: Mapped[str] = mapped_column(String(100), nullable=False) + team: Mapped[str] = mapped_column(String(50), nullable=False) + role: Mapped[str] = mapped_column(String(50), nullable=False) + model: Mapped[str] = mapped_column(String(100), nullable=False) + task_id: Mapped[str | None] = mapped_column(String(36), nullable=True) + started_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + default=lambda: datetime.now(UTC), + nullable=False, + ) + ended_at: Mapped[datetime | None] = mapped_column( + DateTime(timezone=True), nullable=True + ) + # BIGINT — token counts can exceed INT32 for long sessions + tokens_input: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + tokens_output: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + tokens_cache_read: Mapped[int] = mapped_column( + BigInteger, nullable=False, default=0 + ) + tokens_cache_write: Mapped[int] = mapped_column( + BigInteger, nullable=False, default=0 + ) + exit_reason: Mapped[str | None] = mapped_column(String(100), nullable=True) + estimated_cost_usd: Mapped[float | None] = mapped_column(Float, nullable=True) + + # Relationship to snapshots (backref for convenience) + snapshots: Mapped[list["TokenUsageSnapshotTable"]] = relationship( + "TokenUsageSnapshotTable", + back_populates="session", + cascade="all, delete-orphan", + ) + + __table_args__ = ( + Index("ix_agent_spawn_sessions_agent_slug", "agent_slug"), + Index("ix_agent_spawn_sessions_started_at", "started_at"), + Index("ix_agent_spawn_sessions_ended_at", "ended_at"), + Index("ix_agent_spawn_sessions_team", "team"), + ) + + +class TokenUsageSnapshotTable(Base): + """Periodic (every ~60 s) snapshot of cumulative token usage for an + active agent_spawn_session. + + The sweeper inserts one row per active agent per sweep cycle when + token counts are non-zero. Snapshots allow tracking how token usage + grows over session lifetime. + """ + + __tablename__ = "token_usage_snapshots" + + id: Mapped[UUID] = mapped_column( + UUID(as_uuid=True), primary_key=True, default=uuid4 + ) + agent_spawn_session_id: Mapped[UUID] = mapped_column( + UUID(as_uuid=True), + ForeignKey("agent_spawn_sessions.id", ondelete="CASCADE"), + nullable=False, + ) + snapshotted_at: Mapped[datetime] = mapped_column( + DateTime(timezone=True), + default=lambda: datetime.now(UTC), + nullable=False, + ) + tokens_input: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + tokens_output: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + tokens_cache_read: Mapped[int] = mapped_column( + BigInteger, nullable=False, default=0 + ) + tokens_cache_write: Mapped[int] = mapped_column( + BigInteger, nullable=False, default=0 + ) + + session: Mapped["AgentSpawnSessionTable"] = relationship( + "AgentSpawnSessionTable", back_populates="snapshots" + ) + + __table_args__ = ( + Index("ix_token_usage_snapshots_session_id", "agent_spawn_session_id"), + Index("ix_token_usage_snapshots_snapshotted_at", "snapshotted_at"), + ) + + +class DailyUsageRollupTable(Base): + """Pre-aggregated daily token usage per (date, agent_slug, team, model). + + Populated by the orchestrator sweeper via an upsert query over + closed agent_spawn_sessions. Unique constraint on the natural key + enables ON CONFLICT DO UPDATE so the sweep is idempotent. + """ + + __tablename__ = "daily_usage_rollups" + + id: Mapped[UUID] = mapped_column( + UUID(as_uuid=True), primary_key=True, default=uuid4 + ) + date: Mapped[Any] = mapped_column(Date, nullable=False) # datetime.date + agent_slug: Mapped[str] = mapped_column(String(100), nullable=False) + team: Mapped[str] = mapped_column(String(50), nullable=False) + model: Mapped[str] = mapped_column(String(100), nullable=False) + tokens_input: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + tokens_output: Mapped[int] = mapped_column(BigInteger, nullable=False, default=0) + tokens_cache_read: Mapped[int] = mapped_column( + BigInteger, nullable=False, default=0 + ) + tokens_cache_write: Mapped[int] = mapped_column( + BigInteger, nullable=False, default=0 + ) + total_cost_usd: Mapped[float] = mapped_column(Float, nullable=False, default=0.0) + session_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0) + + __table_args__ = ( + UniqueConstraint( + "date", + "agent_slug", + "team", + "model", + name="uq_daily_rollup_date_agent_team_model", + ), + Index("ix_daily_rollups_date", "date"), + Index("ix_daily_rollups_agent_slug", "agent_slug"), + ) + + # ============================================================================= # PROMPTER TABLES # ============================================================================= diff --git a/roboco/events/stream_bus.py b/roboco/events/stream_bus.py index 90188f7f..fc9ff9b2 100644 --- a/roboco/events/stream_bus.py +++ b/roboco/events/stream_bus.py @@ -376,9 +376,9 @@ class StreamEventBus: recovered = 0 for msg in pending_details: - if msg["time_since_delivered"] >= idle_time_ms: + if int(msg["time_since_delivered"]) >= idle_time_ms: recovered += await self._claim_and_handle( - stream, msg["message_id"], idle_time_ms + stream, str(msg["message_id"]), idle_time_ms ) return recovered diff --git a/roboco/models/runtime.py b/roboco/models/runtime.py index f92b58ef..b4e74f5c 100644 --- a/roboco/models/runtime.py +++ b/roboco/models/runtime.py @@ -71,6 +71,10 @@ class AgentInstance: error_count: int = 0 waiting_for: str | None = None # For WAITING_LONG state waiting_context: dict[str, Any] = field(default_factory=dict) + # UUID of the agent_spawn_sessions row created at spawn time. + # Used by _finalize_spawn_session for a direct-by-id lookup instead of a + # fragile (agent_slug, ended_at IS NULL) query. + usage_session_id: UUID | None = None def __post_init__(self) -> None: if not self.id: diff --git a/roboco/runtime/orchestrator.py b/roboco/runtime/orchestrator.py index 9be29d1c..bc500d8c 100644 --- a/roboco/runtime/orchestrator.py +++ b/roboco/runtime/orchestrator.py @@ -27,6 +27,7 @@ import httpx if TYPE_CHECKING: from collections.abc import Callable, Coroutine + from uuid import UUID from roboco.services.llm import AgentRoute from roboco.services.task import TaskService @@ -69,6 +70,11 @@ AgentConfig = OrchestratorAgentConfig AGENT_NETWORK = "roboco_default" AGENT_BASE_IMAGE = "roboco-agent-base" +# Port on which each agent's Claude Code SDK server listens inside its container. +# Referenced by write-hooks (_finalize_spawn_session, _sweep_token_snapshots, +# _sweep_budget_exceeded) to build the SDK health/usage URL. +SDK_PORT: int = 9000 + # The intake (prompter) agent: a single seeded, board-adjacent interviewer. # Unlike delivery agents it is never dispatched and runs ONE persistent # container at a time (single CEO → one live chat). See the INTAKE section @@ -1382,6 +1388,13 @@ class AgentOrchestrator: "model": config.model, }, ) + + # Record a token-usage session row in the DB and bind its UUID to + # the instance so _finalize_spawn_session can look it up directly. + usage_session_id = await self._record_spawn_session(config, task_id) + if usage_session_id is not None: + instance.usage_session_id = usage_session_id + return instance except Exception as e: instance.state = AgentState.OFFLINE @@ -2868,8 +2881,28 @@ class AgentOrchestrator: # AGENT STOPPING # ========================================================================= - async def stop_agent(self, agent_id: str, graceful: bool = True) -> None: - """Stop an agent container.""" + async def stop_agent( + self, + agent_id: str, + graceful: bool = True, + exit_reason: str = "stopped", + ) -> None: + """Stop an agent container. + + Finalization (the HTTP call to the agent SDK's /usage/status endpoint) + is performed BEFORE acquiring self._lock so that the network I/O does + not block other operations that need the lock. + """ + # Finalize the spawn-session row before the container is removed so we + # can still query the SDK's /usage/status endpoint. This must happen + # outside self._lock — the HTTP round-trip would otherwise hold the + # lock for the full network timeout. + instance = self._instances.get(agent_id) + if instance is None: + return + if instance.container_id: + await self._finalize_spawn_session(agent_id, exit_reason=exit_reason) + async with self._lock: if agent_id not in self._instances: return @@ -3013,6 +3046,402 @@ class AgentOrchestrator: error=str(e), ) + # ========================================================================= + # TOKEN USAGE INSTRUMENTATION + # ========================================================================= + + async def _record_spawn_session( + self, + config: "OrchestratorAgentConfig", + task_id: str | None, + ) -> "UUID | None": + """Insert a row into agent_spawn_sessions after a successful spawn. + + Returns the UUID of the created row so the caller can store it on + the AgentInstance for later direct-by-id lookup in + _finalize_spawn_session. Returns None when the insert fails; a + missing session row must never block the spawn path. + """ + try: + from uuid import uuid4 as _uuid4 + + from roboco.db.base import get_session_factory + from roboco.db.tables import AgentSpawnSessionTable + + agent_slug = config.agent_id + team = get_agent_team(agent_slug) or "backend" + role = get_agent_role(agent_slug) or "developer" + + session_id = _uuid4() + session_factory = get_session_factory() + async with session_factory() as db: + row = AgentSpawnSessionTable( + id=session_id, + agent_slug=agent_slug, + team=team, + role=role, + model=config.model or "unknown", + task_id=task_id, + started_at=datetime.now(UTC), + ) + db.add(row) + await db.commit() + logger.debug( + "Spawn session recorded", + agent_slug=agent_slug, + session_id=str(session_id), + task_id=task_id, + ) + return session_id + except Exception as exc: + logger.warning( + "Failed to record spawn session", + agent_slug=config.agent_id, + error=str(exc), + ) + return None + + async def _finalize_spawn_session( + self, + agent_id: str, + exit_reason: str = "stopped", + ) -> None: + """Close the open agent_spawn_sessions row for this agent. + + Fetches final token counts from the agent SDK's /usage/status endpoint, + calculates cost via pricing module, then updates the DB row with + ended_at, token totals, exit_reason, and estimated_cost_usd. + Errors are caught and logged — finalization must never block stop_agent. + """ + try: + from roboco.billing.pricing import calculate_cost + from roboco.db.base import get_session_factory + from roboco.db.tables import AgentSpawnSessionTable + + # Fetch final token counts from the agent's SDK + sdk_url = f"http://roboco-agent-{agent_id}:{SDK_PORT}/usage/status" + tokens_input = 0 + tokens_output = 0 + tokens_cache_read = 0 + tokens_cache_write = 0 + model = "unknown" + + try: + async with httpx.AsyncClient(timeout=3.0) as client: + resp = await client.get(sdk_url) + if resp.status_code == http_status.HTTP_200_OK: + data = resp.json() + tokens_input = data.get("tokens_input", 0) + tokens_output = data.get("tokens_output", 0) + tokens_cache_read = data.get("tokens_cache_read", 0) + tokens_cache_write = data.get("tokens_cache_write", 0) + except Exception as sdk_exc: + logger.debug( + "Could not fetch final token counts from SDK", + agent_id=agent_id, + error=str(sdk_exc), + ) + + # Look up the model and usage_session_id from the running instance config. + instance = self._instances.get(agent_id) + if instance and instance.config: + model = instance.config.model or "unknown" + usage_session_id = instance.usage_session_id if instance else None + + cost = calculate_cost( + model=model, + tokens_input=tokens_input, + tokens_output=tokens_output, + tokens_cache_read=tokens_cache_read, + tokens_cache_write=tokens_cache_write, + ) + + session_factory = get_session_factory() + async with session_factory() as db: + from sqlalchemy import select, update + + # Prefer a direct lookup by the session UUID captured at spawn + # time; fall back to the (agent_slug, ended_at IS NULL) query + # for instances that pre-date the usage_session_id field. + if usage_session_id is not None: + result = await db.execute( + select(AgentSpawnSessionTable).where( + AgentSpawnSessionTable.id == usage_session_id + ) + ) + else: + result = await db.execute( + select(AgentSpawnSessionTable) + .where( + AgentSpawnSessionTable.agent_slug == agent_id, + AgentSpawnSessionTable.ended_at.is_(None), + ) + .order_by(AgentSpawnSessionTable.started_at.desc()) + .limit(1) + ) + session_row = result.scalar_one_or_none() + if session_row is not None: + await db.execute( + update(AgentSpawnSessionTable) + .where(AgentSpawnSessionTable.id == session_row.id) + .values( + ended_at=datetime.now(UTC), + tokens_input=tokens_input, + tokens_output=tokens_output, + tokens_cache_read=tokens_cache_read, + tokens_cache_write=tokens_cache_write, + exit_reason=exit_reason, + estimated_cost_usd=cost, + ) + ) + await db.commit() + logger.debug( + "Spawn session finalized", + agent_id=agent_id, + session_id=str(session_row.id), + tokens_input=tokens_input, + tokens_output=tokens_output, + estimated_cost_usd=cost, + ) + except Exception as exc: + logger.warning( + "Failed to finalize spawn session", + agent_id=agent_id, + error=str(exc), + ) + + async def _sweep_token_snapshots(self) -> None: + """Write a token_usage_snapshots row for each active agent with non-zero tokens. + + Called from _run_sweep() every ~60 s. Also updates the cumulative + token counts on the open agent_spawn_sessions row so the DB reflects + current progress without waiting for session close. + Errors per-agent are caught so one bad agent doesn't abort the whole sweep. + """ + if not self._instances: + return + + try: + from roboco.db.base import get_session_factory + from roboco.db.tables import AgentSpawnSessionTable, TokenUsageSnapshotTable + except ImportError: + return + + session_factory = get_session_factory() + + async with httpx.AsyncClient(timeout=3.0) as client: + for agent_id, instance in list(self._instances.items()): + if instance.state not in ( + AgentState.ACTIVE, + AgentState.WAITING_SHORT, + ): + continue + + sdk_url = f"http://roboco-agent-{agent_id}:{SDK_PORT}/usage/status" + try: + resp = await client.get(sdk_url) + if resp.status_code != http_status.HTTP_200_OK: + continue + data = resp.json() + tokens_input = data.get("tokens_input", 0) + tokens_output = data.get("tokens_output", 0) + tokens_cache_read = data.get("tokens_cache_read", 0) + tokens_cache_write = data.get("tokens_cache_write", 0) + + # Skip agents with no token usage yet + total = ( + tokens_input + + tokens_output + + tokens_cache_read + + tokens_cache_write + ) + if total == 0: + continue + + async with session_factory() as db: + from sqlalchemy import select, update + + # Prefer a direct lookup by the session UUID captured at + # spawn time; fall back to the agent_slug heuristic for + # instances that pre-date the usage_session_id field. + if instance.usage_session_id is not None: + result = await db.execute( + select(AgentSpawnSessionTable).where( + AgentSpawnSessionTable.id + == instance.usage_session_id + ) + ) + else: + result = await db.execute( + select(AgentSpawnSessionTable) + .where( + AgentSpawnSessionTable.agent_slug == agent_id, + AgentSpawnSessionTable.ended_at.is_(None), + ) + .order_by(AgentSpawnSessionTable.started_at.desc()) + .limit(1) + ) + session_row = result.scalar_one_or_none() + if session_row is None: + continue + + # Insert snapshot + from uuid import uuid4 as _uuid4 + + snapshot = TokenUsageSnapshotTable( + id=_uuid4(), + agent_spawn_session_id=session_row.id, + snapshotted_at=datetime.now(UTC), + tokens_input=tokens_input, + tokens_output=tokens_output, + tokens_cache_read=tokens_cache_read, + tokens_cache_write=tokens_cache_write, + ) + db.add(snapshot) + + # Update cumulative totals on the session row + await db.execute( + update(AgentSpawnSessionTable) + .where(AgentSpawnSessionTable.id == session_row.id) + .values( + tokens_input=tokens_input, + tokens_output=tokens_output, + tokens_cache_read=tokens_cache_read, + tokens_cache_write=tokens_cache_write, + ) + ) + await db.commit() + + except Exception as agent_exc: + logger.debug( + "Token snapshot failed for agent", + agent_id=agent_id, + error=str(agent_exc), + ) + + async def _sweep_daily_rollup(self) -> None: + """Upsert daily_usage_rollups from closed agent_spawn_sessions. + + Groups ended sessions by (date, agent_slug, team, model) and sums + their token counts + cost. Uses a Python-side upsert to stay + compatible with asyncpg / SQLAlchemy without raw INSERT ... ON CONFLICT + dialect-specific SQL. + Errors are caught so a bad rollup doesn't abort the sweeper. + """ + try: + from roboco.db.base import get_session_factory + from roboco.db.tables import AgentSpawnSessionTable, DailyUsageRollupTable + except ImportError: + return + + try: + from uuid import uuid4 as _uuid4 + + from sqlalchemy import func, select + + session_factory = get_session_factory() + async with session_factory() as db: + # Aggregate closed sessions by (date, agent_slug, team, model). + # Limit to the last 7 days to avoid re-aggregating all-time + # history on every sweep — older days are already stable. + rollup_window_start = datetime.now(UTC) - timedelta(days=7) + result = await db.execute( + select( + func.date(AgentSpawnSessionTable.started_at).label("date"), + AgentSpawnSessionTable.agent_slug, + AgentSpawnSessionTable.team, + AgentSpawnSessionTable.model, + func.sum(AgentSpawnSessionTable.tokens_input).label( + "tokens_input" + ), + func.sum(AgentSpawnSessionTable.tokens_output).label( + "tokens_output" + ), + func.sum(AgentSpawnSessionTable.tokens_cache_read).label( + "tokens_cache_read" + ), + func.sum(AgentSpawnSessionTable.tokens_cache_write).label( + "tokens_cache_write" + ), + func.sum(AgentSpawnSessionTable.estimated_cost_usd).label( + "total_cost_usd" + ), + func.count(AgentSpawnSessionTable.id).label("session_count"), + ) + .where( + AgentSpawnSessionTable.ended_at.isnot(None), + AgentSpawnSessionTable.started_at >= rollup_window_start, + ) + .group_by( + func.date(AgentSpawnSessionTable.started_at), + AgentSpawnSessionTable.agent_slug, + AgentSpawnSessionTable.team, + AgentSpawnSessionTable.model, + ) + ) + rows = result.fetchall() + + for row in rows: + date_val = row.date + agent_slug = row.agent_slug + team = row.team + model = row.model + + # Look for existing rollup row + existing_result = await db.execute( + select(DailyUsageRollupTable).where( + DailyUsageRollupTable.date == date_val, + DailyUsageRollupTable.agent_slug == agent_slug, + DailyUsageRollupTable.team == team, + DailyUsageRollupTable.model == model, + ) + ) + existing = existing_result.scalar_one_or_none() + + tokens_input = int(row.tokens_input or 0) + tokens_output = int(row.tokens_output or 0) + tokens_cache_read = int(row.tokens_cache_read or 0) + tokens_cache_write = int(row.tokens_cache_write or 0) + total_cost = float(row.total_cost_usd or 0.0) + session_count = int(row.session_count or 0) + + if existing is not None: + from sqlalchemy import update + + await db.execute( + update(DailyUsageRollupTable) + .where(DailyUsageRollupTable.id == existing.id) + .values( + tokens_input=tokens_input, + tokens_output=tokens_output, + tokens_cache_read=tokens_cache_read, + tokens_cache_write=tokens_cache_write, + total_cost_usd=total_cost, + session_count=session_count, + ) + ) + else: + new_row = DailyUsageRollupTable( + id=_uuid4(), + date=date_val, + agent_slug=agent_slug, + team=team, + model=model, + tokens_input=tokens_input, + tokens_output=tokens_output, + tokens_cache_read=tokens_cache_read, + tokens_cache_write=tokens_cache_write, + total_cost_usd=total_cost, + session_count=session_count, + ) + db.add(new_row) + + await db.commit() + logger.debug("Daily usage rollup complete", rows_processed=len(rows)) + + except Exception as exc: + logger.warning("Daily usage rollup failed", error=str(exc)) + async def restore_waiting_records(self) -> int: """Load persisted waiting records into memory on orchestrator start. @@ -3212,6 +3641,11 @@ Start by: # same session. await self._sweep_budget_exceeded() + # Token-usage instrumentation: snapshot active agents and roll up + # closed sessions into the daily aggregation table. + await self._sweep_token_snapshots() + await self._sweep_daily_rollup() + @staticmethod async def _fetch_budget_status( client: httpx.AsyncClient, url: str, agent_id: str @@ -3263,7 +3697,7 @@ Start by: AgentState.WAITING_SHORT, ): continue - url = f"http://roboco-agent-{agent_id}:9000/budget/status" + url = f"http://roboco-agent-{agent_id}:{SDK_PORT}/budget/status" data = await self._fetch_budget_status(client, url, agent_id) if data is None or not data.get("halt"): continue diff --git a/roboco/services/usage.py b/roboco/services/usage.py new file mode 100644 index 00000000..11c3a19d --- /dev/null +++ b/roboco/services/usage.py @@ -0,0 +1,554 @@ +""" +Usage Analytics Service + +Provides token usage analytics over agent_spawn_sessions and +daily_usage_rollups tables. Supports period-based queries (24h, 7d, 30d) +and aggregation by agent, team, and model. +""" + +from __future__ import annotations + +from datetime import UTC, datetime, timedelta +from typing import TYPE_CHECKING, Any + +from sqlalchemy import func, select + +if TYPE_CHECKING: + from sqlalchemy.ext.asyncio import AsyncSession + +from roboco.db.tables import AgentSpawnSessionTable, DailyUsageRollupTable +from roboco.services.base import BaseService + + +def _parse_period(period: str) -> tuple[datetime, int]: + """Parse period string into (start_dt, hours). + + Accepts '24h', '7d', '30d'. Defaults to 24h for unknown values. + Returns (start_datetime_utc, total_hours). + """ + now = datetime.now(UTC) + if period == "7d": + return now - timedelta(days=7), 7 * 24 + if period == "30d": + return now - timedelta(days=30), 30 * 24 + # default 24h + return now - timedelta(hours=24), 24 + + +class UsageService(BaseService): + """Analytics service for token usage data.""" + + # ========================================================================= + # SUMMARY + # ========================================================================= + + async def get_summary(self, period: str = "24h") -> dict[str, Any]: + """Return aggregated token and cost totals for the given period. + + Queries daily_usage_rollups for whole-day periods; falls back to + agent_spawn_sessions for sub-day precision. + + Returns dict with: tokens_input, tokens_output, total_tokens, + total_cost_usd, trend_pct. + """ + start_dt, hours = _parse_period(period) + + # Current period totals from closed sessions + result = await self.session.execute( + select( + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_input), 0).label( + "tokens_input" + ), + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_output), 0).label( + "tokens_output" + ), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_read), 0 + ).label("tokens_cache_read"), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_write), 0 + ).label("tokens_cache_write"), + func.coalesce( + func.sum(AgentSpawnSessionTable.estimated_cost_usd), 0.0 + ).label("total_cost_usd"), + ).where( + AgentSpawnSessionTable.started_at >= start_dt, + AgentSpawnSessionTable.ended_at.isnot(None), + ) + ) + row = result.one() + tokens_input = int(row.tokens_input or 0) + tokens_output = int(row.tokens_output or 0) + total_cost = float(row.total_cost_usd or 0.0) + total_tokens = ( + tokens_input + + tokens_output + + int(row.tokens_cache_read or 0) + + int(row.tokens_cache_write or 0) + ) + + # Previous period for trend calculation. + # Sum all 4 token columns so the comparison is consistent with the + # current-period total_tokens (which also sums all 4 columns). + prev_start = start_dt - timedelta(hours=hours) + prev_result = await self.session.execute( + select( + func.coalesce( + func.sum( + AgentSpawnSessionTable.tokens_input + + AgentSpawnSessionTable.tokens_output + + AgentSpawnSessionTable.tokens_cache_read + + AgentSpawnSessionTable.tokens_cache_write + ), + 0, + ).label("total") + ).where( + AgentSpawnSessionTable.started_at >= prev_start, + AgentSpawnSessionTable.started_at < start_dt, + AgentSpawnSessionTable.ended_at.isnot(None), + ) + ) + prev_row = prev_result.one() + prev_total = int(prev_row.total or 0) + + if prev_total > 0: + trend_pct = round((total_tokens - prev_total) / prev_total * 100, 1) + elif total_tokens > 0: + trend_pct = 100.0 + else: + trend_pct = 0.0 + + return { + "tokens_input": tokens_input, + "tokens_output": tokens_output, + "total_tokens": total_tokens, + "total_cost_usd": round(total_cost, 6), + "trend_pct": trend_pct, + "period": period, + } + + # ========================================================================= + # TIME SERIES + # ========================================================================= + + async def get_time_series(self, period: str = "24h") -> list[dict[str, Any]]: + """Return bucketed time-series data points. + + - 24h → hourly buckets + - 7d / 30d → daily buckets + + Each point has: bucket (ISO string), tokens_input, tokens_output, + total_tokens, cost_usd. + + total_tokens includes all 4 token types (input + output + cache_read + + cache_write) so it is consistent with get_summary()'s total_tokens + field — the two sums must match for the same period. + """ + start_dt, _hours = _parse_period(period) + + if period == "24h": + # Hourly buckets + trunc_fn = func.date_trunc("hour", AgentSpawnSessionTable.started_at) + else: + # Daily buckets + trunc_fn = func.date_trunc("day", AgentSpawnSessionTable.started_at) + + result = await self.session.execute( + select( + trunc_fn.label("bucket"), + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_input), 0).label( + "tokens_input" + ), + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_output), 0).label( + "tokens_output" + ), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_read), 0 + ).label("tokens_cache_read"), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_write), 0 + ).label("tokens_cache_write"), + func.coalesce( + func.sum(AgentSpawnSessionTable.estimated_cost_usd), 0.0 + ).label("cost_usd"), + ) + .where( + AgentSpawnSessionTable.started_at >= start_dt, + AgentSpawnSessionTable.ended_at.isnot(None), + ) + .group_by(trunc_fn) + .order_by(trunc_fn) + ) + rows = result.fetchall() + + points = [] + for r in rows: + ti = int(r.tokens_input or 0) + to_ = int(r.tokens_output or 0) + tcr = int(r.tokens_cache_read or 0) + tcw = int(r.tokens_cache_write or 0) + points.append( + { + "bucket": r.bucket.isoformat() if r.bucket else None, + "tokens_input": ti, + "tokens_output": to_, + "total_tokens": ti + to_ + tcr + tcw, + "cost_usd": round(float(r.cost_usd or 0.0), 6), + } + ) + return points + + # ========================================================================= + # BY-AGENT + # ========================================================================= + + async def get_by_agent(self, period: str = "24h") -> list[dict[str, Any]]: + """Return per-agent token usage with pct_of_total.""" + start_dt, _ = _parse_period(period) + + result = await self.session.execute( + select( + AgentSpawnSessionTable.agent_slug, + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_input), 0).label( + "tokens_input" + ), + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_output), 0).label( + "tokens_output" + ), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_read), 0 + ).label("tokens_cache_read"), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_write), 0 + ).label("tokens_cache_write"), + func.coalesce( + func.sum(AgentSpawnSessionTable.estimated_cost_usd), 0.0 + ).label("cost_usd"), + ) + .where( + AgentSpawnSessionTable.started_at >= start_dt, + AgentSpawnSessionTable.ended_at.isnot(None), + ) + .group_by(AgentSpawnSessionTable.agent_slug) + .order_by( + func.sum( + AgentSpawnSessionTable.tokens_input + + AgentSpawnSessionTable.tokens_output + ).desc() + ) + ) + rows = result.fetchall() + + grand_total = sum( + int(r.tokens_input or 0) + + int(r.tokens_output or 0) + + int(r.tokens_cache_read or 0) + + int(r.tokens_cache_write or 0) + for r in rows + ) + items = [] + for r in rows: + ti = int(r.tokens_input or 0) + to_ = int(r.tokens_output or 0) + tcr = int(r.tokens_cache_read or 0) + tcw = int(r.tokens_cache_write or 0) + total = ti + to_ + tcr + tcw + items.append( + { + "agent_slug": r.agent_slug, + "tokens_input": ti, + "tokens_output": to_, + "total_tokens": total, + "cost_usd": round(float(r.cost_usd or 0.0), 6), + "pct_of_total": round(total / grand_total * 100, 2) + if grand_total > 0 + else 0.0, + } + ) + return items + + # ========================================================================= + # BY-TEAM + # ========================================================================= + + async def get_by_team(self, period: str = "24h") -> list[dict[str, Any]]: + """Return per-team token usage with pct_of_total.""" + start_dt, _ = _parse_period(period) + + result = await self.session.execute( + select( + AgentSpawnSessionTable.team, + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_input), 0).label( + "tokens_input" + ), + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_output), 0).label( + "tokens_output" + ), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_read), 0 + ).label("tokens_cache_read"), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_write), 0 + ).label("tokens_cache_write"), + func.coalesce( + func.sum(AgentSpawnSessionTable.estimated_cost_usd), 0.0 + ).label("cost_usd"), + ) + .where( + AgentSpawnSessionTable.started_at >= start_dt, + AgentSpawnSessionTable.ended_at.isnot(None), + ) + .group_by(AgentSpawnSessionTable.team) + .order_by( + func.sum( + AgentSpawnSessionTable.tokens_input + + AgentSpawnSessionTable.tokens_output + ).desc() + ) + ) + rows = result.fetchall() + + grand_total = sum( + int(r.tokens_input or 0) + + int(r.tokens_output or 0) + + int(r.tokens_cache_read or 0) + + int(r.tokens_cache_write or 0) + for r in rows + ) + items = [] + for r in rows: + ti = int(r.tokens_input or 0) + to_ = int(r.tokens_output or 0) + tcr = int(r.tokens_cache_read or 0) + tcw = int(r.tokens_cache_write or 0) + total = ti + to_ + tcr + tcw + items.append( + { + "team": r.team, + "tokens_input": ti, + "tokens_output": to_, + "total_tokens": total, + "cost_usd": round(float(r.cost_usd or 0.0), 6), + "pct_of_total": round(total / grand_total * 100, 2) + if grand_total > 0 + else 0.0, + } + ) + return items + + # ========================================================================= + # BY-MODEL + # ========================================================================= + + async def get_by_model(self, period: str = "24h") -> list[dict[str, Any]]: + """Return per-model token usage with pct_of_total.""" + start_dt, _ = _parse_period(period) + + result = await self.session.execute( + select( + AgentSpawnSessionTable.model, + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_input), 0).label( + "tokens_input" + ), + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_output), 0).label( + "tokens_output" + ), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_read), 0 + ).label("tokens_cache_read"), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_write), 0 + ).label("tokens_cache_write"), + func.coalesce( + func.sum(AgentSpawnSessionTable.estimated_cost_usd), 0.0 + ).label("cost_usd"), + ) + .where( + AgentSpawnSessionTable.started_at >= start_dt, + AgentSpawnSessionTable.ended_at.isnot(None), + ) + .group_by(AgentSpawnSessionTable.model) + .order_by( + func.sum( + AgentSpawnSessionTable.tokens_input + + AgentSpawnSessionTable.tokens_output + ).desc() + ) + ) + rows = result.fetchall() + + grand_total = sum( + int(r.tokens_input or 0) + + int(r.tokens_output or 0) + + int(r.tokens_cache_read or 0) + + int(r.tokens_cache_write or 0) + for r in rows + ) + items = [] + for r in rows: + ti = int(r.tokens_input or 0) + to_ = int(r.tokens_output or 0) + tcr = int(r.tokens_cache_read or 0) + tcw = int(r.tokens_cache_write or 0) + total = ti + to_ + tcr + tcw + items.append( + { + "model": r.model, + "tokens_input": ti, + "tokens_output": to_, + "total_tokens": total, + "cost_usd": round(float(r.cost_usd or 0.0), 6), + "pct_of_total": round(total / grand_total * 100, 2) + if grand_total > 0 + else 0.0, + } + ) + return items + + # ========================================================================= + # PROJECTION + # ========================================================================= + + async def get_projection(self) -> dict[str, Any]: + """Return projected monthly cost based on 7-day rolling average. + + Computes the average daily cost over the last 7 days and extrapolates + to 30 days. + """ + seven_days_ago = datetime.now(UTC) - timedelta(days=7) + + result = await self.session.execute( + select( + func.coalesce( + func.sum(AgentSpawnSessionTable.estimated_cost_usd), 0.0 + ).label("total_cost_7d"), + func.coalesce(func.count(AgentSpawnSessionTable.id), 0).label( + "session_count" + ), + ).where( + AgentSpawnSessionTable.started_at >= seven_days_ago, + AgentSpawnSessionTable.ended_at.isnot(None), + ) + ) + row = result.one() + total_cost_7d = float(row.total_cost_7d or 0.0) + avg_daily_cost = total_cost_7d / 7.0 + projected_monthly = avg_daily_cost * 30.0 + + return { + "total_cost_7d": round(total_cost_7d, 6), + "avg_daily_cost_usd": round(avg_daily_cost, 6), + "projected_monthly_cost_usd": round(projected_monthly, 4), + "basis_days": 7, + } + + # ========================================================================= + # CACHE EFFICIENCY + # ========================================================================= + + async def get_cache_efficiency(self, period: str = "24h") -> dict[str, Any]: + """Return cache hit rate and estimated savings from prompt caching. + + cache_hit_rate = cache_read_tokens / (input_tokens + cache_read_tokens) + cost_saved = what cache reads would have cost at full input price + minus what they actually cost at cache-read price. + """ + start_dt, _ = _parse_period(period) + + result = await self.session.execute( + select( + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_input), 0).label( + "tokens_input" + ), + func.coalesce(func.sum(AgentSpawnSessionTable.tokens_output), 0).label( + "tokens_output" + ), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_read), 0 + ).label("tokens_cache_read"), + func.coalesce( + func.sum(AgentSpawnSessionTable.tokens_cache_write), 0 + ).label("tokens_cache_write"), + ).where( + AgentSpawnSessionTable.started_at >= start_dt, + AgentSpawnSessionTable.ended_at.isnot(None), + ) + ) + row = result.one() + tokens_input = int(row.tokens_input or 0) + tokens_cache_read = int(row.tokens_cache_read or 0) + tokens_cache_write = int(row.tokens_cache_write or 0) + + total_input_like = tokens_input + tokens_cache_read + cache_hit_rate = ( + tokens_cache_read / total_input_like if total_input_like > 0 else 0.0 + ) + + # Cost saved = (cache_read_tokens * full_input_rate) - actual_cache_read_cost + # Use sonnet as the baseline (most common model) for the aggregate estimate. + # The full input price for sonnet is $3/1M; cache read is $0.30/1M. + _MILLION = 1_000_000.0 + _FULL_INPUT_PRICE = 3.00 # sonnet baseline, USD/1M + _CACHE_READ_PRICE = 0.30 # 10% of input + cost_at_full_price = tokens_cache_read * _FULL_INPUT_PRICE / _MILLION + cost_at_cache_price = tokens_cache_read * _CACHE_READ_PRICE / _MILLION + cost_saved = cost_at_full_price - cost_at_cache_price + + return { + "cache_hit_rate": round(cache_hit_rate, 4), + "tokens_cache_read": tokens_cache_read, + "tokens_cache_write": tokens_cache_write, + "tokens_input": tokens_input, + "cost_saved_by_cache_usd": round(cost_saved, 6), + "period": period, + } + + # ========================================================================= + # TODAY'S USAGE (for CEO dashboard) + # ========================================================================= + + async def get_today_summary(self) -> dict[str, Any]: + """Return today's aggregated usage from daily_usage_rollups. + + Used by the CEO dashboard to populate tokens_today and cost_today_usd. + Falls back to 0 values when no data exists for today. + """ + today = datetime.now(UTC).date() + + result = await self.session.execute( + select( + func.coalesce(func.sum(DailyUsageRollupTable.tokens_input), 0).label( + "tokens_input" + ), + func.coalesce(func.sum(DailyUsageRollupTable.tokens_output), 0).label( + "tokens_output" + ), + func.coalesce( + func.sum(DailyUsageRollupTable.tokens_cache_read), 0 + ).label("tokens_cache_read"), + func.coalesce( + func.sum(DailyUsageRollupTable.tokens_cache_write), 0 + ).label("tokens_cache_write"), + func.coalesce( + func.sum(DailyUsageRollupTable.total_cost_usd), 0.0 + ).label("total_cost_usd"), + ).where(DailyUsageRollupTable.date == today) + ) + row = result.one() + + tokens_today = ( + int(row.tokens_input or 0) + + int(row.tokens_output or 0) + + int(row.tokens_cache_read or 0) + + int(row.tokens_cache_write or 0) + ) + + return { + "tokens_today": tokens_today, + "cost_today_usd": round(float(row.total_cost_usd or 0.0), 6), + } + + +def get_usage_service(db: AsyncSession) -> UsageService: + """Factory function matching the pattern used by other services.""" + return UsageService(db) diff --git a/tests/integration/test_task_service_transitions.py b/tests/integration/test_task_service_transitions.py index a4c36396..2fe512d6 100644 --- a/tests/integration/test_task_service_transitions.py +++ b/tests/integration/test_task_service_transitions.py @@ -623,9 +623,13 @@ async def test_ceo_reject_routes_coordination_task_to_main_pm( rejected = await svc.ceo_reject(task.id, reason="redo the API contract") assert rejected is not None - assert rejected.status == TaskStatus.NEEDS_REVISION + # A coordination root goes to PENDING (the Main PM's claim source), NOT + # needs_revision — that status is developer-claim-only and would deadlock + # the Main PM, which owns the root and must re-plan/re-delegate. + assert rejected.status == TaskStatus.PENDING assert rejected.team == Team.MAIN_PM assert rejected.assigned_to == main_pm_id + assert rejected.claimed_by is None @pytest.mark.asyncio diff --git a/tests/unit/billing/__init__.py b/tests/unit/billing/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/unit/billing/test_pricing.py b/tests/unit/billing/test_pricing.py new file mode 100644 index 00000000..dcfd9cb4 --- /dev/null +++ b/tests/unit/billing/test_pricing.py @@ -0,0 +1,299 @@ +""" +Unit tests for roboco.billing.pricing — calculate_cost(). + +Covers: +- Each model tier (opus, sonnet, haiku) with all 4 token types. +- Unknown model name returns 0.0 without raising. +- Empty model string returns 0.0 without raising. +- Substring match correctness: longer fragment wins + (e.g. 'claude-sonnet-4-6' matches 'claude-sonnet-4' not bare 'sonnet'). +""" + +from __future__ import annotations + +import pytest +from roboco.billing.pricing import calculate_cost + +# --------------------------------------------------------------------------- +# Named constants (ruff PLR2004: magic values in comparisons must be named). +# --------------------------------------------------------------------------- + +# Token counts +_M = 1_000_000 # 1 million tokens + +# Pricing — per-1M USD, matches the _PRICING table in pricing.py +_OPUS_INPUT = 5.00 +_OPUS_OUTPUT = 25.00 +_OPUS_CACHE_READ = 0.50 +_OPUS_CACHE_WRITE = 6.25 + +_SONNET_INPUT = 3.00 +_SONNET_OUTPUT = 15.00 +_SONNET_CACHE_READ = 0.30 +_SONNET_CACHE_WRITE = 0.75 + +_HAIKU_INPUT = 1.00 +_HAIKU_OUTPUT = 5.00 +_HAIKU_CACHE_READ = 0.10 +_HAIKU_CACHE_WRITE = 1.25 + +_HAIKU3_INPUT = 0.25 # claude-haiku-3 is cheaper than haiku-3-5 / haiku-4 + +# Tolerance for floating-point comparisons +_TOL = 1e-4 + + +# --------------------------------------------------------------------------- +# Opus tier +# --------------------------------------------------------------------------- + + +class TestOpusTier: + """claude-opus-4 family pricing.""" + + def test_input_only(self) -> None: + cost = calculate_cost("claude-opus-4-5", tokens_input=_M, tokens_output=0) + assert abs(cost - _OPUS_INPUT) < _TOL + + def test_output_only(self) -> None: + cost = calculate_cost("claude-opus-4-5", tokens_input=0, tokens_output=_M) + assert abs(cost - _OPUS_OUTPUT) < _TOL + + def test_cache_read_only(self) -> None: + cost = calculate_cost( + "claude-opus-4-5", + tokens_input=0, + tokens_output=0, + tokens_cache_read=_M, + ) + assert abs(cost - _OPUS_CACHE_READ) < _TOL + + def test_cache_write_only(self) -> None: + cost = calculate_cost( + "claude-opus-4-5", + tokens_input=0, + tokens_output=0, + tokens_cache_write=_M, + ) + assert abs(cost - _OPUS_CACHE_WRITE) < _TOL + + def test_all_token_types(self) -> None: + cost = calculate_cost( + "claude-opus-4-5", + tokens_input=_M, + tokens_output=_M, + tokens_cache_read=_M, + tokens_cache_write=_M, + ) + expected = _OPUS_INPUT + _OPUS_OUTPUT + _OPUS_CACHE_READ + _OPUS_CACHE_WRITE + assert abs(cost - expected) < _TOL + + def test_short_alias(self) -> None: + """Bare 'opus' alias resolves to the opus tier.""" + cost = calculate_cost("opus", tokens_input=_M, tokens_output=0) + assert abs(cost - _OPUS_INPUT) < _TOL + + def test_returns_float(self) -> None: + cost = calculate_cost("claude-opus-4", tokens_input=100, tokens_output=50) + assert isinstance(cost, float) + + +# --------------------------------------------------------------------------- +# Sonnet tier +# --------------------------------------------------------------------------- + + +class TestSonnetTier: + """claude-sonnet-4 family pricing.""" + + def test_input_only(self) -> None: + cost = calculate_cost("claude-sonnet-4-6", tokens_input=_M, tokens_output=0) + assert abs(cost - _SONNET_INPUT) < _TOL + + def test_output_only(self) -> None: + cost = calculate_cost("claude-sonnet-4-6", tokens_input=0, tokens_output=_M) + assert abs(cost - _SONNET_OUTPUT) < _TOL + + def test_cache_read_only(self) -> None: + cost = calculate_cost( + "claude-sonnet-4-6", + tokens_input=0, + tokens_output=0, + tokens_cache_read=_M, + ) + assert abs(cost - _SONNET_CACHE_READ) < _TOL + + def test_cache_write_only(self) -> None: + cost = calculate_cost( + "claude-sonnet-4-6", + tokens_input=0, + tokens_output=0, + tokens_cache_write=_M, + ) + assert abs(cost - _SONNET_CACHE_WRITE) < _TOL + + def test_all_token_types(self) -> None: + cost = calculate_cost( + "claude-sonnet-4-6", + tokens_input=_M, + tokens_output=_M, + tokens_cache_read=_M, + tokens_cache_write=_M, + ) + expected = ( + _SONNET_INPUT + _SONNET_OUTPUT + _SONNET_CACHE_READ + _SONNET_CACHE_WRITE + ) + assert abs(cost - expected) < _TOL + + def test_short_alias(self) -> None: + """Bare 'sonnet' alias resolves to the sonnet tier.""" + cost = calculate_cost("sonnet", tokens_input=_M, tokens_output=0) + assert abs(cost - _SONNET_INPUT) < _TOL + + def test_35_variant(self) -> None: + """claude-3-5-sonnet resolves to sonnet tier.""" + cost = calculate_cost( + "claude-3-5-sonnet-20241022", tokens_input=_M, tokens_output=0 + ) + assert abs(cost - _SONNET_INPUT) < _TOL + + +# --------------------------------------------------------------------------- +# Haiku tier +# --------------------------------------------------------------------------- + + +class TestHaikuTier: + """claude-haiku family pricing.""" + + def test_input_only(self) -> None: + cost = calculate_cost("claude-haiku-4-5", tokens_input=_M, tokens_output=0) + assert abs(cost - _HAIKU_INPUT) < _TOL + + def test_output_only(self) -> None: + cost = calculate_cost("claude-haiku-4-5", tokens_input=0, tokens_output=_M) + assert abs(cost - _HAIKU_OUTPUT) < _TOL + + def test_cache_read_only(self) -> None: + cost = calculate_cost( + "claude-haiku-4-5", + tokens_input=0, + tokens_output=0, + tokens_cache_read=_M, + ) + assert abs(cost - _HAIKU_CACHE_READ) < _TOL + + def test_cache_write_only(self) -> None: + cost = calculate_cost( + "claude-haiku-4-5", + tokens_input=0, + tokens_output=0, + tokens_cache_write=_M, + ) + assert abs(cost - _HAIKU_CACHE_WRITE) < _TOL + + def test_all_token_types(self) -> None: + cost = calculate_cost( + "claude-haiku-4-5", + tokens_input=_M, + tokens_output=_M, + tokens_cache_read=_M, + tokens_cache_write=_M, + ) + expected = _HAIKU_INPUT + _HAIKU_OUTPUT + _HAIKU_CACHE_READ + _HAIKU_CACHE_WRITE + assert abs(cost - expected) < _TOL + + def test_short_alias(self) -> None: + """Bare 'haiku' alias resolves to the haiku tier.""" + cost = calculate_cost("haiku", tokens_input=_M, tokens_output=0) + assert abs(cost - _HAIKU_INPUT) < _TOL + + def test_haiku3_variant(self) -> None: + """claude-haiku-3 has lower pricing than haiku-3-5.""" + cost = calculate_cost("claude-haiku-3", tokens_input=_M, tokens_output=0) + assert abs(cost - _HAIKU3_INPUT) < _TOL + + +# --------------------------------------------------------------------------- +# Unknown / edge cases — must return 0.0 without raising +# --------------------------------------------------------------------------- + + +class TestUnknownModels: + def test_unknown_model_name_returns_zero(self) -> None: + cost = calculate_cost("gpt-4o", tokens_input=_M, tokens_output=_M) + assert cost == 0.0 + + def test_empty_string_returns_zero(self) -> None: + cost = calculate_cost("", tokens_input=_M, tokens_output=_M) + assert cost == 0.0 + + def test_gibberish_returns_zero(self) -> None: + cost = calculate_cost( + "totally-unknown-model-xyz", tokens_input=100, tokens_output=100 + ) + assert cost == 0.0 + + def test_zero_tokens_with_unknown_model_returns_zero(self) -> None: + cost = calculate_cost("unknown", tokens_input=0, tokens_output=0) + assert cost == 0.0 + + def test_does_not_raise_on_unknown_model(self) -> None: + """Must not raise regardless of token counts.""" + try: + calculate_cost( + "not-a-claude-model", + tokens_input=999_999, + tokens_output=999_999, + ) + except Exception as exc: + pytest.fail(f"calculate_cost raised unexpectedly: {exc}") + + +# --------------------------------------------------------------------------- +# Substring match correctness +# --------------------------------------------------------------------------- + +# Named constants for the comparison floor/ceiling used in these tests. +_ZERO_COST = 0.0 +_SONNET_CHEAPER_THAN_OPUS = True # structural assertion in the test below + + +class TestSubstringMatchPriority: + def test_claude_sonnet_4_resolves_non_zero(self) -> None: + """'claude-sonnet-4-6' must find a match (non-zero cost).""" + cost = calculate_cost("claude-sonnet-4-6", tokens_input=_M, tokens_output=0) + assert cost > _ZERO_COST + + def test_haiku3_cheaper_than_haiku4(self) -> None: + """claude-haiku-3 is cheaper than claude-haiku-4 — longest-match wins.""" + haiku3_cost = calculate_cost("claude-haiku-3", tokens_input=_M, tokens_output=0) + haiku4_cost = calculate_cost("claude-haiku-4", tokens_input=_M, tokens_output=0) + # haiku-3 ($0.25/1M) < haiku-4 ($1.00/1M) + assert haiku3_cost < haiku4_cost + + def test_non_claude_model_returns_zero(self) -> None: + """A random non-Claude model must not match any Claude pricing entry.""" + non_opus_cost = calculate_cost("llama-3-70b", tokens_input=_M, tokens_output=0) + assert non_opus_cost == _ZERO_COST + + def test_opus_model_non_zero(self) -> None: + """Claude opus model resolves to non-zero cost.""" + opus_cost = calculate_cost("claude-opus-4", tokens_input=_M, tokens_output=0) + assert opus_cost > _ZERO_COST + + def test_zero_tokens_returns_zero_for_known_model(self) -> None: + """Known model with 0 tokens has 0 cost.""" + cost = calculate_cost("claude-opus-4", tokens_input=0, tokens_output=0) + assert cost == _ZERO_COST + + def test_case_insensitive_matching(self) -> None: + """Model name matching is case-insensitive.""" + lower_cost = calculate_cost( + "claude-sonnet-4-6", tokens_input=1000, tokens_output=1000 + ) + upper_cost = calculate_cost( + "CLAUDE-SONNET-4-6", tokens_input=1000, tokens_output=1000 + ) + assert lower_cost == upper_cost + assert lower_cost > _ZERO_COST diff --git a/tests/unit/runtime/test_orchestrator_write_hooks.py b/tests/unit/runtime/test_orchestrator_write_hooks.py new file mode 100644 index 00000000..15ea302f --- /dev/null +++ b/tests/unit/runtime/test_orchestrator_write_hooks.py @@ -0,0 +1,543 @@ +""" +Unit tests for orchestrator write-hooks: + _finalize_spawn_session — closes the agent_spawn_sessions DB row on stop + _sweep_token_snapshots — polls active agents and upserts token snapshots + +These tests mock the httpx transport and the SQLAlchemy session factory so no +real network or database is required. + +Coverage: + 1. _finalize_spawn_session success — SDK returns token data → DB update + carries those exact values to calculate_cost and the UPDATE statement. + 2. _finalize_spawn_session HTTP error — SDK unreachable → DB update proceeds + with all-zero token counts (finalization must not raise). + 3. _sweep_token_snapshots active agent — non-zero tokens → snapshot row + inserted and session row updated. + 4. _sweep_token_snapshots per-agent HTTP error — ConnectError on one agent + is caught; the sweep continues and the next agent is still processed. +""" + +from __future__ import annotations + +from contextlib import asynccontextmanager +from pathlib import Path +from typing import Any +from unittest.mock import AsyncMock, MagicMock, patch +from uuid import UUID, uuid4 + +import httpx +from roboco.models.runtime import ( + AgentInstance, + OrchestratorAgentConfig, + OrchestratorAgentState, +) +from roboco.runtime.orchestrator import AgentOrchestrator + +# --------------------------------------------------------------------------- +# Module-level constants (ruff PLR2004: no magic values in comparisons) +# --------------------------------------------------------------------------- + +_AGENT_ID = "be-dev-1" +_AGENT_ID_2 = "be-dev-2" + +# Token counts used in success-path assertions +_TI = 111 # tokens_input +_TO = 222 # tokens_output +_TCR = 33 # tokens_cache_read +_TCW = 44 # tokens_cache_write + +# Token counts for the snapshot test +_SNAP_TI = 50 +_SNAP_TO = 100 +_SNAP_TCR = 10 +_SNAP_TCW = 5 + +# Token counts for the loop-continues test (agent-2) +_LOOP_TI = 25 +_LOOP_TO = 75 + +# Expected number of DB execute() calls for a normal finalize (SELECT + UPDATE) +_FINALIZE_EXEC_CALLS = 2 + + +# --------------------------------------------------------------------------- +# Test helpers +# --------------------------------------------------------------------------- + + +def _make_orchestrator() -> AgentOrchestrator: + """Minimal AgentOrchestrator — no background tasks, no real DB.""" + return AgentOrchestrator(mcp_config_dir=Path("/tmp"), project_root=Path("/tmp")) + + +def _make_instance( + agent_id: str = _AGENT_ID, + usage_session_id: UUID | None = None, +) -> AgentInstance: + """Return an ACTIVE AgentInstance with a running container.""" + return AgentInstance( + agent_id=agent_id, + state=OrchestratorAgentState.ACTIVE, + container_id="abc123def456", + config=OrchestratorAgentConfig( + agent_id=agent_id, + blueprint_path=Path("/tmp/blueprint.md"), + model="sonnet", + ), + usage_session_id=usage_session_id, + ) + + +def _mock_response( + status: int = 200, + json_data: dict[str, Any] | None = None, +) -> MagicMock: + """Build a mock httpx.Response.""" + resp = MagicMock(spec=httpx.Response) + resp.status_code = status + resp.json = MagicMock(return_value=json_data or {}) + return resp + + +def _make_db_factory( + session_row: Any = None, + add_list: list[Any] | None = None, + execute_list: list[Any] | None = None, +) -> Any: + """Return a callable that acts like get_session_factory(). + + The returned callable, when called with no arguments, returns an async + context manager yielding a mock AsyncSession whose execute() returns a + result whose scalar_one_or_none() returns *session_row*. + """ + + @asynccontextmanager + async def _db_context() -> Any: + db = MagicMock() + + result = MagicMock() + result.scalar_one_or_none = MagicMock(return_value=session_row) + + async def _exec(stmt: Any) -> MagicMock: + if execute_list is not None: + execute_list.append(stmt) + return result + + db.execute = AsyncMock(side_effect=_exec) + db.commit = AsyncMock() + + def _add(obj: Any) -> None: + if add_list is not None: + add_list.append(obj) + + db.add = _add if add_list is not None else MagicMock() + yield db + + return _db_context + + +class _FakeHTTPClient: + """Drop-in replacement for ``httpx.AsyncClient`` in tests. + + Accepts a *handler* callable ``(url: str) -> httpx.Response | raises`` + that is invoked by ``get()``. Supports the ``async with`` protocol. + """ + + def __init__(self, handler: Any, **_: Any) -> None: + self._handler = handler + + async def __aenter__(self) -> _FakeHTTPClient: + return self + + async def __aexit__(self, *_: Any) -> None: + pass + + async def get(self, url: str, **_: Any) -> Any: + return self._handler(url) + + +# --------------------------------------------------------------------------- +# _finalize_spawn_session — success path +# --------------------------------------------------------------------------- + + +async def test_finalize_spawn_session_success_calls_calculate_cost() -> None: + """Token values returned by the SDK /usage/status are passed to calculate_cost. + + This verifies the full data-flow: SDK response → token vars → cost calc. + """ + orch = _make_orchestrator() + session_uuid = uuid4() + orch._instances[_AGENT_ID] = _make_instance(usage_session_id=session_uuid) + + token_data = { + "tokens_input": _TI, + "tokens_output": _TO, + "tokens_cache_read": _TCR, + "tokens_cache_write": _TCW, + } + + def _handler(_url: str) -> Any: + return _mock_response(200, token_data) + + session_row = MagicMock() + session_row.id = session_uuid + db_factory = _make_db_factory(session_row=session_row) + + def _client_cls(**_kw: Any) -> _FakeHTTPClient: + return _FakeHTTPClient(_handler) + + with ( + patch("roboco.runtime.orchestrator.httpx.AsyncClient", _client_cls), + patch("roboco.db.base.get_session_factory", return_value=db_factory), + patch("roboco.billing.pricing.calculate_cost", return_value=0.001) as mock_cost, + ): + await orch._finalize_spawn_session(_AGENT_ID, exit_reason="stopped") + + mock_cost.assert_called_once_with( + model="sonnet", + tokens_input=_TI, + tokens_output=_TO, + tokens_cache_read=_TCR, + tokens_cache_write=_TCW, + ) + + +async def test_finalize_spawn_session_success_executes_select_and_update() -> None: + """When a session row exists the function calls execute() twice: SELECT + UPDATE.""" + orch = _make_orchestrator() + session_uuid = uuid4() + orch._instances[_AGENT_ID] = _make_instance(usage_session_id=session_uuid) + + def _handler(_url: str) -> Any: + return _mock_response( + 200, + { + "tokens_input": 10, + "tokens_output": 20, + "tokens_cache_read": 0, + "tokens_cache_write": 0, + }, + ) + + session_row = MagicMock() + session_row.id = session_uuid + execute_calls: list[Any] = [] + db_factory = _make_db_factory(session_row=session_row, execute_list=execute_calls) + + def _client_cls(**_kw: Any) -> _FakeHTTPClient: + return _FakeHTTPClient(_handler) + + with ( + patch("roboco.runtime.orchestrator.httpx.AsyncClient", _client_cls), + patch("roboco.db.base.get_session_factory", return_value=db_factory), + patch("roboco.billing.pricing.calculate_cost", return_value=0.0), + ): + await orch._finalize_spawn_session(_AGENT_ID, exit_reason="completed") + + # SELECT (find the row) + UPDATE (write the values) = 2 execute() calls + assert len(execute_calls) == _FINALIZE_EXEC_CALLS + + +# --------------------------------------------------------------------------- +# _finalize_spawn_session — HTTP-error path +# --------------------------------------------------------------------------- + + +async def test_finalize_spawn_session_http_error_uses_zero_tokens() -> None: + """When the SDK endpoint is unreachable, finalization uses zero tokens. + + The function must not raise; cost must be calculated with all-zero counts. + """ + orch = _make_orchestrator() + session_uuid = uuid4() + orch._instances[_AGENT_ID] = _make_instance(usage_session_id=session_uuid) + + def _boom(_url: str) -> Any: + raise httpx.ConnectError("container not reachable") + + session_row = MagicMock() + session_row.id = session_uuid + db_factory = _make_db_factory(session_row=session_row) + + def _client_cls(**_kw: Any) -> _FakeHTTPClient: + return _FakeHTTPClient(_boom) + + with ( + patch("roboco.runtime.orchestrator.httpx.AsyncClient", _client_cls), + patch("roboco.db.base.get_session_factory", return_value=db_factory), + patch("roboco.billing.pricing.calculate_cost", return_value=0.0) as mock_cost, + ): + # Must not raise even though the SDK is unreachable + await orch._finalize_spawn_session(_AGENT_ID, exit_reason="stopped") + + mock_cost.assert_called_once_with( + model="sonnet", + tokens_input=0, + tokens_output=0, + tokens_cache_read=0, + tokens_cache_write=0, + ) + + +async def test_finalize_spawn_session_non_200_uses_zero_tokens() -> None: + """A non-200 SDK response results in zero-token finalization, no exception.""" + orch = _make_orchestrator() + session_uuid = uuid4() + orch._instances[_AGENT_ID] = _make_instance(usage_session_id=session_uuid) + + def _handler(_url: str) -> Any: + return _mock_response(503) + + session_row = MagicMock() + session_row.id = session_uuid + db_factory = _make_db_factory(session_row=session_row) + + def _client_cls(**_kw: Any) -> _FakeHTTPClient: + return _FakeHTTPClient(_handler) + + with ( + patch("roboco.runtime.orchestrator.httpx.AsyncClient", _client_cls), + patch("roboco.db.base.get_session_factory", return_value=db_factory), + patch("roboco.billing.pricing.calculate_cost", return_value=0.0) as mock_cost, + ): + await orch._finalize_spawn_session(_AGENT_ID, exit_reason="stopped") + + mock_cost.assert_called_once_with( + model="sonnet", + tokens_input=0, + tokens_output=0, + tokens_cache_read=0, + tokens_cache_write=0, + ) + + +# --------------------------------------------------------------------------- +# _sweep_token_snapshots — active agent +# --------------------------------------------------------------------------- + + +async def test_sweep_token_snapshots_inserts_snapshot_for_active_agent() -> None: + """An active agent with non-zero tokens gets a snapshot row added to the DB.""" + orch = _make_orchestrator() + instance = _make_instance(_AGENT_ID) + instance.state = OrchestratorAgentState.ACTIVE + orch._instances[_AGENT_ID] = instance + + token_data = { + "tokens_input": _SNAP_TI, + "tokens_output": _SNAP_TO, + "tokens_cache_read": _SNAP_TCR, + "tokens_cache_write": _SNAP_TCW, + } + + def _handler(_url: str) -> Any: + return _mock_response(200, token_data) + + session_row = MagicMock() + session_row.id = uuid4() + added: list[Any] = [] + db_factory = _make_db_factory(session_row=session_row, add_list=added) + + def _client_cls(**_kw: Any) -> _FakeHTTPClient: + return _FakeHTTPClient(_handler) + + with ( + patch("roboco.runtime.orchestrator.httpx.AsyncClient", _client_cls), + patch("roboco.db.base.get_session_factory", return_value=db_factory), + ): + await orch._sweep_token_snapshots() + + # Exactly one snapshot row must have been passed to db.add() + assert len(added) == 1 + snap = added[0] + assert snap.tokens_input == _SNAP_TI + assert snap.tokens_output == _SNAP_TO + assert snap.tokens_cache_read == _SNAP_TCR + assert snap.tokens_cache_write == _SNAP_TCW + + +async def test_sweep_token_snapshots_skips_zero_token_agents() -> None: + """An agent whose SDK reports all-zero tokens is skipped (no DB writes).""" + orch = _make_orchestrator() + instance = _make_instance(_AGENT_ID) + instance.state = OrchestratorAgentState.ACTIVE + orch._instances[_AGENT_ID] = instance + + def _handler(_url: str) -> Any: + return _mock_response( + 200, + { + "tokens_input": 0, + "tokens_output": 0, + "tokens_cache_read": 0, + "tokens_cache_write": 0, + }, + ) + + added: list[Any] = [] + db_factory = _make_db_factory(add_list=added) + + def _client_cls(**_kw: Any) -> _FakeHTTPClient: + return _FakeHTTPClient(_handler) + + with ( + patch("roboco.runtime.orchestrator.httpx.AsyncClient", _client_cls), + patch("roboco.db.base.get_session_factory", return_value=db_factory), + ): + await orch._sweep_token_snapshots() + + assert added == [] + + +async def test_sweep_token_snapshots_per_agent_error_does_not_abort_loop() -> None: + """A ConnectError for one agent is caught; the next agent is still processed.""" + orch = _make_orchestrator() + + # Agent 1: HTTP error + inst1 = _make_instance(_AGENT_ID) + inst1.state = OrchestratorAgentState.ACTIVE + orch._instances[_AGENT_ID] = inst1 + + # Agent 2: success with non-zero tokens + inst2 = _make_instance(_AGENT_ID_2) + inst2.state = OrchestratorAgentState.ACTIVE + orch._instances[_AGENT_ID_2] = inst2 + + def _handler(url: str) -> Any: + if _AGENT_ID in url and _AGENT_ID_2 not in url: + raise httpx.ConnectError("agent-1 unreachable") + return _mock_response( + 200, + { + "tokens_input": _LOOP_TI, + "tokens_output": _LOOP_TO, + "tokens_cache_read": 0, + "tokens_cache_write": 0, + }, + ) + + session_row = MagicMock() + session_row.id = uuid4() + added: list[Any] = [] + db_factory = _make_db_factory(session_row=session_row, add_list=added) + + def _client_cls(**_kw: Any) -> _FakeHTTPClient: + return _FakeHTTPClient(_handler) + + with ( + patch("roboco.runtime.orchestrator.httpx.AsyncClient", _client_cls), + patch("roboco.db.base.get_session_factory", return_value=db_factory), + ): + await orch._sweep_token_snapshots() + + # Only agent-2's snapshot should be present; agent-1's error was caught. + assert len(added) == 1 + assert added[0].tokens_input == _LOOP_TI + assert added[0].tokens_output == _LOOP_TO + + +# --------------------------------------------------------------------------- +# _sweep_daily_rollup — inserts new row when none exists +# --------------------------------------------------------------------------- + +# Token counts for the rollup test +_ROLLUP_TI = 200 +_ROLLUP_TO = 300 +_ROLLUP_TCR = 20 +_ROLLUP_TCW = 10 + + +async def test_sweep_daily_rollup_inserts_new_row() -> None: + """When no existing DailyUsageRollupTable row exists, db.add() is called + with the correct aggregated token values.""" + orch = _make_orchestrator() + + # Build a fake aggregate result row + agg_row = MagicMock() + agg_row.date = "2026-06-10" + agg_row.agent_slug = _AGENT_ID + agg_row.team = "backend" + agg_row.model = "sonnet" + agg_row.tokens_input = _ROLLUP_TI + agg_row.tokens_output = _ROLLUP_TO + agg_row.tokens_cache_read = _ROLLUP_TCR + agg_row.tokens_cache_write = _ROLLUP_TCW + agg_row.total_cost_usd = 0.0 + agg_row.session_count = 1 + + added: list[Any] = [] + call_count = 0 + + @asynccontextmanager + async def _db_context() -> Any: + nonlocal call_count + + db = MagicMock() + db.commit = AsyncMock() + + def _add(obj: Any) -> None: + added.append(obj) + + db.add = _add + + async def _exec(_stmt: Any) -> MagicMock: + nonlocal call_count + call_count += 1 + result = MagicMock() + if call_count == 1: + # First call: aggregate SELECT — return one agg_row via fetchall() + result.fetchall = MagicMock(return_value=[agg_row]) + result.scalar_one_or_none = MagicMock(return_value=None) + else: + # Second call: lookup SELECT for existing row — return None + result.fetchall = MagicMock(return_value=[]) + result.scalar_one_or_none = MagicMock(return_value=None) + return result + + db.execute = AsyncMock(side_effect=_exec) + yield db + + with patch("roboco.db.base.get_session_factory", return_value=_db_context): + await orch._sweep_daily_rollup() + + # Exactly one new DailyUsageRollupTable row must have been added + assert len(added) == 1 + row = added[0] + assert row.tokens_input == _ROLLUP_TI + assert row.tokens_output == _ROLLUP_TO + assert row.tokens_cache_read == _ROLLUP_TCR + assert row.tokens_cache_write == _ROLLUP_TCW + + +# --------------------------------------------------------------------------- +# stop_agent — _finalize_spawn_session is awaited before acquiring the lock +# --------------------------------------------------------------------------- + + +async def test_stop_agent_finalizes_before_lock() -> None: + """stop_agent awaits _finalize_spawn_session when the instance has a + running container_id (the finalization must happen before the lock).""" + orch = _make_orchestrator() + instance = _make_instance(_AGENT_ID) + instance.container_id = "abc123def456" # non-None → finalize must be called + orch._instances[_AGENT_ID] = instance + + finalized: list[str] = [] + + async def _fake_finalize(agent_id: str, exit_reason: str = "stopped") -> None: # noqa: ARG001 + finalized.append(agent_id) + + # Stub out the Docker subprocess so stop_agent doesn't actually run Docker + mock_proc = MagicMock() + mock_proc.wait = AsyncMock() + + with ( + patch.object(orch, "_finalize_spawn_session", side_effect=_fake_finalize), + patch("asyncio.create_subprocess_exec", AsyncMock(return_value=mock_proc)), + patch.object(orch, "_remove_container", AsyncMock()), + ): + await orch.stop_agent(_AGENT_ID, graceful=True) + + # _finalize_spawn_session must have been called exactly once with our agent id + assert finalized == [_AGENT_ID] diff --git a/tests/unit/services/test_usage.py b/tests/unit/services/test_usage.py new file mode 100644 index 00000000..d7853a61 --- /dev/null +++ b/tests/unit/services/test_usage.py @@ -0,0 +1,761 @@ +""" +Unit tests for roboco.services.usage — UsageService analytics methods. + +These tests mock the SQLAlchemy AsyncSession.execute() boundary and +verify the arithmetic / logic of each analytics method: + +- get_summary: trend_pct edge cases (prev=0, curr=0, both=0, prev>0) +- get_by_agent/team/model: pct_of_total sums to 100% +- get_projection: projected_monthly = avg_daily * 30 +- get_cache_efficiency: cache_hit_rate and cost_saved arithmetic +""" + +from __future__ import annotations + +import datetime +from unittest.mock import AsyncMock, MagicMock + +import pytest +from roboco.services.usage import UsageService + +# --------------------------------------------------------------------------- +# Named constants (ruff PLR2004: magic values in comparisons must be named). +# --------------------------------------------------------------------------- + +# Tolerance for floating-point arithmetic comparisons. +_TOL = 0.001 +# Tolerance for percentage-sum assertions (rounding in pct_of_total). +_PCT_TOL = 0.1 + +# token count helpers +_ZERO = 0 +_M = 1_000_000 + +# Expected values for projection tests +_COST_7D = 70.0 +_EXPECTED_AVG_DAILY = 10.0 # 70 / 7 +_EXPECTED_MONTHLY = 300.0 # 10 * 30 +_DAYS_BASIS = 7 + +# Expected values for cache efficiency tests +_CACHE_READ_TOKENS = 400 +_INPUT_TOKENS = 600 +_EXPECTED_HIT_RATE = 0.4 # 400 / (600 + 400) +_FULL_INPUT_PRICE = 3.00 # sonnet baseline USD/1M +_CACHE_READ_PRICE = 0.30 +_EXPECTED_COST_SAVED = _FULL_INPUT_PRICE - _CACHE_READ_PRICE # = 2.70 per 1M + +# Expected trend_pct values +_TREND_NONE = 0.0 +_TREND_NEW = 100.0 # curr > 0, prev == 0 +_TREND_DOUBLED = 200.0 # curr / prev = 3.0x → +200 % +_TREND_HALVED = -50.0 # curr / prev = 0.5x → -50 % + +# Expected total_tokens when cache tokens are included +_TOTAL_WITH_CACHE = 300 # 100+100+50+50 + +# pct_of_total checks +_FULL_PCT = 100.0 + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _make_row(**kwargs: object) -> MagicMock: + """Return a MagicMock that mimics a SQLAlchemy Row with named attributes.""" + row = MagicMock() + for k, v in kwargs.items(): + setattr(row, k, v) + return row + + +def _result_one(row: MagicMock) -> MagicMock: + """Return a mock execute() result whose .one() returns `row`.""" + result = MagicMock() + result.one = MagicMock(return_value=row) + return result + + +def _result_fetchall(rows: list[MagicMock]) -> MagicMock: + """Return a mock execute() result whose .fetchall() returns `rows`.""" + result = MagicMock() + result.fetchall = MagicMock(return_value=rows) + return result + + +def _service_with_execute(*return_values: object) -> UsageService: + """Build a UsageService whose session.execute() returns the provided + values in sequence (one per call).""" + session = MagicMock() + session.execute = AsyncMock(side_effect=list(return_values)) + return UsageService(session) + + +# --------------------------------------------------------------------------- +# get_summary — trend_pct arithmetic +# --------------------------------------------------------------------------- + + +class TestGetSummaryTrendPct: + @pytest.mark.asyncio + async def test_both_zero_returns_zero_trend(self) -> None: + """When current and previous totals are both 0, trend_pct must be 0.0.""" + current_row = _make_row( + tokens_input=_ZERO, + tokens_output=_ZERO, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + total_cost_usd=0.0, + ) + prev_row = _make_row(total=_ZERO) + svc = _service_with_execute(_result_one(current_row), _result_one(prev_row)) + result = await svc.get_summary("24h") + assert result["trend_pct"] == _TREND_NONE + + @pytest.mark.asyncio + async def test_prev_zero_curr_positive_returns_100(self) -> None: + """When prev period is 0 but current is positive, trend_pct = 100.0.""" + current_row = _make_row( + tokens_input=500, + tokens_output=500, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + total_cost_usd=0.01, + ) + prev_row = _make_row(total=_ZERO) + svc = _service_with_execute(_result_one(current_row), _result_one(prev_row)) + result = await svc.get_summary("24h") + assert result["trend_pct"] == _TREND_NEW + + @pytest.mark.asyncio + async def test_positive_trend_calculation(self) -> None: + """trend_pct = (current - previous) / previous * 100 when prev > 0. + + current = 1500 input + 1500 output = 3000; prev = 1000 + → (3000 - 1000) / 1000 * 100 = 200.0 + """ + current_row = _make_row( + tokens_input=1500, + tokens_output=1500, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + total_cost_usd=0.1, + ) + prev_row = _make_row(total=1000) + svc = _service_with_execute(_result_one(current_row), _result_one(prev_row)) + result = await svc.get_summary("24h") + assert abs(result["trend_pct"] - _TREND_DOUBLED) < _TOL + + @pytest.mark.asyncio + async def test_negative_trend_calculation(self) -> None: + """Negative trend when usage drops. + + current = 250 + 250 = 500; prev = 1000 + → (500 - 1000) / 1000 * 100 = -50.0 + """ + current_row = _make_row( + tokens_input=250, + tokens_output=250, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + total_cost_usd=0.01, + ) + prev_row = _make_row(total=1000) + svc = _service_with_execute(_result_one(current_row), _result_one(prev_row)) + result = await svc.get_summary("24h") + assert abs(result["trend_pct"] - _TREND_HALVED) < _TOL + + @pytest.mark.asyncio + async def test_cache_tokens_included_in_total(self) -> None: + """total_tokens includes cache_read and cache_write tokens.""" + current_row = _make_row( + tokens_input=100, + tokens_output=100, + tokens_cache_read=50, + tokens_cache_write=50, + total_cost_usd=0.005, + ) + prev_row = _make_row(total=_ZERO) + svc = _service_with_execute(_result_one(current_row), _result_one(prev_row)) + result = await svc.get_summary("24h") + assert result["total_tokens"] == _TOTAL_WITH_CACHE + + @pytest.mark.asyncio + async def test_summary_contains_required_fields(self) -> None: + """Response dict must include all required summary fields.""" + current_row = _make_row( + tokens_input=_ZERO, + tokens_output=_ZERO, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + total_cost_usd=0.0, + ) + prev_row = _make_row(total=_ZERO) + svc = _service_with_execute(_result_one(current_row), _result_one(prev_row)) + result = await svc.get_summary("24h") + for field in ("tokens_input", "tokens_output", "total_cost_usd", "trend_pct"): + assert field in result, f"Missing field: {field}" + + +# --------------------------------------------------------------------------- +# get_time_series — total_tokens includes all 4 token types (summary consistency) +# --------------------------------------------------------------------------- + +# Named constants for time-series tests +_TS_INPUT = 100 +_TS_OUTPUT = 200 +_TS_CACHE_READ = 50 +_TS_CACHE_WRITE = 30 +# total = 100 + 200 + 50 + 30 = 380 +_TS_TOTAL_WITH_CACHE = 380 +# Without cache tokens (the old wrong formula): 100 + 200 = 300 +_TS_TOTAL_WITHOUT_CACHE = 300 + + +class TestGetTimeSeries: + @pytest.mark.asyncio + async def test_total_tokens_includes_cache_read_and_write(self) -> None: + """total_tokens in each time-series point must include cache tokens. + + This is the time-series / summary consistency requirement: time-series + total_tokens must sum to the same value as get_summary()'s total_tokens + for the same period. The old implementation used ti + to_ (without + cache), which violated this constraint whenever cache tokens were non-zero. + """ + + bucket_dt = datetime.datetime(2026, 6, 9, 12, 0, 0, tzinfo=datetime.UTC) + row = _make_row( + bucket=bucket_dt, + tokens_input=_TS_INPUT, + tokens_output=_TS_OUTPUT, + tokens_cache_read=_TS_CACHE_READ, + tokens_cache_write=_TS_CACHE_WRITE, + cost_usd=0.01, + ) + svc = _service_with_execute(_result_fetchall([row])) + result = await svc.get_time_series("24h") + assert len(result) == 1 + assert result[0]["total_tokens"] == _TS_TOTAL_WITH_CACHE + + @pytest.mark.asyncio + async def test_total_tokens_without_cache_still_correct(self) -> None: + """When cache tokens are zero, total_tokens == tokens_input + tokens_output.""" + + bucket_dt = datetime.datetime(2026, 6, 9, 12, 0, 0, tzinfo=datetime.UTC) + row = _make_row( + bucket=bucket_dt, + tokens_input=_TS_INPUT, + tokens_output=_TS_OUTPUT, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + cost_usd=0.01, + ) + svc = _service_with_execute(_result_fetchall([row])) + result = await svc.get_time_series("24h") + assert result[0]["total_tokens"] == _TS_INPUT + _TS_OUTPUT + + @pytest.mark.asyncio + async def test_empty_result_returns_empty_list(self) -> None: + svc = _service_with_execute(_result_fetchall([])) + result = await svc.get_time_series("24h") + assert result == [] + + @pytest.mark.asyncio + async def test_point_contains_required_fields(self) -> None: + """Each time-series point must have bucket, tokens_input, tokens_output, + total_tokens, and cost_usd fields.""" + + bucket_dt = datetime.datetime(2026, 6, 9, 12, 0, 0, tzinfo=datetime.UTC) + row = _make_row( + bucket=bucket_dt, + tokens_input=100, + tokens_output=100, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + cost_usd=0.01, + ) + svc = _service_with_execute(_result_fetchall([row])) + result = await svc.get_time_series("24h") + assert len(result) == 1 + point = result[0] + for field in ( + "bucket", + "tokens_input", + "tokens_output", + "total_tokens", + "cost_usd", + ): + assert field in point, f"Missing field: {field}" + + +# --------------------------------------------------------------------------- +# get_by_agent — pct_of_total sums to 100% +# --------------------------------------------------------------------------- + + +class TestGetByAgent: + @pytest.mark.asyncio + async def test_pct_of_total_sums_to_100(self) -> None: + rows = [ + _make_row( + agent_slug="be-dev-1", + tokens_input=600, + tokens_output=400, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.05, + ), + _make_row( + agent_slug="be-dev-2", + tokens_input=300, + tokens_output=200, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.02, + ), + _make_row( + agent_slug="be-qa", + tokens_input=100, + tokens_output=100, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.01, + ), + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_agent("24h") + total_pct = sum(item["pct_of_total"] for item in result) + assert abs(total_pct - _FULL_PCT) < _PCT_TOL + + @pytest.mark.asyncio + async def test_empty_result_returns_empty_list(self) -> None: + svc = _service_with_execute(_result_fetchall([])) + result = await svc.get_by_agent("24h") + assert result == [] + + @pytest.mark.asyncio + async def test_single_agent_has_100_pct(self) -> None: + rows = [ + _make_row( + agent_slug="be-dev-1", + tokens_input=1000, + tokens_output=500, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.1, + ) + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_agent("24h") + assert len(result) == 1 + assert result[_ZERO]["pct_of_total"] == _FULL_PCT + + @pytest.mark.asyncio + async def test_result_contains_agent_slug_field(self) -> None: + rows = [ + _make_row( + agent_slug="be-dev-1", + tokens_input=100, + tokens_output=100, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.01, + ) + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_agent() + assert result[_ZERO]["agent_slug"] == "be-dev-1" + + @pytest.mark.asyncio + async def test_cache_tokens_included_in_total_tokens(self) -> None: + """total_tokens must include cache_read and cache_write. + + Without the fix, total would be 500+300=800 (input+output only). + With the fix, total = 500+300+100+100 = 1000. + """ + _cache_read = 100 + _cache_write = 100 + _expected_total = 500 + 300 + _cache_read + _cache_write # 1000 + rows = [ + _make_row( + agent_slug="be-dev-1", + tokens_input=500, + tokens_output=300, + tokens_cache_read=_cache_read, + tokens_cache_write=_cache_write, + cost_usd=0.05, + ) + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_agent("24h") + assert result[_ZERO]["total_tokens"] == _expected_total + + @pytest.mark.asyncio + async def test_pct_of_total_sums_to_100_with_cache_tokens(self) -> None: + """pct_of_total still sums to 100% when agents have cache tokens.""" + rows = [ + _make_row( + agent_slug="be-dev-1", + tokens_input=400, + tokens_output=200, + tokens_cache_read=150, + tokens_cache_write=50, + cost_usd=0.05, + ), + _make_row( + agent_slug="be-dev-2", + tokens_input=200, + tokens_output=100, + tokens_cache_read=75, + tokens_cache_write=25, + cost_usd=0.02, + ), + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_agent("24h") + total_pct = sum(item["pct_of_total"] for item in result) + assert abs(total_pct - _FULL_PCT) < _PCT_TOL + + +# --------------------------------------------------------------------------- +# get_by_team — pct_of_total sums to 100% +# --------------------------------------------------------------------------- + + +class TestGetByTeam: + @pytest.mark.asyncio + async def test_pct_of_total_sums_to_100(self) -> None: + rows = [ + _make_row( + team="backend", + tokens_input=700, + tokens_output=300, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.05, + ), + _make_row( + team="frontend", + tokens_input=200, + tokens_output=200, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.02, + ), + _make_row( + team="uxui", + tokens_input=100, + tokens_output=100, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.01, + ), + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_team("24h") + total_pct = sum(item["pct_of_total"] for item in result) + assert abs(total_pct - _FULL_PCT) < _PCT_TOL + + @pytest.mark.asyncio + async def test_result_contains_team_field(self) -> None: + rows = [ + _make_row( + team="backend", + tokens_input=100, + tokens_output=100, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.01, + ) + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_team() + assert result[_ZERO]["team"] == "backend" + + @pytest.mark.asyncio + async def test_cache_tokens_included_in_total_tokens(self) -> None: + """total_tokens must include cache_read and cache_write.""" + _cache_read = 200 + _cache_write = 100 + _expected_total = 700 + 300 + _cache_read + _cache_write # 1300 + rows = [ + _make_row( + team="backend", + tokens_input=700, + tokens_output=300, + tokens_cache_read=_cache_read, + tokens_cache_write=_cache_write, + cost_usd=0.05, + ) + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_team("24h") + assert result[_ZERO]["total_tokens"] == _expected_total + + @pytest.mark.asyncio + async def test_pct_of_total_sums_to_100_with_cache_tokens(self) -> None: + """pct_of_total still sums to 100% when teams have cache tokens.""" + rows = [ + _make_row( + team="backend", + tokens_input=600, + tokens_output=200, + tokens_cache_read=120, + tokens_cache_write=80, + cost_usd=0.05, + ), + _make_row( + team="frontend", + tokens_input=300, + tokens_output=100, + tokens_cache_read=60, + tokens_cache_write=40, + cost_usd=0.02, + ), + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_team("24h") + total_pct = sum(item["pct_of_total"] for item in result) + assert abs(total_pct - _FULL_PCT) < _PCT_TOL + + +# --------------------------------------------------------------------------- +# get_by_model — pct_of_total sums to 100% +# --------------------------------------------------------------------------- + + +class TestGetByModel: + @pytest.mark.asyncio + async def test_pct_of_total_sums_to_100(self) -> None: + rows = [ + _make_row( + model="claude-sonnet-4-6", + tokens_input=600, + tokens_output=600, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.1, + ), + _make_row( + model="claude-haiku-4-5", + tokens_input=300, + tokens_output=300, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.02, + ), + _make_row( + model="claude-opus-4-5", + tokens_input=100, + tokens_output=100, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.04, + ), + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_model("24h") + total_pct = sum(item["pct_of_total"] for item in result) + assert abs(total_pct - _FULL_PCT) < _PCT_TOL + + @pytest.mark.asyncio + async def test_result_contains_model_field(self) -> None: + rows = [ + _make_row( + model="claude-sonnet-4-6", + tokens_input=100, + tokens_output=100, + tokens_cache_read=0, + tokens_cache_write=0, + cost_usd=0.01, + ) + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_model() + assert result[_ZERO]["model"] == "claude-sonnet-4-6" + + @pytest.mark.asyncio + async def test_cache_tokens_included_in_total_tokens(self) -> None: + """total_tokens must include cache_read and cache_write.""" + _cache_read = 300 + _cache_write = 100 + _expected_total = 600 + 600 + _cache_read + _cache_write # 1600 + rows = [ + _make_row( + model="claude-sonnet-4-6", + tokens_input=600, + tokens_output=600, + tokens_cache_read=_cache_read, + tokens_cache_write=_cache_write, + cost_usd=0.1, + ) + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_model("24h") + assert result[_ZERO]["total_tokens"] == _expected_total + + @pytest.mark.asyncio + async def test_pct_of_total_sums_to_100_with_cache_tokens(self) -> None: + """pct_of_total still sums to 100% when models have cache tokens.""" + rows = [ + _make_row( + model="claude-sonnet-4-6", + tokens_input=500, + tokens_output=500, + tokens_cache_read=200, + tokens_cache_write=100, + cost_usd=0.1, + ), + _make_row( + model="claude-haiku-4-5", + tokens_input=250, + tokens_output=250, + tokens_cache_read=100, + tokens_cache_write=50, + cost_usd=0.02, + ), + ] + svc = _service_with_execute(_result_fetchall(rows)) + result = await svc.get_by_model("24h") + total_pct = sum(item["pct_of_total"] for item in result) + assert abs(total_pct - _FULL_PCT) < _PCT_TOL + + +# --------------------------------------------------------------------------- +# get_projection — formula: projected_monthly = (total_7d / 7) * 30 +# --------------------------------------------------------------------------- + + +class TestGetProjection: + @pytest.mark.asyncio + async def test_projection_formula_30_day_extrapolation(self) -> None: + """projected_monthly_cost_usd = (total_cost_7d / 7) * 30.""" + row = _make_row(total_cost_7d=_COST_7D, session_count=10) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_projection() + assert abs(result["projected_monthly_cost_usd"] - _EXPECTED_MONTHLY) < _TOL + + @pytest.mark.asyncio + async def test_zero_cost_7d_gives_zero_projection(self) -> None: + row = _make_row(total_cost_7d=0.0, session_count=_ZERO) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_projection() + assert result["projected_monthly_cost_usd"] == 0.0 + + @pytest.mark.asyncio + async def test_avg_daily_cost_equals_total_over_7(self) -> None: + """avg_daily = total_7d / 7.""" + row = _make_row(total_cost_7d=21.0, session_count=5) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_projection() + # 21 / 7 = 3.0 avg daily cost + _avg_daily_21 = 3.0 + assert abs(result["avg_daily_cost_usd"] - _avg_daily_21) < _TOL + + @pytest.mark.asyncio + async def test_projection_contains_required_fields(self) -> None: + row = _make_row(total_cost_7d=7.0, session_count=3) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_projection() + for field in ( + "total_cost_7d", + "avg_daily_cost_usd", + "projected_monthly_cost_usd", + "basis_days", + ): + assert field in result, f"Missing field: {field}" + assert result["basis_days"] == _DAYS_BASIS + + +# --------------------------------------------------------------------------- +# get_cache_efficiency — hit rate and cost_saved arithmetic +# --------------------------------------------------------------------------- + + +class TestGetCacheEfficiency: + @pytest.mark.asyncio + async def test_cache_hit_rate_formula(self) -> None: + """cache_hit_rate = cache_read / (input + cache_read). + + 400 cache reads out of 400+600 total = 0.4 + """ + row = _make_row( + tokens_input=_INPUT_TOKENS, + tokens_output=_ZERO, + tokens_cache_read=_CACHE_READ_TOKENS, + tokens_cache_write=_ZERO, + ) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_cache_efficiency("24h") + assert abs(result["cache_hit_rate"] - _EXPECTED_HIT_RATE) < _TOL + + @pytest.mark.asyncio + async def test_zero_input_tokens_gives_zero_hit_rate(self) -> None: + """When no input or cache_read tokens, hit rate is 0.0.""" + row = _make_row( + tokens_input=_ZERO, + tokens_output=_ZERO, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + ) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_cache_efficiency("24h") + assert result["cache_hit_rate"] == 0.0 + + @pytest.mark.asyncio + async def test_full_cache_hit_gives_rate_of_1(self) -> None: + """When all input-like tokens are cache reads, hit rate = 1.0.""" + _full_rate = 1.0 + row = _make_row( + tokens_input=_ZERO, + tokens_output=_ZERO, + tokens_cache_read=1000, + tokens_cache_write=_ZERO, + ) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_cache_efficiency("24h") + assert abs(result["cache_hit_rate"] - _full_rate) < _TOL + + @pytest.mark.asyncio + async def test_cost_saved_arithmetic(self) -> None: + """cost_saved = cache_read * (full_input_price - cache_read_price) / 1M. + + Sonnet baseline: full=$3.00/1M, cache_read=$0.30/1M. + For 1M cache-read tokens: saved = 3.00 - 0.30 = 2.70. + """ + row = _make_row( + tokens_input=_ZERO, + tokens_output=_ZERO, + tokens_cache_read=_M, + tokens_cache_write=_ZERO, + ) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_cache_efficiency("24h") + assert abs(result["cost_saved_by_cache_usd"] - _EXPECTED_COST_SAVED) < _TOL + + @pytest.mark.asyncio + async def test_zero_cache_reads_gives_zero_savings(self) -> None: + row = _make_row( + tokens_input=1000, + tokens_output=500, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + ) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_cache_efficiency("24h") + assert result["cost_saved_by_cache_usd"] == 0.0 + + @pytest.mark.asyncio + async def test_cache_efficiency_contains_required_fields(self) -> None: + row = _make_row( + tokens_input=_ZERO, + tokens_output=_ZERO, + tokens_cache_read=_ZERO, + tokens_cache_write=_ZERO, + ) + svc = _service_with_execute(_result_one(row)) + result = await svc.get_cache_efficiency("24h") + for field in ("cache_hit_rate", "cost_saved_by_cache_usd"): + assert field in result, f"Missing field: {field}"