2 Commits

Include renames

Author	SHA1	Message	Date
3 people	ff35a646fa	Chore: reduce analytics complexity (#100) ... * refactor(analytics): reduce cyclomatic complexity in usage/pricing/rollup Collapse the three near-identical get_by_* aggregation methods in UsageService into a shared _aggregate_by helper parameterized by group column and key name, and centralize token null-coalescing in a _row_tokens helper. Extract the per-row upsert in _sweep_daily_rollup into _upsert_rollup_row, and the pricing-table lookup into _lookup_prices. All blocks now rank <= B and both modules rank A, so the xenon gate passes; behavior is unchanged and existing tests stay green. * feat(billing): make token pricing provider-aware Distinguish three cases when a model has no per-token rate: a non-Anthropic model (local Ollama, or an Ollama Cloud ":cloud" model billed by flat subscription / GPU-time) legitimately has no per-token cost and returns 0.0 silently; an unpriced Anthropic ("claude"-named) model also returns 0.0 but logs a warning, since that is real spend being undercounted and catches new or renamed Claude models missing from the table. Folds the old ollama/ prefix special-case into the general non-Anthropic path so there is one code path, and replaces the blanket 'no pricing data' warning that fired even for self-hosted models. * fix(tasks): preserve ownership when force-unclaiming to pending The stale-claim reaper and the dependency-blocked release both routed through _force_unclaim_to_pending, which nulled assigned_to and left the task in a pending state owned by nobody — no dispatcher re-spawns an ownerless pending task, so it went dormant. The dispatcher-side claimed_by fallback only masked half the cases. Capture the owner before releasing the claim and keep both assigned_to and claimed_by pointed at it (mirroring the unblock restore), releasing only the live claim (active_claimant_id + heartbeat) and the WorkSession. The same agent now resumes the task once it re-dispatches. Updates the reaper test that asserted the old orphaning behavior and adds owner-preservation coverage for both the reaper and dependency-release paths. * fix(tasks): unblock restores the owner into both ownership fields Audit follow-up to the force-unclaim ownership fix. unblock() only restored assigned_to from blocker_raised_by, which block() stashes solely from assigned_to. A task claimed via give_me_work (claimed_by set, assigned_to null) therefore unblocked into a split-owner state — assigned_to null but claimed_by set — that both the dev dispatcher and the PM pool-router race to pick up. It also left claimed_by pointing at the resolver PM after an escalation. Resolve the owner as blocker_raised_by or assigned_to or claimed_by and write it to both fields, matching the force-unclaim and reassign convention so the original worker resumes cleanly. Adds coverage for the give_me_work-claim case and asserts owner restoration on the existing in_progress-resume test. * test(orchestrator): cover dev owner resolution and the claimed_by fallback _resolve_dev_owner_uuid had no coverage. Add the status-dependent precedence (claimed/blocked prefer the live claimant; other statuses prefer the PM-assigned owner) and the half-reap fallback where a pending task with assigned_to nulled still resolves its owner from claimed_by instead of going dormant. * fix(tasks): wire the pre-block snapshot so unblock(restore=True) works The restore=True path on a PM unblock was a no-op: pre_block_state / pre_block_assignee (migration 006) were read by unblock_with_restore but never written, so it always fell through to legacy unblock() and the restore flag did nothing. Snapshot the resting status + owner at every block entry (dependency block, soft block, escalation) before mutating, capturing only the first block in a chain so a re-block doesn't overwrite the original state. Escalation snapshots the outgoing owner, not the escalation target, so restore returns the original worker. The restore path applies the same branchless guard legacy unblock() relies on — a snapshotted in_progress with no branch diverts to pending instead of looping the dispatcher — and is extracted into _apply_pre_block_restore to keep complexity under the gate. Adds coverage for snapshot capture, restore, the branchless divert, and escalation owner restoration. * test(tasks): update orphan-reconciler and dependency-release tests for owner preservation Both the startup orphan reconciler and the dependency-blocked claim release route through unclaim_for_reaper / _force_unclaim_to_pending, which now preserve the owner instead of nulling assigned_to. Update the two tests that asserted the old orphaning behavior to assert the owner is kept (so the same agent resumes) while the live claim is released. * chore(tests): scrub internal work-item labels from test names, docstrings, comments Rename four test files that carried audit work-item IDs in their filenames (test_p0_7_branch_atomicity, test_p2_8_orphan_reconciler, test_p2_9_autogen_prompt_layer, test_p2_7_attempt_id) to describe what they test, and strip the matching P-/D-/S- cluster labels from docstrings, comments, and assertion messages across the test suite and two orchestrator comments. These are internal references with no meaning in the codebase; behavior is unchanged. * style: reformat assertion line shortened by the internal-ref scrub * build: waive unreachable torch CVE-2025-3000 in pip-audit gate torch is a transitive CPU-pinned dep (piragi / sentence-transformers) never loaded at runtime — the stack uses Ollama over HTTP for all embeddings/LLM, so the vulnerable torch.jit.script path is unreachable. CVE-2025-3000 is MEDIUM, local-only, with no published fix. Documented --ignore-vuln waiver; revisit when a fixed torch ships. * fix(orchestrator): route unplaceable pending tasks to main-pm instead of dropping them _get_routing_target returned None when a 'dev'-classified task had no cell agent (no team, or a non-cell team like fullstack/system) or when the routing classification was unrecognized. _route_unassigned_pm_task logged 'no routing target found' and returned, leaving the task ownerless and pending — and no dispatcher re-spawns an unrouted pending task, so it went dormant for 10+ min until the stuck-task detector caught it. Fall back to main-pm (the same default cell_pm routing and escalation already use) so the task is always owned and triaged, never stranded. Logs the fallback so unplaceable tasks stay visible. Adds a test asserting no (routing, team) combination ever resolves to None. * fix(panel): make intake chat markdown inherit the bubble's text color MarkdownBody is shared by the assistant (text-foreground) and user (text-primary-foreground) bubbles. [&_*]:!text-inherit only colored the prose div's descendants, so the prose div itself kept the prose typography body color (gray) and children inherited that — unreadable on the muted assistant bubble. Add !text-inherit on the prose div itself so it inherits the bubble's color too; descendants then inherit the correct foreground. Fixes both bubbles without hardcoding a color. * fix(prompter): keep a board-reviewed product on the board team so Approve & Start shows A product coordination root confirmed via 'Board review & Start' is assigned to a board reviewer (product-owner) for review, but create_task_from_draft set team=main_pm for every product unconditionally. The CEO's Approve & Start gate keys on team=board, so the button never appeared — and because the owner stayed a board agent while the team said main_pm, the dispatcher routed it to the board path (nothing left to do after review) and the task stranded at pending, with the board agent fruitlessly trying to escalate it up. Route a product by its assignee: a board reviewer keeps it team=board (so the gate appears and approve_and_start later hands it to Main PM), while a main-pm assignee — the 'Approve & Start' straight-through path — is team=main_pm. Adds _assignee_is_board mirroring TaskService's board-role check, and a test. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-06-11 04:36:17 +02:00
5 people	b3057628b0	[499f9eb1] Token Usage & Cost Analytics — Full-Stack Instrumentation, Persistence, and Visualization (#90) ... * [cd2bf666] feat(usage): add token usage types, API client, hooks, and UI components (#87) (#88) - Append 5 TypeScript interfaces to src/types/index.ts: TokenUsageSnapshot, AgentUsageRow, UsageSession, UsageTimePoint, ModelUsageSlice - Create src/lib/api/usage.ts: Axios singleton + isMockMode guards for getUsageSnapshot, getUsageTimeSeries, getAgentUsage, getUsageSessions, getModelUsage - Create src/hooks/use-usage.ts: usageKeys factory + useUsageSnapshot, useUsageTimeSeries, useAgentUsage, useUsageSessions, useModelUsage hooks - Create UsageOverviewPanel (dashboard/usage-overview-panel.tsx): 6 metric rows with Skeleton loading state; week-over-week trend arrow for cost - Update CommandCenter: Metrics+Alerts row expanded from 2-col to 3-col grid adding UsageOverviewPanel - Create src/components/metrics/ folder: UsageTimeSeriesChart (recharts stacked AreaChart with var(--chart-1/2/3)), ModelUsageDonut (PieChart), AgentUsageChart and TeamUsageChart (BarChart), SessionsTable (sortable columns + 10-row Prev/Next pagination) - Update Metrics page: Token Usage & Costs section with 5 rows (summary cards, time series+donut, agent+team bar charts, projection+cache efficiency, sessions table) - Add usage mini-bar to AgentCard: token count + cost + progress bar; AgentGrid and Agents page pass agentUsageMap through - Install recharts 3.8.1 - Export all new symbols through their barrel index.ts files Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> * [10372f0f] Implement full token usage instrumentation: DB migration, SDK endpoints, orchestrator hooks, analytics API, WebSocket events, dashboard integration (#86) (#89) * [10372f0f] feat(token-usage): add Alembic migration 026 for token usage tables Create agent_spawn_sessions, token_usage_snapshots, and daily_usage_rollups tables with correct BIGINT columns, indexes, and unique constraint. Chain: 025_agentrole_prompter → 026_token_usage_tables. * [10372f0f] feat(token-usage): add ORM table classes for token usage instrumentation Add AgentSpawnSessionTable, TokenUsageSnapshotTable, DailyUsageRollupTable to db/tables.py. Import BigInteger and Date from SQLAlchemy. All columns match the migration schema with BIGINT token counts and proper indexes. * [10372f0f] feat(billing): add pricing module with calculate_cost() function Create roboco/billing/__init__.py and roboco/billing/pricing.py with calculate_cost() supporting Claude opus/sonnet/haiku models with input/output/cache pricing. Unknown models return 0.0 without raising. * [10372f0f] feat(sdk): add POST /usage/report and GET /usage/status endpoints to agent SDK Extend _SessionState with token counters. Add TokenReportRequest and TokenUsageStatus models. POST /usage/report additively accumulates token counts; GET /usage/status returns current session totals for sweeper polling. * [10372f0f] feat(orchestrator): add token usage instrumentation hooks - _launch_spawn() calls _record_spawn_session() after successful container spawn - stop_agent() calls _finalize_spawn_session() before container removal - _run_sweep() calls _sweep_token_snapshots() and _sweep_daily_rollup() each tick - New methods: _record_spawn_session, _finalize_spawn_session, _sweep_token_snapshots, _sweep_daily_rollup in TOKEN USAGE section * [10372f0f] feat(api): add token usage analytics API with 7 endpoints Create roboco/services/usage.py (UsageService) and roboco/api/routes/usage.py. Endpoints: GET /api/usage/summary, /time-series, /by-agent, /by-team, /by-model, /projection, /cache-efficiency. Register in app.py. * [10372f0f] feat(dashboard): add usage_summary field to CEO dashboard Add UsageSummary schema (tokens_today, cost_today_usd) to dashboard schemas. Add usage_summary: UsageSummary | None to CEOOverview. Update get_ceo_overview() to populate usage_summary from daily_usage_rollups. * [10372f0f] fix(billing/tests): remove dead except block in _sweep_daily_rollup, add unit tests for pricing.py and services/usage.py - Remove unreachable `except Exception as e` block in orchestrator.py _sweep_daily_rollup() (lines 3376-3381) which referenced undefined `agent_id` and was copy-pasted from _sweep_token_snapshots by mistake - Add tests/unit/billing/test_pricing.py: 31 tests covering opus/sonnet/ haiku tiers with all 4 token types, unknown model → 0.0, empty string → 0.0, and substring-match priority (longer fragment wins) - Add tests/unit/services/test_usage.py: 25 tests covering get_summary trend_pct edge cases (prev=0, both=0, prev>0), get_by_agent/team/model pct_of_total summing to 100%, get_projection formula (avg_daily×30), and get_cache_efficiency hit-rate and cost_saved arithmetic - pricing.py: 100% coverage; services/usage.py: 83% coverage (>80% target) * [10372f0f] fix(usage): include cache tokens in time-series total_tokens to fix AC9 consistency violation get_time_series() previously computed total_tokens as tokens_input + tokens_output only. get_summary() includes all 4 token types (input + output + cache_read + cache_write). AC9 requires both endpoints to agree on their totals for the same period. Fix: add tokens_cache_read and tokens_cache_write to the SELECT query in get_time_series() and include them in the total_tokens calculation. Also adds 4 new unit tests in TestGetTimeSeries covering: - total_tokens includes cache_read and cache_write (the AC9 guard) - zero cache tokens still produces correct total - empty result returns empty list - required fields are present in each point * [10372f0f] fix(usage): remove unused imports and include cache tokens in breakdown totals (AC10) - Remove import math (F401 — never used) - Remove text from sqlalchemy import (F401 — never used) - Remove unused local calculate_cost import inside get_cache_efficiency (F401) - Add tokens_cache_read and tokens_cache_write to SELECT in get_by_agent, get_by_team, and get_by_model; update grand_total and per-item total to include all 4 token types so totals match get_summary() (AC10 fix) - Update test mock rows to include explicit tokens_cache_read=0 and tokens_cache_write=0 so they work with the fixed code - Add new test cases: test_cache_tokens_included_in_total_tokens and test_pct_of_total_sums_to_100_with_cache_tokens for each breakdown class --------- Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * [44b9eb1f] feat(usage): align frontend API client, TS types, and chart components to real backend contract (#92) (#94) Update all usage-related frontend code to match the actual FastAPI backend response shapes and endpoint paths: - panel/src/lib/api/usage.ts: rewrite all 7 API functions to use correct endpoint paths (/usage/summary, /usage/by-agent, /usage/by-model, /usage/by-team, /usage/time-series, /usage/projection, /usage/cache-efficiency); send period query param (24h/7d/30d not hours); mock generators produce data matching real backend shapes exactly; getUsageSessions returns [] in prod (no /usage/sessions endpoint exists) - panel/src/types/index.ts: replace TokenUsageSnapshot with UsageSummary (tokens_input/tokens_output/total_cost_usd/trend_pct); update AgentUsageRow to use agent_slug/total_tokens/cost_usd/pct_of_total; add TeamUsageRow, UsageProjection, CacheEfficiencyResponse; update UsageTimePoint to use bucket field; update UsageSession to use agent_slug - panel/src/hooks/use-usage.ts: rewrite all hooks to match new API and types; add useTeamUsage, useUsageProjection, useCacheEfficiency hooks - panel/src/components/metrics/usage-time-series-chart.tsx: use bucket field (not timestamp) for axis labels - panel/src/components/metrics/agent-usage-chart.tsx: use agent_slug and total_tokens (not agent_name/tokens_today) - panel/src/components/metrics/team-usage-chart.tsx: rewrite to accept TeamUsageRow[] from API directly - panel/src/components/metrics/model-usage-donut.tsx: use total_tokens, cost_usd, pct_of_total (not tokens/cost/percentage) - panel/src/components/metrics/sessions-table.tsx: use agent_slug, sort keys updated - panel/src/components/dashboard/usage-overview-panel.tsx: use useUsageSummary with tokens_input/tokens_output/total_cost_usd/trend_pct - panel/src/app/(dashboard)/metrics/page.tsx: wire all new hooks, add TeamUsageChart, ProjectionCard, CacheEfficiencyCard with correct types - panel/src/app/(dashboard)/agents/page.tsx: key agentUsageMap by agent_slug - panel/src/components/agents/agent-card.tsx: use total_tokens and cost_usd Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> * [2161b832] fix: SDK_PORT constant, stop_agent lock refactor, usage_session_id binding, rollup 7-day window (#93) (#95) - Add SDK_PORT = 9000 module-level constant to orchestrator.py; replace hardcoded 9000 in _sweep_budget_exceeded URL with SDK_PORT - Add UUID to TYPE_CHECKING imports to satisfy ruff F821 - Refactor stop_agent: call _finalize_spawn_session BEFORE acquiring self._lock so the SDK HTTP round-trip does not hold the lock - Add usage_session_id: UUID | None field to AgentInstance dataclass - Change _record_spawn_session to return UUID | None; wire return value back to instance.usage_session_id in _launch_spawn - Update _finalize_spawn_session to use WHERE id=usage_session_id for direct session row lookup when usage_session_id is not None - Add started_at >= (now_utc - 7 days) filter to _sweep_daily_rollup aggregate query to avoid re-aggregating all-time history each sweep Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * [2e0759e1] fix: pricing accuracy, import ordering, session-id binding, rollup cleanup, write-hook tests (#97) (#98) - pricing.py: correct claude-opus-4 prices (5/25/0.50/6.25 not 15/75/1.5/3.75) and haiku family prices (1/5/0.10/1.25 not 0.8/4/0.08/0.20); add Ollama zero-cost early-return; add structlog warning for unmatched model names - app.py: move usage_router import before routes.v1 block (ruff isort fix) - orchestrator.py _sweep_daily_rollup: remove unused calculate_cost import; add blank line between stdlib (uuid4) and third-party (sqlalchemy) imports - orchestrator.py _sweep_token_snapshots: prefer direct lookup by instance.usage_session_id; fall back to agent_slug heuristic only when None - tests: add test_sweep_daily_rollup_inserts_new_row and test_stop_agent_finalizes_before_lock to test_orchestrator_write_hooks.py - usage.py, routes/usage.py, stream_bus.py, test files: ruff format/lint fixes Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * Mypy compliance * fix(migrations,tests): linearize forked migration chain + correct ceo_reject coordination-root expectation The master merge brought in 026_completed_dependency_ids alongside the rework's 026_token_usage_tables — both off 025, forking the alembic head and breaking the enum-parity test. Rebase token-usage onto 026_completed_dependency_ids (linear chain, single head). Also: test_ceo_reject_routes_coordination_task_to_main_pm asserted the old NEEDS_REVISION behavior; the lifecycle fix correctly routes a coordination root to PENDING (Main PM's claim source). Update the assertion. --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-06-10 14:38:44 +02:00