docs: document rate-limit handling, token usage, /ws/system, and the workspace toolchain

Record the features that landed this cycle:
- CHANGELOG: provider rate-limit handling, token usage & cost analytics, the
  /ws/system operator stream; plus the fixes (agent gate toolchain, usage
  capture, panel endpoint shape + WS path, /public 500, provider pricing).
- CLAUDE.md: a WebSocket-streams section (incl. /ws/system + the
  websocket_bridge pattern), a Rate-limiting & usage subsystem note, and the
  'uv sync --extra dev' workspace-toolchain requirement.
- agent API reference: a System & realtime section (/api/system/rate-limits,
  /ws/system, per-resource WS streams).
This commit is contained in:
Renn F
2026-06-11 19:05:44 +02:00
parent 08540107e5
commit 9cbfb5f0dd
3 changed files with 77 additions and 0 deletions
+33
View File
@@ -7,6 +7,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Released] ## [Released]
### Added
- **Provider rate-limit handling.** End-to-end backpressure for LLM-provider
429s: a Redis-backed `RateLimitStateTracker`, a spawn gate that **queues**
(never drops) work while a provider is rate-limited, agent parking via
`i_am_blocked(reason="rate_limited")`, and a background probe-and-resume loop
that auto-revives parked agents when the limit lifts — escalating to the CEO
after repeated failed probes. Surfaced live in the panel via a rate-limit
banner.
- **Token usage & cost analytics.** Per-agent-session token capture read from
the Claude Code transcript (`/usage/sync`), persisted to spawn-session rows
and daily rollups, with provider-aware pricing (Anthropic models priced;
local/Ollama models intentionally $0). Visible on the usage dashboard.
- **`/ws/system` operator WebSocket stream** with a `websocket_bridge` that
forwards system events (the rate-limit lifecycle) from the event bus to panel
clients in real time.
### Fixed
- Agent workspaces now install the project's `dev` extra (`uv sync --extra dev`)
so spawned agents have the full `make quality` toolchain (ruff/mypy/xenon) and
can gate their own work — closing the gap that let lint/type/complexity debt
merge unchecked.
- Token-usage capture: the dashboard previously recorded zeros because nothing
populated the per-session counters.
- Panel rate-limit endpoint shape (`/api/system/rate-limits` returns the
`{ entries: [...] }` envelope the dashboard expects) and the doubled
`/ws/ws/system` WebSocket path.
- Control-panel logo and all `/public` assets returning 500 — the panel image
copied them without chowning to the non-root runtime user.
- Provider-aware pricing (Opus corrected to $5/$25 per 1M; non-Anthropic models
no longer warn or mis-price).
## [0.1.0] - 2026-06-09 ## [0.1.0] - 2026-06-09
### Added ### Added
+36
View File
@@ -120,6 +120,12 @@ repo (no longer a separate `roboco-panel` project or workspace).
- `ROBOCO_WORKSPACE_AUTO_CLONE`: Auto-clone repos on first access (default: `true`) - `ROBOCO_WORKSPACE_AUTO_CLONE`: Auto-clone repos on first access (default: `true`)
- `ROBOCO_WORKSPACE_CLONE_TIMEOUT`: Clone timeout in seconds (default: `300`) - `ROBOCO_WORKSPACE_CLONE_TIMEOUT`: Clone timeout in seconds (default: `300`)
On a Python workspace, `WorkspaceService` runs `uv sync --extra dev` (not plain
`uv sync`) so the clone's `.venv` carries the full gate toolchain
(ruff/mypy/xenon/pytest) — the lint/type/complexity tools live in the `dev`
**extra**, which plain `uv sync` skips. Without it an agent's `make quality`
fails on `ruff: command not found` and the agent can't gate its own work.
## Git Workflow ## Git Workflow
### Branch Naming Convention ### Branch Naming Convention
@@ -458,6 +464,36 @@ The system runs as Docker Compose services. All Dockerfiles live under
This avoids CORS since the browser sees one origin. The Next.js code uses This avoids CORS since the browser sees one origin. The Next.js code uses
relative URLs (`/api`, `/ws`) and lets nginx do the dispatch. relative URLs (`/api`, `/ws`) and lets nginx do the dispatch.
### WebSocket streams
The orchestrator exposes WebSocket endpoints under `/ws` (router in
`roboco/api/websocket.py`, `ConnectionManager` + `broadcast_*` helpers):
| Endpoint | Purpose |
|----------|---------|
| `/ws/channels/{id}`, `/ws/agents/{id}`, `/ws/sessions/{id}`, `/ws/notifications/{id}` | Per-resource live streams |
| `/ws/system` | Operator/system-wide stream (no per-agent keying) — currently the rate-limit lifecycle (`RATE_LIMIT_HIT` / `RATE_LIMIT_LIFTED`) |
Server-side events reach these sockets through `roboco/api/websocket_bridge.py`,
which subscribes to the `StreamEventBus` and forwards each event to the matching
connections. To add a new live event: define an `EventType` (dotted value),
publish it to the bus, add a `_handle_*` forwarder in `websocket_bridge`, and
consume it on the panel via the `useWebSocket("/<endpoint>", …)` hook — do not
stand up a parallel endpoint or client stack.
### Rate limiting & usage
- **Provider rate limits** are tracked in Redis (`RateLimitStateTracker`,
`roboco/services/gateway/`). On a provider 429 an agent calls
`i_am_blocked(reason="rate_limited")`; the spawn gate then **queues** (never
drops) further work for that provider, and a background probe-and-resume loop
in the orchestrator clears the limit and revives parked agents when it lifts.
- **Token usage** is captured per agent session from the Claude Code transcript
via the SDK server's `/usage/sync` (hook → orchestrator finalize →
`agent_spawn_sessions``daily_usage_rollups` → dashboard). Cost uses
provider-aware pricing in `roboco/billing/pricing.py` (Anthropic priced;
local/Ollama intentionally `$0`).
### Startup Sequence ### Startup Sequence
The startup order is critical due to dependencies: The startup order is critical due to dependencies:
+8
View File
@@ -85,6 +85,14 @@ Base URL: `http://{host}:{port}/api/v1`
| GET | `/journals/me/stats` | My stats | | GET | `/journals/me/stats` | My stats |
| GET | `/journals/{agent}/entries` | Read team journal | | GET | `/journals/{agent}/entries` | Read team journal |
## System & realtime
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/system/rate-limits` | Active per-provider rate-limit state (`{ entries: [...] }`) |
| WS | `/ws/system` | Operator stream — rate-limit lifecycle (`RATE_LIMIT_HIT` / `RATE_LIMIT_LIFTED`) |
| WS | `/ws/agents/{id}`, `/ws/channels/{id}`, `/ws/sessions/{id}`, `/ws/notifications/{id}` | Per-resource live streams |
## Documentation ## Documentation
| Method | Endpoint | Description | | Method | Endpoint | Description |