External/roboco
1
0
Fork 0
mirror of https://github.com/rennf93/roboco.git synced 2026-08-03 07:23:24 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: add the user-facing MkDocs documentation site

Browse Source 
Build a complete user-facing documentation site (MkDocs Material) under docs/, served at roboco.dev/docs via a new gh-pages deploy workflow.

- Sections: Get Started, The Company, the Tour, Operating the Panel, Choosing & Running Models, Cost & Observability, Optional Subsystems, Configure & Deploy, API Reference, Troubleshooting & Security (55 pages).
- mkdocs.yml (Material theme; excludes the agent-facing rag/ corpus, internal scratch, and orphaned stub trees) and .github/workflows/docs.yml (mkdocs gh-deploy to gh-pages).
- Retire the stale root usage.md and deployment.md to redirect stubs into the site.
- Fix the docs tooling: add the pymarkdownlnt dependency + .pymarkdown.json, run serve-docs/lint-docs/fix-docs under the docs extra, add a build-docs strict gate.
- Fix the roboco console-script entry point (cli, not the un-awaited async main).
- README: correct the project-structure tree (optimal.py, alembic) and link the docs site.
This commit is contained in:
Renn F
2026-06-22 15:53:00 +02:00
parent 3a3cd69e1e
commit 2fb63fed1f
64 changed files with 3553 additions and 500 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operations/cost-and-usage.md
+97
View File
@@ -0,0 +1,97 @@
# Cost & usage
RoboCo measures what your workforce spends in tokens and dollars, and shows it to you live in the panel. This page explains where those numbers come from, how to read the dashboard, how each provider is priced, and what cost controls exist (and where they don't).
## How spend is measured
Spend is captured **per agent session** — one container, doing one stretch of work. Each spawned agent runs a small in-container SDK server that watches the agent's own transcript and exposes its running token counts (input, output, cache-read, cache-write). The orchestrator runs a background sweep every **~60 seconds**: for every active agent it pulls the live counts, writes a usage snapshot, and updates the open session row so the database reflects progress mid-run. When the container stops, the session is **finalized** — final token counts are resolved, priced through the built-in cost table, and written to the closed session row along with the exit reason.
A separate daily rollup keeps a rolling per-day, per-agent, per-team, per-model tally that feeds the "today" figures.
!!! note "Why mid-run numbers move, and why a crash can read $0"
Because cost is computed per session and firms up only when the session ends, the figures for a still-running agent update roughly every minute and aren't final. A session that crashes or is abandoned before its transcript is read can finalize at a low or zero value. The dashboard is an accurate ledger of *closed* sessions plus a live estimate for the ones still open — not a real-time invoice.
Grok agents have no live SDK hook; the orchestrator reads each Grok container's `usage.json` instead, and uses the same file to enforce the per-agent Grok cost cap (below).
## The Token Usage & Cost dashboard
Two panel surfaces show spend:
- The **command center** carries a **Token Usage & Cost** card that streams live while agents work. It listens on the `/ws/system` WebSocket and shows a **Live** badge when connected; if the socket drops it falls back to HTTP polling and shows **Polling** (or **Connecting**). See [the command center](../panel/command-center.md).
- The **Metrics** page has a dedicated **Token Usage** tab with the full breakdown. See [Metrics](../panel/metrics.md).
A **24h / 7d / 30d** selector drives every panel. What you get:
| Panel | What it shows |
|-------|----------------|
| Summary | Input, output, and total tokens for the period, plus total cost in USD |
| Trend | Percent change vs the immediately prior window of equal length |
| Time series | Hourly points for 24h; daily points for 7d/30d |
| Per-model | Donut of cost share by model |
| Per-agent / per-team | Bar charts of spend, each with its share of the total |
| Monthly projection | Forecast spend (see below) |
| Cache efficiency | Cache hit-rate and the dollars prompt-caching is saving |
| Recent sessions | Raw recent spawn-session rows (default 50) |
!!! info "Totals count all four token classes"
The total-token and cost figures sum **input + output + cache-read + cache-write**. Prompt-caching reads are cheap but not free, so they show up in both the totals and the cache-savings panel.
!!! warning "Projection and cache savings are estimates"
The monthly projection is a naive extrapolation: the average daily cost over the last 7 days × 30. It's a planning forecast, not a bill, and it overstates if you spun up the fleet for a one-off burst. The cache-savings figure uses a single aggregate baseline rate to estimate what the cached tokens *would* have cost at the full input price — treat it as indicative, not exact.
## How each provider is priced
Pricing is provider-aware, from a built-in USD-per-million-token table (`roboco/billing/pricing.py`). Model names are matched on substring, longest fragment wins.
| Provider / model | Input | Output | Priced? |
|------------------|-------|--------|---------|
| Claude Opus 4 | $5.00 | $25.00 | yes (cache read $0.50, write $6.25) |
| Claude Sonnet (4 / 3.7 / 3.5) | $3.00 | $15.00 | yes (cache read $0.30) |
| Claude Haiku 4 / 3.5 | $1.00 | $5.00 | yes |
| xAI `grok-build` | $1.00 | $2.00 | yes (cache read $0.20) |
| Local Ollama (`ollama/…` or bare tag) | — | — | **$0 by design** |
| Ollama Cloud (`:cloud` tag) | — | — | **$0 by design** |
!!! note "$0 for local and Ollama Cloud is not a bug"
Self-hosted Ollama runs on hardware you own and Ollama Cloud is billed by flat subscription, so neither carries a per-token cost. RoboCo intentionally prices them at $0 — it is not undercounting, and you won't see a warning.
!!! warning "A brand-new Claude model can silently undercount"
If a `claude`-named model isn't in the pricing table, RoboCo logs a warning and returns $0 for it rather than crashing. That **is** real spend going uncounted. If you point an agent at a Claude model newer than this build and see $0 cost on a busy fleet, the table needs the new rate added — check the orchestrator logs for the pricing warning.
## Cost controls (and the asymmetry)
There is exactly one built-in dollar cap, and it only covers Grok:
| Setting | Default | What it does |
|---------|---------|--------------|
| `ROBOCO_GROK_MAX_COST_USD` | `0.0` | Per-agent Grok cost ceiling in USD, read from the container's `usage.json`. The orchestrator kills a Grok container once it crosses this, catching runaway-loop token burn. `0` disables it. **Grok agents only.** |
!!! danger "There is no built-in dollar cap for Claude agents"
The Claude path is **observe-only** on cost. RoboCo will show you Claude spend live and historically, but it will not auto-kill a Claude agent for crossing a dollar threshold — there is no `ROBOCO_*_MAX_COST_USD` equivalent for it. Your protection against runaway Claude spend is the structural one: rate-limit and overload **park-and-probe** (a provider 429 or persistent overload queues the agent and probes for recovery instead of retrying in a hot loop — see [provider resilience](../models/resilience.md)), the per-task verb gateway, and watching the dashboard. Budget accordingly.
A related guard reaps abandoned interactive chats so they don't leak a container:
| Setting | Default | What it does |
|---------|---------|--------------|
| `ROBOCO_INTERACTIVE_IDLE_REAP_SECONDS` | `1800` | Reaps an idle live Intake/Secretary chat (by time since last turn) so it stops holding a container and burning tokens. `0` disables. |
## Quick checks from the shell
```bash
# Period summary (totals + cost + trend)
curl -s 'http://localhost:3000/api/usage/summary?period=7d'
# Monthly projection
curl -s http://localhost:3000/api/usage/projection
# Container-level resource use (not token cost)
docker stats
docker system df
```
## Next
- Read the live dashboard on [the command center](../panel/command-center.md) and the full breakdown on [Metrics](../panel/metrics.md).
- See where the live stream comes from in [WebSockets](../api/websockets.md) and the `/api/usage/*` routes in the [REST API](../api/rest-api.md).
- Understand the park-and-probe spend protection in [provider resilience](../models/resilience.md).
- For operational (non-cost) health, see [health & metrics](./health-and-metrics.md).
docs/operations/health-and-metrics.md
+58
View File
@@ -0,0 +1,58 @@
# Health & metrics
Two different questions an operator asks: *is the system up?* and *is the workforce productive?* RoboCo answers the first with two HTTP probes, and the second with an operational Metrics view. Neither has anything to do with token cost — for spend, see [cost & usage](./cost-and-usage.md).
## Liveness and readiness probes
The orchestrator exposes two probes behind nginx (so they're reachable at `localhost:3000/api/...`, or directly on the orchestrator):
| Endpoint | Checks | Returns |
|----------|--------|---------|
| `GET /api/health` | The process is up | Always `200` with `{ status: "ok", version, environment }` once the app is serving |
| `GET /api/ready` | Database (`SELECT 1`) **and** Redis (`PING`) | `200` with `{ status, database, redis }`; `status` is `ok` only when both pass, otherwise `degraded` |
```bash
curl -s http://localhost:3000/api/health
curl -s http://localhost:3000/api/ready
```
!!! info "Readiness is the one to watch"
`/api/health` is a liveness probe — it stays `200` as long as the process answers, so it can't tell you a dependency is down. `/api/ready` is the real readiness signal: it actively pings Postgres and Redis and reports `degraded` (still HTTP `200`, but `status: "degraded"` with the failing dependency's error in the `database`/`redis` field) when either is unreachable. Point your uptime monitor at `/api/ready` and alert on `degraded`.
!!! note "Startup ordering"
The app-level root `/health` used during container startup is additionally gated on the in-house RAG engine being operational, which is why the orchestrator can take a minute to report healthy after a cold start while it indexes documents. See [deployment](../deploy/deployment.md) for the full startup sequence.
## The Metrics → Performance view
The **Metrics** page has a **Performance** tab driven by tasks, messages, and notifications — your read on whether work is actually flowing. See the panel walkthrough in [Metrics](../panel/metrics.md).
What it surfaces:
| Group | Metrics |
|-------|---------|
| Velocity | Tasks completed vs created, average completion time (hours), completion rate |
| Blockers | Active blockers, average and longest blocked time, blockers by team |
| Team & agent | Per-team and per-agent performance |
| Communication | Message/notification volume |
### The ok / slow / critical health signal
Org and per-team health roll up to a single status driven mainly by the **blocked-task ratio**:
```mermaid
flowchart LR
A[blocked / total tasks] -->|ratio > 0.3| C[critical]
A -->|ratio > 0.15| S[slow]
A -->|otherwise| K[ok]
```
A team with more than 30% of its tasks blocked reads **critical**; over 15% reads **slow**; below that, **ok**. A heuristic also flags a team sitting on stale active tasks with zero completions. When you see **slow** or **critical**, the Blockers panel tells you where the work is jammed and which team owns it — chase the longest-blocked tasks first.
!!! tip "Health is about flow, not errors"
This status is computed from task state, not exceptions or crashes. A green org-health with a `degraded` `/api/ready` means the infrastructure is wobbling even though the backlog looks healthy — watch both signals, they answer different questions.
## Next
- Walk the panel surface in [Metrics](../panel/metrics.md) and [the command center](../panel/command-center.md).
- For token spend and the cost dashboard, see [cost & usage](./cost-and-usage.md).
- For the startup sequence and what each container needs, see [deployment](../deploy/deployment.md).
docs/operations/index.md
+19
View File
@@ -0,0 +1,19 @@
# Cost & Observability
RoboCo measures what the company spends and how it's performing, and shows both in the panel. This section explains the cost model and the operational signals.
<div class="grid cards" markdown>
- **[Cost & usage](cost-and-usage.md)**
---
How token spend is measured, the Token Usage & Cost dashboard, provider-aware pricing (local is $0 by design), and the cost caps.
- **[Health & metrics](health-and-metrics.md)**
---
The health and readiness probes, and the delivery-performance metrics — velocity, blockers, and team health.
</div>