Files
roboco/docs/optional/http-security.md
T
0ca9d91b72 v0.16.0: fastapi-guard HTTP security layer — calibrated + scanner honeytrap (#290)
* [fastapi-guard] Phase 1a: gated config flags for the HTTP security layer

Adds the ROBOCO_GUARD_* settings (all default-off / secure-default) for the
upcoming fastapi-guard 7.2.0 hardening — guard_enabled (master switch),
guard_fail_secure (fail-closed default; NAS overrides to false),
guard_telemetry_enabled + guard_agent_api_key + guard_project_id (guard-agent
telemetry, opt-in), guard_emergency + guard_emergency_whitelist (lockdown kill
switch). Inert until consumed: nothing reads them yet, so the request path is
unchanged. Foundation for v0.16.0.

* [fastapi-guard] Phase 1b: security foundation module + gated wiring

Add fastapi-guard 7.2.0 + guard-core 3.3.0 (bare, unpinned) and roboco/security.py:
- build_security_config() from settings — behind-nginx real-IP (trusted_proxies +
  trust_x_forwarded_proto), HSTS/CSP headers, threat-ban + 404-sweep rules,
  redis-backed state, exclude_paths (/ws + health + docs), env-driven
  enforce_https, fail_secure (secure default), emergency lockdown, guard-agent
  telemetry (opt-in), passive-mode calibration switch.
- guard_deco singleton (SecurityDecorator) for per-route decorators (Phase 2+).
- Three custom content validators guard's WAF can't cover: prompt-injection /
  role-override, secret-exfil / credential-in-body, internal-SSRF.
- apply_guard(app) + guarded_lifespan() wired into create_app AFTER settings.
  guard_passive_mode config flag added.

Entirely gated by ROBOCO_GUARD_ENABLED (default off): create_app mounts nothing
and returns the unchanged app when off (verified). make quality GREEN
(cov 95.32%, pip-audit clean, import-linter 2/0). 12 new unit tests.

* [fastapi-guard] Phase 2: critical-path decorators

Apply guard decorators to the highest-value endpoints (metadata-only; enforced
only when the middleware is mounted, so no-op while ROBOCO_GUARD_ENABLED is off):
- provider keys (ollama/grok/self-hosted writes): strict rate_limit +
  max_request_size + block_clouds (no datacenter IP should touch secret writes).
- settings write + release approve/reject (CEO-gated): strict rate_limit.
- intake chat (prompter start/messages/events): rate_limit + max_request_size +
  custom_validation(prompt_injection_validator) — the prompt-facing free-text
  ingress gets the injection/role-override/secret-exfil content scan.

make quality GREEN (cov 95.32%, contracts 2/0). App builds with guard off,
decorators inert (verified).

* [fastapi-guard] Phase 3: wide decorator coverage across ingress + sensitive routes

Targeted-wide application (metadata-only; no-op until ROBOCO_GUARD_ENABLED). The
global SecurityMiddleware already rate-limits + WAF-scans every request, so this
adds the custom content validators on free-text ingress + tight limits on
sensitive ops (not blanket per-route rate_limit on reads):

- agent gateway do verbs (note/say/commit/dm/pitch/progress/draft_playbook/...):
  rate_limit + max_request_size + custom_validation(secret_exfil or prompt_injection).
- a2a message/send + chat writes: rate_limit + size + prompt_injection.
- optimal/RAG (kb/search, rag/query, mentor/ask, errors/decisions/standards/
  learnings): prompt_injection on searches, secret_exfil on record writes; docs
  index → internal_ssrf.
- tasks: create/update → prompt_injection; QA/doc/PM transitions → secret_exfil;
  CEO-gated verbs → tight rate_limit.
- secretary chat → prompt_injection; research → internal_ssrf; orchestrator
  spawn/mutations → rate_limit; git ops + flow verbs → tight rate_limit.
Pure GET/reads left to the global middleware. Applied via a Sonnet workflow,
then verified: app builds with guard off (decorators inert), make quality GREEN
(cov 95.37%, contracts 2/0). Decoy/honeypot-path surface deferred (needs verified
guard ban-API integration — not rushed).

* [fastapi-guard] Phase 5: arm the NAS composes in passive/log-only mode

Arm ROBOCO_GUARD_ENABLED=true + ROBOCO_GUARD_PASSIVE_MODE=true +
ROBOCO_GUARD_FAIL_SECURE=false on the two NAS composes (docker-compose.yaml +
.yml). Passive = guard mounts and logs what it WOULD block but blocks nothing,
so the next NAS deploy calibrates against real traffic; flip PASSIVE_MODE off
after the false-positive review to enforce. fail_secure=false keeps a
guard-internal error from 500ing the personal deploy. The registry (user-facing)
compose is deliberately left unarmed so its published default stays conservative.
Phase 4 (passive calibration) is the operational step this enables.

* feat(security): Phase 3b — full-arsenal per-route guard enrichment

Stack the applicable guard decorators per surface instead of the minimal
rate_limit/max_request_size/custom_validation triad: content_type_filter on
every JSON-body write, honeypot_detection form-traps on human-facing POSTs,
block_clouds on key-writes + CEO release ops, behavior_analysis runaway-rate
rules on the agent flow/do verbs, suspicious_detection + usage_monitor on the
sensitive surfaces. Nine distinct decorators now applied thoughtfully per
endpoint. All metadata-only — no-op while ROBOCO_GUARD_ENABLED is off.

* fix(a2a): permit PR reviewer to deliver gate verdicts to the owning PM

can_a2a_direct had no pr_reviewer rule, so a reviewer (team=board, or a cell
team) fell through to the cell-member path and was cross-cell-denied when the
in-path gate delivered a pr_fail change-request to main-pm (or a cross-cell
cell-pm): "Cannot A2A main-pm ... Ask None to coordinate with None". The
delivery is best-effort, so pr_fail still transitioned but the verdict never
reached the owning PM — the blind-re-submit signal-gap the pr_fail fix closes.

Add an explicit pr_reviewer handler: it may A2A only cell_pm / main_pm (its
sole comms surface — everything else it posts on the PR itself), with a matching
route hint. The cell reviewers kept same-team access by coincidence; this scopes
every reviewer to PM-only, the correct model, with no other A2A caller affected.

Refresh uv.lock to the current resolution.

* feat(models): adopt Claude Sonnet 5 as the sonnet tier

Point the 'sonnet' alias at claude-sonnet-5 (MODEL_MAP) and give pr_reviewer
its own opus tier in ROLE_MODEL_MAP — it was falling through to the sonnet
default, and the role gates untrusted external/fork PRs plus root→master, which
warrants opus.

Price claude-sonnet-5 at the promotional 33% off Sonnet 4.6 ($2.01 / $10.05,
cache 0.201 / 0.5025) through 2026-08-31 via a dedicated pricing fragment that
beats the bare 'sonnet' alias; revert to full rate when the promo ends. Bare
'sonnet' stays full-rate as a conservative fallback (prod prices the resolved
claude-sonnet-5 id from the transcript).

Update the model docs and the billing / usage / manifest / spawn tests.

* feat(security): calibrate the guard WAF for RoboCo traffic + document the layer

The first end-to-end run of the fastapi-guard layer showed active enforcement
would block ~50% of legitimate agent traffic — RoboCo request bodies are code,
SQL, diffs, file paths, HTML, and URLs, which the stock signature WAF reads as
attacks. build_security_config now excludes RoboCo's free-text top-level body
fields (derived from the real request models, including the free-form container
fields whose nested prose is stringified and scanned) from WAF scanning,
dropping the active-mode false-positive rate to zero while keeping the WAF on
every non-excluded (id/enum/slug/branch) field and leaving the
prompt-injection / secret-exfil / internal-SSRF validators — which run
independently of the exclusion — fully in force. enable_penetration_detection
is made explicit.

Only excluded_detection_body_fields is reliable on guard 7.2.1: the per-route
categories knob is bypassed for JSON bodies, and the body scanner excludes
top-level keys only (scanning str(value) of every non-excluded field), so
free-form container fields must be excluded wholesale.

Adds tests/unit/test_security_middleware.py — the first end-to-end exercise of
the middleware (mounts it, drives guard's lifespan, fires real requests):
proves passive mode is log-only, active mode does not false-positive on
realistic agent payloads, threats are still blocked inside excluded fields, and
the WAF still fires on non-excluded fields.

Docs: CHANGELOG (Unreleased); a user-facing Optional-subsystems page + nav +
env reference for the HTTP security layer; the agent-facing RAG corpus
(what it is + why a request could be blocked); and the roboco mapping
(api-core-websocket / deployment-tooling / _complete_map).

* feat(security): Surface N — scanner honeytrap (guard /api auto-ban + nginx edge-drop)

Turns scanner probes against the scanner, in two layers matched to where
traffic lands. Behind nginx only /api, /ws, /health, /ready reach the
orchestrator, so guard can only see (and ban) scanner probes on those paths;
the classic root probes (/.env, /wp-login.php, /phpmyadmin, /.git/config) hit
the panel. So:

- build_security_config's threat_ban_config gains recon / sensitive_file /
  cms_probing categories. A scanner probing those fingerprints on an /api path
  is detected on the URL-path scan; repeated probes from one IP trip an adaptive
  per-IP auto-ban (redis-backed, 24h). Only bans in active mode (passive logs
  the recon hit) and needs redis (the 24h ban exceeds the in-memory cap). The
  spec's decoy-route file is redundant — the WAF url-path scan bans regardless
  of a registered route — so it is intentionally omitted.
- docker/nginx.conf drops the classic root scanner paths at the edge with 444
  (connection closed, no response) before they reach the panel, anchored to
  known scanner fingerprints so /.well-known and every real panel/API route are
  untouched. Always on, independent of ROBOCO_GUARD_ENABLED.

Tests: 2 unit (the exclusion set + the scanner-ban categories are present) and
2 integration (a decoy path is blocked in active mode, passes in passive). The
nginx regex was validated against 15 scanner + 19 legit paths (0 false
positives). Docs: CHANGELOG, the HTTP-security page, the roboco mapping, and the
agent-facing RAG corpus.

* Token optimization — per-role observability, compute policy, spawn preflight (#291)

* test(models): lock the sonnet→claude-sonnet-5 MODEL_MAP invariant

* feat(usage): surface cache tokens + cache_hit_rate in usage breakdowns

* feat(usage): add per-role usage breakdown endpoint

* feat(usage): add spawn-waste signal (per-role unproductive rate + respawn strikes)

* feat(panel): surface per-role cost/cache + spawn-waste on the metrics page

* feat(routing): Phase 2 per-role compute policy — qa→haiku, main_pm→sonnet, per-role effort env mechanism (default-inert)

* feat(orchestrator): Phase 3 flag-gated spawn preflight — refuse non-gateway delivery roles (respawn-forever guard)

* chore(compose): arm ROBOCO_SPAWN_PREFLIGHT_ENABLED on the NAS composes

* docs: per-role usage observability, per-role compute policy, and spawn preflight

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>

* fix(panel): pin outputFileTracingRoot so the standalone build isn't broken by stray lockfiles

* feat(routing): populate ROLE_EFFORT_MAP + wire the verified --effort flag (cell_pm/board/auditor to medium)

* feat(gateway): omit empty context_briefing sections (Phase 4 payload compaction)

* refactor(orchestrator): extract spawn chokepoint guards to restore xenon rank B on spawn_agent

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-07-01 23:54:48 +02:00

6.1 KiB

HTTP security hardening (fastapi-guard)

RoboCo can front its API with a fastapi-guard security layer — IP and rate controls, a signature WAF, security headers, cloud-provider and honeypot checks, an emergency kill switch, and three RoboCo-specific content validators — for when you expose the panel/API beyond a trusted LAN.

The layer is off by default. When ROBOCO_GUARD_ENABLED is unset, create_app never mounts the middleware and the request path is byte-for-byte unchanged; the per-route guard decorators are harmless no-ops (roboco/security.py). A personal NAS deploy needs none of this — it's built for cloud/public hosting.

What it does

Armed, the middleware sits outermost and applies, per route:

  • Rate limiting & size caps — a baseline throttle plus tighter per-endpoint limits on sensitive routes (provider-key writes, CEO release ops, agent verbs).
  • A signature WAF — SQLi/XSS/path-traversal/etc. detection on request bodies, headers, params, and path.
  • Security headers — HSTS, CSP, X-Frame-Options, nosniff, referrer and permissions policies on every response.
  • Cloud-provider & honeypot checks — block datacenter egress on the highest-value routes; form-trap honeypots on human-facing POSTs.
  • Three RoboCo custom validators the stock WAF cannot cover — prompt-injection, secret-exfil, and internal-SSRF scanning on the prompt-facing and agent-content surfaces.
  • An emergency lockdown (ROBOCO_GUARD_EMERGENCY) — a flip-on-without-redeploy switch that blocks every non-whitelisted IP during an active attack.

Behind nginx, guard trusts the single proxy hop for the real client IP, excludes WebSocket upgrades and health/docs paths, and keeps RoboCo's own CORS.

Passive first, then enforce

Guard has two enforcement postures, and the intended rollout is passive → active:

ROBOCO_GUARD_PASSIVE_MODE Behaviour
true (calibrate) Log-only. Detections are logged but never block — legit and malicious requests pass through. Safe to arm on live traffic to surface false positives first.
false (enforce) Detections block the request.

!!! warning "Calibrate before you enforce" RoboCo's request bodies are code, SQL, diffs, file paths, HTML, and URLs — task specs, agent notes and commits, RAG queries, git bodies, chat. A stock WAF reads that legitimate traffic as attacks. RoboCo ships a calibration (excluded_detection_body_fields in build_security_config, derived from the real request models) that excludes those free-text fields from WAF scanning so active mode does not false-positive — while the WAF stays active on every structured (id/enum/slug/branch) field and the prompt-injection / secret-exfil / SSRF validators keep firing regardless. Even so, arm passive first on your own deployment, watch the logs for any straggler false positive, then flip to active.

The NAS composes arm the guard in passive/log-only mode by default (ROBOCO_GUARD_PASSIVE_MODE=true, ROBOCO_GUARD_FAIL_SECURE=false) so a deploy calibrates against real traffic before you enforce.

Scanner honeytrap & auto-ban

Automated scanners probe every host for well-known soft spots (/.env, /wp-login.php, /phpmyadmin, /.git/config, …). RoboCo turns those probes against the scanner, in two layers matched to where the traffic actually lands — behind nginx only /api, /ws, /health, and /ready reach the orchestrator, so guard can only see probes on those paths:

  • Guard adaptive ban (the /api surface). The guard threat_ban_config carries recon / sensitive_file / cms_probing categories. A scanner probing those fingerprints on an /api path is detected on the URL-path scan, and repeated probes from one IP trip a per-IP auto-ban (redis-backed, 24h). This only bans in active mode (passive logs the recon hit) and requires redis (the 24h ban exceeds the in-memory cap).
  • nginx edge-drop (the classic root paths). The classic scanner paths never reach the orchestrator, so nginx drops them at the edge with 444 (closes the connection, returns nothing) before they touch the panel. It's anchored to known scanner fingerprints — /.well-known and every real panel/API route are untouched — and is always on, independent of ROBOCO_GUARD_ENABLED.

Fail-secure

ROBOCO_GUARD_FAIL_SECURE decides what happens if a security check itself errors: true (default) fails closed — block the request — which is the right default for public hosting. The personal NAS compose overrides it to false so a guard-internal bug can never 500 your own deploy.

Enable it

=== "Environment"

```bash
ROBOCO_GUARD_ENABLED=true
ROBOCO_GUARD_PASSIVE_MODE=true      # calibrate first; flip to false to enforce
ROBOCO_GUARD_FAIL_SECURE=true       # false on a personal/NAS deploy
ROBOCO_ENVIRONMENT=production        # drives enforce_https (relaxed in development)
# ROBOCO_GUARD_EMERGENCY=true                 # attack lockdown kill switch
# ROBOCO_GUARD_EMERGENCY_WHITELIST=1.2.3.4    # extra allowed IPs during lockdown
# ROBOCO_GUARD_TELEMETRY_ENABLED=true         # + guard_agent_api_key + guard_project_id
```

=== "Panel"

**Settings → Feature Flags** exposes the master switch. The passive/fail-secure/environment knobs are environment settings, not flags — set them in env.

!!! note "Takes effect on the next backend restart"
    The flag persists in the settings store and applies on the **next backend restart**.

See the environment reference for the full flag list.

Telemetry (optional, off)

ROBOCO_GUARD_TELEMETRY_ENABLED reports security events/metrics to a guard-core platform via guard-agent. It is off by default and no data leaves the box while off; flip it on and set ROBOCO_GUARD_AGENT_API_KEY + ROBOCO_GUARD_PROJECT_ID to enable.

Next

Environment reference for every flag · Deployment for where to set them · back to Optional subsystems.