4 Commits

Include renames

Author	SHA1	Message	Date
3 people	0ca9d91b72	v0.16.0: fastapi-guard HTTP security layer — calibrated + scanner honeytrap (#290) ... * [fastapi-guard] Phase 1a: gated config flags for the HTTP security layer Adds the ROBOCO_GUARD_* settings (all default-off / secure-default) for the upcoming fastapi-guard 7.2.0 hardening — guard_enabled (master switch), guard_fail_secure (fail-closed default; NAS overrides to false), guard_telemetry_enabled + guard_agent_api_key + guard_project_id (guard-agent telemetry, opt-in), guard_emergency + guard_emergency_whitelist (lockdown kill switch). Inert until consumed: nothing reads them yet, so the request path is unchanged. Foundation for v0.16.0. * [fastapi-guard] Phase 1b: security foundation module + gated wiring Add fastapi-guard 7.2.0 + guard-core 3.3.0 (bare, unpinned) and roboco/security.py: - build_security_config() from settings — behind-nginx real-IP (trusted_proxies + trust_x_forwarded_proto), HSTS/CSP headers, threat-ban + 404-sweep rules, redis-backed state, exclude_paths (/ws + health + docs), env-driven enforce_https, fail_secure (secure default), emergency lockdown, guard-agent telemetry (opt-in), passive-mode calibration switch. - guard_deco singleton (SecurityDecorator) for per-route decorators (Phase 2+). - Three custom content validators guard's WAF can't cover: prompt-injection / role-override, secret-exfil / credential-in-body, internal-SSRF. - apply_guard(app) + guarded_lifespan() wired into create_app AFTER settings. guard_passive_mode config flag added. Entirely gated by ROBOCO_GUARD_ENABLED (default off): create_app mounts nothing and returns the unchanged app when off (verified). make quality GREEN (cov 95.32%, pip-audit clean, import-linter 2/0). 12 new unit tests. * [fastapi-guard] Phase 2: critical-path decorators Apply guard decorators to the highest-value endpoints (metadata-only; enforced only when the middleware is mounted, so no-op while ROBOCO_GUARD_ENABLED is off): - provider keys (ollama/grok/self-hosted writes): strict rate_limit + max_request_size + block_clouds (no datacenter IP should touch secret writes). - settings write + release approve/reject (CEO-gated): strict rate_limit. - intake chat (prompter start/messages/events): rate_limit + max_request_size + custom_validation(prompt_injection_validator) — the prompt-facing free-text ingress gets the injection/role-override/secret-exfil content scan. make quality GREEN (cov 95.32%, contracts 2/0). App builds with guard off, decorators inert (verified). * [fastapi-guard] Phase 3: wide decorator coverage across ingress + sensitive routes Targeted-wide application (metadata-only; no-op until ROBOCO_GUARD_ENABLED). The global SecurityMiddleware already rate-limits + WAF-scans every request, so this adds the custom content validators on free-text ingress + tight limits on sensitive ops (not blanket per-route rate_limit on reads): - agent gateway do verbs (note/say/commit/dm/pitch/progress/draft_playbook/...): rate_limit + max_request_size + custom_validation(secret_exfil or prompt_injection). - a2a message/send + chat writes: rate_limit + size + prompt_injection. - optimal/RAG (kb/search, rag/query, mentor/ask, errors/decisions/standards/ learnings): prompt_injection on searches, secret_exfil on record writes; docs index → internal_ssrf. - tasks: create/update → prompt_injection; QA/doc/PM transitions → secret_exfil; CEO-gated verbs → tight rate_limit. - secretary chat → prompt_injection; research → internal_ssrf; orchestrator spawn/mutations → rate_limit; git ops + flow verbs → tight rate_limit. Pure GET/reads left to the global middleware. Applied via a Sonnet workflow, then verified: app builds with guard off (decorators inert), make quality GREEN (cov 95.37%, contracts 2/0). Decoy/honeypot-path surface deferred (needs verified guard ban-API integration — not rushed). * [fastapi-guard] Phase 5: arm the NAS composes in passive/log-only mode Arm ROBOCO_GUARD_ENABLED=true + ROBOCO_GUARD_PASSIVE_MODE=true + ROBOCO_GUARD_FAIL_SECURE=false on the two NAS composes (docker-compose.yaml + .yml). Passive = guard mounts and logs what it WOULD block but blocks nothing, so the next NAS deploy calibrates against real traffic; flip PASSIVE_MODE off after the false-positive review to enforce. fail_secure=false keeps a guard-internal error from 500ing the personal deploy. The registry (user-facing) compose is deliberately left unarmed so its published default stays conservative. Phase 4 (passive calibration) is the operational step this enables. * feat(security): Phase 3b — full-arsenal per-route guard enrichment Stack the applicable guard decorators per surface instead of the minimal rate_limit/max_request_size/custom_validation triad: content_type_filter on every JSON-body write, honeypot_detection form-traps on human-facing POSTs, block_clouds on key-writes + CEO release ops, behavior_analysis runaway-rate rules on the agent flow/do verbs, suspicious_detection + usage_monitor on the sensitive surfaces. Nine distinct decorators now applied thoughtfully per endpoint. All metadata-only — no-op while ROBOCO_GUARD_ENABLED is off. * fix(a2a): permit PR reviewer to deliver gate verdicts to the owning PM can_a2a_direct had no pr_reviewer rule, so a reviewer (team=board, or a cell team) fell through to the cell-member path and was cross-cell-denied when the in-path gate delivered a pr_fail change-request to main-pm (or a cross-cell cell-pm): "Cannot A2A main-pm ... Ask None to coordinate with None". The delivery is best-effort, so pr_fail still transitioned but the verdict never reached the owning PM — the blind-re-submit signal-gap the pr_fail fix closes. Add an explicit pr_reviewer handler: it may A2A only cell_pm / main_pm (its sole comms surface — everything else it posts on the PR itself), with a matching route hint. The cell reviewers kept same-team access by coincidence; this scopes every reviewer to PM-only, the correct model, with no other A2A caller affected. Refresh uv.lock to the current resolution. * feat(models): adopt Claude Sonnet 5 as the sonnet tier Point the 'sonnet' alias at claude-sonnet-5 (MODEL_MAP) and give pr_reviewer its own opus tier in ROLE_MODEL_MAP — it was falling through to the sonnet default, and the role gates untrusted external/fork PRs plus root→master, which warrants opus. Price claude-sonnet-5 at the promotional 33% off Sonnet 4.6 ($2.01 / $10.05, cache 0.201 / 0.5025) through 2026-08-31 via a dedicated pricing fragment that beats the bare 'sonnet' alias; revert to full rate when the promo ends. Bare 'sonnet' stays full-rate as a conservative fallback (prod prices the resolved claude-sonnet-5 id from the transcript). Update the model docs and the billing / usage / manifest / spawn tests. * feat(security): calibrate the guard WAF for RoboCo traffic + document the layer The first end-to-end run of the fastapi-guard layer showed active enforcement would block ~50% of legitimate agent traffic — RoboCo request bodies are code, SQL, diffs, file paths, HTML, and URLs, which the stock signature WAF reads as attacks. build_security_config now excludes RoboCo's free-text top-level body fields (derived from the real request models, including the free-form container fields whose nested prose is stringified and scanned) from WAF scanning, dropping the active-mode false-positive rate to zero while keeping the WAF on every non-excluded (id/enum/slug/branch) field and leaving the prompt-injection / secret-exfil / internal-SSRF validators — which run independently of the exclusion — fully in force. enable_penetration_detection is made explicit. Only excluded_detection_body_fields is reliable on guard 7.2.1: the per-route categories knob is bypassed for JSON bodies, and the body scanner excludes top-level keys only (scanning str(value) of every non-excluded field), so free-form container fields must be excluded wholesale. Adds tests/unit/test_security_middleware.py — the first end-to-end exercise of the middleware (mounts it, drives guard's lifespan, fires real requests): proves passive mode is log-only, active mode does not false-positive on realistic agent payloads, threats are still blocked inside excluded fields, and the WAF still fires on non-excluded fields. Docs: CHANGELOG (Unreleased); a user-facing Optional-subsystems page + nav + env reference for the HTTP security layer; the agent-facing RAG corpus (what it is + why a request could be blocked); and the roboco mapping (api-core-websocket / deployment-tooling / _complete_map). * feat(security): Surface N — scanner honeytrap (guard /api auto-ban + nginx edge-drop) Turns scanner probes against the scanner, in two layers matched to where traffic lands. Behind nginx only /api, /ws, /health, /ready reach the orchestrator, so guard can only see (and ban) scanner probes on those paths; the classic root probes (/.env, /wp-login.php, /phpmyadmin, /.git/config) hit the panel. So: - build_security_config's threat_ban_config gains recon / sensitive_file / cms_probing categories. A scanner probing those fingerprints on an /api path is detected on the URL-path scan; repeated probes from one IP trip an adaptive per-IP auto-ban (redis-backed, 24h). Only bans in active mode (passive logs the recon hit) and needs redis (the 24h ban exceeds the in-memory cap). The spec's decoy-route file is redundant — the WAF url-path scan bans regardless of a registered route — so it is intentionally omitted. - docker/nginx.conf drops the classic root scanner paths at the edge with 444 (connection closed, no response) before they reach the panel, anchored to known scanner fingerprints so /.well-known and every real panel/API route are untouched. Always on, independent of ROBOCO_GUARD_ENABLED. Tests: 2 unit (the exclusion set + the scanner-ban categories are present) and 2 integration (a decoy path is blocked in active mode, passes in passive). The nginx regex was validated against 15 scanner + 19 legit paths (0 false positives). Docs: CHANGELOG, the HTTP-security page, the roboco mapping, and the agent-facing RAG corpus. * Token optimization — per-role observability, compute policy, spawn preflight (#291) * test(models): lock the sonnet→claude-sonnet-5 MODEL_MAP invariant * feat(usage): surface cache tokens + cache_hit_rate in usage breakdowns * feat(usage): add per-role usage breakdown endpoint * feat(usage): add spawn-waste signal (per-role unproductive rate + respawn strikes) * feat(panel): surface per-role cost/cache + spawn-waste on the metrics page * feat(routing): Phase 2 per-role compute policy — qa→haiku, main_pm→sonnet, per-role effort env mechanism (default-inert) * feat(orchestrator): Phase 3 flag-gated spawn preflight — refuse non-gateway delivery roles (respawn-forever guard) * chore(compose): arm ROBOCO_SPAWN_PREFLIGHT_ENABLED on the NAS composes * docs: per-role usage observability, per-role compute policy, and spawn preflight --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> * fix(panel): pin outputFileTracingRoot so the standalone build isn't broken by stray lockfiles * feat(routing): populate ROLE_EFFORT_MAP + wire the verified --effort flag (cell_pm/board/auditor to medium) * feat(gateway): omit empty context_briefing sections (Phase 4 payload compaction) * refactor(orchestrator): extract spawn chokepoint guards to restore xenon rank B on spawn_agent --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-07-01 23:54:48 +02:00
3 people	06682f33c6	Fix: agent workflow hardening (#70) ... * fix(gateway): push the branch before QA handoff so reviewers see the latest commits The commit content tool commits locally without pushing; only open_pr pushed the branch. On the first submission that was fine, but a fix committed while addressing needs_revision never reached origin (open_pr is skipped once the PR exists), so QA — which reviews the remote PR branch — re-reviewed the stale remote and re-failed the task on every cycle, a loop that never converged. i_am_done now pushes the task branch (idempotent; a no-op when nothing is unpushed) as part of the shared submit gate, covering both the normal and resume-from-verifying paths. A push failure blocks the handoff with a clear remediation rather than parking the task in awaiting_qa with commits that exist only in the developer's local workspace. * fix(orchestrator): don't reap a stale claim while the agent's container is alive The stale-claim reaper released any claimed/in_progress task whose last_heartbeat_at exceeded the TTL. The heartbeat only updates on certain gateway calls, so a developer deep in a long edit/test cycle outran the TTL and had its claim reaped mid-work — churning the task and risking a double spawn against the still-running container. The reaper now skips a task whose assignee still holds a live (ACTIVE) agent instance, trusting container liveness — the ground truth — over the heartbeat proxy. The check is defensive on missing fields so a heartbeat-only caller (and the reaper's existing unit tests) behave exactly as before. * fix(gateway): refuse to unblock a task while a dependency is unfinished A PM unblock on a dependency-gated task moved it straight to in_progress, overriding the dependency — letting a dependent proceed without its upstream's work (e.g. a frontend task built before its UX design lands). A dependency block is meant to clear on its own via _unblock_dependents the moment the upstream reaches a terminal state. unblock now refuses while any dependency is still non-terminal, returning a clear remediation that the block resolves automatically. Manual unblock remains available for genuine, non-dependency blockers. * fix(gateway): release a dependency-blocked claim to pending instead of looping A task that reached claimed/in_progress with an unfinished dependency was left in that state when the claim guard rejected, so the orchestrator's respawn loop kept reviving its assignee — which could make no progress — burning work for nothing. The claim guard now releases such a task back to pending. claimed -> blocked is not a legal transition, so pending — held by the dispatch dependency filter — is the lifecycle-correct resting state: the respawn loop ignores pending tasks, and _unblock_dependents re-dispatches it once the upstream reaches a terminal state. release_dependency_blocked_claim shares a _force_unclaim_to_pending core with unclaim_for_reaper so both record a truthful work-session abandon reason. * feat(security): warn at startup in header-trust mode + document the auth posture When ROBOCO_AGENT_AUTH_REQUIRED is not enabled the API accepts the X-Agent-Id / X-Agent-Role headers without a signed token, so any client that can reach it may act as any role (including 'ceo'). The API now logs a clear warning at startup in this mode, and the README gains a Security section documenting the auth posture and how to harden it. Acceptable only on a trusted private network — do not expose the API to untrusted networks. * fix(workspace): scope the refresh fetch to current + default branch ensure_workspace's healthy short-circuit ran an all-refs 'git fetch origin' to keep every origin/<branch> ref current. On a monorepo with many accumulated feature/* branches that exceeds the refresh timeout, the fetch silently fails, and the workspace keeps a stale base — so an agent builds on an out-of-date branch. The refresh now fetches only the workspace's current branch and the repo's default branch (resolved via origin/HEAD), with --no-tags --prune: it transfers near-nothing and can't time out. Readers need their own branch and the default; the integration branch is refreshed at branch-creation time. * fix(git): refresh a dependency-blocked task's branch off the current integration tip A cross-cell dependent (e.g. a frontend task waiting on the UX design) was branched off a base captured before its upstream merged into the integration branch, and the branch was never re-synced — so the agent built on a stale snapshot with none of the upstream's work. Two changes close the gap: - release_dependency_blocked_claim now clears branch_name, so the re-claim (after the dependency clears) re-runs branch creation. - create_branch, when the branch is already on disk with no commits of its own, resets it onto the freshly-pulled base — the dependent now builds on the current integration tip. A branch carrying real commits is left untouched, so no work is discarded; the cell->leaf cascade carries the upstream down to the dev branch automatically. * refactor(gateway): drop the sibling-sequence claim guard Sibling sequence no longer gates a claim. Cross-cell ordering is enforced by task dependencies — a cell task that depends on another is held until its upstream reaches a terminal state, a stronger, status-aware gate than the sequence-number check. That check was dormant in practice anyway: every fan-out child carries sequence 0, on which the guard short-circuited. `sequence` stays a sibling-ordering / dispatch-priority field (list_pending ordering and the panel). Removes sibling_sequence_guard and its _earlier_blocking_sibling helper, the now-unused skip_sequence parameter threaded through the claim verbs, and the sibling fetch that fed it. * feat(gateway): sort a cross-cell dependent after its upstream When the frontend cell task is wired to depend on its UX/UI sibling, set its sequence to the upstream's sequence + 1 so it sorts after the design it waits on — list_pending ordering and the panel now show UX ahead of the implementation it gates, in either delegation order. Adds TaskService.set_sequence (the sibling-ordering field is a service write; it carries no claim-gating semantics — dependencies gate claims). * feat(gateway): make the backend cell depend on UX too UX/UI design defines the screens and API contracts both implementation cells build against, so the backend cell — not just the frontend — waits on the UX/UI cell task in a product fan-out and sorts after it. Wires in either delegation order: a backend task delegated after UX gets the dependency directly; a UX task delegated after a still-pending backend sibling retro-wires it. Mirrors the existing frontend wiring (_depend_backend_on_ux and _depend_pending_backends_on_ux). Backend is held by the same dependency gate, so it costs no extra dispatch churn. * fix(websocket): forward notification acks instead of logging them incomplete The bridge handler serves both notification.sent and notification.acked, but acked events carry `agent_id` (the acking agent) rather than `recipient_id`, so every acknowledgement tripped the missing-field guard and logged "Incomplete notification event" instead of reaching the panel. Accept either field as the recipient. * feat(api): hint the full UUID when a truncated task id fails validation Agents copy the 8-character task prefix the system shows them (the commit prefix, task summaries) and send it as task_id, which fails UUID validation with an opaque "invalid length" 422 and wastes a call. The request-validation handler now detects a task_id UUID error and attaches a `remediate` hint telling the agent to retry with the full 36-character UUID from its task envelope. * fix(audit): record the blocked transition when a task is escalated Escalation sets a task to blocked by writing task.status directly, which bypassed the validated transition helper and so never emitted a task.blocked audit row — the lifecycle moved but the Auditor saw nothing. Extract the audit emit from the central transition helper into _emit_status_transition_audit and call it from the escalate path, capturing the prior status and outgoing owner before reassignment so the row is attributed correctly. * fix(docs): stop doubling the docs path so design specs index into RAG The documenter sometimes hands a doc path already rooted at docs/, and joining it onto DOCS_BASE_PATH (/app/docs) produced /app/docs/docs/..., so the file was never found and the spec never indexed — the frontend cell could not retrieve the UX design over RAG. Normalize the path before joining: trust an absolute path, otherwise strip a single redundant leading docs/ segment. * feat(security): let the control panel authenticate in secure mode With ROBOCO_AGENT_AUTH_REQUIRED=true every request must carry a valid HMAC token, which locked the human control panel out — it sends role headers but no token. nginx, the only trusted hop between the browser and the API, now injects the CEO token on /api and /ws, so the browser never holds the signing secret. The injected value is just the existing per-agent token issued for the CEO identity (issue_panel_token), so the token-verification path is unchanged. An empty value (dev/header-trust mode) renders to no header. `make panel-token` prints the value; set it as ROBOCO_PANEL_AGENT_TOKEN in .env before enabling secure mode. .env.example and the README Security section document the flow. * chore(compose): consolidate the two compose files into one docker-compose.yml and docker-compose.yaml had diverged: .yml — the file Docker actually uses — carried ROBOCO_PUBLIC_BASE_URL but was missing the /app/manifests bind-mount, while .yaml had the manifests mount but not the base URL. Merge the union into docker-compose.yml and delete the duplicate so there is one source of truth and no "multiple config files" warning. This activates the manifests mount in the deployed file: without it the orchestrator writes per-agent tool manifests to its ephemeral container fs, they never reach the host for the daemon to bind-mount, and agents fall back to all-verbs registration. Drop the stale .yaml reference from the config.py docstring, the labeler, and the CI path filters. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-06-05 16:35:22 +02:00
Renn F	9aa30fb945	100% Coverage	2026-05-06 21:02:31 +02:00
Renn F	64c48356d0	test: lift coverage 41% → 76% (+1068 tests across 36 files) ... Service-level tests now exercise provider, permissions, project, journal, messaging, work_session, metrics, kanban, extraction, learning, notification, dashboard, llm_routing, a2a, task, repository_base, audit, db_seed, branch_name, indexed_document, query_helpers, agent. API route tests cover provider, journal, project, sessions, dashboard, work_session, tasks, a2a, groups, notifications, agents, channels, messages, kanban, api_resources. Pure-function helpers covered: handlers, deps_helpers, middleware, middleware_docs, transcription, pr templates, agents_config, errors, logging, journal/notification/channel/a2a access, task_lifecycle, streaming, converters, crypto, schemas (common + websocket), events, permissions extras. pyproject ruff per-file-ignores extended for tests so PLR2004 (status code magic values), PLC0415 (lazy imports), PLR0913 (fixture params), ARG001 (unused fixture deps), SIM105, and E501 don't fight test idioms.	2026-05-06 00:32:52 +02:00