5 Commits

Include renames

Author	SHA1	Message	Date
3 people	c40a7a39c3	feat: Telegram V3 — Mini App cockpit (initData auth + /tg surface) (#554) ... * feat(telegram): Mini App auth — initData validation mints the cloud-auth session cookie * feat(panel): /tg Mini App cockpit — approvals, inbox, read-only board, A2A chat * fix(telegram,panel): unconditional webapp-auth rate limit, future-dated initData rejection, anchored /tg matcher * docs(map,rag): Telegram Mini App auth route, initData validator, (tg) surface --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-07-18 02:47:59 +02:00
3 people	0e9f21de69	fix(api): default event loop to asyncio + cancellation-safe commit — kills the CI segfault (#340) ... * fix(api): default the event loop to asyncio + cancellation-safe commit The recurring CI e2e segfault traced to uvloop: the harness's uvicorn.run() auto-selected it while production's serve() path never consulted Config.loop (stock asyncio, accidentally safe). Every launch site now resolves ROBOCO_UVICORN_LOOP (default asyncio; uvloop opt-in), and DbCommitMiddleware's commit-in-send can no longer be interrupted mid-wire: on cancellation it gets a bounded grace to finish (committed data survives the 504), else invalidate-and-reraise. * feat(runtime): expected-stop breadcrumbs attribute container deaths Two production exit-143s had no attributable source: every orchestrator kill path now records a short reason breadcrumb, and the exit monitor consumes it -- an expected stop logs its reason at info, a genuinely unexpected one logs none_recorded plus docker-inspect diagnostics (OOMKilled, timestamps) so the next mystery SIGTERM self-identifies. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-07-08 16:01:01 +02:00
3 people	3849c1737e	feat(video): switch 0.19.0 renderer Remotion → HyperFrames (HTML-native, Apache-2.0) (#314) ... * feat(video): rewrite sidecar render core to HyperFrames (in place) * feat(video): convert motion compositions from Remotion TSX to HyperFrames HTML * refactor(video): rename render client to video_renderer_client (renderer-agnostic) * chore(video): rename remotion-renderer prose in test_video_pipeline docstrings * chore(video): rename sidecar to video-renderer + add system ffmpeg for HyperFrames * chore(video): rename stray remotion-renderer refs in sidecar + py docstrings (controller cleanup) * chore(video): fix stale Remotion API names in Dockerfile comment (controller cleanup) * docs(video): rewrite video-engine prose for HyperFrames + add map entry + folded prose fixes * docs(video): add trailing newline to docs/map/video-engine.md (controller cleanup) * chore(video): drop internal spec refs + minio/test suppressions (folded hygiene) * fix(video): reclaim outDir on createRenderJob throw + hide empty 4th highlight Final whole-branch review (Opus) triaged two FIX items from the SDD nits ledger; the rest ship as-is. - render.js: a synchronous throw from createRenderJob (post-mkdtemp, not awaited) left an empty outDir on disk — the outer catch only reclaimed extractDir. Reclaim outDir too when it exists, and correct the stale comment that claimed the out dir was never created. - {vertical,square}.html: the 4th highlights <li> lived in the DOM hidden only by JS, so a no-JS / failed-script render would show an empty bullet. Start it style="display:none" and reveal on populate, so an unscripted render shows nothing instead. Vitest smoke (release-announcement.test.js) 4/4 green; render.js syntax checked. Python suite untouched by this fix (JS/HTML only). * fix(video): type _override_db yield as AsyncSession | None T7 widened _build_app's db_session param to AsyncSession | None (to drop the 4x # type: ignore[arg-type] on the DB-independent _build_app(None, ...) calls) but left the inner _override_db fixture typed AsyncIterator[AsyncSession] — so 'yield db_session' yielded AsyncSession | None into a declared AsyncSession, and mypy failed at test_video_routes.py:177 ('Incompatible types in yield'). The DB-independent media tests pass db_session=None deliberately: their route uses a monkeypatched task service and never awaits the session, so yielding None is safe at runtime. Type the override's yield as AsyncSession | None to match — no cast, no # type: ignore, no assert, runtime behavior unchanged. The 3 media tests (3 passed) and the 19 db-gated tests (skipped locally) hold. * chore(gate): skip .superpowers scratch in markdown prose gate reflow_md.py walks the filesystem via rglob('*.md') and skips tooling dirs (.venv, .mypy_cache, .pytest_cache, ...) but not .superpowers/ — the superpowers SDD workflow's scratch dir (briefs, reports, progress ledger, all gitignored). A dev running SDD locally would hit a false markdown-prose gate failure on those transient files. Add .superpowers to SKIP_DIRS, consistent with the existing tooling-scratch exclusions. * fix(video): validate composition_id to close path traversal (CodeQL) compositionId flowed unvalidated from the POST body into path.join under extractDir/motion/compositions/, so a '../..'-style value could escape the composition dir (CodeQL: Uncontrolled data used in path expression). Validate at the trust boundary in server.js (/^[A-Za-z0-9_-]+$/) and add a path.resolve + startsWith containment check in render.js so it stays safe regardless of caller. * fix(mcp): send X-Agent-Token + X-Agent-Team from flow/do servers flow_server._build_headers and do_server._build_headers constructed only X-Agent-ID/Role/Correlation-ID, omitting X-Agent-Token and X-Agent-Team (unlike ApiClient._get_agent_headers used by the other MCP servers). Latent since the gateway refactor — surfaced when ROBOCO_AGENT_AUTH_REQUIRED=true was armed on the NAS, 401-ing every flow/do verb with 'Missing X-Agent-Token header'. Add both headers (mirroring ApiClient) so the HMAC gate passes. Tests assert the headers are now injected. * [video-engine] Per-project video_engine_enabled opt-in toggle Mirrors ci_watch_enabled (migration 048): the global ROBOCO_VIDEO_ENGINE_ENABLED flag arms the subsystem; the new projects.video_engine_enabled column (migration 063) opts a repo into authoring against its motion/ dir. VideoEngine._opted_in_project no-ops open_video_task at the single chokepoint covering all three trigger paths (on-release, on-spotlight, CEO on-demand) until the operator flips it in the panel edit-project dialog. Existing projects stay opted out (server_default=false). * fix(auth): send X-Agent-Token + X-Agent-Team from all agent->API call sites The prior fix (6ed4e139) covered the flow/do MCP servers but missed four other agent->orchestrator call sites that built the header dict by hand and omitted X-Agent-Token and/or X-Agent-Team. With ROBOCO_AGENT_AUTH_REQUIRED armed on the NAS, every one 401s: - agent_sdk/server.py: the session-end post-mortem flush (/api/journals/me/entries), A2A persistence + offline fallback (/api/a2a/*), and the stopped-without-transition auto-substitute (/api/tasks/auto-substitute) — all sent only X-Agent-ID/Role, so each 401'd 'Missing X-Agent-Token'. Add a shared _agent_headers() helper (mirroring flow_server._build_headers) and route all four through it. - agent_sdk/secretary_driver.py: _headers() sent the token but not the team, so the HMAC gate 401'd with signature mismatch (secretary is board-team; token signed with team='board', verified with team=''). Add the team header. - mcp/git_readonly.py: the read-only git MCP sent only X-Agent-ID/Role — no token, no team — so /api/git/* 401'd once auth was armed. Convert the static _HEADERS to a _headers() helper with team + token. - runtime/orchestrator.py: the cell-PM auto-submit self-API call acted as a PM with a hand-built {X-Agent-ID, X-Agent-Role} dict — no token, no team — 401ing under auth-required. Add _agent_api_headers(uuid, role) mirroring _system_api_headers, and use it. Tests: _agent_headers round-trip (token + team, team-omitted when None), _agent_api_headers carries a signed PM token + team. * [auth] Omit UNSIGNED self-call token in dev mode + video-engine test mypy fix _agent_api_headers sent the UNSIGNED sentinel when ROBOCO_AGENT_AUTH_SECRET was unset, but the dev-mode middleware rejects a presented-but-unverifiable token with 401 signature mismatch (while accepting a missing one). The cell-PM auto-submit self-call 401'd in every dev run, regressing test_auto_submit_cuts_the_pm_turn. Attach the token only when a secret is set. Also fix the FromClause.update mypy error in the per-project video-engine opt-out test (ORM row load + flush). * [auth] Omit UNSIGNED agent token at every agent->API call site The orchestrator injects ROBOCO_AGENT_TOKEN=UNSIGNED when the HMAC secret is unset at spawn. The API middleware rejects a presented-but-unverifiable token with 401 'signature mismatch' even in dev mode (auth not required), so forwarding UNSIGNED turned every flow/do/SDK/secretary/git verb into a 401 — the live pr_reviewer/i_am_idle signature-mismatch loop. Omit the header when the token is the UNSIGNED sentinel at all five agent-side header builders; dev accepts a missing token, prod 401s with 'Missing X-Agent-Token' (the clear respawn-with-secret signal). Add a structlog diagnostic on the middleware reject path so the next mismatch logs the exact (id, role, team, token_unsigned, auth_required) inputs. * [auth] Self-heal stale agent tokens at orchestrator startup A token is signed once at spawn. If ROBOCO_AGENT_AUTH_SECRET drifts afterwards (a .env change, a compose recreate that reloads the orchestrator's env without recreating agent containers, an image redeploy), the surviving agent keeps sending its old token and the middleware 401s every verb with 'signature mismatch'. The container stays alive heartbeating, so the reaper never reclaims it and no fresh agent spawns: the fleet stalls. _heal_stale_agent_tokens runs at startup (before _readopt_running_agents) and kills each running agent container whose baked-in token no longer verifies against the current secret, so normal dispatch re-spawns it with a freshly signed token. Inert when the secret is unset (dev): verify fails for every token without a secret, so the heal would kill the whole fleet without this gate. Best-effort: a probe failure leaves the container alone (the reaper still covers it). * [auth] Sign agent token over the UUID, not the slug (pr_reviewer 401 root cause) The token was signed over the agent slug (_append_agent_auth_env) while the MCP servers send X-Agent-ID as the agent UUID (_generate_mcp_config, since 453a7ae2 — gateway v1 parses X-Agent-ID as Annotated[UUID]). The middleware verified HMAC(uuid:role:team) against a slug-signed token → 'signature mismatch', token_unsigned=false. Latent for 2 months until 6ed4e139/53391f22 made the MCP servers forward the token. The c0328971 startup heal missed it: docker exec printenv reads the container-level ROBOCO_AGENT_ID (the slug), so the heal verified the slug-signed token against the slug → matched → didn't kill the stale container, which kept 401ing (its MCP server sends the UUID). Fix: sign the token over the UUID, set the container ROBOCO_AGENT_ID to the UUID too (so the SDK server — which inherits container env, not the MCP manifest env — sends UUID consistently), and resolve the container-env id to its UUID in _heal_stale_agent_tokens so pre-fix stale containers are evicted on next restart. Regression test: test_heal_kills_slug_env_container_with_slug_signed_token. * [scan] gate A2A/notification/stream agent-id deps under cloud auth (C1) * [scan] omit UNSIGNED agent token from MCP server headers (H1) * [scan] fail loud when cloud auth and nginx CEO-token are both armed (H2) * [scan] cache last-known-good auth-probe result in panel proxy (C2) * [respawn] Tripped breaker self-heals after a cooldown A DB-durable PM-respawn counter (migration 051 / e2f7097a) wedges forever once tripped: the only reset was a task status change, which can't happen while the breaker blocks the spawn. So a deploy that fixes the underlying loop (auth/prompt/schema) couldn't clear the wedge without manual DELETE surgery on respawn_tracker — the 2026-07-06 pr-reviewer-1 loop, where the auth fix cleared the 401 but count=63 survived restart and kept skipping the dispatcher spawn for an external-PR task. Freeze last_check at the trip tick and, after pm_respawn_trip_cooldown_seconds (default 300), let ONE spawn through. A still-wedged task re-trips after the threshold (bounded re-burn ~3 spawns per window); a fixed one advances and the status-change path fully resets. Restore re-stamps last_check to now, so a freshly restored row still trips immediately — durability preserved, which is why the migration-051 persistence tests still pass. * [scan] fix test_deps callsites for cloud-auth-gate signature change (C1 followup) * [scan] per-IP rate limit on /auth/login under cloud auth (L31) * [scan] Phase 1 auth/security fixes under 0.19.0 CHANGELOG * [scan] secretary token signs over real team (board) not empty — fixes /api/secretary/* 401 (L31-class) * [scan] LoginRateLimiter: key off X-Forwarded-For first hop + redis-down fail-open test nginx is the single entry point; request.client.host is the nginx peer IP, collapsing every external client into one limiter bucket (self-DoS amp). Read the downstream client IP from X-Forwarded-For (first hop) / X-Real-IP, falling back to the peer. Adds coverage for the XFF keying, the redis-down fail-open branch, and drops a redundant asyncio marker on a sync-TestClient test. * [scan] nits: describe login_max_attempts + replace cast with assert in get_current_agent_slug login_max_attempts was the only bare cloud-auth field; add a Field description matching the surrounding idiom. Replace cast('str', ctx.slug) with a runtime assert that fails loud if the cloud-auth ctx invariant breaks, and drop the now-unused cast import. * [scan] secretary token: use get_agent_team resolver + complete spawn-shutdown mock team (0dfd45ca followup) * [scan] require agent HMAC token under cloud_auth (close v1 flow/do header-trust) * [scan] _require_ceo accepts CEO session cookie under cloud_auth * [scan] HTTP require_panel_token accepts session cookie under cloud_auth * [scan] gate /api/settings behind panel token * [scan] gate unauthenticated /api read routes (agents/a2a-tasks/kanban/usage/rate-limits) * [scan] hoist deferred test imports to top-level (clear PLC0415) * [scan] Phase 1b e2e smoke + CHANGELOG * [scan] add_dependency rejects self-reference + cycle (M18) * [scan] WorkSessionService.create translates IntegrityError to ConflictError (H10) * [scan] _qa_or_doc_claim locks the task row FOR UPDATE (M19) * [scan] docs_complete + mark_pr_created lock the task row FOR UPDATE (H4) * [scan] gate complete() IN_PROGRESS on leaf/branchless only (H3) * [scan] _unclaim_from_blocked clears stale pre-block snapshot (H5) * [scan] admin_set_status terminal guard + skip revision bump under force (M20) * [scan] cell_pm_complete idempotent pre-check before merge (H7) * [scan] wrap gateway post-runner side effects in try/except (H6) * [scan] pass_qa/fail_qa accept AWAITING_QA only (L29) * [scan] mark_pr_created passes audit_agent_id (L30) * [scan] phase 2 e2e smoke - one scenario per finding * [scan] phase 2 quality gate ruff format + check: green mypy roboco/: green (357 files) pytest unit+integration: 6905 passed, 10 pre-existing DB-contamination failures (pass in isolation) e2e smoke: 11 passed, 4 cross-scenario workspace-contamination failures (all 6 state-machine scenarios pass individually) Quality-gate fixes: - move function-local imports to module top (PLC0415) - fix M19 regression: submit_for_qa clears active_claimant_id so the competing-claimant guard lets the QA claim through - fix H7 regression: _StubGit gains is_pr_merged_for_task - fix M19 unit tests: mock session.execute for the FOR UPDATE lock - e2e H3: notes >= 20 chars; e2e H5: rich i_will_work_on inputs + PM unclaims (block reassigns to PM) * [scan] move active_claimant_id clear into pass_qa/fail_qa + admin_set_status (M19 follow-on) Phase 2 opus whole-branch review found the M19 follow-on clear lived in the gateway wrappers (qa_pass/qa_fail) not the transition methods (pass_qa/fail_qa) themselves. The direct REST routes POST /pass-qa and POST /fail-qa call the transitions directly, bypassing the wrappers and leaving the QA's stale active_claimant_id set in AWAITING_DOCUMENTATION / NEEDS_REVISION — the competing-claimant guard then rejects the next legitimate documenter/QA claim. admin_set_status had the same gap for a non-blocked override into a review/queue state (IN_PROGRESS->AWAITING_QA left the dev's id, blocking qa_claim). Root-cause fix: move the clear INTO pass_qa and fail_qa (mirroring submit_for_qa), add a clear in admin_set_status when new_status in _REVIEW_QUEUE_STATES and from_status != BLOCKED, and drop the now-redundant clears + flushes from the qa_pass/qa_fail wrappers. Every caller is covered; the wrappers keep their actor-mismatch warnings. Covering tests: test_pass_qa_clears_active_claimant_for_doc_claim (asserts a subsequent doc_claim succeeds), test_fail_qa_clears_active_claimant, test_admin_set_status_into_review_queue_clears_active_claimant, test_admin_set_status_non_review_queue_keeps_active_claimant. Updated the two wrapper unit tests that asserted the wrapper clears (now the transition's job). * [C3] unindex_journal_entry + call from delete_entry JournalService.delete_entry deleted the DB row but never de-indexed the RAG chunks, so deleted/private journal content bled forever into RAG answers and claim-time briefings. Add OptimalService.unindex_journal_entry mirroring unindex_playbook (vector-store delete_by_source + tracking-row delete via get_db_context, both idempotent + best-effort), and call it from delete_entry after the row commit inside a try/except so a de-index failure never errors the delete. * [M25] learning_id hashes full content to avoid collision The memory distiller emits lessons with a fixed 'Problem: …' opening shape, so two distinct lessons whose first 100 chars match collided on learning_id = f"lrn-{md5(content[:100])[:12]}". replace_on_reingest then routed both to the same source URI and the second ingest's replace_chunks DELETE wiped the first lesson's chunks — silent data loss. Hash the full content (widening the hex slice 12→16) so distinct bodies get distinct ids and each retains its chunks. * [H13] reject non-internal local_llm_base_url at config load * [M28] bulk-insert learning broadcast instead of N+1 * [M27] mark_read/mark_all_read stamp only the unread rows seen at call time mark_read and mark_all_read used to zero the unread counter FIRST, then run a bulk UPDATE … WHERE read_at IS NULL that stamped every inbound unread row. A send_chat_message committing between the counter-zero and the UPDATE inserted a new read_at NULL row that the UPDATE then stamped as read — the new message was silently consumed while the counter stayed 0. Mirrors get_unread_messages (same file): SELECT the unread message IDs at call time, UPDATE exactly those IDs, then recompute the unread counter from the DB via the existing _reset_unread_counter helper. A message arriving mid-call is not in the selected ID set, so the UPDATE skips it and the recomputed counter keeps it unread. * [H12] dedup: exact to_agents predicate + purpose discriminator + ack DEL * [M23] playbook indexed_ok/indexed_at + startup reconcile of unindexed approved * [M24] RAG indexing dead-letter + janitor reclaim + failed_index_count health * [L23] institutional_memory_status sentinel distinguishes below-floor/empty/error/disabled * [L26] sweep_expired_notifications re-escalates stale unacked ack-required * [phase3] e2e smoke + CHANGELOG for 0.19.0 * [M24] _reindex_journal_entry honors is_private (C1 review fix) Dead-letter replay mirrors the original journal._schedule_rag_index path: a private entry is never indexed into the shared JOURNALS corpus, and a private learning is still recorded into LEARNINGS as non-shareable. Previously the replay always called index_journal_entry and skipped record_learning for private learnings, leaking private content on replay and dropping the legitimate non-shared learning. Three regression tests. * [H11] clone via git -c http.extraheader, not URL-embedded PAT * [H11] _sync_read_clone fetch via http.extraheader, not URL-embedded PAT Sibling site to the clone fix: the conventions read-clone refresh ran 'git fetch --tags <https://TOKEN@host> <branch>', exposing the PAT in the fetch argv on the orchestrator host. Mirrors the clone site's per-call '-c http.extraheader=Authorization: Basic …' prefix + bare URL. SSH URLs and tokenless public repos unchanged. * [H11] release_executor clone+push via http.extraheader; delete _inject_token_into_url * [H8] rebase_onto_base gates on clean tree like pull * [H9] _link_commit_to_task flushes, doesn't commit out-of-band * [M38] _pr_is_merged returns None on HTTPError; caller assumes merged * [M39] _cherry_unmerged_entry marker grep anchored to commit-prefix * [L1] thread actor_agent_id through update_pr_for_task * [H8] fix rebase test mocks for clean-tree gate H8 inserted a 'git status --porcelain' dirty-tree gate at the top of rebase_onto_base (mirroring pull). The 3 rebase control-flow tests mocked _run_git with a side_effect list matching the OLD call sequence (no leading status call), so every call shifted by one and the assertions missed. Prepend a clean-status result to each list so the gate passes and the fetch/checkout/reset/rebase/diff/abort/push sequence aligns. Verified: 16 passed (was 3 failed/13 passed post-H8, 16 passed pre-H8). * [L2] push --force-with-lease instead of bare --force * [L1] refresh stale workspace-resolution docstrings pr_target and _workspace_for_branch still documented the actor → assigned_to → created_by fallback chain that L1 removed from _resolve_workspace_agent_id. Update both to the post-L1 actor → assigned_to → None resolver (project.workspace_path as the final fallback) so a future reader doesn't rely on a fallback that no longer exists. * [M37] merge_pr locks the work_session row FOR UPDATE * [phase4] e2e smoke + CHANGELOG for 0.19.0 * [phase4] fix M37 test flake + document H8 skip The opus whole-branch review flagged the M37 concurrency tests as ~50% flaky: both asserted caller A wins the FOR UPDATE race, but which caller wins the lock is non-deterministic. When B won, the 'assert a_row.merged_by == a_merger' branch flipped false even though the production code (M37) was correct — exactly one merger recorded, audit trail intact. Assert the invariant instead: both rows COMPLETED, both report the same merged_by, value in {a_merger, b_merger}. Applied to both the unit test and the e2e twin. Also documents the H8 e2e skip in the module docstring (the report claimed it was documented there but it wasn't) and drops the internal 'Phase 4' label from the docstring header in favor of the public '0.19.0' version anchor. * [H24] wait_for_ci polls through the window on non-success * [H25,L34] release mutex orphan-sweep on start + shared redis client * [M1] tiktok _refresh commits rotated tokens in an independent session * [H25] drop new type:ignore in orphan-sweep test (constraint cleanup) * [M2] feature-spotlight re-arms when exploration stale past 2x interval with no live HoM spawn * [M6,M7] mark_seen after meaningful+project; persist since_id cursor in redis * [M3,M5] reject() guards COMPLETED; edited_body deferred into the single-flight lock * [M4] bound list_completed_video_tasks + ix_tasks_source_status_created index (migration 066) * [M8,M9,L9] pass head_sha to CI gate; _run_git 30s timeout; _commits_since split maxsplit 2 * [M10,L35] dedupe dep_update by (git_url, command); fold redundant per-project queries * [L36] gather ci_watch telemetry sweep instead of sequential iteration * [L11] document self_heal fingerprint is stable per-signal by design * [M11] engine-loop liveness watchdog: heartbeat + 2x-interval staleness alert * [M21] video render loop commits per-task, not one trailing commit * [M22] _detect_stuck_tasks skips held-CEO-source tasks * [L6] video_renderer_client._save writes temp + atomic rename * [phase5] e2e smoke + CHANGELOG for 0.19.0 * [M11] instrument x_mentions + roadmap engine loops with liveness heartbeats * [phase5] fix-wave: correct e2e M11 unit-test filename + strengthen failed-cycle heartbeat assertion * [C4] panel WS: shared /ws/system socket + long-tail retry + pong watchdog * [H15] video-post-queue caption derived per render (mirror x-post-queue) * [C4-fix] panel WS: discriminating long-tail tests + drop dead freeze block + evict dead shared conn on manual disconnect Finding 1 (Critical, websocket.test.ts): the two long-tail-retry tests fired onopen between close cycles, which reset reconnectAttempts to 0 each cycle, so they passed under the pre-fix 3-attempt gate. Rewrote both to NEVER fire onopen between closes, so attempts accumulates: test 1 asserts state stays 'reconnecting' past attempt 3 (old gate would flip 'disconnected' terminal); test 2 asserts a new socket is constructed within 30000ms at attempt 7 where uncapped 5000*1.5^7 ~= 85s (old uncapped code would leave the timer unexpired). Verified both FAIL on a reverted old-shape connection.ts and PASS on the fixed code. Finding 2 (Important, connection.ts): the 'if (raw >= cap) this.reconnectAttempts = exp' block was a no-op (exp was just read from the same field) and the unconditional increment afterwards grew the counter regardless. Deleted the dead block; kept the Math.min cap on the delay. Replaced the misleading ponytail comment with an accurate one: delay is capped, counter grows unbounded but delay is bounded. Finding 3 (Important, use-websocket.ts): manual disconnect() tore down the shared conn for all subscribers but left the dead (manualClose=true, never reconnects) entry in _sharedSockets, so a later mount hit the reuse branch, attached a subscriber, replayed 'disconnected', and never called connect(). Added a urlRef and _sharedSockets.delete(url) in the manual disconnect callback so a later mount reopens a fresh conn. * [H16] settings Save wired to settingsApi (persist + read back) * [H17] tasks page passes status/team/limit to useTasks (server-side filter) * [H18] useAgents roster re-derives on live-status change (statusEpoch in queryKey) * [M40] useMetrics reads agent counts from useAgentStatus cache (dedupe poll) * [H18] tighten useAgents statusEpoch comment (drop spec ref) * [M40] drop spec ref + tighten useMetrics comment * [M41] scorecard refetchInterval 60s -> 5min (25 req/min -> 5) * [M42] feature-flag off-transition confirm + pending-keys Set * [M43] X/TikTok credentials clear-behind confirm dialog * [M44] rate-limit syncFromApi merges (keep fresher hitAt) + A2A reconnect invalidation * [phase6] proxy.ts cookie-check comment + CHANGELOG Fixed entries * [phase6] drop stale WS pin-attempts comment + fix tasks-page lead-in * [H21] type DelegateRequest.estimated_complexity as Complexity (reject critical) * [H22] type SoftBlockRequest.resolver_type as BlockerResolverType (no silent AGENT fallback) * [H23] serialize TaskTable.documents into TaskResponse (DocRefResponse) * [L27] delete SubstituteRequest phantom suggested_role/suggested_team fields * [L14] Envelope.not_found defaults remediate (guide re-fetch + re-issue) * [L28] delete unused ListResponse generic (dead code; pagination deferred) * [H19] _delegate_static_guards allow cell_projects roots (cross-cell MegaTask) * [M13] MegaTask confirm-batch idempotency key from session_id (SETNX guard + result sidecar) * [M14] strip assigned_to from MegaTask drafts (no board-owned root-subtask deadlock) * [H20] thin_routes receiver-gate add/add_all/merge (no false block on set/cache.add) * [M16] tighten noqa code-capture to [A-Z0-9, ]+ (no false block on natural prose) * [M45] conventions read-clone force-refetch on read (no 30s stale map window) * [L25] conventions._resolve returns (root, sha); ORM mutated on the event loop * [M15] open_conventions_pr force-pushes disposable scaffold branch (no silent None) * [L24] roadmap cycle completion emits status-transition audit * [Phase7] CHANGELOG: 15 schema/conventions/MegaTask/API fixed (H21-H23,L27,L14,L28,H19,M13,M14,H20,M16,M45,L25,M15,L24) * [Phase7] lint gate hygiene: shorten docstring (E501), sort imports (I001), hoist AuditLogTable import (PLC0415) * [H14] Enable the GROK provider row in _apply_grok so routing reaches the GrokCliProvider * [M31] Route GROK active-token resolution to usage.json so live usage reflects grok agents * [M32] Pass cache read/write tokens to calculate_cost in the usage sweep so live cost reflects Anthropic cache spend * [M33] Park Ollama-Cloud rate limits via a marker map so a glm-5.2:cloud 429 parks instead of crash-respawning * [M34] Sweep orphan agent_spawn_sessions at startup so crashed-run tokens roll into usage/cost summaries * [L12] Persist revisit_resets (migration 067) so the PM-respawn breaker's revisit counter survives a restart * [L18] Date-gate the Sonnet-5 promo revert so billing returns to list rates after 2026-08-31 * [L20] Warn when ROBOCO_GROK_RUN_LOG yields no session id instead of silently falling back to a zero-usage env id * [phase8] CHANGELOG: LLM provider routing, usage capture, billing fixes * [phase8] Trailing ruff format hygiene (orchestrator marker tuples, token-sweep test signatures) * [phase8] Fix mypy: rename GROK-branch tokens var so transcript fallback stays reachable * [M35] Add an expiring agent-token format (iat/exp) with backward-compatible verify * [M35] Wire agent-token TTL at spawn (config + orchestrator + grok) so tokens are bounded * [M36] Add JWT jti claim and re-mint the sliding cookie only near expiry so a stolen cookie's exp is fixed * [M36] Redis jti revocation: read_token rejects revoked jtis and logout revokes the current jti * [phase9] CHANGELOG: bound agent tokens + sliding-cookie re-mint window + jti revocation * [scan-fix] mypy: type-annotate test files for make-quality gate CI's make quality runs mypy roboco/ tests/; the scan-fix program's local gate ran mypy roboco/ only, so test files were never type-checked. Fix all 67 errors across 23 test files with real annotations/casts/asserts/dead-code removal — no # type: ignore / # noqa added. * [e2e] Per-test DB isolation + dispatcher re-claim before PM complete * [scan] Regenerate verb tables for delegate Complexity type * [scan] Reduce 9 xenon C-ranks to B (auth, orchestrator, gateway, services) * [scan] Restore short-circuit time.time() in verify_agent_token (security path) --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-07-07 10:09:23 +02:00
3 people	3ccc723cd4	v0.17.0 — Wave 3: sandbox DB, DB isolation, mobile UI, cloud auth, X account, roadmap engine (#303) ... * feat(sandbox): throwaway per-agent Postgres/Redis sandbox containers Orchestrator-provisioned sibling containers per agent spawn (SandboxProvisioner, roboco/runtime/sandbox.py). Per-project opt-in via projects.sandbox_services (migration 057); master switch ROBOCO_SANDBOX_DB_ENABLED, default-off, armed in the NAS compose only. When active, ROBOCO_TEST_DB_* / ROBOCO_TEST_REDIS_* point at the sandbox and the prod-creds gate-env injection is suppressed (sandbox replaces, never coexists). Sandbox lifetime tracks the agent container: teardown at every removal path, orphan janitor at startup + each reaper tick with a grace window for mid-flight spawns. The pre-spawn stale-clear spares the just-provisioned sandbox; provision pre-clears stale same-named containers from a crash-missed teardown. Panel: per-project sandbox-service switches in the edit dialog + feature flag card entry. * docs: CLAUDE.md entry for the sandboxed dev DB/Redis subsystem * feat(security): isolate prod Postgres/Redis from agent containers (roboco_data network) Second user-defined bridge roboco_data carries postgres+redis only; the orchestrator is multi-homed (default + data). Spawned agents and their sandbox sidecars stay on roboco_default and can no longer resolve or reach roboco-postgres:5432 / roboco-redis:6379 (redis has no auth — membership is its only containment). Normal bridge, so host-published ports (15432/16379) keep working. Applied to both build composes and the registry compose; docker-compose.yml re-synced byte-identical with docker-compose.yaml (it had drifted by the sandbox flag block). ROBOCO_DB_NETWORK_ISOLATED (config default false, armed alongside the topology) suppresses the legacy _append_gate_env prod-creds injection: under isolation those creds dead-end, and unreachable creds are worse than none. DB-needing projects opt into sandbox_services instead. The flag is deliberately not a panel feature flag - it must travel with the compose networks: stanzas. Preserved by construction: agent<->agent A2A and orchestrator->agent SDK polls on :9000, MCP->orchestrator on :8000, ollama reachability, docker exec/inspect (daemon socket), host port publishing. * feat(panel): full mobile responsiveness pass Shared primitives: useIsMobile (useSyncExternalStore, hydration-safe, memoized matchMedia subscribe), ResponsiveTable table->card switch below md (single subtree mounted, no duplicated interactive rows), scrollable snap TabsList in the base primitive (justify-center-safe so the first tab stays reachable on overflow), persistent md:hidden bottom tab bar (Overview/Tasks/Kanban/Chat, safe-area padded). Applied: card lists for tasks/projects/products/work-sessions/sessions + the three raw metrics tables; CEO approval queue / release proposal / playbook review action rows stack on narrow; command-center reorders approvals above the fold on mobile; task-header metadata wraps; Communications + A2A become URL-driven single-pane drill-downs below lg (fixes the unconstrained-height ScrollArea bug) with dvh heights; recharts label density/radius adapts via useIsMobile; git diff viewer gets mobile font + wrap toggle; vh->dvh sweep; chat composers get safe-area-inset padding; dashboard main p-4 md:p-6 + pb-20 for the bar. Verified at 375px on the built app: bottom bar, drawer, approval-first overview, swipeable kanban tab strip. All gates green (eslint, tsc, vitest 249, next build 24/24 routes). * feat(auth): cloud auth via FastAPI Users (default-off, single-user cookie session) ROBOCO_CLOUD_AUTH_ENABLED (default off) lets the panel/API be exposed beyond localhost without changing the CEO's local no-login flow while off — get_agent_context and the WS gate are byte-for-byte unchanged in off-mode. On: header-trust dies for humans — any agent-role claim (ceo or a privileged PM/board role) with no valid HMAC token or session cookie is 401, closing the header-spoof hole on the host-published :8000 port for every role. The agent-fleet HMAC path and the system self-PATCH keep working unmodified in both modes. Single seeded CEO user (migration 058 users table, UserTable), no registration router — idempotent env-driven upsert at startup by PK. Cookie transport (httponly/secure/samesite=lax) + a JWTStrategy bound to a fingerprint of the current password hash (rotating the password invalidates every prior session). Sliding 30-day session: every authenticated request re-mints the cookie, so an active session never expires — no unexpected logouts. Panel: (auth)/login page + proxy.ts (Next 16 rename of middleware; probes /auth/status over the docker-internal URL, fails open to off) gate the dashboard; client.ts gets withCredentials + 401->/login. nginx unchanged. Review hardening: broadened the on-mode rejection from ceo-only to every non-CEO role without a valid token (was only closed when ROBOCO_AGENT_AUTH_REQUIRED was also armed); Next-16 proxy.ts rename to clear the middleware deprecation warning. * feat(x): RoboCo X account engine — HoM drafts, per-post CEO approval (default-off) ROBOCO_X_ENGINE_ENABLED (default off, inert without creds). Mirrors the ReleaseManagerEngine held-artifact shape: XEngine drafts a post when a release publishes (via a draft_release_post seam on ReleaseProposalService .approve) and drafts replies to meaningful mentions (dedicated poll loop, x_seen_mentions dedup ledger, per-cycle/open caps). Drafting is local-model-only, clamped to 280 chars. Nothing auto-posts — every tweet is a held task (source x_post/x_reply, confirmed_by_human=False, Secretary-owned, dispatcher-skipped) the CEO edits/approves/rejects in a panel queue. The four OAuth 1.0a secrets live Fernet-encrypted in a singleton x_credentials row (migration 059, all-or-nothing, API returns only has_credentials); decryption is server-side, agents never hold creds or egress. Hand-rolled OAuth 1.0a HMAC-SHA1 signer, no new dependency; NullXClient makes the unconfigured path a graceful no-op. XPostService.approve (CEO-only) is the sole caller of post_tweet. Review hardening: closed a double-post race — the approve path now re-reads committed task state inside the Redis lock and commits COMPLETED before releasing, so a concurrent approve that acquires the lock after the winner released can't re-post (SET-NX is non-waiting, and the route-level commit landed after the lock dropped). Added a regression test. * feat(roadmap): board roadmap engine — PO proposes themed cycles, CEO approves per-item (default-off) ROBOCO_ROADMAP_ENGINE_ENABLED (default off). Weekly, RoadmapEngine opens ONE held exploration task (source=board_roadmap, confirmed_by_human=False, Product-Owner-assigned), deduped to one open cycle. A dedicated one-shot _dispatch_roadmap_exploration spawns the PO solo (not the two-reviewer board path, which would also spawn HoM + fire Approve-&-Start). The PO explores read-only (git/KB/metrics/releases/charter/web) and makes one propose_roadmap call (PO-only content verb) authoring a themed cycle — goal + 3-7 item drafts — persisted as a roadmap_cycle marker (no table, no migration; head stays 059). The CEO acts per-item in the panel roadmap queue: approve materializes a BACKLOG task (source=roadmap, no assignee — never auto-starts), reject records a reason; all-items-terminal completes the exploration task. RoadmapService is idempotent per item. Dispatchers skip board_roadmap. Includes a real SQLAlchemy dirty-check fix (deep-copy the JSON marker before mutating, or the in-place edit + reassign compares equal to its own baseline and the UPDATE is skipped). Review hardening: create_task_from_draft now honors a draft-declared source only from a {prompter, roadmap} whitelist — drafts are LLM-authored, so an unbounded source could impersonate a privileged origin (release_manager would even wedge that engine's dedup). * chore(release): 0.17.0 Wave 3 — six default-off subsystems: sandboxed dev DB/Redis, prod Postgres/Redis network isolation, full mobile UI pass, cloud auth (FastAPI Users), the RoboCo X account engine, and the board roadmap engine. Plus the waves 1+2 work already on master since 0.16.0. Version bumped across the canonical set (config.py, __init__.py, pyproject.toml, panel/package.json, uv.lock); CHANGELOG [Unreleased] cut to [0.17.0]; docs/map delta added. Compose: every optional feature armed :-true in the NAS composes, OFF in the user-facing registry compose. Two opt-in exceptions default off (CLOUD_AUTH — needs email/password/secret + TLS, would otherwise fail startup; ROUTING_STRICT — fail-closed spawning). DB_NETWORK_ISOLATED stays on in both (coupled to the roboco_data topology). * chore(compose): arm cloud_auth + routing_strict ON in the NAS composes Every feature defaults ON in the NAS composes per policy — these two were wrongly left off. Both keep the ${VAR:-true} form so the operator controls the real runtime via .env: cloud auth needs ROBOCO_CLOUD_AUTH_EMAIL/_PASSWORD/_SECRET + TLS set there before a boot (else startup fails loud), and routing_strict is fail-closed. Registry compose keeps both off. * fix(ci): reflow board.md prose (quality gate) + document v0.17.0 env creds The roadmap section added hard-wrapped prose that failed the markdown prose gate; reflowed (token-invariant). Also brought .env.example current: cloud auth (now armed — needs SECRET or startup fails), routing strict, the X engine (panel-entered OAuth), and web research. * fix(ci): reduce cyclomatic complexity of five wave-3 blocks (xenon gate) The wave-3 subagents introduced C-rank functions the CI xenon gate rejects (my per-item reviews ran ruff/mypy/pytest but not xenon): - sandbox.janitor_sweep -> extract _list_labeled_sandboxes / _list_live_agent_containers / _prune_grace - x_client.fetch_mentions -> extract _parse_mention_items - x_engine.run_cycle -> extract _process_mentions - orchestrator._dispatch_pm_work -> extract the source-skip into a MODULE-level _is_held_ceo_source (module, not method, so the wholesale-mocked dispatcher unit tests exercise the real logic) - auth/seed.ensure_seed_user -> extract _apply_seed_updates (module avg -> A) Behavior-preserving; full suite green (11902), xenon clean. * fix(ci): declare pyjwt + fastapi-users-db-sqlalchemy as direct deps (deptry) The cloud-auth code imports jwt and fastapi_users_db_sqlalchemy directly but they were only transitive deps (via fastapi-users), which deptry (quality gate, DEP003) rejects. Declared explicitly; deptry roboco/ clean. Missed originally because local make quality stopped at earlier gates before reaching deptry. * feat(x): gate mention replies behind ROBOCO_X_REPLIES_ENABLED (default off) Per CEO decision: the X engine should only post about releases by default. Reading mentions needs a paid X API tier, so the mention-reply half is now a deliberate opt-in on top of release posting. New default-off flag x_replies_enabled gates the mentions poll loop (_x_mentions_poll_loop) and XEngine.run_cycle; release-post drafting (the release-proposal approve hook) is unaffected and still runs when x_engine_enabled + credentials are set. Added to FEATURE_FLAGS + the panel card. Tests: release posting works with replies off; run_cycle + the poll loop are no-ops with replies off. * fix: 401 only redirects to /login when cloud auth is on; panel-token strips .env quotes Two bugs that together dead-ended login in secure mode: - client.ts redirected to /login on ANY 401, so a mismatched panel token (header-trust/secure mode, cloud auth off) bounced the user to a login page whose backend route isn't mounted -> 404. Now it probes /auth/status (bare fetch, no interceptor re-entry) and only redirects when cloud_auth_enabled. - make panel-token read the .env secret with grep|cut without stripping surrounding quotes, so a quoted ROBOCO_AGENT_AUTH_SECRET produced a token signed with the quotes included — which never verifies against the orchestrator (docker-compose/pydantic unquote the secret). Now strips surrounding single/double quotes. * fix: git-log 500 on '|' in commit message; X queue shows an empty state - GET /api/git/log 500'd (ValueError: Invalid isoformat) when a commit SUBJECT contained a '|' (e.g. the 'curl|sh' lockdown commit): the fixed '|' field delimiter let the subject's pipe shift the split so author+date collapsed into one field. Switched to \x1f (Unit Separator), which can't appear in commit content. Regression test with a piped subject. - The X Post Queue returned null when empty, so there was no visible place for the X drafts. It now renders a discoverable empty state pointing at Settings -> X credentials. * docs: bring docs/rag + docs/map current for v0.17.0 (waves 1-3) Agent-facing RAG corpus and codebase map updated for every feature in the 0.17.0 span, code-verified: - wave 3: sandbox DB, DB network isolation, cloud auth, X engine (+ x_replies_enabled sub-flag), board roadmap engine — new RAG architecture pages + role/tool/config-reference updates; new symbols, migrations 057-059, panel surfaces, and the get_agent_context dual-path across the map slices. - waves 1-2: A2A live view + switchboard, prompter memory (search_past_tasks), Secretary edit access + PM-lighter scope, the PR-gate auto-submit turn cut (ROBOCO_PR_GATE_AUTO_SUBMIT_ENABLED). - correctness fix: api-routes-schemas.md no longer claims the A2A admin routes are reachable by any authenticated agent — they carry a _require_ceo gate (wave 2c). docs/internal, _front.md deltas, and the frozen _complete_map.md snapshot untouched. * fix(rag): atomic upsert for indexed-doc tracking (kills e2e segfault) The indexed-document tracking write used check-then-insert in two paths (IndexedDocumentRepository.upsert_batch and the file-source _upsert_doc_record). Under concurrent indexing both callers saw no row and both inserted, so the second violated uq_indexed_doc_source and poisoned its transaction — surfacing in CI as the intermittent _checkin_failed SIGSEGV on the failed connection's pool checkin. Both paths now use INSERT ... ON CONFLICT DO UPDATE against the constraint: coalesce keeps an existing title/preview when the new value is empty (matching the old guards) and metadata is jsonb-merged. The batch dedupes within itself first (ON CONFLICT can't touch a row twice in one statement). expire_all after the Core upsert keeps same-session ORM reads consistent with the merged DB row. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>	2026-07-03 19:24:00 +02:00
Renn F	9aa30fb945	100% Coverage	2026-05-06 21:02:31 +02:00