docs(release): prepare 0.27.0 — curated changelog, rag/map sweep, concat rebuild script (#662)

docs/map/runtime-providers.md

@@ -73,6 +73,28 @@ This slice is the agent-runtime + LLM-provider seam plus the in-container agent
 | GrokCliProvider._append_grok_auth_mount | staticmethod | roboco/llm/providers/grok.py:161 | Mount host ~/.grok DIRECTORY RO to /home/agent/.grok-auth-ro (dir, not file, so tmp+rename refresh propagates); warn if auth.json missing |
 | GrokCliProvider._append_usage_mount | staticmethod | roboco/llm/providers/grok.py:193 | Mount per-agent data dir so entrypoint writes usage.json orchestrator reads at finalize |
 | GrokCliProvider._append_grok_env | method | roboco/llm/providers/grok.py:205 | Append ROBOCO_AGENT_ID/MODEL/MCP_CONFIG/INITIAL_PROMPT/GROK_USAGE_FILE env to docker run |
+| CodexCliProvider | class | roboco/llm/providers/codex.py:110 | Spawns roboco-agent-codex container: mounts host `~/.codex` (RO, DIRECTORY not single file — a bind-mounted single file pins the inode, same concern as grok's mount) with the entrypoint symlinking `~/.codex/auth.json` from it while codex's own writable state (config.toml, rules/, sessions/) lives in the image's own `~/.codex` |
+| default_auth_path (codex) | function | roboco/llm/providers/codex_auth.py:73 | `CODEX_HOME`/`~/.codex/auth.json` path resolution, mirrors grok_auth's own |
+| refresh_if_stale (codex) | function | roboco/llm/providers/codex_auth.py:251 | Orchestrator-side refresh loop for the JWT access token (the ONLY expiry signal is its `exp` claim — unlike grok's bundle there's no sibling `expires_at` field); the refresh-token grant against `auth.openai.com/oauth/token` is single-use, guarded by the same process-wide lock + re-check-inside-the-lock pattern (`_recheck_or_refresh`, line 228) that protects grok's rotation from a concurrent double-burn |
+| _exp_from_jwt | function | roboco/llm/providers/codex_auth.py:91 | Decode the access token's JWT `exp` claim — codex has no `expires_in`/`expires_at` sibling field at all, so this is the ONLY signal (grok has this only as a fallback) |
+| codex_auth.main | function | roboco/llm/providers/codex_auth.py:282 | `--check` backstop CLI (entrypoint refuses to start on a missing/expired token) or bare orchestrator refresh-if-stale |
+| sandbox_level_for_role | function | roboco/llm/providers/codex_cli_config.py:181 | Per-role `--sandbox` level: `workspace-write` for `developer`, `read-only` for every other role — codex's tool-scoping has no CLI-flag equivalent to grok's `--disallowed-tools` |
+| render_execpolicy_rules / write_execpolicy_rules | functions | roboco/llm/providers/codex_cli_config.py:210 / 230 | One shared `~/.codex/rules/default.rules` execpolicy file (Starlark `prefix_rule`s denying git-mutation/destructive/raw-package-manager commands) rendered per role |
+| render_combined_prompt / write_combined_prompt | functions | roboco/llm/providers/codex_cli_config.py:236 / 252 | Codex has no verified system-prompt-file mechanism, so the composed role blueprint is PREPENDED to the task prompt itself rather than mounted separately |
+| codex_cli_config.main | function | roboco/llm/providers/codex_cli_config.py:282 | Entrypoint renderer: mcp-config → `~/.codex/config.toml`, execpolicy rules, combined prompt |
+| aggregate_usage_from_jsonl | function | roboco/llm/providers/codex_cli_usage.py:80 | Sums `turn.completed` events' real input/output/cache-read/cache-write split — a genuine four-bucket split, unlike grok's output-only fallback |
+| usage_and_cost (codex) | function | roboco/llm/providers/codex_cli_usage.py:113 | Prices the aggregated four-bucket usage into `usage.json` |
+| capture_run_usage (codex) | function | roboco/llm/providers/codex_cli_usage.py:133 | Entrypoint: write usage.json for the run |
+| extract_error_text / is_rate_limited / is_auth_failure / classify | functions | roboco/llm/providers/codex_cli_sniff.py:52-94 | Classifies a codex run's terminal state (`rate_limit`/`auth`/none) from ONLY the structured `error.message` JSONL field + stderr — NEVER the model's own transcript (which could false-positive on ordinary on-topic prose; this repo's own prompts use the phrase "quota-limited"). Codex has no exit-code taxonomy (every failure exits 1), so this sniff is the only signal the orchestrator has to park the provider. |
+| GeminiCliProvider | class | roboco/llm/providers/gemini.py:135 | Spawns roboco-agent-gemini container: copies the RO-staged host `~/.gemini` OAuth creds into a container-local WRITABLE copy so the CLI's own in-process token refresh (google-auth-library) can write back locally without ever touching the host copy — unlike grok/codex, Google's refresh token is reusable, so there is NO orchestrator refresh daemon for Gemini at all |
+| policy_rules_for_role / render_policy_toml | functions | roboco/llm/providers/gemini_cli_config.py:158 / 189 | Per-role TOML Policy Engine deny rules (`~/.gemini/policies/roboco.toml`, keyed by `toolName`/`commandPrefix`) — gemini has no CLI-flag equivalent to grok's `--disallowed-tools` either |
+| render_settings_json | function | roboco/llm/providers/gemini_cli_config.py:195 | `~/.gemini/settings.json`: `security.auth.selectedType` for headless OAuth, `experimental.enableAgents=false` (fleet-wide subagent ban), `advanced.autoConfigureMemory=false` |
+| gemini_cli_args | function | roboco/llm/providers/gemini_cli_config.py:225 | `--approval-mode yolo` (universal headless auto-approval) + `--max-turns` cap |
+| write_gemini_memory | function | roboco/llm/providers/gemini_cli_config.py:248 | Writes the composed role blueprint as `GEMINI.md` |
+| gemini_cli_config.main | function | roboco/llm/providers/gemini_cli_config.py:276 | Entrypoint renderer: mcp-config → settings.json + per-role policy TOML + GEMINI.md |
+| extract_model_stats / usage_and_cost (gemini) | functions | roboco/llm/providers/gemini_cli_usage.py:130 / 146 | Reads the run's own `--output-format stream-json` terminal `result` event for per-model FLAT token stats — no session-file scraping — and prices each of the three GA models (`gemini-2.5-pro`/`-flash`/`-flash-lite`) at its own rate |
+| classify_exit_code (gemini) | function | roboco/llm/providers/gemini_cli_usage.py:232 | Remaps a quota/rate-limit error (no dedicated CLI exit code — parsed from the run's JSON `error.type`) to exit 75, mirroring codex/grok's park signal; exit 41 (the CLI's own auth-failure code) passes straight through |
+| gemini_cli_usage.main | function | roboco/llm/providers/gemini_cli_usage.py:268 | Entrypoint: write usage.json for the run |
 | refresh_if_stale | function | roboco/llm/providers/grok_auth.py:307 | Mint fresh access token from refresh_token grant if expiry within skew; acquires _refresh_lock then delegates to _recheck_or_refresh to prevent concurrent double-rotation; returns fresh/refreshed/missing/no_refresh_token/failed (best-effort, never raises) |
 | _recheck_or_refresh | function | roboco/llm/providers/grok_auth.py:283 | Locked body of refresh_if_stale: re-load bundle + re-check staleness inside _refresh_lock, then call _do_refresh if still stale (prevents concurrent double-rotation of the single-use refresh grant, #94) |
 | _atomic_write | function | roboco/llm/providers/grok_auth.py:138 | Rewrite auth.json atomically (tmp+replace) with direct-write fallback so a rotated single-use refresh_token is never lost (F006) |
@@ -147,6 +169,12 @@ Interactive intake (prompter) and secretary containers run intake_main/secretary
 
 The streaming.py callback is set once at bootstrap (websocket_bridge) and invoked by the agent reasoning path to broadcast chunks to /ws/agents/{id}. transcript_retention.select_prunable_transcripts is invoked by the orchestrator's cleanup loop against ~/.claude/projects to delete agent-owned old transcripts (never the operator's own session dirs).
 
+**Codex spawn/refresh.** `ProviderRegistry` also registers `ModelProvider.OPENAI` → `CodexCliProvider`. Spawn mounts the host `~/.codex` directory (from a one-time `codex login`, `ROBOCO_HOST_CODEX_DIR`) read-only; the entrypoint symlinks `~/.codex/auth.json` from that RO mount while codex's own writable state (`config.toml`, `rules/`, `sessions/`) lives in the image's own `~/.codex`, so the orchestrator's atomic tmp+rename refresh can still reach the credential without a single-file-mount inode pin. `roboco/llm/providers/codex_auth.py` runs the orchestrator-side refresh loop (`refresh_if_stale`, mirroring `grok_auth.py`): the access token is a JWT whose `exp` claim is the ONLY expiry signal, and the refresh-token grant is single-use, guarded by the same process-wide lock + re-check-inside-the-lock pattern that protects grok's rotation from a concurrent double-burn. Tool scoping is a per-role `--sandbox` level (`workspace-write` for developer, `read-only` otherwise) plus one shared `~/.codex/rules/default.rules` execpolicy file (Starlark `prefix_rule`s) instead of a `--disallowed-tools` flag; the composed role blueprint is prepended to the task prompt itself (no verified system-prompt-file mechanism). Codex has no exit-code taxonomy (every failure exits 1) — `codex_cli_sniff.py` classifies a run's terminal state from ONLY the structured `error.message` JSONL field + stderr, never the model's own transcript. `codex_cli_usage.py` sums `turn.completed` events into a real four-bucket (input/output/cache-read/cache-write) usage split.
+
+**Gemini spawn/refresh.** `ProviderRegistry` also registers `ModelProvider.GEMINI` → `GeminiCliProvider`. Spawn stages the host `~/.gemini` (from a one-time interactive `gemini` login, `ROBOCO_HOST_GEMINI_DIR`) read-only, then the entrypoint COPIES it into a container-local writable `~/.gemini` so the CLI's own in-process token refresh (google-auth-library) can write back locally without ever touching the host copy — Google's refresh token is REUSABLE (unlike grok's single-use one), so each container refreshing its own copy independently is safe with NO orchestrator refresh daemon for Gemini at all (a deliberate contrast the module docstring spells out against grok/codex). Tool scoping is expressed entirely through a rendered TOML Policy Engine (`~/.gemini/policies/roboco.toml`, deny-only rules keyed by `toolName`/`commandPrefix`) plus `settings.json` (`experimental.enableAgents=false` fleet-wide subagent ban); `--approval-mode yolo` is universal headless auto-approval. `gemini_cli_usage.py` reads the run's own `--output-format stream-json` terminal `result` event for per-model FLAT token stats (no session-file scraping), prices each of the three GA models at its own rate, and remaps a quota/rate-limit error (parsed from the run's JSON `error.type`, no dedicated CLI exit code) to exit 75 — exit 41 (the CLI's own auth-failure code) passes straight through. Both Codex and Gemini are one-shot delivery-role runtimes only in this release — no interactive Intake/Secretary support (only Claude and Grok drive those chats).
+
+**Interactive-role exemption (post-finale sweep, #661).** GLOBAL/ROLE routing rows on `OPENAI`/`GEMINI` are not-applicable to `intake-1`/`secretary-1` — those two agents stay on Anthropic regardless of a fleet-wide mode switch to Codex or Gemini, since neither provider supports the interactive chat driver; an explicit `AGENT_SLUG` pin attempting to route either of them onto `OPENAI`/`GEMINI` is refused loudly by the spawn guard rather than silently spawning a broken interactive session.
+
 ## Mermaid
 ```mermaid
 graph TD
@@ -297,6 +325,11 @@ runtime-providers
 - ROBOCO_GROK_TURN_TIMEOUT_SECONDS (grok_cli_session per-turn watchdog, default 600)
 - ROBOCO_PROMPTER_SESSION_ID / ROBOCO_SECRETARY_SESSION_ID / ROBOCO_WORKSPACE / CLAUDE_CODE_SUBAGENT_MODEL (intake/secretary entrypoints)
 - ROBOCO_AGENT_ROLE / ROBOCO_AGENT_TOKEN (secretary_driver HMAC auth, server, grok_cli_session role resolve)
+- ROBOCO_HOST_CODEX_DIR (codex.py host `~/.codex` mount source, from a one-time `codex login`)
+- ROBOCO_CODEX_CLI_MODEL (default `gpt-5.3-codex` — codex has no reliable default)
+- CODEX_HOME (codex_auth.default_auth_path)
+- ROBOCO_HOST_GEMINI_DIR (gemini.py host `~/.gemini` staging mount source, from a one-time interactive `gemini` login)
+- ROBOCO_GEMINI_CLI_MODEL (pins one of the three GA ids: `gemini-2.5-pro`/`-flash`/`-flash-lite`)
 
 
 ## Gotchas
@@ -316,6 +349,11 @@ runtime-providers
 - grok_cli_config: --always-approve is REQUIRED for headless runs (without it grok cancels the first tool call). Safety holds via --disallowed-tools (removes tools) + --deny (hard-blocks patterns) regardless of approval.
 - grok_cli_config denies git mutation via native --deny (graceful, run continues) but exfil categories via the bash-guard PreToolUse hook (which CANCELS the run on deny) — deliberate split: a reflexive git op must not drop a task, but an exfil attempt should hard-cancel.
 - grok -p ignores -s (requested session id); the real session id is read back from the run log's end event (session_id_from_run_log) and reused via -r on subsequent turns to persist conversation context.
+- codex's RO host mount is the DIRECTORY (like grok's, unlike a naive single-file mount) for the same reason: a single-file bind mount pins the inode, so the orchestrator's atomic tmp+rename refresh would never reach a running container. The Codex CLI's own writable state (config.toml, rules/, sessions/) is NOT on that RO mount — it lives in the image's own `~/.codex`, so only `auth.json` is symlinked from the RO side.
+- codex's refresh token is single-use (like grok's) — `codex_auth.py`'s process-wide lock + re-check-inside-the-lock mirrors `grok_auth._recheck_or_refresh` exactly to prevent a concurrent double-burn of the same grant.
+- gemini's refresh token is REUSABLE (unlike grok/codex) — this is the one structural asymmetry in the whole provider family: no orchestrator-side refresh daemon exists for Gemini at all, by design, because each container can safely refresh its own local copy independently without a single writer serializing rotation.
+- codex has NO exit-code taxonomy — every failure exits 1, so `codex_cli_sniff.py` must classify from structured JSONL `error.message` fields only; it deliberately never inspects the model's own transcript text, since this repo's own prompts legitimately use phrases like "quota-limited" that would false-positive a transcript-text classifier.
+- gemini's quota/rate-limit signal has no dedicated CLI exit code either — `gemini_cli_usage.classify_exit_code` parses the run's own JSON `error.type` and remaps to exit 75 (the same park signal grok's 429 and codex's rate-limit sniff produce), so the orchestrator's park-and-probe loop treats all three providers identically at that seam despite three different underlying signals.
 
 
 ## Drift from CLAUDE.md
@@ -333,6 +371,12 @@ runtime-providers
 
 > Post-snapshot updates (since 2026-06-29): 536bbb64 (Chore/all/logical gaps sweep #286) — grok_auth.py: added module-level `_refresh_lock = threading.Lock()` (line 48) and new `_recheck_or_refresh` helper (line 283); `refresh_if_stale` now acquires `_refresh_lock` before the grant POST and delegates to `_recheck_or_refresh` inside the lock so a concurrent caller that waited finds the already-refreshed token and returns `fresh` instead of re-POSTing the now-dead single-use refresh grant (#94). All other commits touching roboco/runtime/ in this window only modified orchestrator.py (out-of-scope for this slice).
 >
+> **Codex CLI provider (#659, `c70ff3cf` + npm-install fix `13abb2ec`).** New `codex.py`/`codex_auth.py`/`codex_cli_config.py`/`codex_cli_usage.py`/`codex_cli_sniff.py` add `ModelProvider.OPENAI` running OpenAI's official `codex` CLI on a ChatGPT subscription — `ProviderRegistry` now registers OPENAI alongside GROK. Migration 083 seeds the provider row. The install method itself needed a follow-up fix: the CDN-hosted `install.sh` the image first tried is blocked from RoboCo's build network, so the Dockerfile installs the CLI via `npm install -g` instead (no other behavior change).
+>
+> **Gemini CLI provider (#660, `21d67304`).** New `gemini.py`/`gemini_cli_config.py`/`gemini_cli_usage.py` add `ModelProvider.GEMINI` running Google's official `gemini` CLI on an OAuth login — `ProviderRegistry` now registers GEMINI alongside GROK/OPENAI. Migrations 084 (enum value) / 085 (seed row) / 086 (enable the row).
+>
+> **Post-finale completeness sweep (#661, `d4b7e1e7`).** Closes gaps found after the Codex/Gemini rollout: one-click `apply_mode` entries + panel mode cards for both new providers (`docs/map/support-services.md`); the interactive-role exemption (GLOBAL/ROLE rows on OPENAI/GEMINI are not-applicable to `intake-1`/`secretary-1`, an explicit AGENT_SLUG pin onto either is refused loudly by the spawn guard rather than silently breaking the interactive chat driver); compose env passthrough for `ROBOCO_GUARD_TRUSTED_CHAIN_PEERS` and `ROBOCO_TASK_BUDGETS_ENABLED` (the provider host-mount dirs were already wired by the provider PRs themselves); and a `gt=0` validation tightening on the task/project budget fields (`docs/map/gateway-support.md`, `docs/map/orchestrator.md`) so a `0` budget — which would silently block everything — is rejected outright.
+>
 > **v0.18.0** (2026-07-04): Fable mode's grok side — `fable_honesty_nudge_hook_config`/`write_grok_fable_hooks` (grok_cli_config.py:308-345), gated by `fable_mode_enabled` (default off). Deliberately narrower than the Claude path's 5 hooks: only the never-denying PostToolUse honesty-nudge is ported, because a grok `PreToolUse`/`Stop` hook deny cancels the entire run (verified live) — the same asymmetry this file's Gotchas section already documents for the bash-guard's git-deny-vs-exfil-cancel split.
 
 ## Regression Risks
@@ -345,6 +389,8 @@ runtime-providers
 | /usage/sync path guard may reject legit symlinked transcripts | roboco/agent_sdk/server.py:770 | _contained_transcript_path rejects any transcript whose resolved path is not under the resolved transcript root. A legitimately symlinked transcript dir under ~/.claude/projects pointing outside (e.g. a shared NAS location) now returns HTTP 400 and usage_sync silently returns the current snapshot — token usage for that agent stops updating without a loud failure. | low |
 | _coerce_draft drops null/non-array the_work to [] | roboco/agent_sdk/intake_driver.py:116 | New coercion wraps non-list the_work via _coerce_to_list, which returns [] for numbers/bools/None. A draft the agent emitted mid-spec with the_work: null now arrives downstream as the_work: [] (empty list) instead of null/absent. Downstream 'has work?' presence checks that distinguish null from empty may now see an empty list and behave differently (e.g. treat an incomplete draft as a zero-work draft). | low |
 | Missing auth.json now logs warning but still spawns doomed container | roboco/llm/providers/grok.py:185 | When ~/.grok/auth.json is absent, _append_grok_auth_mount now only logs a warning (was: silently skipped the mount). The spawn still proceeds and the container exits 78 at the entrypoint --check. Behavior of the spawn path is unchanged, but an operator who does not tail logs will still diagnose a later exit-78; the warning is only useful if someone reads it. | low |
+| codex_cli_sniff classifies from error.message text only | roboco/llm/providers/codex_cli_sniff.py:94 | Codex has no exit-code taxonomy, so every quota/rate-limit/auth distinction rests entirely on the shape of a structured JSONL `error.message` field. If a future codex CLI version changes that field's wording or moves the signal elsewhere, the sniff silently falls through to "none" and the orchestrator crash-retries straight back into the same rate limit instead of parking the provider — the same failure mode the overload-break feature exists to prevent for the other providers. | medium |
+| Gemini has no orchestrator-side refresh daemon at all | roboco/llm/providers/gemini.py:135 | The reusable-refresh-token design is safe under the CURRENT assumption (each container's local copy refreshes independently, no shared writer to serialize). If Google ever changes the refresh grant to single-use (matching grok/codex), every container refreshing concurrently would race to burn the same grant with no lock protecting it, unlike grok_auth/codex_auth's process-wide lock — this provider has no equivalent safety net because none was ever needed under the current contract. | low |
 
 ## Health
 This slice is coherent and well-factored: the provider ABC + registry cleanly isolates the Grok backend while the Anthropic/Ollama/LOCAL paths stay on the built-in spawn (additive seam, no destabilization), and the agent_sdk sidecar centralizes budget/loop/verb-circuit/token state that hooks share. The single baseline-to-HEAD commit (15effce0) landed three genuine hardening fixes — the F006 refresh_token-loss guard with direct-write fallback, the JWT-exp decode so a refreshed token isn't forever rejected, and the /usage/sync path-traversal guard — plus the grok directory-mount fix that resolves the inode-pinning hang. The main integrity concerns are operational rather than structural: the grok directory mount widens RO exposure to host grok state, the 6h expires_at default can burn the single-use refresh_token on the rare double-miss, the in-process SDK state is lost on every container restart (by design, but means verb-circuit/budget counters reset), and ClaudeCodeProvider is dead reference code whose 'default' label in CLAUDE.md is misleading. Interactive intake/secretary parity between Claude and Grok is real (shared IntakeDriver, only the SessionFactory differs). No obviously broken logic was introduced; the regression risks are edge-case behavior shifts, not holes. Recommend re-running the grok auth refresh test against a token that omits expires_in to confirm the JWT-exp path, and a /usage/sync test with a symlinked transcript to confirm the new guard fails loud where appropriate.