diff --git a/CHANGELOG.md b/CHANGELOG.md index d627296b..7977db84 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Added +- **The eval harness real-spawn path is wired (#701).** `python -m roboco.eval run` now works end-to-end for a developer-role cohort: `OrchestratorStageSpawner.__init__` replaces its `NotImplementedError` with a real `AgentOrchestrator()` constructed the same way the production dispatcher builds it, so `run_stage` drives a real `spawn_agent` per turn. The no-production-reach guarantee lives in `_generate_mcp_config`, which now prefers `settings.api_url` when set (both `PROJECT_HOST_PATH` branches) — the harness's `_bench_environment` patches `settings.api_url` to the disposable stack URL, so a spawned container's MCP servers resolve to the throwaway orchestrator instead of the real production hostname or `127.0.0.1:{port}`, even though `_seed_company` seeds agents under their real production UUIDs (correct — orchestrator-internal helpers keyed by the static registry resolve as in a real deployment; the isolation is the URL, not the UUID). A new `tests/unit/runtime/test_eval_mcp_config_isolation.py` pins the guarantee without a Docker daemon. The injectable scripted `StageSpawner` (`tests/e2e_smoke/test_eval_bench.py`) is unaffected — it injects its own `make_spawner` — and remains the unit-test fallback. The real spawn needs a Docker daemon + built agent images. + - **The Board Program registry — the board finally works (#689, #699 + the Phase-2/3 program train).** Board dormancy root-caused and fixed (#684: the research key never reached the container, the pitch verb was never prompted, auditor playbook curation had no dispatch path), then one generic registry/engine/loop (`BoardProgram`, `BoardProgramEngine`, migrations 087-088) replaces the bespoke roadmap/spotlight loops behavior-preserved, adds a LEARN ledger (per-item CEO decisions feed the next cycle's exploration prompt), per-project scoping with dual polarity (`projects.board_programs`: plain keys opt into project-scoped programs, `!key` opts out of an org-scoped program's outputs), and a single settings-store arming chokepoint (`program_armed`) so the panel toggle can never be silently vetoed by a legacy boot flag. Twelve programs ride it: **Pest Control** (PO bug hunts over the findings ledger and rework hotspots, rework-spike accelerated), **Spackle** (PO gap-fill audits of half-shipped surface area), **Scales** (PO monthly portfolio rebalance whose approvals reprioritize or cancel live backlog tasks), **Dogfood** (the PO walks the product with task-scoped browser tools), **Periscope** (HoM market briefs with mandatory source citations, feeding the roadmap prompt), **Megaphone** (HoM editorial calendar into the X queue), **Mirror** (HoM quarterly positioning audits), **Barfly** (HoM replies to adjacent X conversations, injection-screened, candidate-id-bound), **War Room** (HoM release campaigns as manual-cadence held drafts), **Coroner** (Auditor postmortems on bounce/cancel/budget events, drafting process-change playbooks), **Librarian** (Auditor playbook mining from recurring learnings), and **Sentinel** (Auditor weekly drift-watch quality reports). Every artifact is held; the CEO remains the only path to materialization; nothing auto-posts or auto-starts. ### Fixed diff --git a/CLAUDE.md b/CLAUDE.md index 40b3975b..1da831b7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -436,7 +436,7 @@ The fourteen programs: **Printer**/`roadmap` (PO, weekly cron, org) — the pre- **Ponytail (bundled with Fable-mode).** Rides `ROBOCO_FABLE_MODE_ENABLED` — no separate flag. Vendors the ponytail "lazy senior dev" build-laziness doctrine (`agents/prompts/doctrine/ponytail.md` + ethos sibling, MIT, Copyright (c) 2026 DietrichGebert — trimmed, YAML frontmatter stripped) into every composed system prompt via `ponytail_doctrine_layer` (`roboco/agents/factories/_base.py`), slotted immediately after the Fable doctrine layer and gated on the same flag. Role-scoped: developers (`AgentRole.DEVELOPER`) get the full ladder (YAGNI → reuse-in-this-codebase → stdlib → native-platform → installed-dep → one-line → minimal); every other role gets the ethos-only cut (`ponytail-ethos.md`) — the code-mechanics rungs are dropped so they can't leak into prose artifacts (task plans, review notes, docs). Both files carry a 5-point RoboCo preamble (the ethos sibling adds a 6th: free-text field obligations) that makes the ladder yield to the Architectural Conventions Standard (placement), the 80% coverage gate + QA review + self-verification, the per-team design bar, task hygiene (everything-is-a-task / commits-linked / state-is-sacred), and reviewer feedback (`needs_revision` / `pr_fail` / `request_changes`) — the overlap mitigation is scoping, not deletion, and it rides ponytail's own "when NOT to be lazy" clause. Developer intensity is tunable via `ROBOCO_PONYTAIL_INTENSITY` (`lite` / `full` / `ultra`, default `full`; `roboco/config.py` `ponytail_intensity`, a string value — not a feature flag): `full` enforces the ladder, `lite` builds what's asked and names the lazier alternative, `ultra` is YAGNI-extremist (deletion before addition, challenge the requirement). Non-developers get no dial — `ultra` is wrong for prose artifacts, so the ethos runs a fixed restrained stance. Prompt-only: no hooks, no grok-path changes — ponytail adds no hook surface, so bundling it under the Fable flag changes only the composed prompt, not the spawn hooks. The Fable flag's description in `roboco/config.py` names both doctrines. -**Golden-task eval harness (source-checkout-only offline CLI).** `roboco/eval/` replays a fixed set of `BenchTaskSpec` fixtures (`roboco/eval/fixtures.py`) through the REAL delivery lifecycle in a disposable environment reused from `tests/e2e_smoke/harness.py` (fake GitHub REST, a real local git origin, a throwaway DB) — real isolation, not a mock. `EvalRunner.run_cohort` (`roboco/eval/runner.py`) scores each fixture on deterministic metrics (final status, `revision_count`, cycle time, tokens/cost via the `agent_spawn_sessions` task_id join) plus a local-model judge comparing the final PR diff + notes against the fixture's checked-in expectations, nested under a `"non_deterministic": true`-marked `"judge"` object so a naive cohort diff never mistakes judge noise for a real regression. `agent_spawn_sessions.doctrine_version` (migration 081) is stamped at spawn-session finalize from the composed prompt layers, so a cohort's model + doctrine combination (e.g. Fable-mode on vs. off) is durably identifiable after the fact. **Real-spawn is cut for this release**: `OrchestratorStageSpawner` raises `NotImplementedError` at construction — a real spawn's MCP wiring would resolve to the REAL production orchestrator under real agent UUIDs, unsafe for a bench run — so `python -m roboco.eval run` is wired but not yet functional; the only working path today is driving `EvalRunner` with an injected scripted `StageSpawner` from Python (see `tests/e2e_smoke/test_eval_bench.py`). Scoped to developer-role fixtures only (`run_cohort` refuses any other role) and only runs from a source checkout (`tests/e2e_smoke` isn't shipped in containers or wheels); bench runs also patch every vault flag off so a bench task/note/journal write never lands in the operator's real Obsidian vault. +**Golden-task eval harness (source-checkout-only offline CLI).** `roboco/eval/` replays a fixed set of `BenchTaskSpec` fixtures (`roboco/eval/fixtures.py`) through the REAL delivery lifecycle in a disposable environment reused from `tests/e2e_smoke/harness.py` (fake GitHub REST, a real local git origin, a throwaway DB) — real isolation, not a mock. `EvalRunner.run_cohort` (`roboco/eval/runner.py`) scores each fixture on deterministic metrics (final status, `revision_count`, cycle time, tokens/cost via the `agent_spawn_sessions` task_id join) plus a local-model judge comparing the final PR diff + notes against the fixture's checked-in expectations, nested under a `"non_deterministic": true`-marked `"judge"` object so a naive cohort diff never mistakes judge noise for a real regression. `agent_spawn_sessions.doctrine_version` (migration 081) is stamped at spawn-session finalize from the composed prompt layers, so a cohort's model + doctrine combination (e.g. Fable-mode on vs. off) is durably identifiable after the fact. **Real-spawn is wired**: `OrchestratorStageSpawner` drives a real `AgentOrchestrator.spawn_agent` per turn (constructed the same way the production dispatcher builds it), and `_generate_mcp_config` honors the patched `settings.api_url` (set to the harness's disposable stack URL in `_bench_environment`) so a spawned container's MCP servers resolve to the throwaway orchestrator, never the REAL production one — even though `_seed_company` seeds agents under their REAL production UUIDs (correct: orchestrator-internal helpers keyed by the static registry resolve exactly as in a real deployment; the isolation is about the URL, not the UUID). `python -m roboco.eval run` works end-to-end for a developer-role cohort; it needs a Docker daemon + built agent images for the real spawn path. The injectable scripted `StageSpawner` (see `tests/e2e_smoke/test_eval_bench.py`) remains the unit-test fallback that proves the runner's polling/scoring/DB plumbing without touching Docker; `tests/unit/runtime/test_eval_mcp_config_isolation.py` pins the no-production-reach guarantee without a Docker daemon. Scoped to developer-role fixtures only (`run_cohort` refuses any other role) and only runs from a source checkout (`tests/e2e_smoke` isn't shipped in containers or wheels); bench runs also patch every vault flag off so a bench task/note/journal write never lands in the operator's real Obsidian vault. **Env-branches ladder + EnvSyncEngine (default-off `ROBOCO_ENV_SYNC_ENABLED`).** Replaces a project's single `default_branch` with an ordered environment ladder: nullable `projects.environments` JSONB (migration 073), an ordered `list[{name, branch}]` where index 0 is the **head** rung (where dev/cell/leaf PRs land) and index -1 is the **prod** rung (where the gated release executor commits + tags); middle rungs are intermediates (qa/stag). A null ladder degenerates to a single-branch ladder synthesized from `default_branch` at read time (`roboco/models/env_branches.py`: `head_branch` / `prod_branch` / `ladder_pairs` / `promotion_chain`) — no backfill, byte-for-byte legacy behavior until the CEO declares a real split. Every former `default_branch` consumer now routes through the shim: the PR target and per-agent clone (`WorkspaceService.ensure_workspace` / `ensure_read_clone`), the CI branch, the release executor's clone/commit/tag target (`_ReleaseContext.prod_branch`) plus its full-chain head→…→prod promotion before bumping (`promote_env_chain`, fail-closed `promotion_failed` on a merge conflict), and `release_readiness`'s diff baseline (`prod..head` instead of `last_tag..HEAD`) with a tag-drift cross-check (`_tag_drift_gaps` — the last tag's commit vs. prod tip disagreeing flags a hotfix that landed on prod after the tag). `EnvSyncEngine` (`roboco/services/env_sync_engine.py`) cascades the ladder prod→…→head via GitHub's merges API: a clean merge auto-pushes straight to the lower rung, a conflict opens ONE idempotent sync PR + a Main-PM coordination task and stops that project's cascade for the cycle — the cascade's target is never the prod rung by construction, so "only the CEO merges master" still holds. Bounded + deduped per repo (one open env_sync task at a time). Panel: an environment-ladder editor on the project edit dialog. diff --git a/docs/map/tests.md b/docs/map/tests.md index d947051b..d4b848a3 100644 --- a/docs/map/tests.md +++ b/docs/map/tests.md @@ -28,7 +28,7 @@ The pytest test suite for RoboCo: 571 test_*.py files across tests/foundation, t | tests/e2e_smoke/harness.py | E2E harness: E2EStack app + orchestrator client + per-test agent manifests; used by the e2e_smoke tier | ~520 | | tests/e2e_smoke/test_gitea_live.py | Live-Gitea contract suite for `GiteaProvider` — fully self-seeding (creates its own uniquely-named repo, pushes real commits) against a real Gitea instance; skipped unless `ROBOCO_GITEA_E2E_URL`/`ROBOCO_GITEA_E2E_TOKEN` are both set; exercises PR open → duplicate-409→422 reshape → list/filter → diff → comment review → commit-status CI reshape → squash merge → branch delete → release, plus the git-CLI Basic-auth extraheader claim | 250 | | tests/e2e_smoke/test_gitlab_live.py | Live-GitLab contract suite for `GitLabProvider` — mirrors `test_gitea_live.py`, self-seeding against a real GitLab instance (gitlab.com works, project deleted afterward best-effort); skipped unless `ROBOCO_GITLAB_E2E_URL`/`ROBOCO_GITLAB_E2E_TOKEN` are both set; exercises MR open → duplicate reshape → GitHub-shape adaptation → diff reassembly → note review → commit-status CI reshape → squash merge → branch delete → release → the oauth2 Basic-auth git-CLI claim | 265 | -| roboco/eval/ (`fixtures.py`/`runner.py`/`__main__.py`) | Golden-task offline CLI bench harness (source-checkout-only, NOT a pytest suite) — replays fixed `BenchTaskSpec` fixtures through the real delivery lifecycle in a disposable environment REUSED from `tests/e2e_smoke/harness.py` (fake GitHub REST, a real local git origin, a throwaway DB), scoring on deterministic metrics + a local-model judge. `python -m roboco.eval run` is wired but not functional this release — `OrchestratorStageSpawner` raises `NotImplementedError` at construction (a real spawn would resolve to the production orchestrator under real agent UUIDs, unsafe for a bench run); the only working path today is driving `EvalRunner` with an injected scripted `StageSpawner` from Python. Scoped to developer-role fixtures only; only runs from a source checkout. See `tests/e2e_smoke/test_eval_bench.py`. | — | +| roboco/eval/ (`fixtures.py`/`runner.py`/`__main__.py`) | Golden-task offline CLI bench harness (source-checkout-only, NOT a pytest suite) — replays fixed `BenchTaskSpec` fixtures through the real delivery lifecycle in a disposable environment REUSED from `tests/e2e_smoke/harness.py` (fake GitHub REST, a real local git origin, a throwaway DB), scoring on deterministic metrics + a local-model judge. `python -m roboco.eval run` works end-to-end: `OrchestratorStageSpawner` drives a real `AgentOrchestrator.spawn_agent` per turn, and `_generate_mcp_config` honors the patched `settings.api_url` (set to the harness's disposable stack URL in `_bench_environment`) so a spawned container's MCP servers resolve to the throwaway orchestrator, never the real production one — even though `_seed_company` seeds agents under their REAL production UUIDs (correct: orchestrator-internal helpers keyed by the static registry resolve as in a real deployment). Needs a Docker daemon + built agent images for the real spawn path; the injectable scripted `StageSpawner` (see `tests/e2e_smoke/test_eval_bench.py`) remains the unit-test fallback that proves the runner's plumbing without touching Docker. Scoped to developer-role fixtures only; only runs from a source checkout. | — | ## E2E smoke harness diff --git a/roboco/eval/__main__.py b/roboco/eval/__main__.py index af66b3be..1e331edb 100644 --- a/roboco/eval/__main__.py +++ b/roboco/eval/__main__.py @@ -3,15 +3,14 @@ python -m roboco.eval run --role --cohort \\ [--fixtures a,b] [--json-out path] -NOT YET FUNCTIONAL: the real-spawn path (``OrchestratorStageSpawner``) is -deliberately cut — see ``roboco/eval/runner.py``'s module docstring's -"Real-spawn status" section — because a real container spawn's MCP wiring -would authenticate against the REAL production orchestrator, not this -harness's disposable one. ``run`` will raise ``NotImplementedError`` once it -reaches the first fixture. The only working path today is driving -``EvalRunner`` with an injected scripted ``StageSpawner`` from Python (see -``tests/e2e_smoke/test_eval_bench.py``); this CLI is wired for the day the -follow-up lands, not for use today. +Replays the golden-task fixtures against one agent through the REAL delivery +lifecycle: ``OrchestratorStageSpawner`` drives a real ``AgentOrchestrator`` +container spawn per turn, and ``_generate_mcp_config`` honors the patched +``settings.api_url`` so spawned MCP servers resolve to the harness's +disposable orchestrator, never the real production one. Needs a Docker daemon ++ built agent images. The injectable scripted ``StageSpawner`` (see +``tests/e2e_smoke/test_eval_bench.py``) remains the unit-test fallback that +proves the runner's plumbing without touching Docker. Offline dev/ops tool: no panel surface, no feature flag. Also runs from a source checkout only (needs ``tests/e2e_smoke``, not shipped in containers @@ -35,10 +34,7 @@ def _parse_args(argv: list[str]) -> argparse.Namespace: run = subparsers.add_parser( "run", - help=( - "Replay the golden-task fixtures against one agent " - "[NOT YET FUNCTIONAL — real-spawn path is cut, see module docstring]" - ), + help="Replay the golden-task fixtures against one agent", ) run.add_argument( "--role", required=True, help="Agent slug under test (e.g. be-dev-1)" diff --git a/roboco/eval/runner.py b/roboco/eval/runner.py index b54c02dd..f0181fdf 100644 --- a/roboco/eval/runner.py +++ b/roboco/eval/runner.py @@ -28,23 +28,22 @@ Vault safety: ``_bench_environment`` also patches ``obsidian_vault_enabled`` bench task/note/journal write never lands in the operator's real Obsidian vault even when the ambient deployment has vault flags armed. -Real-spawn status: CUT for this release. ``StageSpawner`` is the seam -between a real container spawn and a scripted stand-in, and -``OrchestratorStageSpawner`` — what would be the default, real -implementation — raises ``NotImplementedError`` at construction: a spawned -container's MCP servers resolve their orchestrator URL via -``_generate_mcp_config`` (``PROJECT_HOST_PATH`` -> the REAL production -hostname, or ``settings.port``), never the patched ``settings.api_url`` this -harness's disposable stack listens on — combined with ``_seed_company`` -seeding agents under their REAL production UUIDs, a real spawn here could -authenticate as e.g. be-dev-1 against the production orchestrator and act on -real tasks. Fixing the spawn-env wiring is a dedicated follow-up. The ONLY -working ``StageSpawner`` today is an injected scripted one (see -``tests/e2e_smoke/test_eval_bench.py``) that drives the SAME real MCP flow/do -tool functions e2e_smoke's ``ScriptedAgent`` uses — proving the runner's +Real-spawn status: ``OrchestratorStageSpawner`` is the default, real +``StageSpawner``: it drives one turn via the REAL +``AgentOrchestrator.spawn_agent`` — the exact method the production +dispatcher calls. ``_generate_mcp_config`` honors the patched +``settings.api_url`` (set to the harness's disposable stack URL in +``_bench_environment``), so a spawned container's MCP servers resolve to the +throwaway orchestrator, never the real production one — even though +``_seed_company`` seeds agents under their REAL production UUIDs (which is +correct: orchestrator-internal helpers keyed by the static registry resolve +exactly as they would in a real deployment). The injectable scripted +``StageSpawner`` (see ``tests/e2e_smoke/test_eval_bench.py``) remains the +unit-test fallback — it drives the SAME real MCP flow/do tool functions +e2e_smoke's ``ScriptedAgent`` uses, proving the runner's polling/scoring/DB plumbing without touching Docker. ``python -m roboco.eval -run`` therefore does not work yet; it is wired for the day the follow-up -lands, not for use today. +run`` works end to end for a developer-role cohort; it needs a Docker daemon ++ built agent images for the real spawn path. Scope cut: only developer-role fixtures are supported (``run_cohort`` refuses any other role). QA/documenter/cell-PM only ever pick up a task a @@ -491,40 +490,36 @@ class StageSpawner(Protocol): class OrchestratorStageSpawner: - """CUT for this release — do not construct. See the ``NotImplementedError`` - raised below for exactly why, and the module docstring's "Real-spawn - status" section. - - This was meant to be the default, real ``StageSpawner``: drive one turn - via the REAL ``AgentOrchestrator.spawn_agent`` — the exact method the - production dispatcher calls — reusing its own ``_get_prompt_for_agent`` / + """The default, real ``StageSpawner``: drive one turn via the REAL + ``AgentOrchestrator.spawn_agent`` — the exact method the production + dispatcher calls — reusing its own ``_get_prompt_for_agent`` / ``_task_git_context`` helpers so the prompt and workspace mount are byte-for-byte what a real dispatch tick would build, then wait for the - container to exit (or the stage timeout). The ``run_stage`` body below is - otherwise correct and is left in place for the follow-up that fixes the - wiring (see ``__init__``) rather than deleted — re-enable it there by - removing the raise. + container to exit (or the stage timeout). + + Safe because ``_generate_mcp_config`` honors the patched + ``settings.api_url`` (set to the harness's disposable stack URL in + ``_bench_environment``), so a spawned container's MCP servers resolve to + the throwaway orchestrator, never the real production one — even though + ``_seed_company`` seeds agents under their REAL production UUIDs (which + is correct: orchestrator-internal helpers keyed by the static registry + resolve exactly as they would in a real deployment). """ _orchestrator: Any _stage_timeout_seconds: float def __init__(self, stage_timeout_seconds: float = 900.0) -> None: - raise NotImplementedError( - "OrchestratorStageSpawner (the real-spawn path) is cut from this " - "release: a spawned container's MCP servers connect via " - "_generate_mcp_config, which resolves the orchestrator URL from " - "PROJECT_HOST_PATH ('http://roboco-orchestrator:8000', the REAL " - "production hostname) or settings.port — NEVER the patched " - "settings.api_url this harness's disposable stack listens on. " - "Combined with _seed_company seeding agents under their REAL " - "production UUIDs, a real spawn here would authenticate as e.g. " - "be-dev-1 against the production orchestrator and could act on " - "real tasks. Fixing this belongs in a dedicated follow-up that " - "makes the spawn env honor the patched stack; until then only " - "the injectable scripted StageSpawner (see " - "tests/e2e_smoke/test_eval_bench.py) is a working path." - ) + from roboco.runtime.orchestrator import AgentOrchestrator + + self._stage_timeout_seconds = stage_timeout_seconds + # Constructed the same way the production dispatcher does + # (bootstrap.py: ``AgentOrchestrator()``); the harness's + # ``_bench_environment`` has already patched ``settings.database_*`` + # to the throwaway DB and ``settings.api_url`` to the disposable + # stack URL, so the orchestrator's DB + MCP-config wiring resolve to + # the bench's own environment, not production. + self._orchestrator = AgentOrchestrator() async def run_stage(self, *, task: dict[str, Any], agent_slug: str) -> None: from roboco.models.runtime import OrchestratorAgentState diff --git a/roboco/runtime/orchestrator.py b/roboco/runtime/orchestrator.py index b62d0199..08aab22c 100644 --- a/roboco/runtime/orchestrator.py +++ b/roboco/runtime/orchestrator.py @@ -3889,8 +3889,16 @@ class AgentOrchestrator: not-authorized error rather than 404. Git context is forwarded only as a fallback for tools that resolve project/branch from env. """ - # MCP servers run inside agent containers, need to connect via Docker network - if PROJECT_HOST_PATH: + # MCP servers run inside agent containers, need to connect to the + # orchestrator API. Prefer an explicit settings.api_url override — + # production sets it to the container hostname, and the eval harness + # patches it to its disposable in-process stack (see runner.py's + # _bench_environment) so a spawned container's MCP servers resolve to + # the throwaway orchestrator, never the real production one. Fall back + # to the PROJECT_HOST_PATH / settings.port logic when it is unset. + if settings.api_url: + api_url = settings.api_url + elif PROJECT_HOST_PATH: api_url = "http://roboco-orchestrator:8000" else: api_url = f"http://127.0.0.1:{settings.port}" diff --git a/tests/unit/runtime/test_eval_mcp_config_isolation.py b/tests/unit/runtime/test_eval_mcp_config_isolation.py new file mode 100644 index 00000000..68cba8df --- /dev/null +++ b/tests/unit/runtime/test_eval_mcp_config_isolation.py @@ -0,0 +1,60 @@ +"""The eval harness patches ``settings.api_url`` to its disposable stack URL +(see ``roboco/eval/runner.py``'s ``_bench_environment``). ``_generate_mcp_config`` +must honor that patch so a spawned container's MCP servers resolve to the +throwaway orchestrator, never the real production hostname +(``http://roboco-orchestrator:8000``) or ``127.0.0.1:{port}`` — the +no-production-reach guarantee. The agent UUID in the config is the REAL fixed +UUID from ``foundation.identity.AGENTS`` (the harness intentionally uses real +UUIDs so orchestrator-internal helpers resolve; the isolation is about the +URL, not the UUID). +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import TYPE_CHECKING + +from roboco.config import settings +from roboco.foundation import identity as _foundation +from roboco.runtime.orchestrator import AgentOrchestrator + +if TYPE_CHECKING: + import pytest + +_AGENT_SLUG = "be-dev-1" +_DISPOSABLE_URL = "http://localhost:9999" + + +async def test_mcp_config_uses_disposable_api_url_when_set( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """With ``settings.api_url`` patched to a disposable URL, the generated + MCP config's ROBOCO_API_URL / ROBOCO_ORCHESTRATOR_URL point at the + disposable URL — not the production hostname or 127.0.0.1:port.""" + monkeypatch.setattr(settings, "api_url", _DISPOSABLE_URL) + orch = AgentOrchestrator.__new__(AgentOrchestrator) + config_path = await orch._generate_mcp_config(_AGENT_SLUG) + config = json.loads(Path(config_path).read_text()) + # Every MCP server shares the same env dict; sample the first one. + first_env = next(iter(config["mcpServers"].values()))["env"] + assert first_env["ROBOCO_API_URL"] == _DISPOSABLE_URL + assert first_env["ROBOCO_ORCHESTRATOR_URL"] == _DISPOSABLE_URL + assert "roboco-orchestrator" not in first_env["ROBOCO_API_URL"] + assert "127.0.0.1" not in first_env["ROBOCO_API_URL"] + + +async def test_mcp_config_preserves_real_agent_uuid( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """The agent UUID in the config is the REAL fixed UUID from + ``foundation.identity.AGENTS`` — the harness intentionally uses real UUIDs + so orchestrator-internal helpers keyed by the static registry resolve + exactly as they would in a real deployment.""" + monkeypatch.setattr(settings, "api_url", _DISPOSABLE_URL) + orch = AgentOrchestrator.__new__(AgentOrchestrator) + config_path = await orch._generate_mcp_config(_AGENT_SLUG) + config = json.loads(Path(config_path).read_text()) + first_env = next(iter(config["mcpServers"].values()))["env"] + expected_uuid = str(_foundation.AGENTS[_AGENT_SLUG].uuid) + assert first_env["ROBOCO_AGENT_ID"] == expected_uuid