Files
roboco/tests/unit/agents/test_prompts_use_makefile.py
a9dee3b34e feat(agents): force agents to the Makefile — deny raw uv/pip/conda/poetry (CEO #15) (#518)
* fix(prompts): point agents at Makefile, drop raw uv run instructions

backend.md:23-26 literally instructed raw uv run ruff/mypy/pytest (copied from
the human-facing CLAUDE.md), so agents bypassed the Makefile's UV_NO_SYNC=1 +
private UV_CACHE_DIR venv-corruption guard. Replace with make targets across
backend/developer/qa/cell_pm + a universal rule in base.md. Regenerate verbs.md
from the updated regen script (baked instruction now make foundation-check) and
align the Makefile drift message. Ships with the bash-guard deny in the next
commit so agents don't loop fighting the guard.

* feat(bash-guard): deny raw uv/pip/conda/poetry, point at Makefile

When a Makefile is present, deny raw uv run/uv pip/uv lock/add/remove, pip/pip3
install/uninstall, conda install/create/run, poetry run/install/add and remediate
to make quality/gate/lint/test. Skipped when no Makefile (Makefile-less projects
not blocked). ROBOCO_GUARD_SKIP_PM=1 (grok path) nudges exit 0 instead of the
run-canceling exit 2. Overrides the prior bare-uv-run-allowed stance by CEO
direction; the /app-targeted blocks above keep priority.

* feat(grok): deny raw uv/pip/conda/poetry via native --deny + PM-skip nudge

Add _RAW_PM_DENY (uv run/pip install/lock/add/remove, pip/pip3 install, conda
install/create/run, poetry run/install/add) to _deny_rules so grok's graceful
native --deny blocks raw package-manager commands (model adapts to make, run
continues — unlike a hook deny which cancels the run). The bash-guard hook
keeps the compound-command fallback (cd x && uv run) and nudges exit 0 there via
ROBOCO_GUARD_SKIP_PM=1 in the grok hook env, never canceling.

* test(bash-guard): align existing tests with W1 Makefile-gate policy

Raw uv run / pip install are now Makefile-gated (W1, CEO item #15), so two
existing bash-guard invariants reverse:

- test_allows_pytest_even_if_suite_uses_requests keeps its HTTP-injection
  allow-path intent but uses bare `python -m pytest` (raw `uv run` is now
  denied); the deny case is covered by test_bash_guard_makefile_guardrail.
- test_allows_pip_install_in_workspace -> test_denies_pip_install_when_makefile_
  present: a workspace clone carries a Makefile, so bare pip install is now
  denied -> agents use `make` / `uv sync --extra dev`. Makefile-less skips
  stay covered.

Gate: 12994 passed, 439 skipped, 94.81% cov (DB env :55432 user renzof);
the lone flaky integration error passes in isolation (DB-state race, not W1).

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-07-15 04:32:27 +02:00

41 lines
1.6 KiB
Python

"""Agent prompts must steer the fleet at the Makefile, not raw package managers.
`agents/prompts/teams/backend.md` used to literally instruct `uv run ruff ...`
(copied from the human-facing CLAUDE.md), so agents ran raw `uv run` and bypassed
the Makefile's `UV_NO_SYNC=1` + private `UV_CACHE_DIR` — the guard against the
concurrent-venv-corruption race. This grep invariant keeps the prompts honest:
no agent-facing prompt instructs a raw `uv run`/`pip`/`conda`/`poetry`, and the
backend standards block names the make targets.
"""
from __future__ import annotations
from pathlib import Path
PROMPTS = Path(__file__).resolve().parents[3] / "agents" / "prompts"
RAW_PM = ["uv run ", "uv pip install", "pip install ", "conda install ", "poetry run "]
def _prompt_texts() -> dict[str, str]:
texts: dict[str, str] = {}
for path in PROMPTS.rglob("*.md"):
texts[str(path.relative_to(PROMPTS))] = path.read_text()
return texts
def test_no_prompt_instructs_raw_package_managers() -> None:
"""No agent-facing prompt tells an agent to run raw uv/pip/conda/poetry."""
offenders: list[str] = []
for name, body in _prompt_texts().items():
for bad in RAW_PM:
if bad in body:
offenders.append(f"{name}: '{bad.strip()}'")
assert not offenders, f"prompts still instruct raw PM commands: {offenders}"
def test_backend_prompt_names_make_targets() -> None:
"""The backend standards block must point at make targets, not uv run."""
body = (PROMPTS / "teams" / "backend.md").read_text()
assert "make quality" in body
assert "make gate" in body