feat(agents): force agents to the Makefile — deny raw uv/pip/conda/poetry (CEO #15) (#518)

* fix(prompts): point agents at Makefile, drop raw uv run instructions backend.md:23-26 literally instructed raw uv run ruff/mypy/pytest (copied from the human-facing CLAUDE.md), so agents bypassed the Makefile's UV_NO_SYNC=1 + private UV_CACHE_DIR venv-corruption guard. Replace with make targets across backend/developer/qa/cell_pm + a universal rule in base.md. Regenerate verbs.md from the updated regen script (baked instruction now make foundation-check) and align the Makefile drift message. Ships with the bash-guard deny in the next commit so agents don't loop fighting the guard. * feat(bash-guard): deny raw uv/pip/conda/poetry, point at Makefile When a Makefile is present, deny raw uv run/uv pip/uv lock/add/remove, pip/pip3 install/uninstall, conda install/create/run, poetry run/install/add and remediate to make quality/gate/lint/test. Skipped when no Makefile (Makefile-less projects not blocked). ROBOCO_GUARD_SKIP_PM=1 (grok path) nudges exit 0 instead of the run-canceling exit 2. Overrides the prior bare-uv-run-allowed stance by CEO direction; the /app-targeted blocks above keep priority. * feat(grok): deny raw uv/pip/conda/poetry via native --deny + PM-skip nudge Add _RAW_PM_DENY (uv run/pip install/lock/add/remove, pip/pip3 install, conda install/create/run, poetry run/install/add) to _deny_rules so grok's graceful native --deny blocks raw package-manager commands (model adapts to make, run continues — unlike a hook deny which cancels the run). The bash-guard hook keeps the compound-command fallback (cd x && uv run) and nudges exit 0 there via ROBOCO_GUARD_SKIP_PM=1 in the grok hook env, never canceling. * test(bash-guard): align existing tests with W1 Makefile-gate policy Raw uv run / pip install are now Makefile-gated (W1, CEO item #15), so two existing bash-guard invariants reverse: - test_allows_pytest_even_if_suite_uses_requests keeps its HTTP-injection allow-path intent but uses bare `python -m pytest` (raw `uv run` is now denied); the deny case is covered by test_bash_guard_makefile_guardrail. - test_allows_pip_install_in_workspace -> test_denies_pip_install_when_makefile_ present: a workspace clone carries a Makefile, so bare pip install is now denied -> agents use `make` / `uv sync --extra dev`. Makefile-less skips stay covered. Gate: 12994 passed, 439 skipped, 94.81% cov (DB env :55432 user renzof); the lone flaky integration error passes in isolation (DB-state race, not W1). --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-08-03 07:23:24 +02:00 · 2026-07-15 04:32:27 +02:00
parent fe9940dec1
commit a9dee3b34e
14 changed files with 238 additions and 17 deletions
@@ -558,7 +558,7 @@ foundation-check:
 	@echo "==> verb tables up-to-date (renders + git diff)"
 	@uv run python scripts/regenerate_verb_tables.py
 	@git diff --exit-code -- agents/prompts/_generated/ ':!agents/prompts/_generated/lifecycle-*.md' \
-		|| (echo "Verb tables are out of date. Run 'uv run python scripts/regenerate_verb_tables.py' and commit the diff." && exit 1)
+		|| (echo "Verb tables are out of date. Run 'make foundation-check' and commit the diff." && exit 1)
 	@echo "==> postgres enum parity (skip if no migrated DB)"
 	uv run python scripts/verify_postgres_enums.py
 	@echo "All foundation drift checks passed."
@@ -3,7 +3,7 @@

 # Per-role verb shapes (autogenerated)

-Run `uv run python scripts/regenerate_verb_tables.py` after changing
+Run `make foundation-check` after changing
 any role config or schema. Role prompts reference this file's sections
 as the source of truth for verb signatures.

@@ -87,3 +87,7 @@ The briefing also carries `company_goals` — the company's charter (north star,
 ## Substitute reasons (for `i_am_blocked`)

 `low_context`, `out_of_scope_team`, `out_of_scope_role`, `task_complete`, `max_retries`, `blocked_external`.
+
+## Makefile is the gate surface
+
+Run `make <target>` for lint/test/typecheck — never raw `uv run`/`pip`/`conda`/`poetry`. The Makefile sets `UV_NO_SYNC=1` and a private `UV_CACHE_DIR` to prevent the concurrent-venv-corruption race; bare `uv run` bypasses both. Targets: `make quality` (full gate), `make gate` (fast pre-submit), `make lint`, `make test`, `make panel-gate` (frontend).
@@ -177,7 +177,7 @@ The PM journal is what makes the cell legible to Main PM and CEO. Skipping entri
 1. ✅ Every subtask under your task is in a terminal state (`completed` or `cancelled`) — gateway-enforced.
 2. ✅ You inspected each child's PR (already merged into your branch via `complete`) — call `evidence(your_task_id)` for the aggregate diff.
 3. ✅ Each acceptance criterion on YOUR cell-PM task is met by something in the aggregate (commit / merged PR / doc).
-4. ✅ Tests/lint on the aggregate are green — your branch is the integration point for the cell, so run `make quality` (or equivalent) before submitting up.
+4. ✅ Tests/lint on the aggregate are green — your branch is the integration point for the cell, so run `make quality` before submitting up.
 5. ✅ `note(scope='reflect', task_id=...)` written — aggregate review.
 6. ✅ `note(scope='decision', task_id=...)` written — submit-up rationale (gateway-required).
 7. ✅ `notes` argument to `submit_up` >= 20 chars (gateway-enforced).
@@ -103,7 +103,7 @@ The gateway enforces some of these; the rest are convention but failing one of t

 1. ✅ At least one `commit()` on this branch (gateway-enforced).
 2. ✅ Every acceptance criterion is met by actual code or test, not just intention. Re-read them via `evidence(task_id)`.
-3. ✅ Tests/lint/typecheck pass locally — run them via `Bash`. If your project has `make quality` (or equivalent), run it. **`i_am_done` runs the fast gate (lint + typecheck) in your workspace and rejects the submit if it's red** — so run it yourself first and submit green on the first try; QA and CI run the full gate (incl. tests) too.
+3. ✅ Tests/lint/typecheck pass locally — run `make quality` (or `make gate` for the fast pre-submit gate). Never raw `uv run`. **`i_am_done` runs the fast gate (lint + typecheck) in your workspace and rejects the submit if it's red** — so run it yourself first and submit green on the first try; QA and CI run the full gate (incl. tests) too.
 4. ✅ `git diff` (call `evidence(task_id)` to inspect) shows nothing stray — no `print()` debugging, no commented-out code, no unrelated edits.
 5. ✅ `note(scope='reflect', task_id=...)` walks through every criterion (gateway-enforced as `journal:reflect`).
 6. ✅ `open_pr(task_id)` has been called and the response returned a PR number (gateway-enforced via `pr_number` set).
@@ -51,7 +51,7 @@ A pass without evidence is a betrayal of your role: the entire downstream chain
 3. If you need to re-inspect anything, call `evidence(task_id)`. **Do not** grep the workspace or run `Bash git diff` — the diff is in the response.
 4. **Read the dev's `reflect` note** — it walks through every acceptance criterion and explains how each is met. Cross-check those claims against the actual diff.
 5. For each acceptance criterion individually: confirm there is a referencing artifact (commit, progress entry, or file change) AND that the change actually meets it. Don't batch-approve criteria; check them one at a time.
-6. Run tests/lint via `Bash` (e.g. `make quality` or `pytest`) — even if the dev says they passed, you re-run.
+6. Run `make quality` — even if the dev says they passed, you re-run.
 7. `note(scope='struggle', text='...')` if you can't decide — flag the ambiguity rather than guess. Then `dm(recipient=<dev>, text='<question>')` to ask before failing.
 8. `note(scope='learning', text="<what worked / what would have caught the issue earlier / what pattern this work establishes>")` — required before pass/fail.
 9. Pass: `pass(task_id, notes="<>=80 chars: overall review summary, edge cases tested, any caveats>", ac_verdicts=["criterion 1 — verified by <commit/file/line>", "criterion 2 — verified by <artifact>", ...])` — **one entry per acceptance criterion, in the task's criterion order**; the gateway rejects a pass that leaves any criterion uncovered. If even one criterion does not hold, do NOT pass — `fail` instead. Fail: `fail(task_id, findings=[{"file": "path", "line": 42, "severity": "major", "criterion": "<ac id if applicable>", "expected": "...", "actual": "..."}, ...])` — one object per issue, capped at 10 (nudge above 5 — split or prioritize).
@@ -19,11 +19,13 @@

 ## Development Standards
 ```bash
-# Before any commit
-uv run ruff format .
-uv run ruff check .
-uv run mypy roboco/
-uv run pytest
+# Before any commit — use the Makefile, never raw `uv run`/`pip`/`conda`/`poetry`.
+# The Makefile sets UV_NO_SYNC=1 + a private UV_CACHE_DIR to prevent venv
+# corruption; bare `uv run` bypasses both.
+make lint        # ruff format + ruff check + mypy + vulture (formats in place)
+make gate        # fast pre-submit: ruff format --check + ruff check + mypy + xenon
+make quality     # full merge gate (lint+types+tests+cov+xenon+bandit+audit+...)
+make test        # pytest with coverage

 # Coverage target: 80%
 ```
@@ -383,4 +383,23 @@ if echo "$low" | grep -qE '(^|[[:space:];&|])(uv[[:space:]]+run|uvx)([[:space:]]
    exit 2
 fi

+# --- raw package-manager / test-runner commands — use the Makefile -----------
+# CEO direction: force the fleet to the Makefile. The blocks above deliberately
+# allowed bare `uv run` (workspace .venv); this overrides that when a Makefile is
+# present, denying raw uv/pip/conda/poetry and remediating to the make targets.
+# The Makefile sets UV_NO_SYNC=1 + a private UV_CACHE_DIR for consistent gate
+# behaviour; bare `uv run` bypasses both. Skipped when no Makefile exists so
+# Makefile-less projects aren't blocked. `make`-internal uv (hook inspects the
+# agent's command string, not subprocesses) and WorkspaceService's uv sync
+# (subprocess, not the agent Bash tool) are untouched. On grok a deny cancels
+# the whole run, so ROBOCO_GUARD_SKIP_PM=1 nudges (exit 0) instead.
+if test -f Makefile && echo "$low" | grep -qE '(^|[[:space:];&|])(uv[[:space:]]+(run|pip[[:space:]]+(install|uninstall)|lock|add|remove)|pip3?[[:space:]]+(install|uninstall)|conda[[:space:]]+(install|create|run)|poetry[[:space:]]+(run|install|add))([[:space:]]|$)'; then
+    if [ -n "${ROBOCO_GUARD_SKIP_PM:-}" ]; then
+        echo "Nudge: raw package-manager commands are blocked — use \`make quality\` / \`make gate\` / \`make lint\` / \`make test\`. The Makefile sets UV_NO_SYNC=1 + a private cache; bare \`uv run\` bypasses that." >&2
+        exit 0
+    fi
+    echo "Denied: raw package-manager commands are blocked — use the Makefile. Run \`make quality\` (full gate), \`make gate\` (fast pre-submit), \`make lint\`, or \`make test\`. The Makefile sets UV_NO_SYNC=1 + a private cache to prevent venv corruption; bare \`uv run\` bypasses that." >&2
+    exit 2
+fi
+
 exit 0
@@ -126,6 +126,29 @@ _GIT_MUTATE_DENY = (
 )
 _DESTRUCTIVE_DENY = ("Bash(rm -rf*)",)

+# Raw package-manager / test-runner commands — use the Makefile (CEO direction).
+# Native --deny is graceful (model adapts to `make`, run continues), so this is
+# the primary gate on grok; the bash-guard hook only catches the compound
+# commands the globs miss (cd x && uv run) and there nudges via
+# ROBOCO_GUARD_SKIP_PM=1 instead of canceling the run.
+_RAW_PM_DENY = (
+    "Bash(uv run*)",
+    "Bash(uv pip install*)",
+    "Bash(uv pip uninstall*)",
+    "Bash(uv lock*)",
+    "Bash(uv add*)",
+    "Bash(uv remove*)",
+    "Bash(pip install*)",
+    "Bash(pip3 install*)",
+    "Bash(pip uninstall*)",
+    "Bash(conda install*)",
+    "Bash(conda create*)",
+    "Bash(conda run*)",
+    "Bash(poetry run*)",
+    "Bash(poetry install*)",
+    "Bash(poetry add*)",
+)
+

 def render_config_toml(mcp_config: dict[str, Any]) -> str:
    """Translate Claude Code ``mcpServers`` into grok's ``[mcp_servers]`` TOML.
@@ -171,7 +194,7 @@ def _deny_rules(role: str) -> list[str]:
    """``--deny`` permission rules for a role (only bash-capable roles need any)."""
    if role not in _BASH_ROLES:
        return []  # bash removed entirely → nothing left to gate
-    return [*_DESTRUCTIVE_DENY, *_GIT_MUTATE_DENY]
+    return [*_DESTRUCTIVE_DENY, *_GIT_MUTATE_DENY, *_RAW_PM_DENY]


 def _effort() -> str | None:
@@ -266,7 +289,13 @@ def bash_guard_hook_config(hook_path: str = BASH_GUARD_HOOK) -> dict[str, Any]:
                        {
                            "type": "command",
                            "command": hook_path,
-                            "env": {"ROBOCO_GUARD_SKIP_GIT": "1"},
+                            "env": {
+                                "ROBOCO_GUARD_SKIP_GIT": "1",
+                                # Raw-PM is gated by graceful native --deny above;
+                                # the hook only nudges (exit 0) for compound
+                                # commands the globs miss, never cancels the run.
+                                "ROBOCO_GUARD_SKIP_PM": "1",
+                            },
                        }
                    ],
                }
@@ -191,7 +191,7 @@ def main() -> None:
        "",
        "# Per-role verb shapes (autogenerated)",
        "",
-        "Run `uv run python scripts/regenerate_verb_tables.py` after changing",
+        "Run `make foundation-check` after changing",
        "any role config or schema. Role prompts reference this file's sections",
        "as the source of truth for verb signatures.",
        "",
@@ -0,0 +1,40 @@
+"""Agent prompts must steer the fleet at the Makefile, not raw package managers.
+
+`agents/prompts/teams/backend.md` used to literally instruct `uv run ruff ...`
+(copied from the human-facing CLAUDE.md), so agents ran raw `uv run` and bypassed
+the Makefile's `UV_NO_SYNC=1` + private `UV_CACHE_DIR` — the guard against the
+concurrent-venv-corruption race. This grep invariant keeps the prompts honest:
+no agent-facing prompt instructs a raw `uv run`/`pip`/`conda`/`poetry`, and the
+backend standards block names the make targets.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+PROMPTS = Path(__file__).resolve().parents[3] / "agents" / "prompts"
+RAW_PM = ["uv run ", "uv pip install", "pip install ", "conda install ", "poetry run "]
+
+
+def _prompt_texts() -> dict[str, str]:
+    texts: dict[str, str] = {}
+    for path in PROMPTS.rglob("*.md"):
+        texts[str(path.relative_to(PROMPTS))] = path.read_text()
+    return texts
+
+
+def test_no_prompt_instructs_raw_package_managers() -> None:
+    """No agent-facing prompt tells an agent to run raw uv/pip/conda/poetry."""
+    offenders: list[str] = []
+    for name, body in _prompt_texts().items():
+        for bad in RAW_PM:
+            if bad in body:
+                offenders.append(f"{name}: '{bad.strip()}'")
+    assert not offenders, f"prompts still instruct raw PM commands: {offenders}"
+
+
+def test_backend_prompt_names_make_targets() -> None:
+    """The backend standards block must point at make targets, not uv run."""
+    body = (PROMPTS / "teams" / "backend.md").read_text()
+    assert "make quality" in body
+    assert "make gate" in body
@@ -0,0 +1,32 @@
+"""Grok native --deny covers raw package-manager commands (Makefile guardrail).
+
+Native ``--deny`` is graceful on grok (the model gets a permission error and
+adapts to ``make``; the run continues — unlike a hook deny, which cancels the
+whole run). So raw-PM commands are denied via ``_RAW_PM_DENY`` in
+``_deny_rules()``, mirroring ``_GIT_MUTATE_DENY``. The bash-guard hook handles
+only the compound-command case the globs miss, and there
+``ROBOCO_GUARD_SKIP_PM=1`` nudges instead of canceling.
+"""
+
+from __future__ import annotations
+
+from roboco.llm.providers.grok_cli_config import _RAW_PM_DENY, _deny_rules
+
+
+def test_raw_pm_deny_rules_present() -> None:
+    rules = _deny_rules("developer")
+    assert "Bash(uv run*)" in rules
+    assert "Bash(uv pip install*)" in rules
+    assert "Bash(pip install*)" in rules
+    assert "Bash(conda install*)" in rules
+    assert "Bash(poetry run*)" in rules
+
+
+def test_raw_pm_deny_is_subset_of_deny_rules() -> None:
+    rules = set(_deny_rules("developer"))
+    assert set(_RAW_PM_DENY) <= rules
+
+
+def test_raw_pm_deny_only_for_bash_roles() -> None:
+    """Non-bash roles have bash removed entirely — no deny rules at all."""
+    assert _deny_rules("auditor") == []
@@ -0,0 +1,86 @@
+"""bash-guard Makefile guardrail — deny raw uv/pip/conda/poetry, point at make.
+
+CEO item: force agents to the Makefile. The existing hook deliberately allowed
+bare ``uv run`` (workspace .venv, cwd-relative); this guard overrides that by
+CEO direction when a ``Makefile`` is present, denying raw package-manager /
+test-runner commands and remediating to the make targets. Skipped when no
+Makefile exists so Makefile-less projects aren't blocked. On the grok path
+(``ROBOCO_GUARD_SKIP_PM=1``) a deny cancels the whole run, so it nudges (exit 0)
+instead.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import subprocess
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+HOOK = REPO_ROOT / "docker" / "scripts" / "bash-guard-hook.sh"
+
+# Hook exits 2 to deny, 0 to allow. Named (not magic) for ruff PLR2004.
+_DENIED = 2
+_ALLOWED = 0
+
+
+def _run_hook(
+    command: str, cwd: Path, env_over: dict[str, str] | None = None
+) -> tuple[int, str]:
+    payload = json.dumps({"tool_input": {"command": command}})
+    env = dict(os.environ)
+    if env_over:
+        env.update(env_over)
+    proc = subprocess.run(
+        ["bash", str(HOOK)],
+        input=payload,
+        capture_output=True,
+        text=True,
+        cwd=str(cwd),
+        env=env,
+        check=False,
+    )
+    return proc.returncode, proc.stderr
+
+
+def test_denies_uv_run_when_makefile_present() -> None:
+    rc, err = _run_hook("uv run pytest", REPO_ROOT)
+    assert rc == _DENIED
+    assert "make" in err.lower()
+
+
+def test_denies_pip_install() -> None:
+    rc, _ = _run_hook("pip install requests", REPO_ROOT)
+    assert rc == _DENIED
+
+
+def test_denies_compound_uv_run() -> None:
+    rc, _ = _run_hook("cd svc && uv run ruff check .", REPO_ROOT)
+    assert rc == _DENIED
+
+
+def test_denies_conda_and_poetry() -> None:
+    assert _run_hook("conda install numpy", REPO_ROOT)[0] == _DENIED
+    assert _run_hook("poetry run pytest", REPO_ROOT)[0] == _DENIED
+
+
+def test_allows_make_quality() -> None:
+    rc, _ = _run_hook("make quality", REPO_ROOT)
+    assert rc != _DENIED
+
+
+def test_allows_pnpm() -> None:
+    rc, _ = _run_hook("pnpm lint", REPO_ROOT)
+    assert rc != _DENIED
+
+
+def test_skips_deny_without_makefile(tmp_path: Path) -> None:
+    rc, _ = _run_hook("uv run pytest", tmp_path)
+    assert rc != _DENIED
+
+
+def test_grok_path_nudges_not_denies() -> None:
+    """ROBOCO_GUARD_SKIP_PM=1 (grok) -> exit 0 nudge, not run-canceling exit 2."""
+    rc, err = _run_hook("uv run pytest", REPO_ROOT, {"ROBOCO_GUARD_SKIP_PM": "1"})
+    assert rc == _ALLOWED
+    assert "make" in err.lower()
@@ -379,8 +379,13 @@ def test_allows_python_httpx_import_without_internal_host() -> None:

 def test_allows_pytest_even_if_suite_uses_requests() -> None:
    """The command string is just the runner — no http-client token and
-    no internal host literal — so it must pass."""
-    assert _run("uv run python -m pytest tests/unit/ -q") == _ALLOWED
+    no internal host literal — so it must pass.
+
+    Bare ``python -m pytest`` (not ``uv run``): raw ``uv run`` is now
+    Makefile-gated (W1), so the runner here is the bare interpreter to keep
+    this test about the HTTP-injection allow path, not package-manager policy.
+    """
+    assert _run("python -m pytest tests/unit/ -q") == _ALLOWED


 # ---------------------------------------------------------------------------
@@ -435,8 +440,12 @@ def test_allows_uv_sync_in_workspace() -> None:
    )


-def test_allows_pip_install_in_workspace() -> None:
-    assert _run("pip install -r requirements.txt") == _ALLOWED
+def test_denies_pip_install_when_makefile_present() -> None:
+    """W1: raw ``pip install`` is Makefile-gated. A workspace clone carries
+    a ``Makefile`` (same repo), so a bare ``pip install`` is denied — agents
+    use ``make`` / ``uv sync --extra dev``. Makefile-less projects skip the
+    deny (covered in test_bash_guard_makefile_guardrail.py)."""
+    assert _run("pip install -r requirements.txt") == _DENIED


 def test_allows_reading_files_under_app() -> None: