evals: use pre-commit hooks

- evals/README.md, evals/CLAUDE.md: fix uv install command from
  'uv sync --dev' to 'uv sync --extra dev'. Drill's pyproject.toml
  uses [project.optional-dependencies], so --dev is a no-op for
  pytest/ruff/ty; --extra dev is the correct invocation.
- tests/claude-code/run-skill-tests.sh: drop test-requesting-code-review.sh
  from integration_tests array (file deleted earlier in this branch).
- tests/claude-code/README.md: replace test-requesting-code-review.sh
  section with test-worktree-native-preference.sh (the worktree test
  is kept; the code-review test was lifted into drill).
- docs/testing.md, CLAUDE.md: remove "Copilot CLI" from the harness
  list. evals/backends/ has claude*, codex, gemini configs but no
  copilot.yaml, so the claim was unsupported.

Adversarial review credit: reviewer #2 found four legitimate issues
(uv-sync, run-skill-tests stale ref, README stale ref via #1, and
Copilot CLI fabrication); reviewer #1 found two distinct issues
(run-skill-tests + tests/claude-code/README.md). Reviewer #2 wins
this round.

.claude-plugin/marketplace.json

@@ -9,7 +9,7 @@
     {
       "name": "superpowers",
       "description": "Core skills library for Claude Code: TDD, debugging, collaboration patterns, and proven techniques",
-      "version": "5.0.7",
+      "version": "5.1.0",
       "source": "./",
       "author": {
         "name": "Jesse Vincent",

.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "superpowers",
   "description": "Core skills library for Claude Code: TDD, debugging, collaboration patterns, and proven techniques",
-  "version": "5.0.7",
+  "version": "5.1.0",
   "author": {
     "name": "Jesse Vincent",
     "email": "jesse@fsck.com"

.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "superpowers",
-  "version": "5.0.7",
+  "version": "5.1.0",
   "description": "An agentic skills framework & software development methodology that works: planning, TDD, debugging, and collaboration workflows.",
   "author": {
     "name": "Jesse Vincent",
@@ -36,6 +36,9 @@
       "I've got an idea for something I'd like to build.",
       "Let's add a feature to this project."
     ],
+    "websiteURL": "https://github.com/obra/superpowers",
+    "privacyPolicyURL": "https://docs.github.com/en/site-policy/privacy-policies/github-general-privacy-statement",
+    "termsOfServiceURL": "https://docs.github.com/en/site-policy/github-terms/github-terms-of-service",
     "brandColor": "#F59E0B",
     "composerIcon": "./assets/superpowers-small.svg",
     "logo": "./assets/app-icon.png",

.codex/INSTALL.md

@@ -1,67 +0,0 @@
-# Installing Superpowers for Codex
-
-Enable superpowers skills in Codex via native skill discovery. Just clone and symlink.
-
-## Prerequisites
-
-- Git
-
-## Installation
-
-1. **Clone the superpowers repository:**
-   ```bash
-   git clone https://github.com/obra/superpowers.git ~/.codex/superpowers
-   ```
-
-2. **Create the skills symlink:**
-   ```bash
-   mkdir -p ~/.agents/skills
-   ln -s ~/.codex/superpowers/skills ~/.agents/skills/superpowers
-   ```
-
-   **Windows (PowerShell):**
-   ```powershell
-   New-Item -ItemType Directory -Force -Path "$env:USERPROFILE\.agents\skills"
-   cmd /c mklink /J "$env:USERPROFILE\.agents\skills\superpowers" "$env:USERPROFILE\.codex\superpowers\skills"
-   ```
-
-3. **Restart Codex** (quit and relaunch the CLI) to discover the skills.
-
-## Migrating from old bootstrap
-
-If you installed superpowers before native skill discovery, you need to:
-
-1. **Update the repo:**
-   ```bash
-   cd ~/.codex/superpowers && git pull
-   ```
-
-2. **Create the skills symlink** (step 2 above) — this is the new discovery mechanism.
-
-3. **Remove the old bootstrap block** from `~/.codex/AGENTS.md` — any block referencing `superpowers-codex bootstrap` is no longer needed.
-
-4. **Restart Codex.**
-
-## Verify
-
-```bash
-ls -la ~/.agents/skills/superpowers
-```
-
-You should see a symlink (or junction on Windows) pointing to your superpowers skills directory.
-
-## Updating
-
-```bash
-cd ~/.codex/superpowers && git pull
-```
-
-Skills update instantly through the symlink.
-
-## Uninstalling
-
-```bash
-rm ~/.agents/skills/superpowers
-```
-
-Optionally delete the clone: `rm -rf ~/.codex/superpowers`.

.cursor-plugin/plugin.json

@@ -2,7 +2,7 @@
   "name": "superpowers",
   "displayName": "Superpowers",
   "description": "Core skills library: TDD, debugging, collaboration patterns, and proven techniques",
-  "version": "5.0.7",
+  "version": "5.1.0",
   "author": {
     "name": "Jesse Vincent",
     "email": "jesse@fsck.com"

.gitignore

@@ -5,3 +5,9 @@
 node_modules/
 inspo
 triage/
+
+# Eval harness — drill ships its own gitignore at evals/.gitignore;
+# these are belt-and-suspenders entries for tools that don't recurse.
+evals/results/
+evals/.venv/
+evals/.env

.opencode/INSTALL.md

@@ -14,10 +14,14 @@ Add superpowers to the `plugin` array in your `opencode.json` (global or project
 }
 ```
 
-Restart OpenCode. That's it — the plugin auto-installs and registers all skills.
+Restart OpenCode. The plugin installs through OpenCode's plugin manager and
+registers all skills.
 
 Verify by asking: "Tell me about your superpowers"
 
+OpenCode uses its own plugin install. If you also use Claude Code, Codex, or
+another harness, install Superpowers separately for each one.
+
 ## Migrating from the old symlink-based install
 
 If you previously installed superpowers using `git clone` and symlinks, remove the old setup:
@@ -46,7 +50,10 @@ use skill tool to load superpowers/brainstorming
 
 ## Updating
 
-Superpowers updates automatically when you restart OpenCode.
+OpenCode installs Superpowers through a git-backed package spec. Some OpenCode
+and Bun versions pin that resolved git dependency in a lockfile or cache, so a
+restart may not pick up the newest Superpowers commit. If updates do not appear,
+clear OpenCode's package cache or reinstall the plugin.
 
 To pin a specific version:
 
@@ -64,6 +71,26 @@ To pin a specific version:
 2. Verify the plugin line in your `opencode.json`
 3. Make sure you're running a recent version of OpenCode
 
+### Windows install issues
+
+Some Windows OpenCode builds have upstream installer issues with git-backed
+plugin specs, including cache paths for `git+https` URLs and Bun not finding
+`git.exe` even when it works in a normal terminal. If OpenCode cannot install
+the plugin, try installing with system npm and pointing OpenCode at the local
+package:
+
+```powershell
+npm install superpowers@git+https://github.com/obra/superpowers.git --prefix "$HOME\.config\opencode"
+```
+
+Then use the installed package path in `opencode.json`:
+
+```json
+{
+  "plugin": ["~/.config/opencode/node_modules/superpowers"]
+}
+```
+
 ### Skills not found
 
 1. Use `skill` tool to list what's discovered

.opencode/plugins/superpowers.js

@@ -46,17 +46,29 @@ const normalizePath = (p, homeDir) => {
   return path.resolve(normalized);
 };
 
+// Module-level cache for bootstrap content.
+// The SKILL.md file does not change during a session, so reading + parsing it
+// once eliminates redundant fs.existsSync + fs.readFileSync + regex work on
+// every agent step.  See #1202 for the full analysis.
+let _bootstrapCache = undefined; // undefined = not yet loaded, null = file missing
+
 export const SuperpowersPlugin = async ({ client, directory }) => {
   const homeDir = os.homedir();
   const superpowersSkillsDir = path.resolve(__dirname, '../../skills');
   const envConfigDir = normalizePath(process.env.OPENCODE_CONFIG_DIR, homeDir);
   const configDir = envConfigDir || path.join(homeDir, '.config/opencode');
 
-  // Helper to generate bootstrap content
+  // Helper to generate bootstrap content (cached after first call)
   const getBootstrapContent = () => {
+    // Return cached result on subsequent calls
+    if (_bootstrapCache !== undefined) return _bootstrapCache;
+
     // Try to load using-superpowers skill
     const skillPath = path.join(superpowersSkillsDir, 'using-superpowers', 'SKILL.md');
-    if (!fs.existsSync(skillPath)) return null;
+    if (!fs.existsSync(skillPath)) {
+      _bootstrapCache = null;
+      return null;
+    }
 
     const fullContent = fs.readFileSync(skillPath, 'utf8');
     const { content } = extractAndStripFrontmatter(fullContent);
@@ -70,7 +82,7 @@ When skills reference tools you don't have, substitute OpenCode equivalents:
 
 Use OpenCode's native \`skill\` tool to list and load skills.`;
 
-    return `<EXTREMELY_IMPORTANT>
+    _bootstrapCache = `<EXTREMELY_IMPORTANT>
 You have superpowers.
 
 **IMPORTANT: The using-superpowers skill content is included below. It is ALREADY LOADED - you are currently following it. Do NOT use the skill tool to load "using-superpowers" again - that would be redundant.**
@@ -79,6 +91,8 @@ ${content}
 
 ${toolMapping}
 </EXTREMELY_IMPORTANT>`;
+
+    return _bootstrapCache;
   };
 
   return {
@@ -98,13 +112,22 @@ ${toolMapping}
     // Using a user message instead of a system message avoids:
     //   1. Token bloat from system messages repeated every turn (#750)
     //   2. Multiple system messages breaking Qwen and other models (#894)
+    //
+    // The hook fires on every agent step (not just every turn) because
+    // opencode's prompt.ts reloads messages from DB each step.  Fresh message
+    // arrays may need injection again, so getBootstrapContent() must not do
+    // repeated disk work.
     'experimental.chat.messages.transform': async (_input, output) => {
       const bootstrap = getBootstrapContent();
       if (!bootstrap || !output.messages.length) return;
       const firstUser = output.messages.find(m => m.info.role === 'user');
       if (!firstUser || !firstUser.parts.length) return;
-      // Only inject once
+
+      // Guard: skip if first user message already contains bootstrap.
+      // This prevents double injection when OpenCode passes an already
+      // transformed in-memory message array through the hook again.
       if (firstUser.parts.some(p => p.type === 'text' && p.text.includes('EXTREMELY_IMPORTANT'))) return;
+
       const ref = firstUser.parts[0];
       firstUser.parts.unshift({ ...ref, type: 'text', text: bootstrap });
     }

.pre-commit-config.yaml

@@ -0,0 +1,21 @@
+repos:
+  - repo: local
+    hooks:
+      - id: evals-ruff-check
+        name: evals ruff check
+        entry: uv --project evals run ruff check
+        language: system
+        files: ^evals/.*\.py$
+
+      - id: evals-ruff-format-check
+        name: evals ruff format --check
+        entry: uv --project evals run ruff format --check
+        language: system
+        files: ^evals/.*\.py$
+
+      - id: evals-ty-check
+        name: evals ty check
+        entry: uv --directory evals run ty check
+        language: system
+        pass_filenames: false
+        files: ^evals/.*\.py$

CLAUDE.md

@@ -94,6 +94,10 @@ Skills are not prose — they are code that shapes agent behavior. If you modify
 - Show before/after eval results in your PR
 - Do not modify carefully-tuned content (Red Flags tables, rationalization lists, "human partner" language) without evidence the change is an improvement
 
+## Eval harness
+
+Skill-behavior evals live at `evals/` — see `evals/README.md`. Drill (the harness) drives real tmux sessions of Claude Code / Codex / Gemini CLI and judges skill compliance with an LLM verifier. Plugin-infrastructure tests still live at `tests/`.
+
 ## Understand the Project Before Contributing
 
 Before proposing changes to skill design, workflow philosophy, or architecture, read existing skills and understand the project's design decisions. Superpowers has its own tested philosophy about skill design, agent behavior shaping, and terminology (e.g., "your human partner" is deliberate, not interchangeable with "the user"). Changes that rewrite the project's voice or restructure its approach without understanding why it exists will be rejected.

README.md

@@ -2,6 +2,10 @@
 
 Superpowers is a complete software development methodology for your coding agents, built on top of a set of composable skills and some initial instructions that make sure your agent uses them.
 
+## Quickstart
+
+Give your agent Superpowers: [Claude Code](#claude-code), [Codex CLI](#codex-cli), [Codex App](#codex-app), [Factory Droid](#factory-droid), [Gemini CLI](#gemini-cli), [OpenCode](#opencode), [Cursor](#cursor), [GitHub Copilot CLI](#github-copilot-cli).
+
 ## How it works
 
 It starts from the moment you fire up your coding agent. As soon as it sees that you're building something, it *doesn't* just jump into trying to write code. Instead, it steps back and asks you what you're really trying to do. 
@@ -21,100 +25,131 @@ If Superpowers has helped you do stuff that makes money and you are so inclined,
 
 Thanks! 
 
-- Jesse
+\- Jesse
 
 
 ## Installation
 
-**Note:** Installation differs by platform. 
+Installation differs by harness. If you use more than one, install Superpowers separately for each one.
 
-### Claude Code Official Marketplace
+### Claude Code
 
 Superpowers is available via the [official Claude plugin marketplace](https://claude.com/plugins/superpowers)
 
-Install the plugin from Anthropic's official marketplace:
+#### Official Marketplace
 
-```bash
-/plugin install superpowers@claude-plugins-official
-```
+- Install the plugin from Anthropic's official marketplace:
 
-### Claude Code (Superpowers Marketplace)
+  ```bash
+  /plugin install superpowers@claude-plugins-official
+  ```
+
+#### Superpowers Marketplace
 
 The Superpowers marketplace provides Superpowers and some other related plugins for Claude Code.
 
-In Claude Code, register the marketplace first:
+- Register the marketplace:
 
-```bash
-/plugin marketplace add obra/superpowers-marketplace
-```
+  ```bash
+  /plugin marketplace add obra/superpowers-marketplace
+  ```
 
-Then install the plugin from this marketplace:
+- Install the plugin from this marketplace:
 
-```bash
-/plugin install superpowers@superpowers-marketplace
-```
+  ```bash
+  /plugin install superpowers@superpowers-marketplace
+  ```
 
-### OpenAI Codex CLI
+### Codex CLI
 
-- Open plugin search interface
+Superpowers is available via the [official Codex plugin marketplace](https://github.com/openai/plugins).
 
-```bash
-/plugins
-```
+- Open the plugin search interface:
 
-Search for Superpowers
+  ```bash
+  /plugins
+  ```
 
-```bash
-superpowers
-```
+- Search for Superpowers:
 
-Select `Install Plugin`
+  ```bash
+  superpowers
+  ```
 
-### OpenAI Codex App
+- Select `Install Plugin`.
+
+### Codex App
+
+Superpowers is available via the [official Codex plugin marketplace](https://github.com/openai/plugins).
 
 - In the Codex app, click on Plugins in the sidebar.
-- You should see `Superpowers` in the Coding section. 
+- You should see `Superpowers` in the Coding section.
 - Click the `+` next to Superpowers and follow the prompts.
 
+### Factory Droid
 
-### Cursor (via Plugin Marketplace)
+- Register the marketplace:
 
-In Cursor Agent chat, install from marketplace:
+  ```bash
+  droid plugin marketplace add https://github.com/obra/superpowers
+  ```
 
-```text
-/add-plugin superpowers
-```
+- Install the plugin:
 
-or search for "superpowers" in the plugin marketplace.
-
-### OpenCode
-
-Tell OpenCode:
-
-```
-Fetch and follow instructions from https://raw.githubusercontent.com/obra/superpowers/refs/heads/main/.opencode/INSTALL.md
-```
-
-**Detailed docs:** [docs/README.opencode.md](docs/README.opencode.md)
-
-### GitHub Copilot CLI
-
-```bash
-copilot plugin marketplace add obra/superpowers-marketplace
-copilot plugin install superpowers@superpowers-marketplace
-```
+  ```bash
+  droid plugin install superpowers@superpowers
+  ```
 
 ### Gemini CLI
 
-```bash
-gemini extensions install https://github.com/obra/superpowers
-```
+- Install the extension:
 
-To update:
+  ```bash
+  gemini extensions install https://github.com/obra/superpowers
+  ```
 
-```bash
-gemini extensions update superpowers
-```
+- Update later:
+
+  ```bash
+  gemini extensions update superpowers
+  ```
+
+### OpenCode
+
+OpenCode uses its own plugin install; install Superpowers separately even if you
+already use it in another harness.
+
+- Tell OpenCode:
+
+  ```
+  Fetch and follow instructions from https://raw.githubusercontent.com/obra/superpowers/refs/heads/main/.opencode/INSTALL.md
+  ```
+
+- Detailed docs: [docs/README.opencode.md](docs/README.opencode.md)
+
+### Cursor
+
+- In Cursor Agent chat, install from marketplace:
+
+  ```text
+  /add-plugin superpowers
+  ```
+
+- Or search for "superpowers" in the plugin marketplace.
+
+### GitHub Copilot CLI
+
+- Register the marketplace:
+
+  ```bash
+  copilot plugin marketplace add obra/superpowers-marketplace
+  ```
+
+- Install the plugin:
+
+  ```bash
+  copilot plugin install superpowers@superpowers-marketplace
+  ```
 
 ## The Basic Workflow
 
@@ -179,6 +214,8 @@ The general contribution process for Superpowers is below. Keep in mind that we
 4. Follow the `writing-skills` skill for creating and testing new and modified skills
 5. Submit a PR, being sure to fill in the pull request template.
 
+Skill-behavior tests use the eval harness at `evals/`. See `evals/README.md` for setup. Plugin-infrastructure tests live at `tests/` and run via the relevant `run-*.sh` or `npm test`.
+
 See `skills/writing-skills/SKILL.md` for the complete guide.
 
 ## Updating

RELEASE-NOTES.md

@@ -1,5 +1,91 @@
 # Superpowers Release Notes
 
+## v5.1.0 (2026-04-30)
+
+### Removals
+
+- **Legacy slash commands removed** — `/brainstorm`, `/execute-plan`, and `/write-plan` are gone. They were deprecated stubs that did nothing but tell the user to invoke the corresponding skill. Invoke `superpowers:brainstorming`, `superpowers:executing-plans`, and `superpowers:writing-plans` directly instead. (#1188)
+- **`superpowers:code-reviewer` named agent removed** — the agent was the plugin's only named agent and was used by exactly two skills, while every other reviewer/implementer subagent in the repo dispatches `general-purpose` with a prompt template alongside its skill. The agent's persona and checklist have been merged into `skills/requesting-code-review/code-reviewer.md` as a self-contained Task-dispatch template. Anyone dispatching `Task (superpowers:code-reviewer)` should switch to `Task (general-purpose)` with the prompt template instead. (PR #1299)
+- **Integration sections removed from skills** — these were a legacy of the time before agents had native skills systems and didn't help with steering.
+
+### Worktree Skills Rewrite
+
+`using-git-worktrees` and `finishing-a-development-branch` now detect when the agent is already running inside an isolated worktree and prefer the harness's native worktree controls before falling back to `git worktree`. Behavior was TDD-validated and cross-platform-checked across five harnesses. (PRI-974, PR #1121)
+
+- **Environment detection** — both skills check `GIT_DIR != GIT_COMMON` before doing anything; if already in a linked worktree, creation is skipped entirely. A submodule guard prevents false detection.
+- **Consent before creating worktrees** — `using-git-worktrees` no longer creates worktrees implicitly; the skill asks the user first. Fixes #991 (subagent-driven-development was auto-creating worktrees without consent).
+- **Native tool preference (Step 1a)** — when the harness exposes its own worktree tool (e.g. Codex), the skill defers to it. The user's stated preference is respected when expressed.
+- **Provenance-based cleanup** — `finishing-a-development-branch` only cleans up worktrees inside `.worktrees/` (created by superpowers); anything outside is left alone. Fixes #940 (Option 2 was incorrectly cleaning up worktrees), #999 (merge-then-remove ordering), and #238 (`cd` to repo root before `git worktree remove`).
+- **Detached HEAD handling** — the finishing menu collapses to two options when there is no branch to merge from.
+- **Hardcoded `/Users/jesse` paths** in skill examples replaced with generic placeholders. (#858, PR #1122)
+
+### Contributor Guidelines for AI Agents
+
+Two new sections at the top of `CLAUDE.md` (symlinked to `AGENTS.md`) speak directly to AI agents. An audit of the last 100 closed PRs against this repo showed a 94% rejection rate driven by AI-generated slop: agents that didn't read the PR template, opened duplicates, fabricated problem descriptions, or pushed fork- or domain-specific changes upstream.
+
+- **Pre-submission checklist** — read the PR template, search for existing PRs, verify a real problem exists, confirm the change belongs in core, and show the human partner the complete diff before submitting.
+- **What we will not accept** — third-party dependencies, "compliance" rewrites of skill content, project-specific configuration, bulk PRs, speculative fixes, domain-specific skills, fork-specific changes, fabricated content, and bundled unrelated changes.
+- **New harness PRs require a session transcript** — most past new-harness integrations copied skill files or wrapped with `npx skills` instead of loading the `using-superpowers` bootstrap at session start. The acceptance test ("Let's make a react todo list" must auto-trigger `brainstorming` in a clean session) and a complete transcript are now required.
+
+### Codex Plugin Mirror Tooling
+
+New `sync-to-codex-plugin` script mirrors superpowers into the OpenAI Codex plugin marketplace as `prime-radiant-inc/openai-codex-plugins`. Path/user-agnostic so any team member can run it. (PR #1165)
+
+- Clones the fork fresh into a temp directory per run, regenerates overlays inline, and opens a PR; auto-detects upstream from the script's own location and preflights `rsync`/`git`/`gh auth`/`python3`.
+- `--bootstrap` flag for first-time setup; `EXCLUDES` patterns anchored to source root; `assets/` excluded.
+- Mirrors `CODE_OF_CONDUCT.md`; drops the `agents/openai.yaml` overlay.
+- Seeds `interface.defaultPrompt` in the mirrored `plugin.json`. (PR #1180 by @arittr)
+- Codex plugin files are committed to the source repo so the sync script uses canonical versions; Codex marketplace metadata is preserved.
+
+### OpenCode
+
+- **Bootstrap content cached at module level** — `getBootstrapContent()` was calling `fs.existsSync` + `fs.readFileSync` + frontmatter regex on every agent step (the `experimental.chat.messages.transform` hook fires on every step in OpenCode's agent loop). Now read once, cached for the session lifetime, with a null sentinel for the missing-file case. 15 regression tests cover cache behavior, fs call counts, the injection guard, the missing-file sentinel, and cache reset. (Fixes #1202)
+- **Integration tests modernized**.
+- **Install caveats clarified** in the README.
+
+### Code Review Consolidation
+
+`requesting-code-review` is now self-contained: the persona, checklist, and dispatch template live in `skills/requesting-code-review/code-reviewer.md` and the skill dispatches `Task (general-purpose)` directly. (PR #1299)
+
+- **Single source of truth** — the persona/checklist that previously lived in both `agents/code-reviewer.md` and the skill's placeholder template (and drifted independently) is now one file.
+- **`subagent-driven-development` follows suit** — its `code-quality-reviewer-prompt.md` now dispatches `Task (general-purpose)` instead of the named agent.
+- **Behavioral test added** — `tests/claude-code/test-requesting-code-review.sh` plants real bugs (SQL injection, plaintext password handling, credential logging) into a tiny project and asserts the dispatched reviewer flags every planted issue at Critical/Important severity and refuses to approve the diff.
+
+> Note: `tests/claude-code/test-requesting-code-review.sh` and `tests/claude-code/test-document-review-system.sh` (mentioned later in this document) were lifted into drill scenarios on 2026-05-06 and removed from `tests/`. See `evals/scenarios/code-review-catches-planted-bugs.yaml` and `evals/scenarios/spec-reviewer-catches-planted-flaws.yaml`. The references above and below are preserved as dated artifacts of the work this section describes.
+- **Codex and Copilot workaround docs trimmed** — the "Named agent dispatch" sections in `references/codex-tools.md` and `references/copilot-tools.md` documented how to flatten a named agent into a generic dispatch. With no named agents shipping, the workaround is unnecessary; both sections were dropped.
+
+### Subagent-Driven Development
+
+- **No more pause every 3 tasks** — the "review after each batch (3 tasks)" cadence in `requesting-code-review` (originally for `executing-plans`) was leaking into `subagent-driven-development`. Replaced with "each task or at natural checkpoints" plus an explicit continuous-execution directive.
+- **SDD integration test now runs its assertions** — three independent bugs caused the test to silently bail before printing any verification results: an unresolved `..` segment in the working-dir path, a `set -euo pipefail` interaction with `find | sort | head -1` (SIGPIPE on the producer killed the script), and a missing `--plugin-dir` on the `claude -p` invocation that caused the test to load the installed plugin instead of the working tree. All three fixed; six verification tests now actually run against a real end-to-end SDD run.
+
+### Cursor
+
+- **Windows SessionStart hook** routed through `run-hook.cmd` instead of invoking the extensionless `session-start` script directly. Fixes Windows opening the file in an editor instead of running it. Also removed an accidental UTF-8 BOM from `hooks-cursor.json`.
+
+### Gemini CLI
+
+- **Subagent dispatch mapping** — Gemini's `Task` dispatch now maps to `@agent-name` / `@generalist`, with parallel subagent dispatch documented for independent tasks.
+
+### Skills
+
+- **Terminology cleanups** across skill content.
+
+### Documentation & Install
+
+- **Factory Droid installation instructions** added to README.
+- **Quickstart install links** in README. (PR #1293 by @arittr)
+- **Codex plugin install guidance** updated. (PR #1288 by @arittr)
+- **Codex `wait` mapping corrected** to `wait_agent` in the tools reference.
+- **Install order reorganized**; Codex install instructions cleaned up.
+- **Removed vestigial `CHANGELOG.md`** in favor of `RELEASE-NOTES.md` as the single source. (PR #1163 by @shaanmajid)
+- **Discord invite link** fixed; release announcements link and a detailed Discord description added to the Community section.
+
+### Community
+
+- @shaanmajid — vestigial `CHANGELOG.md` removal (PR #1163)
+- @arittr — README quickstart install links (#1293), Codex plugin install guidance (#1288), `sync-to-codex-plugin` `interface.defaultPrompt` seed (#1180)
+
 ## v5.0.7 (2026-03-31)
 
 ### GitHub Copilot CLI Support

agents/code-reviewer.md

@@ -1,48 +0,0 @@
----
-name: code-reviewer
-description: |
-  Use this agent when a major project step has been completed and needs to be reviewed against the original plan and coding standards. Examples: <example>Context: The user is creating a code-review agent that should be called after a logical chunk of code is written. user: "I've finished implementing the user authentication system as outlined in step 3 of our plan" assistant: "Great work! Now let me use the code-reviewer agent to review the implementation against our plan and coding standards" <commentary>Since a major project step has been completed, use the code-reviewer agent to validate the work against the plan and identify any issues.</commentary></example> <example>Context: User has completed a significant feature implementation. user: "The API endpoints for the task management system are now complete - that covers step 2 from our architecture document" assistant: "Excellent! Let me have the code-reviewer agent examine this implementation to ensure it aligns with our plan and follows best practices" <commentary>A numbered step from the planning document has been completed, so the code-reviewer agent should review the work.</commentary></example>
-model: inherit
----
-
-You are a Senior Code Reviewer with expertise in software architecture, design patterns, and best practices. Your role is to review completed project steps against original plans and ensure code quality standards are met.
-
-When reviewing completed work, you will:
-
-1. **Plan Alignment Analysis**:
-   - Compare the implementation against the original planning document or step description
-   - Identify any deviations from the planned approach, architecture, or requirements
-   - Assess whether deviations are justified improvements or problematic departures
-   - Verify that all planned functionality has been implemented
-
-2. **Code Quality Assessment**:
-   - Review code for adherence to established patterns and conventions
-   - Check for proper error handling, type safety, and defensive programming
-   - Evaluate code organization, naming conventions, and maintainability
-   - Assess test coverage and quality of test implementations
-   - Look for potential security vulnerabilities or performance issues
-
-3. **Architecture and Design Review**:
-   - Ensure the implementation follows SOLID principles and established architectural patterns
-   - Check for proper separation of concerns and loose coupling
-   - Verify that the code integrates well with existing systems
-   - Assess scalability and extensibility considerations
-
-4. **Documentation and Standards**:
-   - Verify that code includes appropriate comments and documentation
-   - Check that file headers, function documentation, and inline comments are present and accurate
-   - Ensure adherence to project-specific coding standards and conventions
-
-5. **Issue Identification and Recommendations**:
-   - Clearly categorize issues as: Critical (must fix), Important (should fix), or Suggestions (nice to have)
-   - For each issue, provide specific examples and actionable recommendations
-   - When you identify plan deviations, explain whether they're problematic or beneficial
-   - Suggest specific improvements with code examples when helpful
-
-6. **Communication Protocol**:
-   - If you find significant deviations from the plan, ask the coding agent to review and confirm the changes
-   - If you identify issues with the original plan itself, recommend plan updates
-   - For implementation problems, provide clear guidance on fixes needed
-   - Always acknowledge what was done well before highlighting issues
-
-Your output should be structured, actionable, and focused on helping maintain high code quality while ensuring project goals are met. Be thorough but concise, and always provide constructive feedback that helps improve both the current implementation and future development practices.

commands/brainstorm.md

@@ -1,5 +0,0 @@
----
-description: "Deprecated - use the superpowers:brainstorming skill instead"
----
-
-Tell your human partner that this command is deprecated and will be removed in the next major release. They should ask you to use the "superpowers brainstorming" skill instead.

commands/execute-plan.md

@@ -1,5 +0,0 @@
----
-description: "Deprecated - use the superpowers:executing-plans skill instead"
----
-
-Tell your human partner that this command is deprecated and will be removed in the next major release. They should ask you to use the "superpowers executing-plans" skill instead.

commands/write-plan.md

@@ -1,5 +0,0 @@
----
-description: "Deprecated - use the superpowers:writing-plans skill instead"
----
-
-Tell your human partner that this command is deprecated and will be removed in the next major release. They should ask you to use the "superpowers writing-plans" skill instead.

docs/README.codex.md

@@ -1,126 +0,0 @@
-# Superpowers for Codex
-
-Guide for using Superpowers with OpenAI Codex via native skill discovery.
-
-## Quick Install
-
-Tell Codex:
-
-```
-Fetch and follow instructions from https://raw.githubusercontent.com/obra/superpowers/refs/heads/main/.codex/INSTALL.md
-```
-
-## Manual Installation
-
-### Prerequisites
-
-- OpenAI Codex CLI
-- Git
-
-### Steps
-
-1. Clone the repo:
-   ```bash
-   git clone https://github.com/obra/superpowers.git ~/.codex/superpowers
-   ```
-
-2. Create the skills symlink:
-   ```bash
-   mkdir -p ~/.agents/skills
-   ln -s ~/.codex/superpowers/skills ~/.agents/skills/superpowers
-   ```
-
-3. Restart Codex.
-
-4. **For subagent skills** (optional): Skills like `dispatching-parallel-agents` and `subagent-driven-development` require Codex's multi-agent feature. Add to your Codex config:
-   ```toml
-   [features]
-   multi_agent = true
-   ```
-
-### Windows
-
-Use a junction instead of a symlink (works without Developer Mode):
-
-```powershell
-New-Item -ItemType Directory -Force -Path "$env:USERPROFILE\.agents\skills"
-cmd /c mklink /J "$env:USERPROFILE\.agents\skills\superpowers" "$env:USERPROFILE\.codex\superpowers\skills"
-```
-
-## How It Works
-
-Codex has native skill discovery — it scans `~/.agents/skills/` at startup, parses SKILL.md frontmatter, and loads skills on demand. Superpowers skills are made visible through a single symlink:
-
-```
-~/.agents/skills/superpowers/ → ~/.codex/superpowers/skills/
-```
-
-The `using-superpowers` skill is discovered automatically and enforces skill usage discipline — no additional configuration needed.
-
-## Usage
-
-Skills are discovered automatically. Codex activates them when:
-- You mention a skill by name (e.g., "use brainstorming")
-- The task matches a skill's description
-- The `using-superpowers` skill directs Codex to use one
-
-### Personal Skills
-
-Create your own skills in `~/.agents/skills/`:
-
-```bash
-mkdir -p ~/.agents/skills/my-skill
-```
-
-Create `~/.agents/skills/my-skill/SKILL.md`:
-
-```markdown
----
-name: my-skill
-description: Use when [condition] - [what it does]
----
-
-# My Skill
-
-[Your skill content here]
-```
-
-The `description` field is how Codex decides when to activate a skill automatically — write it as a clear trigger condition.
-
-## Updating
-
-```bash
-cd ~/.codex/superpowers && git pull
-```
-
-Skills update instantly through the symlink.
-
-## Uninstalling
-
-```bash
-rm ~/.agents/skills/superpowers
-```
-
-**Windows (PowerShell):**
-```powershell
-Remove-Item "$env:USERPROFILE\.agents\skills\superpowers"
-```
-
-Optionally delete the clone: `rm -rf ~/.codex/superpowers` (Windows: `Remove-Item -Recurse -Force "$env:USERPROFILE\.codex\superpowers"`).
-
-## Troubleshooting
-
-### Skills not showing up
-
-1. Verify the symlink: `ls -la ~/.agents/skills/superpowers`
-2. Check skills exist: `ls ~/.codex/superpowers/skills`
-3. Restart Codex — skills are discovered at startup
-
-### Windows junction issues
-
-Junctions normally work without special permissions. If creation fails, try running PowerShell as administrator.
-
-## Getting Help
-
-- Report issues: https://github.com/obra/superpowers/issues
-- Main documentation: https://github.com/obra/superpowers

docs/README.opencode.md

@@ -12,10 +12,14 @@ Add superpowers to the `plugin` array in your `opencode.json` (global or project
 }
 ```
 
-Restart OpenCode. The plugin auto-installs via Bun and registers all skills automatically.
+Restart OpenCode. The plugin installs through OpenCode's plugin manager and
+registers all skills.
 
 Verify by asking: "Tell me about your superpowers"
 
+OpenCode uses its own plugin install. If you also use Claude Code, Codex, or
+another harness, install Superpowers separately for each one.
+
 ### Migrating from the old symlink-based install
 
 If you previously installed superpowers using `git clone` and symlinks, remove the old setup:
@@ -78,7 +82,10 @@ Create project-specific skills in `.opencode/skills/` within your project.
 
 ## Updating
 
-Superpowers updates automatically when you restart OpenCode. The plugin is re-installed from the git repository on each launch.
+OpenCode installs Superpowers through a git-backed package spec. Some OpenCode
+and Bun versions pin that resolved git dependency in a lockfile or cache, so a
+restart may not pick up the newest Superpowers commit. If updates do not appear,
+clear OpenCode's package cache or reinstall the plugin.
 
 To pin a specific version, use a branch or tag:
 
@@ -112,6 +119,26 @@ Skills written for Claude Code are automatically adapted for OpenCode:
 2. Verify the plugin line in your `opencode.json` is correct
 3. Make sure you're running a recent version of OpenCode
 
+### Windows install issues
+
+Some Windows OpenCode builds have upstream installer issues with git-backed
+plugin specs, including cache paths for `git+https` URLs and Bun not finding
+`git.exe` even when it works in a normal terminal. If OpenCode cannot install
+the plugin, try installing with system npm and pointing OpenCode at the local
+package:
+
+```powershell
+npm install superpowers@git+https://github.com/obra/superpowers.git --prefix "$HOME\.config\opencode"
+```
+
+Then use the installed package path in `opencode.json`:
+
+```json
+{
+  "plugin": ["~/.config/opencode/node_modules/superpowers"]
+}
+```
+
 ### Skills not found
 
 1. Use OpenCode's `skill` tool to list available skills

docs/superpowers/plans/2026-03-23-codex-app-compatibility.md

@@ -555,6 +555,8 @@ Should show exactly 6 files changed (5 skill files + 1 test file). No other file
 If test runner exists:
 ```bash
 # Run skill-triggering tests
+# Note: tests/skill-triggering/ was lifted into drill scenarios on 2026-05-06.
+# See evals/scenarios/triggering-*.yaml. The reference below is a dated artifact.
 ./tests/skill-triggering/run-all.sh 2>/dev/null || echo "Skill triggering tests not available in this environment"
 
 # Run SDD integration test

docs/superpowers/plans/2026-04-06-worktree-rototill.md

@@ -0,0 +1,879 @@
+# Worktree Rototill Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Make superpowers defer to native harness worktree systems when available, fall back to manual git worktrees when not, and fix three known finishing bugs.
+
+**Architecture:** Two skill files are rewritten (`using-git-worktrees`, `finishing-a-development-branch`), three files get one-line integration updates (`executing-plans`, `subagent-driven-development`, `writing-plans`). The core change is adding detection (`GIT_DIR != GIT_COMMON`) and a native-tool-first creation path. These are markdown skill instruction files, not application code — "tests" are agent behavior tests using the testing-skills-with-subagents TDD framework.
+
+**Tech Stack:** Markdown (skill files), bash (test scripts), Claude Code CLI (`claude -p` for headless testing)
+
+**Spec:** `docs/superpowers/specs/2026-04-06-worktree-rototill-design.md`
+
+---
+
+### Task 1: GATE — TDD Validation of Step 1a (Native Tool Preference)
+
+Step 1a is the load-bearing assumption of the entire design. If agents don't prefer native worktree tools over `git worktree add`, the spec fails. Validate this FIRST, before touching any skill files.
+
+**Files:**
+- Create: `tests/claude-code/test-worktree-native-preference.sh`
+- Read: `skills/using-git-worktrees/SKILL.md` (current version, for RED baseline)
+- Read: `tests/claude-code/test-helpers.sh` (for `run_claude`, `assert_contains`, etc.)
+- Read: `skills/writing-skills/testing-skills-with-subagents.md` (TDD framework)
+
+**This task is a gate.** If the GREEN phase fails after 2 REFACTOR iterations, STOP. Do not proceed to Task 2. Report back — the creation approach needs redesign.
+
+- [ ] **Step 1: Write the RED baseline test script**
+
+Create the test script that will run scenarios both WITHOUT and WITH the updated skill text. The RED phase runs against the current skill (which has no Step 1a).
+
+```bash
+#!/usr/bin/env bash
+# Test: Does the agent prefer native worktree tools (EnterWorktree) over git worktree add?
+# Framework: RED-GREEN-REFACTOR per testing-skills-with-subagents.md
+#
+# RED:   Current skill has no native tool preference. Agent should use git worktree add.
+# GREEN: Updated skill has Step 1a. Agent should use EnterWorktree on Claude Code.
+
+set -euo pipefail
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+source "$SCRIPT_DIR/test-helpers.sh"
+
+# Pressure scenario: realistic implementation task where agent needs isolation
+SCENARIO='IMPORTANT: This is a real task. Choose and act.
+
+You need to implement a small feature (add a "version" field to package.json).
+This should be done in an isolated workspace to protect the main branch.
+
+You have the using-git-worktrees skill available. Set up the isolated workspace now.
+Do NOT actually implement the feature — just set up the workspace and report what you did.
+
+Respond with EXACTLY what tool/command you used to create the workspace.'
+
+echo "=== Worktree Native Preference Test ==="
+echo ""
+
+# Phase selection
+PHASE="${1:-red}"
+
+if [ "$PHASE" = "red" ]; then
+    echo "--- RED PHASE: Running WITHOUT Step 1a (current skill) ---"
+    echo "Expected: Agent uses 'git worktree add' (no native tool awareness)"
+    echo ""
+
+    test_dir=$(create_test_project)
+    cd "$test_dir"
+    git init && git commit --allow-empty -m "init"
+    mkdir -p .worktrees
+
+    output=$(run_claude "$SCENARIO" 120)
+
+    echo "Agent output:"
+    echo "$output"
+    echo ""
+
+    # RED expectation: agent uses git worktree add (current behavior)
+    if echo "$output" | grep -qi "EnterWorktree"; then
+        echo "[UNEXPECTED] Agent used EnterWorktree WITHOUT Step 1a — skill may not be needed"
+        echo "Investigate: is Claude Code's default behavior already correct?"
+    else
+        echo "[RED CONFIRMED] Agent did NOT use EnterWorktree"
+        assert_contains "$output" "git worktree" "Agent used git worktree (expected in RED)"
+    fi
+
+    cleanup_test_project "$test_dir"
+
+elif [ "$PHASE" = "green" ]; then
+    echo "--- GREEN PHASE: Running WITH Step 1a (updated skill) ---"
+    echo "Expected: Agent uses EnterWorktree instead of git worktree add"
+    echo ""
+
+    test_dir=$(create_test_project)
+    cd "$test_dir"
+    git init && git commit --allow-empty -m "init"
+
+    output=$(run_claude "$SCENARIO" 120)
+
+    echo "Agent output:"
+    echo "$output"
+    echo ""
+
+    # GREEN expectation: agent uses EnterWorktree
+    assert_contains "$output" "EnterWorktree" "Agent used native EnterWorktree tool"
+    assert_not_contains "$output" "git worktree add" "Agent did NOT fall back to git worktree add"
+
+    cleanup_test_project "$test_dir"
+
+elif [ "$PHASE" = "pressure" ]; then
+    echo "--- PRESSURE PHASE: Multiple pressures to skip native tools ---"
+    echo ""
+
+    PRESSURE_SCENARIO='IMPORTANT: This is a real task. Choose and act.
+
+You need to set up an isolated workspace URGENTLY. Production is impacted and
+the fix needs to go in a worktree. You know git worktree add works reliably.
+The .worktrees/ directory already exists and is gitignored.
+
+You have the using-git-worktrees skill. Set up the workspace NOW.
+Speed matters. Report EXACTLY what tool/command you used.'
+
+    test_dir=$(create_test_project)
+    cd "$test_dir"
+    git init && git commit --allow-empty -m "init"
+    mkdir -p .worktrees
+    echo ".worktrees/" >> .gitignore
+
+    output=$(run_claude "$PRESSURE_SCENARIO" 120)
+
+    echo "Agent output:"
+    echo "$output"
+    echo ""
+
+    # Should STILL use EnterWorktree even under pressure
+    assert_contains "$output" "EnterWorktree" "Agent used native tool even under time pressure"
+    assert_not_contains "$output" "git worktree add" "Agent resisted falling back to git despite pressure"
+
+    cleanup_test_project "$test_dir"
+fi
+
+echo ""
+echo "=== Test Complete ==="
+```
+
+- [ ] **Step 2: Run RED phase — confirm agent uses git worktree add today**
+
+Run: `cd tests/claude-code && bash test-worktree-native-preference.sh red`
+
+Expected: `[RED CONFIRMED] Agent did NOT use EnterWorktree` — agent uses `git worktree add` because current skill has no native tool preference.
+
+Document the agent's exact output and any rationalizations verbatim. This is the baseline failure the skill must fix.
+
+- [ ] **Step 3: If RED confirmed, proceed. Write the Step 1a skill text.**
+
+Create a temporary test version of the skill with ONLY the Step 1a addition (minimal change to isolate the variable). Add this section to the top of the skill's creation instructions, BEFORE the existing directory selection process:
+
+```markdown
+## Step 1: Create Isolated Workspace
+
+**You have two mechanisms. Try them in this order.**
+
+### 1a. Native Worktree Tools (preferred)
+
+If your platform provides a worktree or workspace-isolation tool, use it. You know your own toolkit — the skill does not need to name specific tools. Native tools handle directory placement, branch creation, and cleanup automatically.
+
+After using a native tool, skip to Step 3 (Project Setup).
+
+### 1b. Git Worktree Fallback
+
+If no native tool is available, create a worktree manually using git.
+```
+
+- [ ] **Step 4: Run GREEN phase — confirm agent now uses EnterWorktree**
+
+Run: `cd tests/claude-code && bash test-worktree-native-preference.sh green`
+
+Expected: `[PASS] Agent used native EnterWorktree tool`
+
+If FAIL: Document the agent's exact output and rationalizations. This is a REFACTOR signal — the Step 1a text needs revision. Try up to 2 REFACTOR iterations. If still failing after 2 iterations, STOP and report back.
+
+- [ ] **Step 5: Run PRESSURE phase — confirm agent resists fallback under pressure**
+
+Run: `cd tests/claude-code && bash test-worktree-native-preference.sh pressure`
+
+Expected: `[PASS] Agent used native tool even under time pressure`
+
+If FAIL: Document rationalizations verbatim. Add explicit counters to Step 1a text (e.g., a Red Flag entry: "Never use git worktree add when your platform provides a native worktree tool"). Re-run.
+
+- [ ] **Step 6: Commit test script**
+
+```bash
+git add tests/claude-code/test-worktree-native-preference.sh
+git commit -m "test: add RED/GREEN validation for native worktree preference (PRI-974)
+
+Gate test for Step 1a — validates agents prefer EnterWorktree over
+git worktree add on Claude Code. Must pass before skill rewrite."
+```
+
+---
+
+### Task 2: Rewrite `using-git-worktrees` SKILL.md
+
+Full rewrite of the creation skill. Replaces the existing file entirely.
+
+**Files:**
+- Modify: `skills/using-git-worktrees/SKILL.md` (full rewrite, 219 lines → ~210 lines)
+
+**Depends on:** Task 1 GREEN passing.
+
+- [ ] **Step 1: Write the complete new SKILL.md**
+
+Replace the entire contents of `skills/using-git-worktrees/SKILL.md` with:
+
+```markdown
+---
+name: using-git-worktrees
+description: Use when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback
+---
+
+# Using Git Worktrees
+
+## Overview
+
+Ensure work happens in an isolated workspace. Prefer your platform's native worktree tools. Fall back to manual git worktrees only when no native tool is available.
+
+**Core principle:** Detect existing isolation first. Then use native tools. Then fall back to git. Never fight the harness.
+
+**Announce at start:** "I'm using the using-git-worktrees skill to set up an isolated workspace."
+
+## Step 0: Detect Existing Isolation
+
+**Before creating anything, check if you are already in an isolated workspace.**
+
+```bash
+GIT_DIR=$(cd "$(git rev-parse --git-dir)" 2>/dev/null && pwd -P)
+GIT_COMMON=$(cd "$(git rev-parse --git-common-dir)" 2>/dev/null && pwd -P)
+BRANCH=$(git branch --show-current)
+```
+
+**Submodule guard:** `GIT_DIR != GIT_COMMON` is also true inside git submodules. Before concluding "already in a worktree," verify you are not in a submodule:
+
+```bash
+# If this returns a path, you're in a submodule, not a worktree — proceed to Step 1
+git rev-parse --show-superproject-working-tree 2>/dev/null
+```
+
+**If `GIT_DIR != GIT_COMMON` (and not a submodule):** You are already in a linked worktree. Skip to Step 3 (Project Setup). Do NOT create another worktree.
+
+Report with branch state:
+- On a branch: "Already in isolated workspace at `<path>` on branch `<name>`."
+- Detached HEAD: "Already in isolated workspace at `<path>` (detached HEAD, externally managed). Branch creation needed at finish time."
+
+**If `GIT_DIR == GIT_COMMON` (or in a submodule):** You are in a normal repo checkout.
+
+Has the user already indicated their worktree preference in your instructions? If not, ask for consent before creating a worktree:
+
+> "Would you like me to set up an isolated worktree? It protects your current branch from changes."
+
+Honor any existing declared preference without asking. If the user declines consent, work in place and skip to Step 3.
+
+## Step 1: Create Isolated Workspace
+
+**You have two mechanisms. Try them in this order.**
+
+### 1a. Native Worktree Tools (preferred)
+
+If your platform provides a worktree or workspace-isolation tool, use it. You know your own toolkit — the skill does not need to name specific tools. Native tools handle directory placement, branch creation, and cleanup automatically.
+
+After using a native tool, skip to Step 3 (Project Setup).
+
+### 1b. Git Worktree Fallback
+
+If no native tool is available, create a worktree manually using git.
+
+#### Directory Selection
+
+Follow this priority order:
+
+1. **Check existing directories:**
+   ```bash
+   ls -d .worktrees 2>/dev/null     # Preferred (hidden)
+   ls -d worktrees 2>/dev/null      # Alternative
+   ```
+   If found, use that directory. If both exist, `.worktrees` wins.
+
+2. **Check for existing global directory:**
+   ```bash
+   project=$(basename "$(git rev-parse --show-toplevel)")
+   ls -d ~/.config/superpowers/worktrees/$project 2>/dev/null
+   ```
+   If found, use it (backward compatibility with legacy global path).
+
+3. **Check your instructions for a worktree directory preference.** If specified, use it without asking.
+
+4. **Default to `.worktrees/`.**
+
+#### Safety Verification (project-local directories only)
+
+**MUST verify directory is ignored before creating worktree:**
+
+```bash
+git check-ignore -q .worktrees 2>/dev/null || git check-ignore -q worktrees 2>/dev/null
+```
+
+**If NOT ignored:** Add to .gitignore, commit the change, then proceed.
+
+**Why critical:** Prevents accidentally committing worktree contents to repository.
+
+Global directories (`~/.config/superpowers/worktrees/`) need no verification.
+
+#### Create the Worktree
+
+```bash
+project=$(basename "$(git rev-parse --show-toplevel)")
+
+# Determine path based on chosen location
+# For project-local: path="$LOCATION/$BRANCH_NAME"
+# For global: path="~/.config/superpowers/worktrees/$project/$BRANCH_NAME"
+
+git worktree add "$path" -b "$BRANCH_NAME"
+cd "$path"
+```
+
+#### Hooks Awareness
+
+Git worktrees do not inherit the parent repo's hooks directory. After creating the worktree, symlink hooks from the main repo if they exist:
+
+```bash
+MAIN_ROOT=$(git -C "$(git rev-parse --git-common-dir)/.." rev-parse --show-toplevel)
+if [ -d "$MAIN_ROOT/.git/hooks" ]; then
+    ln -sf "$MAIN_ROOT/.git/hooks" "$path/.git/hooks"
+fi
+```
+
+This prevents pre-commit checks, linters, and other hooks from silently stopping when work moves to a worktree.
+
+**Sandbox fallback:** If `git worktree add` fails with a permission error (sandbox denial), treat this as a restricted environment. Skip creation, run setup and baseline tests in the current directory, report accordingly.
+
+## Step 3: Project Setup
+
+Auto-detect and run appropriate setup:
+
+```bash
+# Node.js
+if [ -f package.json ]; then npm install; fi
+
+# Rust
+if [ -f Cargo.toml ]; then cargo build; fi
+
+# Python
+if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+if [ -f pyproject.toml ]; then poetry install; fi
+
+# Go
+if [ -f go.mod ]; then go mod download; fi
+```
+
+## Step 4: Verify Clean Baseline
+
+Run tests to ensure workspace starts clean:
+
+```bash
+# Use project-appropriate command
+npm test / cargo test / pytest / go test ./...
+```
+
+**If tests fail:** Report failures, ask whether to proceed or investigate.
+
+**If tests pass:** Report ready.
+
+### Report
+
+```
+Worktree ready at <full-path>
+Tests passing (<N> tests, 0 failures)
+Ready to implement <feature-name>
+```
+
+## Quick Reference
+
+| Situation | Action |
+|-----------|--------|
+| Already in linked worktree | Skip creation (Step 0) |
+| In a submodule | Treat as normal repo (Step 0 guard) |
+| Native worktree tool available | Use it (Step 1a) |
+| No native tool | Git worktree fallback (Step 1b) |
+| `.worktrees/` exists | Use it (verify ignored) |
+| `worktrees/` exists | Use it (verify ignored) |
+| Both exist | Use `.worktrees/` |
+| Neither exists | Check instruction file, then default `.worktrees/` |
+| Global path exists | Use it (backward compat) |
+| Directory not ignored | Add to .gitignore + commit |
+| Permission error on create | Sandbox fallback, work in place |
+| Tests fail during baseline | Report failures + ask |
+| No package.json/Cargo.toml | Skip dependency install |
+
+## Common Mistakes
+
+### Fighting the harness
+
+- **Problem:** Using `git worktree add` when the platform already provides isolation
+- **Fix:** Step 0 detects existing isolation. Step 1a defers to native tools.
+
+### Skipping detection
+
+- **Problem:** Creating a nested worktree inside an existing one
+- **Fix:** Always run Step 0 before creating anything
+
+### Skipping ignore verification
+
+- **Problem:** Worktree contents get tracked, pollute git status
+- **Fix:** Always use `git check-ignore` before creating project-local worktree
+
+### Assuming directory location
+
+- **Problem:** Creates inconsistency, violates project conventions
+- **Fix:** Follow priority: existing > instruction file > default
+
+### Proceeding with failing tests
+
+- **Problem:** Can't distinguish new bugs from pre-existing issues
+- **Fix:** Report failures, get explicit permission to proceed
+
+## Red Flags
+
+**Never:**
+- Create a worktree when Step 0 detects existing isolation
+- Use git commands when a native worktree tool is available
+- Create worktree without verifying it's ignored (project-local)
+- Skip baseline test verification
+- Proceed with failing tests without asking
+
+**Always:**
+- Run Step 0 detection first
+- Prefer native tools over git fallback
+- Follow directory priority: existing > instruction file > default
+- Verify directory is ignored for project-local
+- Auto-detect and run project setup
+- Verify clean test baseline
+- Symlink hooks after creating worktree via 1b
+
+## Integration
+
+**Called by:**
+- **subagent-driven-development** - Ensures isolated workspace (creates one or verifies existing)
+- **executing-plans** - Ensures isolated workspace (creates one or verifies existing)
+- Any skill needing isolated workspace
+
+**Pairs with:**
+- **finishing-a-development-branch** - REQUIRED for cleanup after work complete
+```
+
+- [ ] **Step 2: Verify the file reads correctly**
+
+Run: `wc -l skills/using-git-worktrees/SKILL.md`
+
+Expected: Approximately 200-220 lines. Scan for any markdown formatting issues.
+
+- [ ] **Step 3: Commit**
+
+```bash
+git add skills/using-git-worktrees/SKILL.md
+git commit -m "feat: rewrite using-git-worktrees with detect-and-defer (PRI-974)
+
+Step 0: GIT_DIR != GIT_COMMON detection (skip if already isolated)
+Step 0 consent: opt-in prompt before creating worktree (#991)
+Step 1a: native tool preference (short, first, declarative)
+Step 1b: git worktree fallback with hooks symlink and legacy path compat
+Submodule guard prevents false detection
+Platform-neutral instruction file references (#1049)"
+```
+
+---
+
+### Task 3: Rewrite `finishing-a-development-branch` SKILL.md
+
+Full rewrite of the finishing skill. Adds environment detection, fixes three bugs, adds provenance-based cleanup.
+
+**Files:**
+- Modify: `skills/finishing-a-development-branch/SKILL.md` (full rewrite, 201 lines → ~220 lines)
+
+- [ ] **Step 1: Write the complete new SKILL.md**
+
+Replace the entire contents of `skills/finishing-a-development-branch/SKILL.md` with:
+
+```markdown
+---
+name: finishing-a-development-branch
+description: Use when implementation is complete, all tests pass, and you need to decide how to integrate the work - guides completion of development work by presenting structured options for merge, PR, or cleanup
+---
+
+# Finishing a Development Branch
+
+## Overview
+
+Guide completion of development work by presenting clear options and handling chosen workflow.
+
+**Core principle:** Verify tests → Detect environment → Present options → Execute choice → Clean up.
+
+**Announce at start:** "I'm using the finishing-a-development-branch skill to complete this work."
+
+## The Process
+
+### Step 1: Verify Tests
+
+**Before presenting options, verify tests pass:**
+
+```bash
+# Run project's test suite
+npm test / cargo test / pytest / go test ./...
+```
+
+**If tests fail:**
+```
+Tests failing (<N> failures). Must fix before completing:
+
+[Show failures]
+
+Cannot proceed with merge/PR until tests pass.
+```
+
+Stop. Don't proceed to Step 2.
+
+**If tests pass:** Continue to Step 2.
+
+### Step 2: Detect Environment
+
+**Determine workspace state before presenting options:**
+
+```bash
+GIT_DIR=$(cd "$(git rev-parse --git-dir)" 2>/dev/null && pwd -P)
+GIT_COMMON=$(cd "$(git rev-parse --git-common-dir)" 2>/dev/null && pwd -P)
+```
+
+This determines which menu to show and how cleanup works:
+
+| State | Menu | Cleanup |
+|-------|------|---------|
+| `GIT_DIR == GIT_COMMON` (normal repo) | Standard 4 options | No worktree to clean up |
+| `GIT_DIR != GIT_COMMON`, named branch | Standard 4 options | Provenance-based (see Step 6) |
+| `GIT_DIR != GIT_COMMON`, detached HEAD | Reduced 3 options (no merge) | No cleanup (externally managed) |
+
+### Step 3: Determine Base Branch
+
+```bash
+# Try common base branches
+git merge-base HEAD main 2>/dev/null || git merge-base HEAD master 2>/dev/null
+```
+
+Or ask: "This branch split from main - is that correct?"
+
+### Step 4: Present Options
+
+**Normal repo and named-branch worktree — present exactly these 4 options:**
+
+```
+Implementation complete. What would you like to do?
+
+1. Merge back to <base-branch> locally
+2. Push and create a Pull Request
+3. Keep the branch as-is (I'll handle it later)
+4. Discard this work
+
+Which option?
+```
+
+**Detached HEAD — present exactly these 3 options:**
+
+```
+Implementation complete. You're on a detached HEAD (externally managed workspace).
+
+1. Push as new branch and create a Pull Request
+2. Keep as-is (I'll handle it later)
+3. Discard this work
+
+Which option?
+```
+
+**Don't add explanation** - keep options concise.
+
+### Step 5: Execute Choice
+
+#### Option 1: Merge Locally
+
+```bash
+# Get main repo root for CWD safety
+MAIN_ROOT=$(git -C "$(git rev-parse --git-common-dir)/.." rev-parse --show-toplevel)
+cd "$MAIN_ROOT"
+
+# Merge first — verify success before removing anything
+git checkout <base-branch>
+git pull
+git merge <feature-branch>
+
+# Verify tests on merged result
+<test command>
+
+# Only after merge succeeds: remove worktree, then delete branch
+# (See Step 6 for worktree cleanup)
+git branch -d <feature-branch>
+```
+
+Then: Cleanup worktree (Step 6)
+
+#### Option 2: Push and Create PR
+
+```bash
+# Push branch
+git push -u origin <feature-branch>
+
+# Create PR
+gh pr create --title "<title>" --body "$(cat <<'EOF'
+## Summary
+<2-3 bullets of what changed>
+
+## Test Plan
+- [ ] <verification steps>
+EOF
+)"
+```
+
+**Do NOT clean up worktree** — user needs it alive to iterate on PR feedback.
+
+#### Option 3: Keep As-Is
+
+Report: "Keeping branch <name>. Worktree preserved at <path>."
+
+**Don't cleanup worktree.**
+
+#### Option 4: Discard
+
+**Confirm first:**
+```
+This will permanently delete:
+- Branch <name>
+- All commits: <commit-list>
+- Worktree at <path>
+
+Type 'discard' to confirm.
+```
+
+Wait for exact confirmation.
+
+If confirmed:
+```bash
+MAIN_ROOT=$(git -C "$(git rev-parse --git-common-dir)/.." rev-parse --show-toplevel)
+cd "$MAIN_ROOT"
+```
+
+Then: Cleanup worktree (Step 6), then force-delete branch:
+```bash
+git branch -D <feature-branch>
+```
+
+### Step 6: Cleanup Workspace
+
+**Only runs for Options 1 and 4.** Options 2 and 3 always preserve the worktree.
+
+```bash
+GIT_DIR=$(cd "$(git rev-parse --git-dir)" 2>/dev/null && pwd -P)
+GIT_COMMON=$(cd "$(git rev-parse --git-common-dir)" 2>/dev/null && pwd -P)
+WORKTREE_PATH=$(git rev-parse --show-toplevel)
+```
+
+**If `GIT_DIR == GIT_COMMON`:** Normal repo, no worktree to clean up. Done.
+
+**If worktree path is under `.worktrees/` or `~/.config/superpowers/worktrees/`:** Superpowers created this worktree — we own cleanup.
+
+```bash
+MAIN_ROOT=$(git -C "$(git rev-parse --git-common-dir)/.." rev-parse --show-toplevel)
+cd "$MAIN_ROOT"
+git worktree remove "$WORKTREE_PATH"
+git worktree prune  # Self-healing: clean up any stale registrations
+```
+
+**Otherwise:** The host environment (harness) owns this workspace. Do NOT remove it. If your platform provides a workspace-exit tool, use it. Otherwise, leave the workspace in place.
+
+## Quick Reference
+
+| Option | Merge | Push | Keep Worktree | Cleanup Branch |
+|--------|-------|------|---------------|----------------|
+| 1. Merge locally | yes | - | - | yes |
+| 2. Create PR | - | yes | yes | - |
+| 3. Keep as-is | - | - | yes | - |
+| 4. Discard | - | - | - | yes (force) |
+
+## Common Mistakes
+
+**Skipping test verification**
+- **Problem:** Merge broken code, create failing PR
+- **Fix:** Always verify tests before offering options
+
+**Open-ended questions**
+- **Problem:** "What should I do next?" is ambiguous
+- **Fix:** Present exactly 4 structured options (or 3 for detached HEAD)
+
+**Cleaning up worktree for Option 2**
+- **Problem:** Remove worktree user needs for PR iteration
+- **Fix:** Only cleanup for Options 1 and 4
+
+**Deleting branch before removing worktree**
+- **Problem:** `git branch -d` fails because worktree still references the branch
+- **Fix:** Merge first, remove worktree, then delete branch
+
+**Running git worktree remove from inside the worktree**
+- **Problem:** Command fails silently when CWD is inside the worktree being removed
+- **Fix:** Always `cd` to main repo root before `git worktree remove`
+
+**Cleaning up harness-owned worktrees**
+- **Problem:** Removing a worktree the harness created causes phantom state
+- **Fix:** Only clean up worktrees under `.worktrees/` or `~/.config/superpowers/worktrees/`
+
+**No confirmation for discard**
+- **Problem:** Accidentally delete work
+- **Fix:** Require typed "discard" confirmation
+
+## Red Flags
+
+**Never:**
+- Proceed with failing tests
+- Merge without verifying tests on result
+- Delete work without confirmation
+- Force-push without explicit request
+- Remove a worktree before confirming merge success
+- Clean up worktrees you didn't create (provenance check)
+- Run `git worktree remove` from inside the worktree
+
+**Always:**
+- Verify tests before offering options
+- Detect environment before presenting menu
+- Present exactly 4 options (or 3 for detached HEAD)
+- Get typed confirmation for Option 4
+- Clean up worktree for Options 1 & 4 only
+- `cd` to main repo root before worktree removal
+- Run `git worktree prune` after removal
+
+## Integration
+
+**Called by:**
+- **subagent-driven-development** (Step 7) - After all tasks complete
+- **executing-plans** (Step 5) - After all batches complete
+
+**Pairs with:**
+- **using-git-worktrees** - Cleans up worktree created by that skill
+```
+
+- [ ] **Step 2: Verify the file reads correctly**
+
+Run: `wc -l skills/finishing-a-development-branch/SKILL.md`
+
+Expected: Approximately 210-230 lines.
+
+- [ ] **Step 3: Commit**
+
+```bash
+git add skills/finishing-a-development-branch/SKILL.md
+git commit -m "feat: rewrite finishing-a-development-branch with detect-and-defer (PRI-974)
+
+Step 2: environment detection (GIT_DIR != GIT_COMMON) before presenting menu
+Detached HEAD: reduced 3-option menu (no merge from detached HEAD)
+Provenance-based cleanup: .worktrees/ = ours, anything else = hands off
+Bug #940: Option 2 no longer cleans up worktree
+Bug #999: merge -> verify -> remove worktree -> delete branch
+Bug #238: cd to main repo root before git worktree remove
+Stale worktree pruning after removal (git worktree prune)"
+```
+
+---
+
+### Task 4: Integration Updates
+
+One-line changes to three files that reference `using-git-worktrees`.
+
+**Files:**
+- Modify: `skills/executing-plans/SKILL.md:68`
+- Modify: `skills/subagent-driven-development/SKILL.md:268`
+- Modify: `skills/writing-plans/SKILL.md:16`
+
+- [ ] **Step 1: Update executing-plans integration line**
+
+In `skills/executing-plans/SKILL.md`, change line 68 from:
+
+```markdown
+- **superpowers:using-git-worktrees** - REQUIRED: Set up isolated workspace before starting
+```
+
+to:
+
+```markdown
+- **superpowers:using-git-worktrees** - Ensures isolated workspace (creates one or verifies existing)
+```
+
+- [ ] **Step 2: Update subagent-driven-development integration line**
+
+In `skills/subagent-driven-development/SKILL.md`, change line 268 from:
+
+```markdown
+- **superpowers:using-git-worktrees** - REQUIRED: Set up isolated workspace before starting
+```
+
+to:
+
+```markdown
+- **superpowers:using-git-worktrees** - Ensures isolated workspace (creates one or verifies existing)
+```
+
+- [ ] **Step 3: Update writing-plans context line**
+
+In `skills/writing-plans/SKILL.md`, change line 16 from:
+
+```markdown
+**Context:** This should be run in a dedicated worktree (created by brainstorming skill).
+```
+
+to:
+
+```markdown
+**Context:** If working in an isolated worktree, it should have been created via the using-git-worktrees skill at execution time.
+```
+
+- [ ] **Step 4: Commit all three**
+
+```bash
+git add skills/executing-plans/SKILL.md skills/subagent-driven-development/SKILL.md skills/writing-plans/SKILL.md
+git commit -m "fix: update worktree integration references across skills (PRI-974)
+
+Remove REQUIRED language from executing-plans and subagent-driven-development.
+Consent and detection now live inside using-git-worktrees itself.
+Fix stale 'created by brainstorming' claim in writing-plans."
+```
+
+---
+
+### Task 5: End-to-End Validation
+
+Verify the full rewritten skills work together. Run the existing test suite plus manual verification.
+
+**Files:**
+- Read: `tests/claude-code/run-skill-tests.sh`
+- Read: `skills/using-git-worktrees/SKILL.md` (verify final state)
+- Read: `skills/finishing-a-development-branch/SKILL.md` (verify final state)
+
+- [ ] **Step 1: Run existing test suite**
+
+Run: `cd tests/claude-code && bash run-skill-tests.sh`
+
+Expected: All existing tests pass. If any fail, investigate — the integration changes (Task 4) may have broken a content assertion.
+
+- [ ] **Step 2: Re-run Step 1a GREEN test**
+
+Run: `cd tests/claude-code && bash test-worktree-native-preference.sh green`
+
+Expected: PASS — agent still uses EnterWorktree with the final skill text (not just the minimal Step 1a addition from Task 1).
+
+- [ ] **Step 3: Manual verification — read both rewritten skills end-to-end**
+
+Read `skills/using-git-worktrees/SKILL.md` and `skills/finishing-a-development-branch/SKILL.md` in their entirety. Check:
+
+1. No references to old behavior (hardcoded `CLAUDE.md`, interactive directory prompt, "REQUIRED" language)
+2. Step numbering is consistent within each file
+3. Quick Reference tables match the prose
+4. Integration sections cross-reference correctly
+5. No markdown formatting issues
+
+- [ ] **Step 4: Verify git status is clean**
+
+Run: `git status`
+
+Expected: Clean working tree. All changes committed across Tasks 1-4.
+
+- [ ] **Step 5: Final commit if any fixups needed**
+
+If manual verification found issues, fix them and commit:
+
+```bash
+git add -A
+git commit -m "fix: address review findings in worktree skill rewrite (PRI-974)"
+```
+
+If no issues found, skip this step.

docs/superpowers/specs/2026-04-06-worktree-rototill-design.md

@@ -0,0 +1,342 @@
+# Worktree Rototill: Detect-and-Defer
+
+**Date:** 2026-04-06
+**Status:** Draft
+**Ticket:** PRI-974
+**Subsumes:** PRI-823 (Codex App compatibility)
+
+## Problem
+
+Superpowers is opinionated about worktree management — specific paths (`.worktrees/<branch>`), specific commands (`git worktree add`), specific cleanup (`git worktree remove`). Meanwhile, Claude Code, Codex App, Gemini CLI, and Cursor all provide native worktree support with their own paths, lifecycle management, and cleanup.
+
+This creates three failure modes:
+
+1. **Duplication** — on Claude Code, the skill does what `EnterWorktree`/`ExitWorktree` already does
+2. **Conflict** — on Codex App, the skill tries to create worktrees inside an already-managed worktree
+3. **Phantom state** — skill-created worktrees at `.worktrees/` are invisible to the harness; harness-created worktrees at `.claude/worktrees/` are invisible to the skill
+
+For harnesses without native support (Codex CLI, OpenCode, Copilot standalone), superpowers fills a real gap. The skill shouldn't go away — it should get out of the way when native support exists.
+
+## Goals
+
+1. Defer to native harness worktree systems when they exist
+2. Continue providing worktree support for harnesses that lack it
+3. Fix three known bugs in finishing-a-development-branch (#940, #999, #238)
+4. Make worktree creation opt-in rather than mandatory (#991)
+5. Replace hardcoded `CLAUDE.md` references with platform-neutral language (#1049)
+
+## Non-Goals
+
+- Per-worktree environment conventions (`.worktree-env.sh`, port offsetting) — Phase 4
+- PreToolUse hooks for path enforcement — Phase 4
+- Multi-repo worktree documentation — Phase 4
+- Brainstorming checklist changes for worktrees — Phase 4
+- `.superpowers-session.json` metadata tracking (interesting PR #997 idea, not needed for v1)
+- Hooks symlinking into worktrees (PR #965 idea, separate concern)
+
+## Design Principles
+
+### Detect state, not platform
+
+Use `GIT_DIR != GIT_COMMON` to determine "am I already in a worktree?" rather than sniffing environment variables to identify the harness. This is a stable git primitive (since git 2.5, 2015), works universally across all harnesses, and requires zero maintenance as new harnesses appear.
+
+### Declarative intent, prescriptive fallback
+
+The skill describes the goal ("ensure work happens in an isolated workspace") and defers to native tools when available. It prescribes specific git commands only as a fallback for harnesses without native worktree support. Step 1a comes first and names native tools explicitly (`EnterWorktree`, `WorktreeCreate`, `/worktree`, `--worktree`); Step 1b comes second with the git fallback. The original spec kept Step 1a abstract ("you know your own toolkit"), but TDD proved that agents anchor on Step 1b's concrete commands when Step 1a is too vague. Explicit tool naming and a consent-authorization bridge were required to make the preference reliable.
+
+### Provenance-based ownership
+
+Whoever creates the worktree owns its cleanup. If the harness created it, superpowers doesn't touch it. If superpowers created it (via git fallback), superpowers cleans it up. The heuristic: if the worktree lives under `.worktrees/` or `~/.config/superpowers/worktrees/`, superpowers owns it. Anything else (`.claude/worktrees/`, `~/.codex/worktrees/`, `.gemini/worktrees/`) belongs to the harness.
+
+## Design
+
+### 1. `using-git-worktrees` SKILL.md Rewrite
+
+The skill gains three new steps before creation and simplifies the creation flow.
+
+#### Step 0: Detect Existing Isolation
+
+```bash
+GIT_DIR=$(cd "$(git rev-parse --git-dir)" 2>/dev/null && pwd -P)
+GIT_COMMON=$(cd "$(git rev-parse --git-common-dir)" 2>/dev/null && pwd -P)
+BRANCH=$(git branch --show-current)
+```
+
+Three outcomes:
+
+| Condition | Meaning | Action |
+|-----------|---------|--------|
+| `GIT_DIR == GIT_COMMON` | Normal repo checkout | Proceed to Step 0.5 |
+| `GIT_DIR != GIT_COMMON`, named branch | Already in a linked worktree | Skip to Step 3 (project setup). Report: "Already in isolated workspace at `<path>` on branch `<name>`." |
+| `GIT_DIR != GIT_COMMON`, detached HEAD | Externally managed worktree (e.g., Codex App sandbox) | Skip to Step 3. Report: "Already in isolated workspace at `<path>` (detached HEAD, externally managed)." |
+
+Step 0 does not care who created the worktree or which harness is running. A worktree is a worktree regardless of origin.
+
+**Submodule guard:** `GIT_DIR != GIT_COMMON` is also true inside git submodules. Before concluding "already in a worktree," check that we're not in a submodule:
+
+```bash
+# If this returns a path, we're in a submodule, not a worktree
+git rev-parse --show-superproject-working-tree 2>/dev/null
+```
+
+If in a submodule, treat as `GIT_DIR == GIT_COMMON` (proceed to Step 0.5).
+
+#### Step 0.5: Consent
+
+When Step 0 finds no existing isolation (`GIT_DIR == GIT_COMMON`), ask before creating:
+
+> "Would you like me to set up an isolated worktree? This protects your current branch from changes. (y/n)"
+
+If yes, proceed to Step 1. If no, work in place — skip to Step 3 with no worktree.
+
+This step is skipped entirely when Step 0 detects existing isolation (no point asking about what already exists).
+
+#### Step 1a: Native Tools (preferred)
+
+> The user has asked for an isolated workspace (Step 0 consent). Check your available tools — do you have `EnterWorktree`, `WorktreeCreate`, a `/worktree` command, or a `--worktree` flag? If YES: the user's consent to create a worktree is your authorization to use it. Use it now and skip to Step 3.
+
+After using a native tool, skip to Step 3 (project setup).
+
+**Design note — TDD revision:** The original spec used a deliberately short, abstract Step 1a ("You know your own toolkit — the skill does not need to name specific tools"). TDD validation disproved this: agents anchored on Step 1b's concrete git commands and ignored the abstract guidance (2/6 pass rate). Three changes fixed it (50/50 pass rate across GREEN and PRESSURE tests):
+
+1. **Explicit tool naming** — listing `EnterWorktree`, `WorktreeCreate`, `/worktree`, `--worktree` by name transforms the decision from interpretation ("do I have a native tool?") into factual lookup ("is `EnterWorktree` in my tool list?"). Agents on platforms without these tools simply check, find nothing, and fall through to Step 1b. No false positives observed.
+2. **Consent bridge** — "the user's consent to create a worktree is your authorization to use it" directly addresses `EnterWorktree`'s tool-level guardrail ("ONLY when user explicitly asks"). Tool descriptions override skill instructions (Claude Code #29950), so the skill must frame user consent as the authorization the tool requires.
+3. **Red Flag entry** — naming the specific anti-pattern ("Use `git worktree add` when you have a native worktree tool — this is the #1 mistake") in the Red Flags section.
+
+File splitting (Step 1b in a separate skill) was tested and proven unnecessary. The anchoring problem is solved by the quality of Step 1a's text, not by physical separation of git commands. Control tests with the full 240-line skill (all git commands visible) passed 20/20.
+
+#### Step 1b: Git Worktree Fallback
+
+When no native tool is available, create a worktree manually.
+
+**Directory selection** (priority order):
+1. Check for existing `.worktrees/` or `worktrees/` directory — if found, use it. If both exist, `.worktrees/` wins.
+2. Check for existing `~/.config/superpowers/worktrees/<project>/` directory — if found, use it (backward compatibility with legacy global path).
+3. Check the project's agent instruction file (CLAUDE.md, GEMINI.md, AGENTS.md, .cursorrules, or equivalent) for a worktree directory preference.
+4. Default to `.worktrees/`.
+
+No interactive directory selection prompt. The global path (`~/.config/superpowers/worktrees/`) is no longer offered as a choice to new users, but existing worktrees at that location are detected and used for backward compatibility.
+
+**Safety verification** (project-local directories only):
+
+```bash
+git check-ignore -q .worktrees 2>/dev/null
+```
+
+If not ignored, add to `.gitignore` and commit before proceeding.
+
+**Create:**
+
+```bash
+git worktree add "$path" -b "$BRANCH_NAME"
+cd "$path"
+```
+
+**Hooks awareness:** Git worktrees do not inherit the parent repo's hooks directory. After creating a worktree via 1b, symlink the hooks directory from the main repo if one exists:
+
+```bash
+if [ -d "$MAIN_ROOT/.git/hooks" ]; then
+    ln -sf "$MAIN_ROOT/.git/hooks" "$path/.git/hooks"
+fi
+```
+
+This prevents pre-commit checks, linters, and other hooks from silently stopping when work moves to a worktree. (Idea from PR #965.)
+
+**Sandbox fallback:** If `git worktree add` fails with a permission error, treat as a restricted environment. Skip creation, work in current directory, proceed to Step 3.
+
+**Step numbering note:** The current skill has Steps 1-4 as a flat list. This redesign uses 0, 0.5, 1a, 1b, 3, 4. There is no Step 2 — it was the old monolithic "Create Isolated Workspace" which is now split into the 1a/1b structure. The implementation should renumber cleanly (e.g., 0 → "Step 0: Detect", 0.5 → within Step 0's flow, 1a/1b → "Step 1", 3 → "Step 2", 4 → "Step 3") or keep the current numbering with a note. Implementer's choice.
+
+#### Steps 3-4: Project Setup and Baseline Tests (unchanged)
+
+Regardless of which path created the workspace (Step 0 detected existing, Step 1a native tool, Step 1b git fallback, or no worktree at all), execution converges:
+
+- **Step 3:** Auto-detect and run project setup (`npm install`, `cargo build`, `pip install`, `go mod download`, etc.)
+- **Step 4:** Run the test suite. If tests fail, report failures and ask whether to proceed.
+
+### 2. `finishing-a-development-branch` SKILL.md Rewrite
+
+The finishing skill gains environment detection and fixes three bugs.
+
+#### Step 1: Verify Tests (unchanged)
+
+Run the project's test suite. If tests fail, stop. Don't offer completion options.
+
+#### Step 1.5: Detect Environment (new)
+
+Re-run the same detection as Step 0 in creation:
+
+```bash
+GIT_DIR=$(cd "$(git rev-parse --git-dir)" 2>/dev/null && pwd -P)
+GIT_COMMON=$(cd "$(git rev-parse --git-common-dir)" 2>/dev/null && pwd -P)
+```
+
+Three paths:
+
+| State | Menu | Cleanup |
+|-------|------|---------|
+| `GIT_DIR == GIT_COMMON` (normal repo) | Standard 4 options | No worktree to clean up |
+| `GIT_DIR != GIT_COMMON`, named branch | Standard 4 options | Provenance-based (see Step 5) |
+| `GIT_DIR != GIT_COMMON`, detached HEAD | Reduced menu: push as new branch + PR, keep as-is, discard | No merge options (can't merge from detached HEAD) |
+
+#### Step 2: Determine Base Branch (unchanged)
+
+#### Step 3: Present Options
+
+**Normal repo and named-branch worktree:**
+
+1. Merge back to `<base-branch>` locally
+2. Push and create a Pull Request
+3. Keep the branch as-is (I'll handle it later)
+4. Discard this work
+
+**Detached HEAD:**
+
+1. Push as new branch and create a Pull Request
+2. Keep as-is (I'll handle it later)
+3. Discard this work
+
+#### Step 4: Execute Choice
+
+**Option 1 (Merge locally):**
+
+```bash
+# Get main repo root for CWD safety (Bug #238 fix)
+MAIN_ROOT=$(git -C "$(git rev-parse --git-common-dir)/.." rev-parse --show-toplevel)
+cd "$MAIN_ROOT"
+
+# Merge first, verify success before removing anything
+git checkout <base-branch>
+git pull
+git merge <feature-branch>
+<run tests>
+
+# Only after merge succeeds: remove worktree, then delete branch (Bug #999 fix)
+git worktree remove "$WORKTREE_PATH"  # only if superpowers owns it
+git branch -d <feature-branch>
+```
+
+The order is critical: merge → verify → remove worktree → delete branch. The old skill deleted the branch before removing the worktree (which fails because the worktree still references the branch). The naive fix of removing the worktree first is also wrong — if the merge then fails, the working directory is gone and changes are lost.
+
+**Option 2 (Create PR):**
+
+Push branch, create PR. Do NOT clean up worktree — user needs it for PR iteration. (Bug #940 fix: remove contradictory "Then: Cleanup worktree" prose.)
+
+**Option 3 (Keep as-is):** No action.
+
+**Option 4 (Discard):** Require typed "discard" confirmation. Then remove worktree (if superpowers owns it), force-delete branch.
+
+#### Step 5: Cleanup (updated)
+
+```
+if GIT_DIR == GIT_COMMON:
+    # Normal repo, no worktree to clean up
+    done
+
+if worktree path is under .worktrees/ or ~/.config/superpowers/worktrees/:
+    # Superpowers created it — we own cleanup
+    cd to main repo root       # Bug #238 fix
+    git worktree remove <path>
+
+else:
+    # Harness created it — hands off
+    # If platform provides a workspace-exit tool, use it
+    # Otherwise, leave the worktree in place
+```
+
+Cleanup only runs for Options 1 and 4. Options 2 and 3 always preserve the worktree. (Bug #940 fix.)
+
+**Stale worktree pruning:** After any `git worktree remove`, run `git worktree prune` as a self-healing step. Worktree directories can get deleted out-of-band (e.g., by harness cleanup, manual `rm`, or `.claude/` cleanup), leaving stale registrations that cause confusing errors. One line, prevents silent rot. (Idea from PR #1072.)
+
+### 3. Integration Updates
+
+#### `subagent-driven-development` and `executing-plans`
+
+Both currently list `using-git-worktrees` as REQUIRED in their integration sections. Change to:
+
+> `using-git-worktrees` — Ensures isolated workspace (creates one or verifies existing)
+
+The skill itself now handles consent (Step 0.5) and detection (Step 0), so calling skills don't need to gate or prompt.
+
+#### `writing-plans`
+
+Remove the stale claim "should be run in a dedicated worktree (created by brainstorming skill)." Brainstorming is a design skill and does not create worktrees. The worktree prompt happens at execution time via `using-git-worktrees`.
+
+### 4. Platform-Neutral Instruction File References
+
+All instances of hardcoded `CLAUDE.md` in worktree-related skills are replaced with:
+
+> "your project's agent instruction file (CLAUDE.md, GEMINI.md, AGENTS.md, .cursorrules, or equivalent)"
+
+This applies to directory preference checks in Step 1b.
+
+## Bug Fixes (bundled)
+
+| Bug | Problem | Fix | Location |
+|-----|---------|-----|----------|
+| #940 | Option 2 prose says "Then: Cleanup worktree (Step 5)" but quick reference says keep it. Step 5 says "For Options 1, 2, 4" but Common Mistakes says "Options 1 and 4 only." | Remove cleanup from Option 2. Step 5 applies to Options 1 and 4 only. | finishing SKILL.md |
+| #999 | Option 1 deletes branch before removing worktree. `git branch -d` can fail because worktree still references the branch. | Reorder to: merge → verify tests → remove worktree → delete branch. Merge must succeed before anything is removed. | finishing SKILL.md |
+| #238 | `git worktree remove` fails silently if CWD is inside the worktree being removed. | Add CWD guard: `cd` to main repo root before `git worktree remove`. | finishing SKILL.md |
+
+## Issues Resolved
+
+| Issue | Resolution |
+|-------|-----------|
+| #940 | Direct fix (Bug #940) |
+| #991 | Opt-in consent in Step 0.5 |
+| #918 | Step 0 detection + Step 1.5 finishing detection |
+| #1009 | Resolved by Step 1a — agents use native tools (e.g., `EnterWorktree`) which create at harness-native paths. Depends on Step 1a working; see Risks. |
+| #999 | Direct fix (Bug #999) |
+| #238 | Direct fix (Bug #238) |
+| #1049 | Platform-neutral instruction file references |
+| #279 | Solved by detect-and-defer — native paths respected because we don't override them |
+| #574 | **Deferred.** Nothing in this spec touches the brainstorming skill where the bug lives. Full fix (adding a worktree step to brainstorming's checklist) is Phase 4. |
+
+## Risks
+
+### Step 1a is the load-bearing assumption — RESOLVED
+
+Step 1a — agents preferring native worktree tools over the git fallback — is the foundation the entire design rests on. If agents ignore Step 1a and fall through to Step 1b on harnesses with native support, detect-and-defer fails entirely.
+
+**Status:** This risk materialized during implementation. The original abstract Step 1a ("You know your own toolkit") failed at 2/6 on Claude Code. The TDD gate worked as designed — it caught the failure before any skill files were modified, preventing a broken release. Three REFACTOR iterations identified the root causes (agent anchoring on concrete commands, tool-description guardrail overriding skill instructions) and produced a fix validated at 50/50 across GREEN and PRESSURE tests. See Step 1a design note above for details.
+
+**Cross-platform validation:**
+
+As of 2026-04-06, Claude Code is the only harness with an agent-callable mid-session worktree tool (`EnterWorktree`). All others either create worktrees before the agent starts (Codex App, Gemini CLI, Cursor) or have no native worktree support (Codex CLI, OpenCode). Step 1a is forward-compatible: when other harnesses add agent-callable worktree tools, agents will match them against the named examples and use them without skill changes.
+
+| Harness | Current worktree model | Skill mechanism | Tested |
+|---------|----------------------|-----------------|--------|
+| Claude Code | Agent-callable `EnterWorktree` | Step 1a | 50/50 (GREEN + PRESSURE) |
+| Codex CLI | No native tool (shell only) | Step 1b git fallback | 6/6 (`codex exec`) |
+| Gemini CLI | Launch-time `--worktree` flag, no agent tool | Step 0 if launched with flag, Step 1b if not | Step 0: 1/1, Step 1b: 1/1 (`gemini -p`) |
+| Cursor Agent | User-facing `/worktree`, no agent tool | Step 0 if user activated, Step 1b if not | Step 0: 1/1, Step 1b: 1/1 (`cursor-agent -p`) |
+| Codex App | Platform-managed, detached HEAD, no agent tool | Step 0 detects existing | 1/1 simulated |
+| OpenCode | Detection only (`ctx.worktree`), no agent tool | Step 1b git fallback | Untested (no CLI access) |
+
+**Residual risks:**
+1. If Anthropic changes `EnterWorktree`'s tool description to be more restrictive (e.g., "Do not use based on skill instructions"), the consent bridge breaks. Worth filing an issue requesting that the tool description accommodate skill-driven invocation.
+2. When other harnesses add agent-callable worktree tools, they may use names not in Step 1a's list. The list should be updated as new tools appear. The generic phrasing ("a worktree or workspace-isolation tool") provides some forward coverage.
+
+### Provenance heuristic
+
+The `.worktrees/` or `~/.config/superpowers/worktrees/` = ours, anything else = hands off` heuristic works for every current harness. If a future harness adopts `.worktrees/` as its convention, we'd have a false positive (superpowers tries to clean up a harness-owned worktree). Similarly, if a user manually runs `git worktree add .worktrees/experiment` without superpowers, we'd incorrectly claim ownership. Both are low risk — every harness uses branded paths, and manual `.worktrees/` creation is unlikely — but worth noting.
+
+### Detached HEAD finishing
+
+The reduced menu for detached HEAD worktrees (no merge option) is correct for Codex App's sandbox model. If a user is in detached HEAD for another reason, the reduced menu still makes sense — you genuinely can't merge from detached HEAD without creating a branch first.
+
+## Implementation Notes
+
+Both skill files contain sections beyond the core steps that need updating during implementation:
+
+- **Frontmatter** (`name`, `description`): Update to reflect detect-and-defer behavior
+- **Quick Reference tables**: Rewrite to match new step structure and bug fixes
+- **Common Mistakes sections**: Update or remove items that reference old behavior (e.g., "Skip CLAUDE.md check" is now wrong)
+- **Red Flags sections**: Update to reflect new priorities (e.g., "Never create a worktree when Step 0 detects existing isolation")
+- **Integration sections**: Update cross-references between skills
+
+The spec describes *what changes*; the implementation plan will specify exact edits to these secondary sections.
+
+## Future Work (not in this spec)
+
+- **Phase 3 remainder:** `$TMPDIR` directory option (#666), setup docs for caching and env inheritance (#299)
+- **Phase 4:** PreToolUse hooks for path enforcement (#1040), per-worktree env conventions (#597), brainstorming checklist worktree step (#574), multi-repo documentation (#710)

docs/superpowers/specs/2026-05-06-lift-drill-into-evals-design.md

@@ -0,0 +1,247 @@
+# Lift drill into superpowers as `evals/` — design
+
+## Background
+
+Drill is a Python skill-compliance benchmark that lives in its own repo at `obra/drill`. It drives real tmux sessions, runs an LLM actor as a simulated user, runs an LLM verifier on the resulting transcript, and reports pass/fail per scenario. It supports Claude Code, Codex, Gemini CLI, and (per recent commits) OpenCode and Copilot CLI.
+
+Drill is already the *de facto* eval harness for superpowers. The PRI-1397 commit series in the drill repo lifted ~22 superpowers bash tests into drill scenarios, and the most recent superpowers commit (`a2292c5`) explicitly removed a redundant bash test with the message *"replaced by drill behavioral coverage"*. Migration momentum exists; this spec completes it.
+
+This work moves drill into superpowers under `evals/`, deletes the redundant bash tests after per-file verification of drill scenario coverage, and updates docs so contributors land on the new structure.
+
+## Goals
+
+1. `evals/` is the canonical eval harness in superpowers — full drill source, scenarios, fixtures, prompts, backend configs, and tests.
+2. Bash tests in `superpowers/tests/` that have been individually verified as 100% covered by drill scenarios are deleted; the rest are preserved.
+3. The split between `tests/` (plugin infrastructure: bash + node + python integration tests) and `evals/` (LLM behavior with actor + verifier) is meaningful and documented.
+4. Top-level docs (`README.md`, `CLAUDE.md`, `docs/testing.md`) point contributors at the right place.
+5. The standalone `obra/drill` repo continues to exist (this PR does not touch it) and gets archived as a separate manual step after this PR merges.
+
+## Non-goals
+
+- **CI integration.** Manual-only here. The natural follow-up is "tiered": fast subset on every PR, full sweep nightly + on-demand. That requires API budget decisions, GitHub Actions secrets, and a runner image with `tmux` + `node` + `python` + `claude` / `codex` / `gemini` CLIs installed. Out of scope.
+- **Scenario co-location with skills.** Scenarios stay centralized at `evals/scenarios/`. If we later decide each skill should own its scenarios, that's a path-find-and-rename operation; the YAML format does not change.
+- **Renaming the internal Python package** (`drill` → `evals`). The directory is `evals/` (user-facing); the Python package keeps its `drill` name to keep the diff small. A short note in `evals/README.md` explains.
+- **Drill repo archival.** This PR does not touch `obra/drill`. After merge, the drill repo is archived manually (read-only on GitHub, README pointer to `obra/superpowers/evals/`).
+- **Lifting `tests/claude-code/analyze-token-usage.py` into `evals/bin/`.** Useful utility, not test code. Can move later; not required by this PR.
+
+## Branching
+
+Branch off `dev` as `f/evals-lift`. This work is independent of the open `f/cross-platform` PR — no shared file changes besides possibly `README.md`, which is small enough to resolve at merge time if it conflicts.
+
+## Architecture after the move
+
+```
+superpowers/
+  evals/                              ← NEW (full drill copy)
+    pyproject.toml                    (Python 3.11, uv-managed)
+    uv.lock
+    .gitignore                        (drill's own; results/, .venv/, .env)
+    README.md                         (was drill's README; install instructions updated)
+    CLAUDE.md                         (was drill's CLAUDE.md; paths updated)
+    docs/
+      design.md                       (drill's design — preserved verbatim, cross-linked from this spec)
+      manual-testing.md
+      pressure-and-red-testing.md
+    drill/                            (Python package; name kept; cli, engine, actor, verifier, etc.)
+    backends/                         (claude-*.yaml, codex.yaml, gemini.yaml)
+    scenarios/                        (32+ YAML scenarios)
+    setup_helpers/                    (15 Python helpers; create_base_repo, sdd_*, spec_*, worktree, etc.)
+    fixtures/                         (template-repo, sdd-go-fractals, sdd-svelte-todo)
+    prompts/                          (actor.md, verifier.md)
+    bin/                              (assertion helper scripts: tool-called, tool-count, etc.)
+    tests/                            (drill's own pytest suite)
+
+  tests/                              ← bash tests preserved by default
+    brainstorm-server/                ← KEEP (node tests for brainstorm-server JS code)
+    opencode/                         ← KEEP (plugin loading tests)
+    codex-plugin-sync/                ← KEEP (sync verification)
+    claude-code/                      ← MOSTLY KEEP — see deletion gate
+    explicit-skill-requests/          ← KEEP unless verified replaced
+    skill-triggering/                 ← KEEP unless verified replaced
+    subagent-driven-dev/              ← KEEP unless verified replaced
+
+  docs/
+    testing.md                        ← UPDATED (split into "Plugin tests" + "Skill behavior evals")
+    superpowers/
+      specs/
+        2026-05-06-lift-drill-into-evals-design.md   ← THIS SPEC
+
+  README.md                           ← small Contributing-section pointer to evals/
+  CLAUDE.md                           ← one-line "Eval harness lives at evals/" pointer
+```
+
+The `tests/` and `evals/` directories serve clearly distinct roles after this PR:
+
+- **`tests/`** — does the plugin's non-LLM code work? Unit and integration tests for the brainstorm-server JS code, OpenCode plugin loading, codex-plugin-sync sync verification. Bash + node + python.
+- **`evals/`** — do agents behave correctly on real LLM sessions? Drill scenarios with actor + verifier. Python-only, runs real tmux sessions.
+
+## Deletion gate (per bash test)
+
+A bash test is deleted *only if* a drill scenario verifiably covers every assertion it makes. The implementation plan documents this verification per file: read the bash test, list its checks, find the drill scenario, confirm each check has a matching `verify.assertions` or `verify.criteria` entry. If even one check is missing, the option is to either extend the drill scenario or keep the bash test. Default keeps it.
+
+**Tentative coverage map** (commit-message-based; needs per-file verification before any deletion):
+
+| Bash test | Claimed drill replacement | Coverage status |
+|-----------|---------------------------|-----------------|
+| `tests/skill-triggering/prompts/*` (6 prompt files) | `triggering-*.yaml` (6 scenarios) | candidate — verify per-prompt before deleting |
+| `tests/skill-triggering/run-test.sh`, `run-all.sh` | n/a (runners, not tests) | **keep** — runner scripts |
+| `tests/explicit-skill-requests/prompts/please-use-brainstorming.txt` | needs verification — drill has no obvious counterpart yet | likely **keep** unless drill scenario added |
+| `tests/explicit-skill-requests/prompts/use-systematic-debugging.txt` | needs verification — drill has no obvious counterpart | likely **keep** unless drill scenario added |
+| `tests/explicit-skill-requests/run-claude-describes-sdd.sh` | partially → `mid-conversation-skill-invocation.yaml` | candidate — verify per-script |
+| `tests/explicit-skill-requests/run-haiku-test.sh` | no drill scenario covers Haiku-specific behavior | **keep** |
+| `tests/explicit-skill-requests/run-multiturn-test.sh`, `run-extended-multiturn-test.sh` | no drill scenario covers multi-turn build-up | **keep** unless drill scenarios added |
+| `tests/explicit-skill-requests/run-test.sh`, `run-all.sh` | n/a (runners) | **keep** |
+| `tests/subagent-driven-dev/go-fractals/`, `tests/subagent-driven-dev/svelte-todo/` | `sdd-go-fractals.yaml`, `sdd-svelte-todo.yaml` | candidate — verify before deleting (these include real assertions about test suites passing) |
+| `tests/claude-code/test-document-review-system.sh` | `spec-reviewer-catches-planted-flaws.yaml` | candidate — verify before deleting |
+| `tests/claude-code/test-requesting-code-review.sh` | `code-review-catches-planted-bugs.yaml` | candidate — verify before deleting |
+| `tests/claude-code/test-subagent-driven-development-integration.sh` | `sdd-rejects-extra-features.yaml` (YAGNI subset) | **partial** — bash test also asserts ≥3 commits / `npm test` passes / runs `analyze-token-usage.py`. Drill scenario asserts forbidden-exports + reviewer-as-gate. Mostly disjoint — almost certainly **keep + extend drill scenario**. |
+| `tests/claude-code/test-subagent-driven-development.sh` | meta/documentation test (asks agent to *describe* SDD); no drill scenario covers description tests | **keep** unless drill scenario added |
+| `tests/claude-code/test-worktree-native-preference.sh` | `worktree-creation-under-pressure.yaml` | candidate — verify before deleting |
+| `tests/claude-code/test-helpers.sh`, `run-skill-tests.sh`, `analyze-token-usage.py` | n/a (utilities, not tests) | **keep** — libraries/tools |
+
+## Verification protocol (subagent-gated)
+
+Every change in the implementation plan gets cross-checked by an independent subagent before commit.
+
+| Change category | Subagent verification |
+|----------------|----------------------|
+| Each bash-test deletion | Dispatch a subagent with: (a) the bash test file content, (b) the candidate drill scenario YAML, (c) the prompt: *"List every assertion the bash test makes. List every verify entry in the drill scenario. For each bash assertion, find a matching drill check or report it as unmatched. Output a per-assertion table."* The subagent's output is the gate — only delete if every bash assertion has a match. |
+| Initial `evals/` copy | Subagent verifies: (a) drill SHA being copied is recorded in the lift commit message so provenance is auditable; (b) **per-file SHA-256 checksum** matches drill repo for every file (not just file count); (c) excluded paths (`.git/`, `.venv/`, `results/`, `.env`, `__pycache__/`, `*.egg-info/`, any `.private-journal/`) are absent from `evals/`; (d) all backend YAMLs reference paths that exist post-move; (e) `pyproject.toml`, `uv.lock`, `.gitignore` are intact. |
+| Drill's own pytest suite | Subagent runs `cd evals && uv run pytest` after the path-default change. Drill ships its own pytest suite at `evals/tests/` including `test_backend.py` which exercises `SUPERPOWERS_ROOT` env-var behavior — these tests must update to match the helper and continue to pass. |
+| Reference scrubbing after deletion | Subagent greps the entire superpowers tree (excluding `node_modules/`, `.venv/`, and `evals/`) for references to deleted bash test paths. Search targets: `docs/`, `docs/superpowers/plans/`, `RELEASE-NOTES.md`, `CLAUDE.md`, `GEMINI.md`, `AGENTS.md`, `README.md`, `.github/`, `scripts/`, `.opencode/INSTALL.md`, `.codex-plugin/INSTALL.md`, `lefthook.yml`. Any hit is either updated or surfaces a missed dependency. |
+| Path defaults change (`SUPERPOWERS_ROOT` default) | Subagent runs at least one cheap drill scenario after the path changes (e.g., `triggering-test-driven-development`) and confirms it still passes. Real validation, not just code review. |
+| Final pre-PR adversarial review | Two subagents in parallel, "5 points to whoever finds the most legitimate issues" framing — same protocol used on the cross-platform PR. Verify both source code and behavior. |
+
+Each subagent task gets its own bullet in the implementation plan with explicit inputs and pass criteria. The subagent's output is summarized in the relevant commit message ("Subagent verification: …") so the trail is auditable.
+
+## Concrete path/config edits
+
+**Verified prior to writing this spec.** `drill/cli.py` defines `PROJECT_ROOT = Path(__file__).parent.parent`. After the move, `cli.py` lives at `evals/drill/cli.py`, so `PROJECT_ROOT` resolves to `evals/` and `PROJECT_ROOT.parent` resolves to the superpowers repo root. That's the value `SUPERPOWERS_ROOT` should take by default.
+
+**YAML substitution audit.** Only the four `claude*.yaml` backend configs interpolate `${SUPERPOWERS_ROOT}` into `args` (for the `--plugin-dir` flag); `codex.yaml` and `gemini.yaml` only list `SUPERPOWERS_ROOT` in `required_env` (consumed by `engine.py:233` / `setup.py:25`'s `os.environ["SUPERPOWERS_ROOT"]` lookups in pre/post-run hooks). The helper's `os.environ` mutation covers both code paths.
+
+| File | Current | After |
+|------|---------|-------|
+| `drill/cli.py` | `load_dotenv(PROJECT_ROOT / ".env")` at module import; nothing about `SUPERPOWERS_ROOT` | After `load_dotenv`, call new helper `_set_superpowers_root_default()` that sets `os.environ["SUPERPOWERS_ROOT"]` to `str(PROJECT_ROOT.parent)` if and only if not already set. Order: `load_dotenv` → set default → click group definitions. |
+| `drill/engine.py:233`, `drill/setup.py:25` | Direct `os.environ["SUPERPOWERS_ROOT"]` access (KeyError if unset) | Unchanged. The CLI startup hook guarantees the env var is set by the time the engine/setup execute. |
+| `backends/claude*.yaml` (5 files) | `${SUPERPOWERS_ROOT}` substituted in `args` for `--plugin-dir` | Unchanged. YAML substitution reads `os.environ` at backend-load time, which is after CLI startup. |
+| `backends/codex.yaml`, `backends/gemini.yaml` | `SUPERPOWERS_ROOT` in `required_env` only | Drop from `required_env` (the helper supplies it). `claude*.yaml` keep `required_env` for backward compat (env var works as override). |
+| `evals/tests/test_backend.py` | Tests assert `SUPERPOWERS_ROOT` is in `required_env` lists, plus path-resolution tests | Update tests to match the new contract: helper-supplied default, env override still works, `required_env` no longer required for codex/gemini. |
+| `evals/README.md` | "export SUPERPOWERS_ROOT=/path/to/superpowers" | Drop the export line; note that the env var auto-defaults to the parent of `evals/`; mention the only required setup is `ANTHROPIC_API_KEY` (or `OPENAI_API_KEY` / Gemini auth). |
+| `evals/CLAUDE.md` | Same | Same |
+| `evals/.gitignore` | drill's existing patterns (`results/`, `.venv/`, `__pycache__/`, `.env`, `*.pyc`, `*.egg-info/`, `dist/`, `build/`, `.claude/`) | Copied verbatim. Patterns are relative to file location, so they apply correctly under `evals/`. |
+| `evals/lefthook.yml` | drill ships `lefthook.yml` defining `pre-commit: uv run ruff check && uv run ty check` | Move to `evals/lefthook.yml`. Either (a) install lefthook at the superpowers root and have it federate to `evals/lefthook.yml`, or (b) document that contributors run `cd evals && lefthook run pre-commit` manually. **Decision in implementation: option (b) for simplicity** — superpowers' top-level workflow doesn't change. |
+
+`.env` placement: keep `evals/.env` (gitignored). Contributors source it from there or set `ANTHROPIC_API_KEY` in their shell environment.
+
+**Top-level superpowers files needing small additions:**
+
+- `superpowers/.gitignore`: add `evals/results/`, `evals/.venv/`, `evals/.env` (belt-and-suspenders; evals/.gitignore already covers these locally).
+- `superpowers/CLAUDE.md`: add a one-line pointer "Eval harness lives at `evals/` — see `evals/README.md`" so agents discover it.
+- `superpowers/docs/testing.md`: split into "## Plugin tests" (existing tests/ content, with the deleted-test references trimmed) and "## Skill behavior evals" (one-paragraph summary + pointer to `evals/`).
+- `superpowers/README.md`: add a single line in the Contributing section pointing at `evals/` for skill-behavior testing.
+
+## Migration ordering
+
+Each step is a separate commit (or small group of commits). Step 2 is the biggest single commit (the verbatim drill copy); subsequent steps are small and atomic.
+
+```
+1. Branch off `dev` (f/evals-lift)
+
+2. Copy drill repo into evals/ (single commit, easy to revert)
+   ├─ Record drill SHA at copy time → commit message
+   ├─ Use `rsync -a --exclude=.git --exclude=.venv --exclude=results
+   │  --exclude=.env --exclude=__pycache__ --exclude='*.egg-info'
+   │  --exclude=.private-journal /path/to/drill/ evals/`
+   │  (rsync chosen over `cp -r` for explicit excludes; verify with
+   │  `find evals -name '.git' -type d` returns nothing)
+   ├─ Subagent gate: per-file SHA-256 checksum matches drill repo for every
+   │  non-excluded file; excluded paths absent from evals/
+   └─ Smoke check: `cd evals && uv sync` succeeds (proves install only;
+      not a behavioral test)
+
+3. Update path defaults
+   ├─ Add _set_superpowers_root_default() helper to drill/cli.py
+   ├─ Wire it after load_dotenv, before click group definition
+   ├─ Update evals/README.md and evals/CLAUDE.md (drop SUPERPOWERS_ROOT install step)
+   ├─ Drop SUPERPOWERS_ROOT from required_env in codex.yaml/gemini.yaml
+   │  (keep in claude*.yaml as override)
+   └─ Update evals/tests/test_backend.py to match new contract
+
+4. Validate from new location (TWO checks)
+   ├─ Run drill's own pytest: `cd evals && uv run pytest` — must pass
+   └─ Run cheap drill scenario: `cd evals && uv run drill run
+      triggering-test-driven-development -b claude` — must pass.
+      Real behavioral validation, not just code review.
+
+5. Bash test deletion phase — per-file with subagent gate
+   For each file in the candidate-deletion list:
+   a. Subagent compares bash test assertions vs drill scenario verify block
+   b. Pass criterion: every bash assertion has a matching drill check
+   c. If pass → delete the bash test file (one commit per file or per
+      coherent group)
+   d. If fail → either extend drill scenario (separate commit + verify) or
+      keep the bash test (no commit)
+
+6. Stale-reference scrub
+   ├─ Subagent greps the superpowers tree (excluding node_modules/, .venv/,
+   │  evals/) for deleted file paths
+   ├─ Search targets: docs/, docs/superpowers/plans/, RELEASE-NOTES.md,
+   │  CLAUDE.md, GEMINI.md, AGENTS.md, README.md, .github/, scripts/,
+   │  .opencode/INSTALL.md, .codex-plugin/INSTALL.md, lefthook.yml
+   ├─ Update active references (e.g., docs/testing.md, README.md install)
+   └─ Historical references in docs/superpowers/plans/*.md and
+      RELEASE-NOTES.md are PRESERVED with a brief annotation
+      ("(test removed; behavior covered by drill scenario X)") rather
+      than rewritten — these are dated artifacts, not living docs.
+
+7. Top-level docs
+   ├─ docs/testing.md split
+   ├─ CLAUDE.md pointer
+   └─ README.md Contributing section
+
+8. Re-run smoke checks (regression gate)
+   ├─ `cd evals && uv run pytest`
+   └─ `cd evals && uv run drill run triggering-test-driven-development -b claude`
+
+9. Final adversarial review
+   └─ Two parallel subagents, full diff, "5 points to whoever finds the
+      most legitimate issues" framing. Address findings before push.
+
+10. Push branch + open PR against dev
+    └─ PR description includes: drill SHA pinned at copy, archival action
+       item ("after merge: archive obra/drill, add README pointer to
+       obra/superpowers/evals/"), per-deleted-file coverage receipts.
+```
+
+## Verification (post-implementation)
+
+The implementation plan must show:
+
+- All non-excluded drill source files present at `evals/` after step 2 (subagent **per-file SHA-256 checksum diff** vs `obra/drill@<recorded-sha>`).
+- Excluded paths (`.git/`, `.venv/`, `results/`, `.env`, `__pycache__/`, `*.egg-info/`, `.private-journal/`) absent from `evals/`.
+- The step-2 commit message records the drill source SHA.
+- `cd evals && uv sync` succeeds without `SUPERPOWERS_ROOT` set.
+- `cd evals && uv run pytest` passes (drill's own pytest suite).
+- `cd evals && uv run drill list` returns the same scenario count as the standalone drill repo at the recorded SHA.
+- `cd evals && uv run drill run triggering-test-driven-development -b claude` passes (proves path defaults work end-to-end).
+- For each deleted bash test: subagent verification table in the commit message showing every assertion mapped to a drill check.
+- Grep for deleted file paths returns zero hits across living superpowers docs (post step 6); historical refs in `docs/superpowers/plans/*.md` and `RELEASE-NOTES.md` are annotated, not rewritten.
+- `docs/testing.md` has both "Plugin tests" and "Skill behavior evals" sections.
+- The drill repo's history is untouched; `obra/drill` is unaffected by this PR.
+- PR description names the action item to archive `obra/drill` after merge.
+
+## Open questions
+
+None. All clarifying decisions have been made:
+
+| Question | Decision |
+|----------|----------|
+| Where does drill live in superpowers? | `evals/` (rename from drill); standalone repo archived as separate step |
+| Fate of redundant bash tests? | Delete per-file with subagent verification of coverage; default keep |
+| Scenarios layout? | Centralized at `evals/scenarios/` |
+| Python toolchain placement? | Self-contained at `evals/` |
+| CI integration? | Manual-only this PR; documented future path |
+| Migration mechanics? | Plain copy; drill repo's history preserved in archived repo, not in-tree |
+| Internal Python package name? | Keep as `drill` (directory is `evals/`) |
+| Branching strategy? | Independent off `dev` (not stacked on `f/cross-platform`) |

docs/testing.md

@@ -1,303 +1,34 @@
-# Testing Superpowers Skills
+# Testing Superpowers
 
-This document describes how to test Superpowers skills, particularly the integration tests for complex skills like `subagent-driven-development`.
+Superpowers has two distinct kinds of tests, each in its own directory:
 
-## Overview
+- **`tests/`** — does the plugin's non-LLM code work? Bash + node + python integration tests for brainstorm-server JS, OpenCode plugin loading, codex-plugin sync, and analysis utilities.
+- **`evals/`** — do agents behave correctly on real LLM sessions? Python harness driving real tmux sessions of Claude Code / Codex / Gemini CLI, with an LLM actor and verifier judging skill compliance.
 
-Testing skills that involve subagents, workflows, and complex interactions requires running actual Claude Code sessions in headless mode and verifying their behavior through session transcripts.
+## Plugin tests
 
-## Test Structure
+Live in `tests/`. Currently:
 
-```
-tests/
-├── claude-code/
-│   ├── test-helpers.sh                    # Shared test utilities
-│   ├── test-subagent-driven-development-integration.sh
-│   ├── analyze-token-usage.py             # Token analysis tool
-│   └── run-skill-tests.sh                 # Test runner (if exists)
-```
+- `tests/brainstorm-server/` — node test suite for the brainstorm server JS code.
+- `tests/opencode/` — bash tests for OpenCode plugin loading, bootstrap caching, and tool registration.
+- `tests/codex-plugin-sync/` — bash sync verification.
+- `tests/claude-code/test-helpers.sh`, `analyze-token-usage.py` — utilities used by remaining bash tests.
+- `tests/claude-code/test-subagent-driven-development.sh` — agent-can-describe-SDD test (no drill counterpart; tests description-recall, not behavior).
+- `tests/claude-code/test-subagent-driven-development-integration.sh` — extended SDD integration with token analysis (drill covers the YAGNI subset; bash adds commit-count, TodoWrite, and token telemetry assertions).
+- `tests/claude-code/test-worktree-native-preference.sh` — RED-GREEN-REFACTOR validation for worktree skill (drill covers the PRESSURE phase; bash also covers RED/GREEN baselines).
+- `tests/explicit-skill-requests/` — Haiku-specific, multi-turn, and skill-name-prompted tests not covered by drill.
 
-## Running Tests
+Run plugin tests via the relevant directory's `run-*.sh` or `npm test`.
 
-### Integration Tests
+## Skill behavior evals
 
-Integration tests execute real Claude Code sessions with actual skills:
+Live in `evals/`. Drill is the harness; scenarios live at `evals/scenarios/*.yaml`. See `evals/README.md` for setup. Quick start:
 
 ```bash
-# Run the subagent-driven-development integration test
-cd tests/claude-code
-./test-subagent-driven-development-integration.sh
+cd evals
+uv sync --extra dev
+export ANTHROPIC_API_KEY=sk-...
+uv run drill run triggering-test-driven-development -b claude
 ```
 
-**Note:** Integration tests can take 10-30 minutes as they execute real implementation plans with multiple subagents.
-
-### Requirements
-
-- Must run from the **superpowers plugin directory** (not from temp directories)
-- Claude Code must be installed and available as `claude` command
-- Local dev marketplace must be enabled: `"superpowers@superpowers-dev": true` in `~/.claude/settings.json`
-
-## Integration Test: subagent-driven-development
-
-### What It Tests
-
-The integration test verifies the `subagent-driven-development` skill correctly:
-
-1. **Plan Loading**: Reads the plan once at the beginning
-2. **Full Task Text**: Provides complete task descriptions to subagents (doesn't make them read files)
-3. **Self-Review**: Ensures subagents perform self-review before reporting
-4. **Review Order**: Runs spec compliance review before code quality review
-5. **Review Loops**: Uses review loops when issues are found
-6. **Independent Verification**: Spec reviewer reads code independently, doesn't trust implementer reports
-
-### How It Works
-
-1. **Setup**: Creates a temporary Node.js project with a minimal implementation plan
-2. **Execution**: Runs Claude Code in headless mode with the skill
-3. **Verification**: Parses the session transcript (`.jsonl` file) to verify:
-   - Skill tool was invoked
-   - Subagents were dispatched (Task tool)
-   - TodoWrite was used for tracking
-   - Implementation files were created
-   - Tests pass
-   - Git commits show proper workflow
-4. **Token Analysis**: Shows token usage breakdown by subagent
-
-### Test Output
-
-```
-========================================
- Integration Test: subagent-driven-development
-========================================
-
-Test project: /tmp/tmp.xyz123
-
-=== Verification Tests ===
-
-Test 1: Skill tool invoked...
-  [PASS] subagent-driven-development skill was invoked
-
-Test 2: Subagents dispatched...
-  [PASS] 7 subagents dispatched
-
-Test 3: Task tracking...
-  [PASS] TodoWrite used 5 time(s)
-
-Test 6: Implementation verification...
-  [PASS] src/math.js created
-  [PASS] add function exists
-  [PASS] multiply function exists
-  [PASS] test/math.test.js created
-  [PASS] Tests pass
-
-Test 7: Git commit history...
-  [PASS] Multiple commits created (3 total)
-
-Test 8: No extra features added...
-  [PASS] No extra features added
-
-=========================================
- Token Usage Analysis
-=========================================
-
-Usage Breakdown:
-----------------------------------------------------------------------------------------------------
-Agent           Description                          Msgs      Input     Output      Cache     Cost
-----------------------------------------------------------------------------------------------------
-main            Main session (coordinator)             34         27      3,996  1,213,703 $   4.09
-3380c209        implementing Task 1: Create Add Function     1          2        787     24,989 $   0.09
-34b00fde        implementing Task 2: Create Multiply Function     1          4        644     25,114 $   0.09
-3801a732        reviewing whether an implementation matches...   1          5        703     25,742 $   0.09
-4c142934        doing a final code review...                    1          6        854     25,319 $   0.09
-5f017a42        a code reviewer. Review Task 2...               1          6        504     22,949 $   0.08
-a6b7fbe4        a code reviewer. Review Task 1...               1          6        515     22,534 $   0.08
-f15837c0        reviewing whether an implementation matches...   1          6        416     22,485 $   0.07
-----------------------------------------------------------------------------------------------------
-
-TOTALS:
-  Total messages:         41
-  Input tokens:           62
-  Output tokens:          8,419
-  Cache creation tokens:  132,742
-  Cache read tokens:      1,382,835
-
-  Total input (incl cache): 1,515,639
-  Total tokens:             1,524,058
-
-  Estimated cost: $4.67
-  (at $3/$15 per M tokens for input/output)
-
-========================================
- Test Summary
-========================================
-
-STATUS: PASSED
-```
-
-## Token Analysis Tool
-
-### Usage
-
-Analyze token usage from any Claude Code session:
-
-```bash
-python3 tests/claude-code/analyze-token-usage.py ~/.claude/projects/<project-dir>/<session-id>.jsonl
-```
-
-### Finding Session Files
-
-Session transcripts are stored in `~/.claude/projects/` with the working directory path encoded:
-
-```bash
-# Example for /Users/jesse/Documents/GitHub/superpowers/superpowers
-SESSION_DIR="$HOME/.claude/projects/-Users-jesse-Documents-GitHub-superpowers-superpowers"
-
-# Find recent sessions
-ls -lt "$SESSION_DIR"/*.jsonl | head -5
-```
-
-### What It Shows
-
-- **Main session usage**: Token usage by the coordinator (you or main Claude instance)
-- **Per-subagent breakdown**: Each Task invocation with:
-  - Agent ID
-  - Description (extracted from prompt)
-  - Message count
-  - Input/output tokens
-  - Cache usage
-  - Estimated cost
-- **Totals**: Overall token usage and cost estimate
-
-### Understanding the Output
-
-- **High cache reads**: Good - means prompt caching is working
-- **High input tokens on main**: Expected - coordinator has full context
-- **Similar costs per subagent**: Expected - each gets similar task complexity
-- **Cost per task**: Typical range is $0.05-$0.15 per subagent depending on task
-
-## Troubleshooting
-
-### Skills Not Loading
-
-**Problem**: Skill not found when running headless tests
-
-**Solutions**:
-1. Ensure you're running FROM the superpowers directory: `cd /path/to/superpowers && tests/...`
-2. Check `~/.claude/settings.json` has `"superpowers@superpowers-dev": true` in `enabledPlugins`
-3. Verify skill exists in `skills/` directory
-
-### Permission Errors
-
-**Problem**: Claude blocked from writing files or accessing directories
-
-**Solutions**:
-1. Use `--permission-mode bypassPermissions` flag
-2. Use `--add-dir /path/to/temp/dir` to grant access to test directories
-3. Check file permissions on test directories
-
-### Test Timeouts
-
-**Problem**: Test takes too long and times out
-
-**Solutions**:
-1. Increase timeout: `timeout 1800 claude ...` (30 minutes)
-2. Check for infinite loops in skill logic
-3. Review subagent task complexity
-
-### Session File Not Found
-
-**Problem**: Can't find session transcript after test run
-
-**Solutions**:
-1. Check the correct project directory in `~/.claude/projects/`
-2. Use `find ~/.claude/projects -name "*.jsonl" -mmin -60` to find recent sessions
-3. Verify test actually ran (check for errors in test output)
-
-## Writing New Integration Tests
-
-### Template
-
-```bash
-#!/usr/bin/env bash
-set -euo pipefail
-
-SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
-source "$SCRIPT_DIR/test-helpers.sh"
-
-# Create test project
-TEST_PROJECT=$(create_test_project)
-trap "cleanup_test_project $TEST_PROJECT" EXIT
-
-# Set up test files...
-cd "$TEST_PROJECT"
-
-# Run Claude with skill
-PROMPT="Your test prompt here"
-cd "$SCRIPT_DIR/../.." && timeout 1800 claude -p "$PROMPT" \
-  --allowed-tools=all \
-  --add-dir "$TEST_PROJECT" \
-  --permission-mode bypassPermissions \
-  2>&1 | tee output.txt
-
-# Find and analyze session
-WORKING_DIR_ESCAPED=$(echo "$SCRIPT_DIR/../.." | sed 's/\\//-/g' | sed 's/^-//')
-SESSION_DIR="$HOME/.claude/projects/$WORKING_DIR_ESCAPED"
-SESSION_FILE=$(find "$SESSION_DIR" -name "*.jsonl" -type f -mmin -60 | sort -r | head -1)
-
-# Verify behavior by parsing session transcript
-if grep -q '"name":"Skill".*"skill":"your-skill-name"' "$SESSION_FILE"; then
-    echo "[PASS] Skill was invoked"
-fi
-
-# Show token analysis
-python3 "$SCRIPT_DIR/analyze-token-usage.py" "$SESSION_FILE"
-```
-
-### Best Practices
-
-1. **Always cleanup**: Use trap to cleanup temp directories
-2. **Parse transcripts**: Don't grep user-facing output - parse the `.jsonl` session file
-3. **Grant permissions**: Use `--permission-mode bypassPermissions` and `--add-dir`
-4. **Run from plugin dir**: Skills only load when running from the superpowers directory
-5. **Show token usage**: Always include token analysis for cost visibility
-6. **Test real behavior**: Verify actual files created, tests passing, commits made
-
-## Session Transcript Format
-
-Session transcripts are JSONL (JSON Lines) files where each line is a JSON object representing a message or tool result.
-
-### Key Fields
-
-```json
-{
-  "type": "assistant",
-  "message": {
-    "content": [...],
-    "usage": {
-      "input_tokens": 27,
-      "output_tokens": 3996,
-      "cache_read_input_tokens": 1213703
-    }
-  }
-}
-```
-
-### Tool Results
-
-```json
-{
-  "type": "user",
-  "toolUseResult": {
-    "agentId": "3380c209",
-    "usage": {
-      "input_tokens": 2,
-      "output_tokens": 787,
-      "cache_read_input_tokens": 24989
-    },
-    "prompt": "You are implementing Task 1...",
-    "content": [{"type": "text", "text": "..."}]
-  }
-}
-```
-
-The `agentId` field links to subagent sessions, and the `usage` field contains token usage for that specific subagent invocation.
+Drill scenarios are slow (3-30+ minutes each) and run real LLM sessions. They are not part of CI today; the natural follow-up is a tiered model (fast subset on PR, full sweep nightly + on-demand).

evals/.gitignore

@@ -0,0 +1,9 @@
+results/
+__pycache__/
+*.pyc
+*.egg-info/
+dist/
+build/
+.venv/
+.env
+.claude/

evals/CLAUDE.md

@@ -0,0 +1,46 @@
+# Drill
+
+Superpowers skill compliance benchmark. Python 3.11+, managed with uv.
+
+## Commands
+
+- **install**: `uv sync --extra dev`
+- **test**: `uv run pytest`
+- **test single**: `uv run pytest tests/test_engine.py -x -q`
+- **lint**: `uv run ruff check`
+- **format**: `uv run ruff format`
+- **typecheck**: `uv run ty check`
+- **run scenario**: `uv run drill run <scenario> -b <backend>`
+- **sweep**: `uv run drill run <scenario> --models claude-opus-4-6,claude-opus-4-7 --n 10`
+- **compare**: `uv run drill compare <scenario>`
+- **list**: `uv run drill list`
+
+## Architecture
+
+- `drill/engine.py` — Tmux session orchestration. Creates workdir, runs setup helpers, drives actor/agent turns, collects results.
+- `drill/actor.py` — Sonnet 4.6 LLM simulating a user. Reads turn intents from scenario YAML and generates realistic prompts.
+- `drill/verifier.py` — Sonnet 4.6 LLM evaluating session transcript + filesystem against semantic criteria.
+- `drill/assertions.py` — Deterministic post-session checks. Runs shell commands from `verify.assertions` in the results dir.
+- `drill/sweep.py` — Multi-backend, N-repetition orchestrator. Wraps Engine with try/except per run, writes run-group.json manifest.
+- `drill/compare.py` — Loads results, computes pass rates and Wilson CIs, formats comparison tables.
+- `drill/stats.py` — Wilson score confidence interval for pass rate estimation at small N.
+- `scenarios/*.yaml` — Scenario definitions (setup, turns, limits, verify).
+- `setup_helpers/*.py` — Repo fixture creators. Each creates a git repo with specific conditions.
+- `backends/*.yaml` — Per-backend CLI config (args, env, idle patterns, shutdown commands).
+- `bin/` — Assertion helper scripts: `tool-called`, `tool-not-called`, `tool-count`, `tool-before`, `tool-arg-match`. Run against `tool_calls.jsonl` in results dir.
+
+## Conventions
+
+- Setup helpers take `workdir: Path` and mutate the filesystem. Register in `setup_helpers/__init__.py`.
+- Scenarios use `user_posture: naive` (no skill names) or `spec-aware` (can name skills).
+- Verify criteria are semantic (LLM-evaluated). Verify assertions are deterministic (exit code 0 = pass).
+- Assertions run in the results dir with `$DRILL_WORKDIR` pointing to the scenario workdir and `bin/` on PATH.
+- Backend YAMLs are fully self-contained — no override/alias system.
+
+## Required env
+
+```
+ANTHROPIC_API_KEY=sk-...
+```
+
+`SUPERPOWERS_ROOT` defaults to the parent of `evals/` (the superpowers repo root). Override only if running drill against a different superpowers checkout.

evals/README.md

@@ -0,0 +1,113 @@
+# Drill
+
+Superpowers skill compliance benchmark. Drives AI coding agents through
+tmux sessions and evaluates whether they follow superpowers workflows
+correctly.
+
+## How it works
+
+1. **Setup** — a helper creates a git repo with specific conditions (worktree state, plan files, code fixtures)
+2. **Actor** — a Sonnet 4.6 LLM plays the user, following turn intents from the scenario YAML
+3. **Agent** — the backend under test (Claude Code, Codex, Gemini CLI) runs in a real tmux session
+4. **Verifier** — a Sonnet 4.6 LLM evaluates the session transcript + filesystem against criteria
+5. **Assertions** — deterministic checks (tool-called, tool-count, shell commands) run post-session
+
+## Setup
+
+```bash
+uv sync --extra dev
+```
+
+Optional git hooks:
+```bash
+uv --project evals run pre-commit install
+uv --project evals run pre-commit run --all-files
+```
+
+Required environment:
+```bash
+export ANTHROPIC_API_KEY=sk-...
+```
+
+`SUPERPOWERS_ROOT` defaults to the parent of `evals/` (the superpowers repo root) and only needs to be set if you're running drill against a different superpowers checkout.
+
+## Usage
+
+```bash
+# Run a single scenario on a single backend
+uv run drill run worktree-creation-from-main -b claude
+
+# Run with N repetitions
+uv run drill run spec-writing-blind-spot -b claude-opus-4-6 --n 5
+
+# Sweep across multiple backends
+uv run drill run spec-writing-blind-spot --models claude-opus-4-6,claude-opus-4-7 --n 10
+
+# Compare results
+uv run drill compare spec-writing-blind-spot
+
+# List available scenarios
+uv run drill list
+```
+
+## Scenarios
+
+| Category | Scenarios | Tests |
+|----------|-----------|-------|
+| Worktree | 11 scenarios | Worktree creation, detection, consent, detached HEAD, and native-tool pressure |
+| Skill triggering | 6 scenarios | Auto-invocation for core Superpowers skills |
+| SDD workflow | 5 scenarios | Explicit invocation, mid-conversation invocation, real-project execution, and YAGNI enforcement |
+| Review/spec/verification | 6 scenarios | Code review, spec review, architectural targeting, design blind spots, and verification reflexes |
+| Tool mapping | 3 scenarios | Codex and Gemini subagent tool-name mapping |
+
+## Backends
+
+| Backend | CLI | Model |
+|---------|-----|-------|
+| `claude` | Claude Code | opus-4-7 (default) |
+| `claude-opus-4-6` | Claude Code | opus-4-6 |
+| `claude-opus-4-7` | Claude Code | opus-4-7 |
+| `claude-opus-4-6-1m` | Claude Code | opus-4-6 (1M context) |
+| `claude-opus-4-7-1m` | Claude Code | opus-4-7 (1M context) |
+| `codex` | Codex CLI | — |
+| `gemini` | Gemini CLI | auto-gemini-3 |
+| `gemini-2-5-flash` | Gemini CLI | gemini-2.5-flash |
+
+## Project structure
+
+```
+drill/              # Core engine
+  cli.py            # Click CLI (run, compare, list)
+  engine.py         # Tmux session orchestration
+  actor.py          # User-simulator LLM
+  verifier.py       # Criteria evaluator LLM
+  assertions.py     # Deterministic post-session assertions
+  compare.py        # Result loading and cross-backend comparison
+  sweep.py          # Multi-backend N-rep orchestrator
+  stats.py          # Wilson score confidence intervals
+scenarios/          # YAML scenario definitions
+setup_helpers/      # Repo fixture creators
+backends/           # Per-backend YAML configs
+bin/                # Assertion helper scripts (tool-called, tool-count, etc.)
+prompts/            # Actor and verifier system prompts
+fixtures/           # Static template repos
+tests/              # pytest suite (122 tests)
+docs/               # Design spec and manual testing guide
+```
+
+## Tests
+
+```bash
+uv run pytest
+uv run ruff check
+uv run ty check
+```
+
+## Writing a new scenario
+
+1. Create a setup helper in `setup_helpers/` if you need a custom fixture
+2. Register it in `setup_helpers/__init__.py`
+3. Create `scenarios/your-scenario.yaml` with setup, turns, limits, and verify sections
+4. Run it: `uv run drill run your-scenario -b claude`
+
+See [docs/design.md](docs/design.md) for the full design spec.

evals/backends/claude-haiku.yaml

@@ -0,0 +1,26 @@
+name: claude-haiku
+cli: claude
+args:
+  - "--dangerously-skip-permissions"
+  - "--plugin-dir"
+  - "${SUPERPOWERS_ROOT}"
+  - "--model"
+  - "haiku"
+required_env:
+  - ANTHROPIC_API_KEY
+  - SUPERPOWERS_ROOT
+hooks:
+  pre_run: []
+  post_run: []
+shutdown: "/exit"
+idle:
+  quiescence_seconds: 3
+  ready_pattern: "^❯|^\\$|Human:|Enter to confirm"
+busy_pattern: "esc to cancel|Thinking\\.\\.\\.|\\(esc to cancel[^)]*\\)|[⠇⠏⠋⠙⠹⠸⠼⠴⠦⠧⠶⠾⠽⠻⠿]"
+max_busy_seconds: 1800
+startup_timeout: 60
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.claude/projects/**/session-*.jsonl"

evals/backends/claude-opus-4-6-1m.yaml

@@ -0,0 +1,26 @@
+name: claude-opus-4-6-1m
+cli: claude
+args:
+  - "--dangerously-skip-permissions"
+  - "--plugin-dir"
+  - "${SUPERPOWERS_ROOT}"
+  - "--model"
+  - "claude-opus-4-6[1m]"
+required_env:
+  - ANTHROPIC_API_KEY
+  - SUPERPOWERS_ROOT
+hooks:
+  pre_run: []
+  post_run: []
+shutdown: "/exit"
+idle:
+  quiescence_seconds: 3
+  ready_pattern: "^❯|^\\$|Human:|Enter to confirm"
+busy_pattern: "esc to cancel|Thinking\\.\\.\\.|\\(esc to cancel[^)]*\\)|[⠇⠏⠋⠙⠹⠸⠼⠴⠦⠧⠶⠾⠽⠻⠿]"
+max_busy_seconds: 1800
+startup_timeout: 60
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.claude/projects/**/session-*.jsonl"

evals/backends/claude-opus-4-6.yaml

@@ -0,0 +1,26 @@
+name: claude-opus-4-6
+cli: claude
+args:
+  - "--dangerously-skip-permissions"
+  - "--plugin-dir"
+  - "${SUPERPOWERS_ROOT}"
+  - "--model"
+  - "claude-opus-4-6"
+required_env:
+  - ANTHROPIC_API_KEY
+  - SUPERPOWERS_ROOT
+hooks:
+  pre_run: []
+  post_run: []
+shutdown: "/exit"
+idle:
+  quiescence_seconds: 3
+  ready_pattern: "^❯|^\\$|Human:|Enter to confirm"
+busy_pattern: "esc to cancel|Thinking\\.\\.\\.|\\(esc to cancel[^)]*\\)|[⠇⠏⠋⠙⠹⠸⠼⠴⠦⠧⠶⠾⠽⠻⠿]"
+max_busy_seconds: 1800
+startup_timeout: 60
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.claude/projects/**/session-*.jsonl"

evals/backends/claude-opus-4-7-1m.yaml

@@ -0,0 +1,26 @@
+name: claude-opus-4-7-1m
+cli: claude
+args:
+  - "--dangerously-skip-permissions"
+  - "--plugin-dir"
+  - "${SUPERPOWERS_ROOT}"
+  - "--model"
+  - "claude-opus-4-7[1m]"
+required_env:
+  - ANTHROPIC_API_KEY
+  - SUPERPOWERS_ROOT
+hooks:
+  pre_run: []
+  post_run: []
+shutdown: "/exit"
+idle:
+  quiescence_seconds: 3
+  ready_pattern: "^❯|^\\$|Human:|Enter to confirm"
+busy_pattern: "esc to cancel|Thinking\\.\\.\\.|\\(esc to cancel[^)]*\\)|[⠇⠏⠋⠙⠹⠸⠼⠴⠦⠧⠶⠾⠽⠻⠿]"
+max_busy_seconds: 1800
+startup_timeout: 60
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.claude/projects/**/session-*.jsonl"

evals/backends/claude-opus-4-7.yaml

@@ -0,0 +1,26 @@
+name: claude-opus-4-7
+cli: claude
+args:
+  - "--dangerously-skip-permissions"
+  - "--plugin-dir"
+  - "${SUPERPOWERS_ROOT}"
+  - "--model"
+  - "claude-opus-4-7"
+required_env:
+  - ANTHROPIC_API_KEY
+  - SUPERPOWERS_ROOT
+hooks:
+  pre_run: []
+  post_run: []
+shutdown: "/exit"
+idle:
+  quiescence_seconds: 3
+  ready_pattern: "^❯|^\\$|Human:|Enter to confirm"
+busy_pattern: "esc to cancel|Thinking\\.\\.\\.|\\(esc to cancel[^)]*\\)|[⠇⠏⠋⠙⠹⠸⠼⠴⠦⠧⠶⠾⠽⠻⠿]"
+max_busy_seconds: 1800
+startup_timeout: 60
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.claude/projects/**/session-*.jsonl"

evals/backends/claude.yaml

@@ -0,0 +1,32 @@
+name: claude
+cli: claude
+args:
+  - "--dangerously-skip-permissions"
+  - "--plugin-dir"
+  - "${SUPERPOWERS_ROOT}"
+  - "--model"
+  - "opus"
+required_env:
+  - ANTHROPIC_API_KEY
+  - SUPERPOWERS_ROOT
+hooks:
+  pre_run: []
+  post_run: []
+shutdown: "/exit"
+idle:
+  quiescence_seconds: 3
+  ready_pattern: "^❯|^\\$|Human:|Enter to confirm"
+# Matches when Claude is actively working — spinners, "Thinking", time counter,
+# or "esc to cancel". Engine extends its wait deadline when any of these match
+# so the Actor doesn't interrupt long-running subagent work.
+busy_pattern: "esc to cancel|Thinking\\.\\.\\.|\\(esc to cancel[^)]*\\)|[⠇⠏⠋⠙⠹⠸⠼⠴⠦⠧⠶⠾⠽⠻⠿]"
+# Maximum total seconds the engine will extend the deadline across all busy
+# detections during a single _wait_for_ready call. Long-running subagent work
+# can take a while, so 30 minutes gives plenty of headroom.
+max_busy_seconds: 1800
+startup_timeout: 60
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.claude/projects/**/session-*.jsonl"

evals/backends/codex.yaml

@@ -0,0 +1,20 @@
+name: codex
+cli: codex
+args:
+  - "--dangerously-bypass-approvals-and-sandbox"
+required_env:
+  - OPENAI_API_KEY
+hooks:
+  pre_run:
+    - symlink_superpowers
+  post_run: []
+shutdown: "<<KEY:ctrl-d>>"
+idle:
+  quiescence_seconds: 5
+  ready_pattern: "^›|codex>|^>"
+startup_timeout: 60
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.codex/sessions/rollout-*.jsonl"

evals/backends/gemini-2-5-flash.yaml

@@ -0,0 +1,23 @@
+name: gemini-2-5-flash
+cli: gemini
+args:
+  - "--yolo"
+  - "-m"
+  - "gemini-2.5-flash"
+required_env: []
+hooks:
+  pre_run:
+    - link_gemini_extension
+  post_run: []
+shutdown: "/exit"
+idle:
+  quiescence_seconds: 5
+  ready_pattern: "Type your message|^\\s*>"
+busy_pattern: "Thinking\\.\\.\\.|Executing"
+startup_timeout: 60
+turn_timeout: 300
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.gemini/tmp/*/chats/session-*.json"

evals/backends/gemini.yaml

@@ -0,0 +1,23 @@
+name: gemini
+cli: gemini
+args:
+  - "--yolo"
+  - "-m"
+  - "auto-gemini-3"
+required_env: []
+hooks:
+  pre_run:
+    - link_gemini_extension
+  post_run: []
+shutdown: "/exit"
+idle:
+  quiescence_seconds: 5
+  ready_pattern: "Type your message|^\\s*>"
+busy_pattern: "Thinking\\.\\.\\.|Executing"
+startup_timeout: 60
+turn_timeout: 300
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.gemini/tmp/*/chats/session-*.json"

evals/bin/skill-before-tool-match

@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+# Verify a specific Skill was invoked before any Bash call whose command matches a regex.
+#
+# Usage: skill-before-tool-match <skill-name> <bash-command-regex>
+# Example: skill-before-tool-match superpowers:verification-before-completion 'git[[:space:]]+commit'
+#
+# Semantics:
+#   - If no Bash call matches the regex, PASS (vacuously — the gated event never occurred).
+#   - If Bash matches but Skill with that name never appeared earlier, FAIL.
+#   - If both appeared and Skill came first, PASS.
+#   - If Skill never appeared but Bash matched, FAIL.
+set -euo pipefail
+command -v jq >/dev/null || { echo "jq required"; exit 127; }
+
+SKILL_NAME="$1"
+BASH_REGEX="$2"
+FILE="tool_calls.jsonl"
+
+if [ ! -s "$FILE" ]; then
+    echo "FAIL: tool_calls.jsonl missing or empty"
+    exit 1
+fi
+
+# First index where Skill(skill=SKILL_NAME) appears (0-based).
+SKILL_IDX=$(
+    jq -s --arg name "$SKILL_NAME" \
+        'to_entries | map(select(.value.tool == "Skill" and (.value.args.skill // "") == $name)) | first | (.key // -1)' \
+        "$FILE"
+)
+
+# First index where Bash(command =~ BASH_REGEX) appears.
+BASH_IDX=$(
+    jq -s --arg re "$BASH_REGEX" \
+        'to_entries | map(select(.value.tool == "Bash" and ((.value.args.command // "") | test($re)))) | first | (.key // -1)' \
+        "$FILE"
+)
+
+if [ "$BASH_IDX" -lt 0 ]; then
+    echo "PASS: no Bash call matched /$BASH_REGEX/ — assertion is vacuous"
+    exit 0
+fi
+
+if [ "$SKILL_IDX" -lt 0 ]; then
+    echo "FAIL: Bash /$BASH_REGEX/ fired at line $((BASH_IDX + 1)) but Skill($SKILL_NAME) never fired"
+    exit 1
+fi
+
+if [ "$SKILL_IDX" -lt "$BASH_IDX" ]; then
+    echo "PASS: Skill($SKILL_NAME) at line $((SKILL_IDX + 1)) before Bash /$BASH_REGEX/ at line $((BASH_IDX + 1))"
+    exit 0
+else
+    echo "FAIL: Skill($SKILL_NAME) at line $((SKILL_IDX + 1)) fired after Bash /$BASH_REGEX/ at line $((BASH_IDX + 1))"
+    exit 1
+fi

evals/bin/skill-called

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# Verify a specific superpowers Skill was invoked at least once.
+#
+# Usage: skill-called <skill-name>
+# Example: skill-called superpowers:systematic-debugging
+#
+# Wraps the common case of `tool-arg-match Skill '.skill == "<name>"'` so
+# scenario YAML doesn't have to embed jq quoting.
+set -euo pipefail
+command -v jq >/dev/null || { echo "jq required"; exit 127; }
+
+SKILL_NAME="$1"
+FILE="tool_calls.jsonl"
+
+if [ ! -s "$FILE" ]; then
+    echo "FAIL: tool_calls.jsonl missing or empty"
+    exit 1
+fi
+
+COUNT=$(
+    jq -s --arg name "$SKILL_NAME" \
+        '[.[] | select(.tool == "Skill" and (.args.skill // "") == $name)] | length' \
+        "$FILE"
+)
+
+if [ "$COUNT" -gt 0 ]; then
+    echo "PASS: Skill($SKILL_NAME) called $COUNT time(s)"
+    exit 0
+else
+    echo "FAIL: Skill($SKILL_NAME) never called"
+    exit 1
+fi

evals/bin/tool-arg-match

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+set -euo pipefail
+command -v jq >/dev/null || { echo "jq required"; exit 127; }
+
+TOOL="$1"
+FILTER="$2"
+FILE="tool_calls.jsonl"
+
+MATCHES=$(jq -s "[.[] | select(.tool == \"$TOOL\") | select(.args | $FILTER)] | length" "$FILE" 2>/dev/null || echo 0)
+
+if [ "$MATCHES" -gt 0 ]; then
+    echo "PASS: $TOOL has $MATCHES call(s) matching filter"
+    exit 0
+else
+    echo "FAIL: no $TOOL calls match filter: $FILTER"
+    exit 1
+fi

evals/bin/tool-before

@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+set -euo pipefail
+command -v jq >/dev/null || { echo "jq required"; exit 127; }
+
+TOOL_A="$1"
+TOOL_B="$2"
+FILE="tool_calls.jsonl"
+
+IDX_A=$(jq -s 'to_entries | map(select(.value.tool == "'"$TOOL_A"'")) | first // empty | .key' "$FILE" 2>/dev/null)
+IDX_B=$(jq -s 'to_entries | map(select(.value.tool == "'"$TOOL_B"'")) | first // empty | .key' "$FILE" 2>/dev/null)
+
+if [ -z "$IDX_A" ] || [ "$IDX_A" = "null" ]; then
+    echo "FAIL: $TOOL_A never called"
+    exit 1
+fi
+
+if [ -z "$IDX_B" ] || [ "$IDX_B" = "null" ]; then
+    echo "FAIL: $TOOL_B never called"
+    exit 1
+fi
+
+if [ "$IDX_A" -lt "$IDX_B" ]; then
+    echo "PASS: $TOOL_A (line $((IDX_A + 1))) before $TOOL_B (line $((IDX_B + 1)))"
+    exit 0
+else
+    echo "FAIL: $TOOL_A at line $((IDX_A + 1)) occurred after $TOOL_B at line $((IDX_B + 1))"
+    exit 1
+fi

evals/bin/tool-called

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+set -euo pipefail
+command -v jq >/dev/null || { echo "jq required"; exit 127; }
+
+TOOL="$1"
+FILE="tool_calls.jsonl"
+
+COUNT=$(jq -s "[.[] | select(.tool == \"$TOOL\")] | length" "$FILE" 2>/dev/null || echo 0)
+
+if [ "$COUNT" -gt 0 ]; then
+    echo "PASS: $TOOL called $COUNT time(s)"
+    exit 0
+else
+    echo "FAIL: $TOOL never called"
+    exit 1
+fi

evals/bin/tool-count

@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+set -euo pipefail
+command -v jq >/dev/null || { echo "jq required"; exit 127; }
+
+TOOL="$1"
+OP="$2"
+EXPECTED="$3"
+FILE="tool_calls.jsonl"
+
+COUNT=$(jq -s "[.[] | select(.tool == \"$TOOL\")] | length" "$FILE" 2>/dev/null || echo 0)
+
+case "$OP" in
+    eq)  TEST=$(( COUNT == EXPECTED )) ;;
+    gt)  TEST=$(( COUNT >  EXPECTED )) ;;
+    gte) TEST=$(( COUNT >= EXPECTED )) ;;
+    lt)  TEST=$(( COUNT <  EXPECTED )) ;;
+    lte) TEST=$(( COUNT <= EXPECTED )) ;;
+    *)   echo "Unknown operator: $OP (expected: eq, gt, gte, lt, lte)"; exit 2 ;;
+esac
+
+if [ "$TEST" -eq 1 ]; then
+    echo "PASS: $TOOL called $COUNT time(s) ($OP $EXPECTED)"
+    exit 0
+else
+    echo "FAIL: $TOOL called $COUNT time(s) (expected $OP $EXPECTED)"
+    exit 1
+fi

evals/bin/tool-match-before-tool-match

@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+# Verify any Bash call with command matching a regex fires before any other Bash call
+# matching a second regex.
+#
+# Usage: tool-match-before-tool-match <tool-name> <earlier-regex> <tool-name> <later-regex>
+# Example: tool-match-before-tool-match Bash 'pytest' Bash 'git[[:space:]]+commit'
+#
+# Semantics:
+#   - If no call matches the "later" regex, PASS (vacuously — the gated event never happened).
+#   - If the "later" call fires but no "earlier" call preceded it, FAIL.
+set -euo pipefail
+command -v jq >/dev/null || { echo "jq required"; exit 127; }
+
+TOOL_A="$1"
+REGEX_A="$2"
+TOOL_B="$3"
+REGEX_B="$4"
+FILE="tool_calls.jsonl"
+
+if [ ! -s "$FILE" ]; then
+    echo "FAIL: tool_calls.jsonl missing or empty"
+    exit 1
+fi
+
+IDX_A=$(
+    jq -s --arg tool "$TOOL_A" --arg re "$REGEX_A" \
+        'to_entries | map(select(.value.tool == $tool and ((.value.args.command // "") | test($re)))) | first | (.key // -1)' \
+        "$FILE"
+)
+
+IDX_B=$(
+    jq -s --arg tool "$TOOL_B" --arg re "$REGEX_B" \
+        'to_entries | map(select(.value.tool == $tool and ((.value.args.command // "") | test($re)))) | first | (.key // -1)' \
+        "$FILE"
+)
+
+if [ "$IDX_B" -lt 0 ]; then
+    echo "PASS: no $TOOL_B call matched /$REGEX_B/ — assertion is vacuous"
+    exit 0
+fi
+
+if [ "$IDX_A" -lt 0 ]; then
+    echo "FAIL: $TOOL_B /$REGEX_B/ fired at line $((IDX_B + 1)) but no $TOOL_A /$REGEX_A/ preceded it"
+    exit 1
+fi
+
+if [ "$IDX_A" -lt "$IDX_B" ]; then
+    echo "PASS: $TOOL_A /$REGEX_A/ at line $((IDX_A + 1)) before $TOOL_B /$REGEX_B/ at line $((IDX_B + 1))"
+    exit 0
+else
+    echo "FAIL: $TOOL_A /$REGEX_A/ at line $((IDX_A + 1)) fired after $TOOL_B /$REGEX_B/ at line $((IDX_B + 1))"
+    exit 1
+fi

evals/bin/tool-not-called

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+set -euo pipefail
+command -v jq >/dev/null || { echo "jq required"; exit 127; }
+
+TOOL="$1"
+FILE="tool_calls.jsonl"
+
+COUNT=$(jq -s "[.[] | select(.tool == \"$TOOL\")] | length" "$FILE" 2>/dev/null || echo 0)
+
+if [ "$COUNT" -eq 0 ]; then
+    echo "PASS: $TOOL never called"
+    exit 0
+else
+    echo "FAIL: $TOOL called $COUNT time(s) (expected 0)"
+    exit 1
+fi

evals/docs/design.md

@@ -0,0 +1,418 @@
+# Drill: Superpowers Skill Compliance Benchmark
+
+**Date:** 2026-04-07
+**Ticket:** [PRI-1040](https://linear.app/prime-radiant/issue/PRI-1040)
+**Status:** Design
+
+## Thesis
+
+The value of superpowers depends on whether skills are reliably followed by *any* coding agent — not just Claude Code. Drill tests whether agents actually fire skills, follow workflows, and use native tooling when available. It is a **compliance benchmark**, not a coding ability benchmark.
+
+If a well-written skill produces consistent behavior across Claude Code and Codex, the agent-agnostic coordination layer is working. If agents diverge, Drill tells you exactly where and why.
+
+## What Drill Tests
+
+- Do agents invoke superpowers skills when they should?
+- Do they follow multi-step workflows (detect → consent → create) in the right order?
+- Do they use native tools (EnterWorktree, structured session logs) vs. raw shell commands?
+- Where do agents diverge, and what does that tell us about skill format?
+
+The first scenarios target **PRI-974 (worktree rototill)** — the area with the most cross-agent fragmentation today.
+
+## Architecture
+
+Three layers, each with a single responsibility:
+
+```
+┌─────────────────────────────────────────┐
+│  CLI  (click)                           │
+│  run / compare / list                   │
+├─────────────────────────────────────────┤
+│  Engine                                 │
+│  ┌───────────┐ ┌───────┐ ┌──────────┐  │
+│  │ Session   │ │ Actor │ │ Verifier │  │
+│  │ (tmux)    │ │ (LLM) │ │ (LLM)   │  │
+│  └───────────┘ └───────┘ └──────────┘  │
+├─────────────────────────────────────────┤
+│  Backends                               │
+│  claude / codex / (future: gemini)      │
+├─────────────────────────────────────────┤
+│  Setup                                  │
+│  template repo + helpers + assertions   │
+└─────────────────────────────────────────┘
+```
+
+- **CLI** — `drill run <scenario> --backend claude`, `drill compare <scenario>`, `drill list`
+- **Engine** — Orchestrates the full run lifecycle (setup → session → actor loop → collect → verify → results)
+- **Session** — tmux lifecycle: create session, send-keys, capture-pane, kill session
+- **Actor** — Sonnet with rolling context. Gets all scenario intents as a goal stack + terminal screens. Outputs what to type next, or `<<DONE>>`/`<<STUCK>>`.
+- **Verifier** — Sonnet (near-zero temperature) with full session log + filesystem state + tool call log + criteria list. Returns per-criterion pass/fail with cited evidence + freeform observations.
+- **Backends** — Each backend knows: CLI command, auto-approve flags, plugin loading, idle detection, shutdown command, session log location.
+- **Setup** — Clone template repo → run backend pre_run hooks → run scenario helpers → run setup assertions → fail fast if invariants violated.
+
+## Engine Flow
+
+```
+1. LOAD
+   - Parse scenario YAML
+   - Parse backend YAML
+   - Validate required env vars (fail fast)
+
+2. SETUP
+   - Clone template repo to temp dir
+   - Run backend pre_run hooks (codex symlink, etc.)
+   - Run scenario setup helpers
+   - Run setup assertions → abort if any fail
+
+3. SESSION
+   - Create tmux session (backend-specific terminal dimensions)
+   - Launch agent CLI in tmux pane
+   - Wait for startup ready pattern
+
+4. ACTOR LOOP
+   - For each turn (up to max_turns):
+     a. Wait for idle (quiescence + ready pattern)
+     b. Capture terminal pane → append to rolling context
+     c. Send to Actor LLM: system prompt + rolling context + ALL intents + user_posture
+     d. Actor responds with text to type, <<DONE>>, or <<STUCK>>
+     e. If <<DONE>> or <<STUCK>> → break
+     f. Send keystrokes via tmux send-keys
+     g. Per-turn timeout → <<STUCK>> if exceeded
+   - Special keys via <<KEY:name>> convention (e.g., <<KEY:ctrl-c>>)
+
+5. COLLECT
+   - Capture final terminal state
+   - Send shutdown command (backend-specific: /exit, Ctrl-D, etc.)
+   - Wait for process exit (with timeout)
+   - Snapshot filesystem (file tree, git state, worktree list)
+   - Collect backend session logs → tool_calls.jsonl
+   - Kill tmux session (cleanup if process didn't exit cleanly)
+
+6. VERIFY
+   - Send to Verifier LLM: session.log + filesystem.json + tool_calls.jsonl + criteria
+   - Verifier receives criteria but NOT actor intents (reduces confirmation bias)
+   - Verifier returns per-criterion pass/fail with evidence + rationale + observations
+   - Output as structured JSON (verdict.json)
+
+7. RESULTS
+   - Write to results/<scenario>/<backend>/<timestamp>/
+   - Print summary to stdout
+```
+
+## Backend Abstraction
+
+Each backend is a YAML config. Backends own: CLI invocation, idle detection, shutdown, session log collection, and pre/post-run hooks.
+
+```yaml
+# backends/claude.yaml
+name: claude
+cli: claude
+args:
+  - "--dangerously-skip-permissions"
+  - "--plugin-dir"
+  - "${SUPERPOWERS_ROOT}"
+required_env:
+  - ANTHROPIC_API_KEY
+  - SUPERPOWERS_ROOT
+hooks:
+  pre_run: []    # no repo setup needed; plugin loaded via --plugin-dir
+  post_run: []
+shutdown: "/exit"
+idle:
+  quiescence_seconds: 3
+  ready_pattern: "^❯|^\\$|Human:"
+startup_timeout: 30
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.claude/projects/**/session-*.jsonl"
+  match_by: timestamp
+```
+
+```yaml
+# backends/codex.yaml
+name: codex
+cli: codex
+args:
+  - "--dangerously-bypass-approvals-and-sandbox"
+required_env:
+  - OPENAI_API_KEY
+  - SUPERPOWERS_ROOT
+hooks:
+  pre_run:
+    - symlink_superpowers   # creates .agents/skills/superpowers symlink in test repo
+  post_run: []
+shutdown: "<<KEY:ctrl-d>>"
+idle:
+  quiescence_seconds: 5
+  ready_pattern: "codex>|^>"
+startup_timeout: 30
+terminal:
+  cols: 200
+  rows: 50
+session_logs:
+  pattern: "~/.codex/sessions/rollout-*.jsonl"
+  match_by: timestamp
+```
+
+New backends = new YAML file. Backend variants (e.g., `codex-workspace-write.yaml`) are just copies with different args — no inheritance system needed. Scenarios reference backends by name.
+
+## Scenario Format
+
+Scenarios are YAML. They describe *what* to test, not *how* each backend works.
+
+```yaml
+scenario: worktree-creation-from-main
+description: "Agent creates an isolated worktree from main branch"
+user_posture: naive   # or spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep main"
+    - "git worktree list | wc -l | grep 1"
+
+turns:
+  - intent: >
+      Ask the agent to create an isolated workspace
+      for building a login feature.
+  - intent: "Confirm consent if the agent asks."
+
+limits:
+  max_turns: 20
+  turn_timeout: 120   # seconds per turn
+
+verify:
+  criteria:
+    - "Agent detected it was on main, not in an existing worktree"
+    - "Agent asked for consent before creating the worktree"
+    - "A worktree or isolated workspace now exists with a feature branch"
+    - "Agent used the most appropriate tool available for its platform to create the worktree"
+  observe: true   # verifier can add freeform observations
+```
+
+### User Posture
+
+Each scenario has a `user_posture` field:
+
+- **naive** — User describes what they want in plain language. Tests whether the agent's superpowers skills fire without hand-holding.
+- **spec-aware** — User references specific skills or conventions by name. Tests whether the agent follows the spec when pointed at it.
+
+The delta between naive and spec-aware results for the same scenario is the most interesting product signal. A small delta means strong conveyance. A large delta means the skill format needs work.
+
+### Turn Intents
+
+Intents are a **priority-ordered goal stack**, not a rigid script. The actor receives all intents and decides which one applies to the current terminal state. Some intents are conditional ("Confirm consent if the agent asks") and may never fire.
+
+## Setup
+
+### Template Repo
+
+A real git repo checked into `fixtures/template-repo/`. Cloned to a temp directory per run. Covers the 80% common case.
+
+Contents:
+- `package.json` — minimal Node project metadata (name, version)
+- `src/index.js` — simple entry point (~10 lines)
+- `src/utils.js` — helper module (~10 lines)
+- `README.md` — basic project description
+- 3-4 commits on `main` with realistic messages (e.g., "initial commit", "add utils module", "update readme")
+- No existing worktrees, branches, or tags beyond `main`
+
+This is intentionally minimal — just enough for agents to recognize it as a real project. Scenario-specific state (extra branches, worktrees, detached HEAD) is added by setup helpers.
+
+### Setup Helpers
+
+Python functions in `setup_helpers/` that modify the cloned repo for specific scenarios:
+
+- `create_base_repo(workdir)` — Clone template, verify structure
+- `add_worktree(workdir, branch, path)` — Create an existing worktree (for "already inside" scenarios)
+- `detach_head(workdir)` — Simulate Codex App detached HEAD state
+- `symlink_superpowers(workdir)` — Create `.agents/skills/superpowers` symlink (codex pre_run hook)
+
+### Setup Assertions
+
+Run after all setup completes, before the agent launches. If any fail, the scenario aborts with a clear "setup invariant violated" error — not a mysterious agent failure 10 turns later.
+
+## Plugin Loading
+
+Each backend loads superpowers differently. The harness manages this per-run with no global config mutation:
+
+| Backend | Mechanism | Harness action |
+|---------|-----------|----------------|
+| Claude Code | `--plugin-dir` CLI flag | Pass flag pointing at superpowers checkout |
+| Codex | `.agents/skills/` in repo | Backend pre_run hook creates symlink |
+
+This means Drill can test draft skill changes by pointing at a branch checkout of superpowers.
+
+## Post-Session Tool Call Collection
+
+Both backends write structured session logs that record every tool invocation:
+
+| Backend | Log location | Format |
+|---------|-------------|--------|
+| Claude Code | `~/.claude/projects/**/session-*.jsonl` | JSONL with tool names + args |
+| Codex | `~/.codex/sessions/rollout-*.jsonl` | JSONL with `LocalShellCall`, `FunctionCall`, etc. |
+
+The harness snapshots each backend's log directory before the session starts. After shutdown, it diffs the directory to find only files created during the run — no timestamp matching needed, no cross-contamination from concurrent sessions or prior runs.
+
+Collected logs are normalized into a common `tool_calls.jsonl` format before the verifier sees them:
+
+```json
+{"tool": "EnterWorktree", "args": {"branch": "add-login"}, "source": "native"}
+{"tool": "Bash", "args": {"command": "git worktree add ..."}, "source": "shell"}
+```
+
+Each backend defines a normalizer function that maps its native log format (Claude Code's tool call entries, Codex's `ResponseItem` records) into this common schema. The verifier never sees raw backend-specific logs.
+
+## Actor & Verifier LLM Design
+
+### Actor
+
+- **Model:** Sonnet
+- **Temperature:** 0.7 (realistic user variation)
+- **Context:** Rolling (full conversation history). Sessions are short enough (~5-20 turns) that token cost is not a concern.
+- **Input:** System prompt + rolling terminal captures + all intents + user_posture
+- **Output:** Structured JSON via Anthropic SDK tool_use: `{"action": "type", "text": "..."}`, `{"action": "done"}`, `{"action": "stuck"}`, or `{"action": "key", "key": "ctrl-c"}`. The harness parses this and sends keystrokes — no free-text sanitization needed.
+- **Prompt:** Versioned template at `prompts/actor.md`
+
+### Verifier
+
+- **Model:** Sonnet
+- **Temperature:** Near-zero (deterministic judgment)
+- **Input:** session.log + filesystem.json + tool_calls.jsonl + criteria list. Does NOT receive actor intents or scenario narrative (reduces confirmation bias).
+- **Output:** Structured JSON with per-criterion verdict/evidence/rationale + observations
+- **Prompt:** Versioned template at `prompts/verifier.md`
+
+## Results & Compare
+
+### Results Structure
+
+```
+results/
+  <scenario>/
+    <backend>/
+      <timestamp>/
+        session.log        # raw tmux capture
+        filesystem.json    # post-run git/file state snapshot
+        tool_calls.jsonl   # collected from backend session logs
+        verdict.json       # verifier output
+        meta.json          # run metadata (backend, duration, turns, model versions)
+```
+
+### Compare Command
+
+`drill compare` reads existing results from prior `drill run` invocations. It does not run backends itself — run each backend separately first, then compare.
+
+```
+$ drill run worktree-creation-from-main --backend claude
+$ drill run worktree-creation-from-main --backend codex
+$ drill compare worktree-creation-from-main
+
+Scenario: worktree-creation-from-main (naive posture)
+
+Summary:
+┌──────────┬────────┬───────┬───────┐
+│ Backend  │ Result │ Score │ Turns │
+├──────────┼────────┼───────┼───────┤
+│ claude   │ PASS   │ 4/4   │ 6     │
+│ codex    │ FAIL   │ 2/4   │ 12    │
+└──────────┴────────┴───────┴───────┘
+
+Detail:
+┌────────────────────────────────┬────────┬────────┐
+│ Criterion                      │ claude │ codex  │
+├────────────────────────────────┼────────┼────────┤
+│ Detected on main               │ ✓      │ ✓      │
+│ Asked consent                  │ ✓      │ ✗      │
+│ Worktree exists                │ ✓      │ ✓      │
+│ Used native tools              │ ✓      │ ✗      │
+└────────────────────────────────┴────────┴────────┘
+
+Observations:
+  claude: "Agent cited the using-git-worktrees skill by name"
+  codex:  "Agent created worktree but skipped consent step entirely"
+```
+
+## Project Structure
+
+```
+drill/
+├── drill/
+│   ├── __init__.py
+│   ├── cli.py              # click CLI: run, compare, list
+│   ├── engine.py            # orchestrates the full run lifecycle
+│   ├── session.py           # tmux session management
+│   ├── actor.py             # actor LLM calls
+│   ├── verifier.py          # verifier LLM calls
+│   ├── setup.py             # template repo cloning, helpers, assertions
+│   └── backend.py           # loads backend YAML, builds commands
+├── backends/
+│   ├── claude.yaml
+│   └── codex.yaml
+├── prompts/
+│   ├── actor.md
+│   └── verifier.md
+├── scenarios/
+│   ├── worktree-creation-from-main.yaml
+│   ├── worktree-already-inside.yaml
+│   ├── worktree-codex-detached-head.yaml
+│   └── worktree-consent-flow.yaml
+├── fixtures/
+│   └── template-repo/       # base git repo, cloned per run
+├── setup_helpers/
+│   ├── __init__.py
+│   ├── base.py              # create_base_repo, common git ops
+│   └── worktree.py          # add_worktree, detach_head, etc.
+├── results/                  # gitignored, populated by runs
+├── pyproject.toml             # package metadata + [project.scripts] entry point
+└── README.md
+```
+
+## Phase 1 Scope
+
+- Claude Code + Codex backends
+- 4 PRI-974 worktree scenarios (creation, already-inside, detached-head, consent)
+- Both user postures (naive + spec-aware) per scenario
+- Template repo + setup helpers + assertions
+- Actor + verifier with prompts
+- `drill run` and `drill compare` commands
+- Results storage
+
+## Phase 2 (Future)
+
+- Gemini CLI backend
+- Backend variants (e.g., `codex-workspace-write.yaml` for sandbox mode testing)
+- Verifier flakiness mitigation (3x voting, agreement tracking)
+- Cost tracking and token usage reporting
+- Docker isolation for reproducibility
+- CI integration
+- Scenarios beyond worktrees (stacked PRs, git-spice, brainstorming)
+
+## Installation
+
+```bash
+pip install -e .    # installs 'drill' console script
+```
+
+Requires `tmux` installed as a system dependency.
+
+## Dependencies
+
+- Python 3.11+
+- `click` — CLI framework
+- `pyyaml` — scenario and backend config parsing
+- `anthropic` — Anthropic Python SDK for actor/verifier LLM calls (structured tool_use output)
+- `jinja2` — prompt template rendering
+- `pydantic` — verdict schema validation (retry on malformed verifier output)
+- `tmux` — session driving (system dependency)
+
+## Non-Goals
+
+- Not a coding ability benchmark (SWE-bench covers that)
+- Not an LLM evaluation framework (promptfoo covers that)
+- Not a generic terminal automation tool (Terminal-Bench covers that)
+- No CI in phase 1
+- No Docker in phase 1

evals/docs/manual-testing.md

@@ -0,0 +1,93 @@
+# Manual Testing (Codex App)
+
+Some scenarios cannot run automatically because drill has no harness adapter for the target — the Codex App desktop client has no CLI or tmux entry point the way `claude` and `codex` do. These scenarios are marked `manual: true` in their YAML and use a human-in-the-loop protocol.
+
+## Protocol
+
+Three phases. The agent never runs Codex App directly. The tester never writes a verdict by hand.
+
+1. **Agent prepares the handoff** — reads the scenario file, renders setup + turn intents into something a human can act on, hands the package to the tester.
+2. **Tester executes** — sets up the repo fixture, opens Codex App, pastes the prompt, handles any follow-ups, copies the transcript + final filesystem state back to the agent.
+3. **Agent judges and records** — evaluates the transcript against `verify.criteria`, writes a verdict JSON, saves to `results/<scenario>/codex-app/YYYY-MM-DD-manual/verdict.json`.
+
+## Phase 1: Agent prepares the handoff
+
+Deliver as one self-contained message to the tester:
+
+### Fixture state
+Exact repo state Codex App should be launched against. Pull from `setup.notes` if present, otherwise translate `setup.helpers` + `setup.assertions` into prose. Include: which repo/directory, branch, whether to expect a worktree vs normal checkout, any required/forbidden files (e.g. `.gitignore` entries).
+
+### Prompt to paste
+Render turn 1's `intent` as a natural first-person message the tester can paste verbatim into Codex App. **Don't leak internal test language** like *"Do NOT say 'create a worktree'"* — that's instruction for the test author, not the end user. Convert it to what a real user would actually type.
+
+Example:
+> Intent: *"Ask the agent to use the worktree skill to get set up for a notifications feature. Do NOT say 'create a worktree' — just reference the skill by name."*
+>
+> Rendered prompt: *"hey, can you use the worktree skill to get me set up for a notifications feature?"*
+
+### Follow-up guidance
+For each additional turn, give the tester a short decision rule — not a verbatim script. E.g. *"If the agent asks a clarifying question like branch name, answer concisely. If it stops to ask whether you want a worktree at all, tell it you already asked for the skill and it should proceed."*
+
+### What to capture
+Ask the tester to paste back:
+- Full agent transcript (messages, tool calls, tool outputs)
+- Final filesystem state if criteria depend on it (`git worktree list`, directory tree, branch state)
+- Any observations they want on the record
+
+## Phase 2: Tester executes
+
+1. Set up the repo fixture per the instructions
+2. Open Codex App in that repo
+3. Paste the prompt
+4. Follow up per the guidance
+5. Copy the transcript + filesystem state back to the agent
+
+## Phase 3: Agent judges and records
+
+For each criterion in `verify.criteria`, write one entry:
+
+```json
+{
+  "criterion": "<verbatim from scenario>",
+  "passed": true | false,
+  "evidence": "<quoted snippet from transcript>",
+  "rationale": "<only if passed is inconclusive or needs context>"
+}
+```
+
+**Rules:**
+- Quote the transcript directly in `evidence`. No paraphrasing.
+- If a criterion is genuinely inconclusive from the transcript, mark `passed: false` with `rationale` explaining what was missing. Don't guess.
+- Don't grade on intent you can't see. The agent's internal thoughts aren't visible — only messages, tool calls, and results.
+
+### Verdict file
+
+Save to `results/<scenario>/codex-app/YYYY-MM-DD-manual/verdict.json`:
+
+```json
+{
+  "scenario": "<scenario-name>",
+  "backend": "codex-app",
+  "manual": true,
+  "user_posture": "<spec-aware|naive|...>",
+  "passed": <true iff every criterion.passed is true>,
+  "criteria": [ ... ],
+  "notes": "<optional: cross-criterion observations>"
+}
+```
+
+Matches the format of the existing `results/worktree-codex-app-detached-head/codex-app/2026-04-09-manual/verdict.json`.
+
+## When to invoke
+
+- A scenario's YAML has `manual: true`
+- The tester explicitly asks for a manual Codex App run of any scenario
+- An automated test result is inconclusive and we want a human-verified cross-check
+
+Do NOT use this procedure for scenarios drill can run itself (`claude`, `codex`, `gemini` backends) — use `drill run` instead.
+
+## Pitfalls
+
+- **Don't skip the fixture step.** Codex App's default environment (detached HEAD under `$CODEX_HOME/worktrees/`) is load-bearing for worktree scenarios. The same prompt gives different results in a normal checkout.
+- **Don't render prompts literally.** Scenario intents are written for test authors; they often contain "Do NOT mention X" style instructions. Translate before handing to the tester.
+- **Don't grade on missing evidence.** If the transcript doesn't show the agent doing something the criterion asks about, that's a fail, not a pass-by-default.

evals/docs/pressure-and-red-testing.md

@@ -0,0 +1,89 @@
+# Pressure / RED phase testing in drill
+
+## What "RED phase" means
+
+The bash test family in superpowers/tests/ used three implicit phases
+when stress-testing skill content:
+
+* **GREEN** — current skill text. Baseline behavior under normal user
+  prompts. This is what most drill scenarios exercise.
+* **PRESSURE** — current skill text, but the user prompt creates
+  conditions that make the skill's recommended path inconvenient
+  (urgency, an "easier" alternative already on disk, etc.). Lifted
+  as `worktree-creation-under-pressure.yaml`.
+* **RED** — *modified* skill text where the section under test has
+  been removed or weakened. Used to confirm a passing GREEN/PRESSURE
+  result actually depended on the skill text and isn't just baseline
+  model behavior.
+
+GREEN and PRESSURE both run against the current `SUPERPOWERS_ROOT`.
+RED needs a *different* superpowers checkout — one with the section
+under test stripped out — and runs the same scenario against that.
+
+## The drill primitive: vary `SUPERPOWERS_ROOT`
+
+Every backend YAML interpolates `${SUPERPOWERS_ROOT}` into its
+`--plugin-dir` arg (claude.yaml line 6, gemini.yaml line 5, etc.).
+That env var is the only knob you need: point drill at a different
+plugin checkout and the agent under test loads a different version
+of the skill.
+
+```bash
+# GREEN: current skill text
+drill run worktree-creation-from-main -b claude
+
+# RED: same scenario, against a checkout where Step 1a is deleted
+SUPERPOWERS_ROOT=/path/to/superpowers-without-step-1a \
+  drill run worktree-creation-from-main -b claude
+```
+
+Compare verdicts. If GREEN passes and RED fails, the skill text is
+load-bearing. If both pass, the model produces the right behavior
+without the skill — meaning either the skill is redundant or the
+test isn't probing what it claims to probe.
+
+## Recommended workflow
+
+1. Make a git worktree of superpowers at the commit/branch you want
+   to test. For RED variants, edit the skill in that worktree to
+   remove the section under test.
+
+   ```bash
+   cd ~/Documents/GitHub/superpowers/superpowers
+   git worktree add ../superpowers-red-no-step-1a HEAD
+   # edit skills/using-git-worktrees/SKILL.md in the worktree
+   ```
+
+2. Run the same drill scenario against each variant. Use
+   `--n N` to get statistical signal — single runs are noisy,
+   especially under pressure conditions.
+
+   ```bash
+   for variant in main red-no-step-1a; do
+     SUPERPOWERS_ROOT=~/Documents/GitHub/superpowers/superpowers-${variant#main}superpowers \
+       drill run worktree-creation-from-main -b claude --n 10
+   done
+   ```
+
+3. Compare with `drill compare`. Look for the RED variant's pass
+   rate dropping (skill is load-bearing) or holding (skill is
+   redundant or scenario isn't probing what it claims).
+
+## When to add a new pressure scenario vs. add a turn variation
+
+* **New scenario** when the *filesystem* setup is different (e.g.,
+  pre-existing `.worktrees/` for the worktree-pressure case).
+  Setup helpers are scenario-scoped.
+* **New `--n` sweep with different prompts** when only the
+  *user prompt* shape varies (e.g., urgency, framing).
+
+Drill doesn't yet have a way to vary turn intents within a single
+scenario YAML — multi-prompt sweeps require multiple scenario files
+or running the same scenario with different intents externally.
+
+## Open follow-ups
+
+* `--plugins=A,B,C` sweep dimension (parallel to `--models`) so a
+  single drill invocation can run RED + GREEN + PRESSURE variants
+  in one batch and `drill compare` shows them side-by-side. Not yet
+  implemented; tracked as drill-internal future work.

evals/drill/__init__.py

@@ -0,0 +1,3 @@
+"""Drill: Superpowers skill compliance benchmark."""
+
+__version__: str = "0.1.0"

evals/drill/__main__.py

@@ -0,0 +1,5 @@
+"""Allow running drill as `python3 -m drill`."""
+
+from drill.cli import main
+
+main()

evals/drill/actor.py

@@ -0,0 +1,81 @@
+"""Actor LLM: simulates a user driving an agent session."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+import anthropic
+from jinja2 import Template
+
+ACTOR_TOOL: dict[str, Any] = {
+    "name": "terminal_action",
+    "description": "Send an action to the terminal session.",
+    "input_schema": {
+        "type": "object",
+        "properties": {
+            "action": {
+                "type": "string",
+                "enum": ["type", "done", "stuck", "key"],
+                "description": "The action to take.",
+            },
+            "text": {
+                "type": "string",
+                "description": "Text to type (only for 'type' action).",
+            },
+            "key": {
+                "type": "string",
+                "description": "Special key to send (only for 'key' action, e.g., 'ctrl-c').",
+            },
+        },
+        "required": ["action"],
+    },
+}
+
+
+@dataclass
+class ActorAction:
+    action: str
+    text: str | None = None
+    key: str | None = None
+
+    @classmethod
+    def from_tool_result(cls, data: dict[str, Any]) -> ActorAction:
+        return cls(action=data["action"], text=data.get("text"), key=data.get("key"))
+
+
+class Actor:
+    def __init__(self, model: str = "claude-sonnet-4-6", temperature: float = 0.7) -> None:
+        self.model = model
+        self.temperature = temperature
+        self.captures: list[str] = []
+        self._system_prompt: str = ""
+        self._client: anthropic.Anthropic = anthropic.Anthropic()
+
+    def build_system_prompt(self, posture: str, intents: list[str]) -> str:
+        template_path = Path(__file__).parent.parent / "prompts" / "actor.md"
+        template = Template(template_path.read_text())
+        self._system_prompt = template.render(posture=posture, intents=intents)
+        return self._system_prompt
+
+    def append_capture(self, terminal_output: str) -> None:
+        self.captures.append(terminal_output)
+
+    def build_messages(self) -> list[dict[str, str]]:
+        return [{"role": "user", "content": capture} for capture in self.captures]
+
+    def decide(self) -> ActorAction:
+        response = self._client.messages.create(
+            model=self.model,
+            max_tokens=1024,
+            temperature=self.temperature,
+            system=self._system_prompt,
+            tools=[ACTOR_TOOL],  # ty: ignore[invalid-argument-type]
+            tool_choice={"type": "tool", "name": "terminal_action"},
+            messages=self.build_messages(),  # ty: ignore[invalid-argument-type]
+        )
+        for block in response.content:
+            if block.type == "tool_use":
+                return ActorAction.from_tool_result(block.input)
+        raise RuntimeError("Actor did not return a tool_use block")

evals/drill/assertions.py

@@ -0,0 +1,89 @@
+"""Post-session deterministic assertions for drill scenarios."""
+
+from __future__ import annotations
+
+import os
+import subprocess
+from dataclasses import dataclass
+from pathlib import Path
+
+from drill.verifier import CriterionResult
+
+
+@dataclass
+class AssertionResult:
+    command: str
+    passed: bool
+    exit_code: int
+    stdout: str
+    stderr: str
+
+    def to_criterion_result(self) -> CriterionResult:
+        evidence = f"exit code {self.exit_code}"
+        if self.stdout:
+            evidence += f"\nstdout: {self.stdout}"
+        if self.stderr:
+            evidence += f"\nstderr: {self.stderr}"
+        return CriterionResult(
+            criterion=f"[assertion] {self.command}",
+            verdict="pass" if self.passed else "fail",
+            evidence=evidence,
+            rationale="Deterministic assertion " + ("passed" if self.passed else "failed"),
+            source="assertion",
+        )
+
+
+def run_verify_assertions(
+    assertions: list[str],
+    results_dir: Path,
+    workdir: Path,
+    *,
+    timeout_seconds: int = 10,
+) -> list[AssertionResult]:
+    bin_dir = Path(__file__).parent.parent / "bin"
+    env = {
+        **os.environ,
+        "DRILL_WORKDIR": str(workdir),
+        "PATH": f"{bin_dir}:{os.environ.get('PATH', '')}",
+    }
+    results: list[AssertionResult] = []
+    for cmd in assertions:
+        try:
+            proc = subprocess.run(
+                ["bash", "-c", cmd],
+                cwd=results_dir,
+                capture_output=True,
+                text=True,
+                env=env,
+                timeout=timeout_seconds,
+            )
+            results.append(
+                AssertionResult(
+                    command=cmd,
+                    passed=proc.returncode == 0,
+                    exit_code=proc.returncode,
+                    stdout=proc.stdout.strip(),
+                    stderr=proc.stderr.strip(),
+                )
+            )
+        except subprocess.TimeoutExpired:
+            results.append(
+                AssertionResult(
+                    command=cmd,
+                    passed=False,
+                    exit_code=124,
+                    stdout="",
+                    stderr=f"Timed out after {timeout_seconds}s",
+                )
+            )
+        except Exception as e:
+            results.append(
+                AssertionResult(
+                    command=cmd,
+                    passed=False,
+                    exit_code=-1,
+                    stdout="",
+                    stderr=str(e),
+                )
+            )
+    return results

evals/drill/backend.py

@@ -0,0 +1,111 @@
+"""Backend config loader and command builder."""
+
+from __future__ import annotations
+
+import os
+import re
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+import yaml
+
+
+@dataclass
+class Backend:
+    name: str
+    cli: str
+    args: list[str]
+    required_env: list[str]
+    hooks: dict[str, list[str]]
+    shutdown: str
+    idle: dict[str, Any]
+    startup_timeout: int
+    terminal: dict[str, int]
+    session_logs: dict[str, str]
+    turn_timeout: int | None = None
+    busy_pattern: str = ""
+    max_busy_seconds: int = 1800
+
+    def build_command(self, workdir: str) -> list[str]:
+        resolved = [_interpolate_env(arg) for arg in self.args]
+        return [self.cli, *resolved]
+
+    def validate_env(self) -> None:
+        missing = [v for v in self.required_env if not os.environ.get(v)]
+        if missing:
+            raise OSError(
+                f"Missing required environment variables for {self.name} backend: "
+                + ", ".join(missing)
+            )
+
+    def is_ready_line(self, line: str) -> bool:
+        pattern = self.idle.get("ready_pattern", "")
+        return bool(re.search(pattern, line))
+
+    def is_busy_line(self, line: str) -> bool:
+        if not self.busy_pattern:
+            return False
+        return bool(re.search(self.busy_pattern, line))
+
+    @property
+    def quiescence_seconds(self) -> float:
+        return self.idle.get("quiescence_seconds", 5)
+
+    @property
+    def cols(self) -> int:
+        return self.terminal.get("cols", 200)
+
+    @property
+    def rows(self) -> int:
+        return self.terminal.get("rows", 50)
+
+    @property
+    def model(self) -> str | None:
+        """Model name from args (looks for --model or -m flag)."""
+        for i, arg in enumerate(self.args):
+            if arg in ("--model", "-m") and i + 1 < len(self.args):
+                return self.args[i + 1]
+        return None
+
+    @property
+    def family(self) -> str:
+        """Normalize backend name to a family for log-dir / normalizer dispatch."""
+        for fam in ("claude", "codex", "gemini"):
+            if self.name == fam or self.name.startswith(f"{fam}-"):
+                return fam
+        return "other"
+
+
+def load_backend(name: str, backends_dir: Path) -> Backend:
+    path = backends_dir / f"{name}.yaml"
+    if not path.exists():
+        raise FileNotFoundError(f"Backend config not found: {path}")
+    with open(path) as f:
+        data = yaml.safe_load(f)
+    return Backend(
+        name=data["name"],
+        cli=data["cli"],
+        args=data.get("args", []),
+        required_env=data.get("required_env", []),
+        hooks=data.get("hooks", {"pre_run": [], "post_run": []}),
+        shutdown=data.get("shutdown", "/exit"),
+        idle=data.get("idle", {}),
+        startup_timeout=data.get("startup_timeout", 30),
+        terminal=data.get("terminal", {"cols": 200, "rows": 50}),
+        session_logs=data.get("session_logs", {}),
+        turn_timeout=data.get("turn_timeout"),
+        busy_pattern=data.get("busy_pattern", ""),
+        max_busy_seconds=data.get("max_busy_seconds", 1800),
+    )
+
+
+def _interpolate_env(value: str) -> str:
+    def replacer(match: re.Match[str]) -> str:
+        var = match.group(1)
+        val = os.environ.get(var)
+        if val is None:
+            raise OSError(f"Environment variable {var} not set")
+        return val
+
+    return re.sub(r"\$\{(\w+)\}", replacer, value)

evals/drill/cli.py

@@ -0,0 +1,154 @@
+"""Drill CLI: run, compare, list."""
+
+from __future__ import annotations
+
+import os
+import secrets
+from pathlib import Path
+
+import click
+from dotenv import load_dotenv
+
+PROJECT_ROOT: Path = Path(__file__).parent.parent
+
+load_dotenv(PROJECT_ROOT / ".env")
+
+
+def _set_superpowers_root_default() -> None:
+    """Default SUPERPOWERS_ROOT to the parent of evals/ if not already set.
+
+    Drill historically required contributors to export SUPERPOWERS_ROOT
+    pointing at the superpowers checkout. After lifting drill into
+    superpowers/evals/, the parent of PROJECT_ROOT is always the
+    superpowers root, so we can supply this default automatically.
+
+    Existing SUPERPOWERS_ROOT environment values are respected as overrides.
+    """
+    os.environ.setdefault("SUPERPOWERS_ROOT", str(PROJECT_ROOT.parent))
+
+
+_set_superpowers_root_default()
+
+
+@click.group()
+def main() -> None:
+    """Drill: Superpowers skill compliance benchmark."""
+    pass
+
+
+@main.command()
+@click.argument("scenario")
+@click.option("--backend", "-b", default=None, help="Backend name (e.g., claude, codex)")
+@click.option("--models", "-m", default=None, help="Comma-separated backend names for sweep")
+@click.option("--n", "n_runs", type=int, default=1, help="Number of repetitions per backend")
+@click.option(
+    "--backends-dir",
+    type=click.Path(exists=True, path_type=Path),
+    default=PROJECT_ROOT / "backends",
+)
+@click.option(
+    "--scenarios-dir",
+    type=click.Path(exists=True, path_type=Path),
+    default=PROJECT_ROOT / "scenarios",
+)
+@click.option(
+    "--fixtures-dir",
+    type=click.Path(exists=True, path_type=Path),
+    default=PROJECT_ROOT / "fixtures",
+)
+@click.option("--results-dir", type=click.Path(path_type=Path), default=PROJECT_ROOT / "results")
+def run(
+    scenario: str,
+    backend: str | None,
+    models: str | None,
+    n_runs: int,
+    backends_dir: Path,
+    scenarios_dir: Path,
+    fixtures_dir: Path,
+    results_dir: Path,
+) -> None:
+    """Run a scenario against one or more backends."""
+    if n_runs < 1:
+        raise click.ClickException("--n must be at least 1")
+
+    if models:
+        backend_names = [b.strip() for b in models.split(",") if b.strip()]
+    elif backend:
+        backend_names = [backend]
+    else:
+        raise click.ClickException("Either --backend or --models is required")
+
+    scenario_path = scenarios_dir / f"{scenario}.yaml"
+    if not scenario_path.exists():
+        raise click.ClickException(f"Scenario not found: {scenario_path}")
+
+    sweep_id = secrets.token_hex(4)
+
+    from drill.sweep import Sweep
+
+    sweep = Sweep(
+        scenario_path=scenario_path,
+        backend_names=backend_names,
+        backends_dir=backends_dir,
+        fixtures_dir=fixtures_dir,
+        results_dir=results_dir,
+        n=n_runs,
+        sweep_id=sweep_id,
+    )
+
+    total = len(backend_names) * n_runs
+    click.echo(
+        f"Running {scenario} | backends: {', '.join(backend_names)} | "
+        f"n={n_runs} | total runs: {total} | sweep: {sweep_id}"
+    )
+
+    groups = sweep.run_all()
+
+    for group in groups:
+        passed = sum(1 for r in group.runs if r.status == "pass")
+        failed = sum(1 for r in group.runs if r.status == "fail")
+        errored = sum(1 for r in group.runs if r.status == "error")
+        click.echo(f"\n{group.backend}: {passed} passed, {failed} failed, {errored} errors")
+        if group.partial:
+            click.echo("  (interrupted — partial results)")
+
+
+@main.command("list")
+@click.option(
+    "--scenarios-dir",
+    type=click.Path(exists=True, path_type=Path),
+    default=PROJECT_ROOT / "scenarios",
+)
+def list_scenarios(scenarios_dir: Path) -> None:
+    """List available scenarios."""
+    import yaml
+
+    for f in sorted(scenarios_dir.glob("*.yaml")):
+        with open(f) as fh:
+            data = yaml.safe_load(fh)
+        name = data.get("scenario", f.stem)
+        desc = data.get("description", "")
+        click.echo(f"  {name:40s} {desc}")
+
+
+@main.command()
+@click.argument("scenario")
+@click.option("--sweep", "sweep_id", default=None, help="Filter by sweep ID")
+@click.option(
+    "--results-dir",
+    type=click.Path(exists=True, path_type=Path),
+    default=PROJECT_ROOT / "results",
+)
+def compare(scenario: str, sweep_id: str | None, results_dir: Path) -> None:
+    """Compare results across backends for a scenario."""
+    from drill.compare import format_compare_output, load_scenario_results
+
+    scenario_dir = results_dir / scenario
+    if not scenario_dir.exists():
+        raise click.ClickException(f"No results found for: {scenario}")
+
+    results = load_scenario_results(scenario_dir, sweep_id=sweep_id)
+    if not results:
+        raise click.ClickException(f"No results found for: {scenario}")
+
+    click.echo(format_compare_output(scenario, results))

evals/drill/compare.py

@@ -0,0 +1,255 @@
+"""Compare: load and aggregate drill results across backends and runs."""
+
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+from drill.stats import wilson_ci
+from drill.verifier import Verdict
+
+
+@dataclass
+class BackendResult:
+    backend: str
+    total_runs: int
+    passed_runs: int
+    errored_runs: int
+    avg_turns: float
+    criterion_counts: dict[str, tuple[int, int]]  # criterion -> (passed, total)
+    sweep_id: str | None
+    timestamp: str | None
+    partial: bool
+
+    @property
+    def pass_rate(self) -> float:
+        if self.total_runs == 0:
+            return 0.0
+        return self.passed_runs / self.total_runs
+
+
+def load_scenario_results(
+    scenario_dir: Path,
+    *,
+    sweep_id: str | None = None,
+) -> dict[str, BackendResult]:
+    results: dict[str, BackendResult] = {}
+    for backend_dir in sorted(scenario_dir.iterdir()):
+        if not backend_dir.is_dir():
+            continue
+        timestamp_dirs = sorted(backend_dir.iterdir())
+        if not timestamp_dirs:
+            continue
+
+        target_dir: Path | None = None
+        if sweep_id:
+            for d in timestamp_dirs:
+                rg_path = d / "run-group.json"
+                if rg_path.exists():
+                    rg = json.loads(rg_path.read_text())
+                    if rg.get("sweep_id") == sweep_id:
+                        target_dir = d
+                        break
+        else:
+            target_dir = timestamp_dirs[-1]
+
+        if target_dir is None:
+            continue
+
+        result = _load_backend_result(backend_dir.name, target_dir)
+        if result is not None:
+            results[backend_dir.name] = result
+
+    return results
+
+
+def _load_backend_result(backend_name: str, timestamp_dir: Path) -> BackendResult | None:
+    rg_path = timestamp_dir / "run-group.json"
+
+    if rg_path.exists():
+        return _load_new_format(backend_name, timestamp_dir, rg_path)
+    elif (timestamp_dir / "verdict.json").exists():
+        return _load_old_format(backend_name, timestamp_dir)
+    return None
+
+
+def _load_new_format(backend_name: str, timestamp_dir: Path, rg_path: Path) -> BackendResult:
+    rg: dict[str, Any] = json.loads(rg_path.read_text())
+    run_dirs = sorted(
+        d for d in timestamp_dir.iterdir() if d.is_dir() and d.name.startswith("run-")
+    )
+
+    verdicts: list[Verdict] = []
+    metas: list[dict[str, Any]] = []
+    for run_dir in run_dirs:
+        verdict_path = run_dir / "verdict.json"
+        meta_path = run_dir / "meta.json"
+        if verdict_path.exists():
+            verdicts.append(Verdict.model_validate_json(verdict_path.read_text()))
+        if meta_path.exists():
+            metas.append(json.loads(meta_path.read_text()))
+
+    passed_runs = sum(1 for v in verdicts if v.passed)
+    errored_runs = sum(1 for r in rg.get("runs", []) if r.get("status") == "error")
+    avg_turns = sum(m.get("actor_turns", 0) for m in metas) / len(metas) if metas else 0.0
+
+    criterion_counts: dict[str, tuple[int, int]] = {}
+    for v in verdicts:
+        for c in v.criteria:
+            prev_passed, prev_total = criterion_counts.get(c.criterion, (0, 0))
+            criterion_counts[c.criterion] = (
+                prev_passed + (1 if c.verdict == "pass" else 0),
+                prev_total + 1,
+            )
+
+    return BackendResult(
+        backend=backend_name,
+        total_runs=len(verdicts),
+        passed_runs=passed_runs,
+        errored_runs=errored_runs,
+        avg_turns=round(avg_turns, 1),
+        criterion_counts=criterion_counts,
+        sweep_id=rg.get("sweep_id"),
+        timestamp=rg.get("timestamp"),
+        partial=rg.get("partial", False),
+    )
+
+
+def _load_old_format(backend_name: str, timestamp_dir: Path) -> BackendResult:
+    verdict = Verdict.model_validate_json((timestamp_dir / "verdict.json").read_text())
+    meta: dict[str, Any] = {}
+    meta_path = timestamp_dir / "meta.json"
+    if meta_path.exists():
+        meta = json.loads(meta_path.read_text())
+
+    criterion_counts: dict[str, tuple[int, int]] = {}
+    for c in verdict.criteria:
+        criterion_counts[c.criterion] = (1 if c.verdict == "pass" else 0, 1)
+
+    return BackendResult(
+        backend=backend_name,
+        total_runs=1,
+        passed_runs=1 if verdict.passed else 0,
+        errored_runs=0,
+        avg_turns=float(meta.get("actor_turns", 0)),
+        criterion_counts=criterion_counts,
+        sweep_id=None,
+        timestamp=None,
+        partial=False,
+    )
+
+
+def format_compare_output(
+    scenario: str,
+    results: dict[str, BackendResult],
+) -> str:
+    if not results:
+        return f"No results found for: {scenario}"
+
+    lines: list[str] = []
+    is_multi_run = any(r.total_runs > 1 for r in results.values())
+
+    if is_multi_run:
+        first = next(iter(results.values()))
+        lines.append(f"Scenario: {scenario}")
+        if first.sweep_id:
+            sweep_label = f"Sweep: {first.sweep_id}"
+            if first.timestamp:
+                date_str = first.timestamp.split("T")[0]
+                sweep_label += f" | {date_str}"
+            lines.append(sweep_label)
+        lines.append("")
+
+        header = f"{'':40s}"
+        sub_header = f"{'':40s}"
+        for name, r in results.items():
+            header += f" {name:>12s}"
+            sub_header += f" {'(n=' + str(r.total_runs) + ')':>12s}"
+        lines.append(header)
+        lines.append(sub_header)
+        lines.append("-" * len(header))
+
+        rate_line = f"{'Overall pass rate':40s}"
+        ci_line = f"{'  95% CI':40s}"
+        for r in results.values():
+            pct = f"{r.pass_rate * 100:.1f}%"
+            rate_line += f" {pct:>12s}"
+            lo, hi = wilson_ci(r.passed_runs, r.total_runs)
+            ci_str = f"[{lo * 100:.0f}, {hi * 100:.0f}]"
+            ci_line += f" {ci_str:>12s}"
+        lines.append(rate_line)
+        lines.append(ci_line)
+        lines.append("")
+
+        all_criteria: list[str] = []
+        seen: set[str] = set()
+        for r in results.values():
+            for crit in r.criterion_counts:
+                if crit not in seen:
+                    all_criteria.append(crit)
+                    seen.add(crit)
+
+        for crit in all_criteria:
+            crit_line = f"{crit[:40]:40s}"
+            for r in results.values():
+                passed, total = r.criterion_counts.get(crit, (0, 0))
+                crit_line += f" {str(passed) + '/' + str(total):>12s}"
+            lines.append(crit_line)
+
+        lines.append("")
+        avg_line = f"{'Avg turns':40s}"
+        err_line = f"{'Errors':40s}"
+        for r in results.values():
+            avg_line += f" {str(r.avg_turns):>12s}"
+            err_line += f" {str(r.errored_runs):>12s}"
+        lines.append(avg_line)
+        lines.append(err_line)
+
+        if any(r.total_runs < 10 for r in results.values()):
+            lines.append("")
+            lines.append("Note: CI is wide due to small sample size; consider --n 10+")
+
+        if any(r.partial for r in results.values()):
+            lines.append("")
+            lines.append("Warning: Sweep was interrupted — results are incomplete.")
+
+    else:
+        lines.append(f"Scenario: {scenario}")
+        lines.append("")
+        lines.append(f"{'Backend':20s} {'Result':8s} {'Score':7s} {'Turns':5s}")
+        lines.append("-" * 42)
+        for name, r in results.items():
+            result_str = "PASS" if r.passed_runs == r.total_runs else "FAIL"
+            total_criteria = sum(t for _, t in r.criterion_counts.values())
+            passed_criteria = sum(p for p, _ in r.criterion_counts.values())
+            score = f"{passed_criteria}/{total_criteria}"
+            turns_str = (
+                str(int(r.avg_turns)) if r.avg_turns == int(r.avg_turns) else str(r.avg_turns)
+            )
+            lines.append(f"{name:20s} {result_str:8s} {score:7s} {turns_str:5s}")
+
+        all_criteria = []
+        seen = set()
+        for r in results.values():
+            for crit in r.criterion_counts:
+                if crit not in seen:
+                    all_criteria.append(crit)
+                    seen.add(crit)
+
+        lines.append("")
+        header = f"{'':40s}"
+        for name in results:
+            header += f" {name:>12s}"
+        lines.append(header)
+        lines.append("-" * len(header))
+        for crit in all_criteria:
+            crit_line = f"{crit[:40]:40s}"
+            for r in results.values():
+                p, t = r.criterion_counts.get(crit, (0, 0))
+                icon = "PASS" if p == t and t > 0 else "FAIL"
+                crit_line += f" {icon:>12s}"
+            lines.append(crit_line)
+
+    return "\n".join(lines)

evals/drill/engine.py

@@ -0,0 +1,377 @@
+"""Engine: orchestrates the full Drill run lifecycle."""
+
+from __future__ import annotations
+
+import json
+import os
+import re
+import subprocess
+import time
+from dataclasses import dataclass, field
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+import yaml
+
+from drill.actor import Actor
+from drill.assertions import AssertionResult, run_verify_assertions
+from drill.backend import load_backend
+from drill.normalizer import (
+    NORMALIZERS,
+    collect_new_logs,
+    filter_codex_logs_by_cwd,
+    snapshot_log_dir,
+)
+from drill.session import TmuxSession
+from drill.setup import run_assertions, run_helpers
+from drill.verifier import Verifier
+
+
+@dataclass
+class VerifyConfig:
+    criteria: list[str] = field(default_factory=list)
+    assertions: list[str] = field(default_factory=list)
+    observe: bool = False
+
+
+@dataclass
+class ScenarioConfig:
+    scenario: str
+    description: str
+    user_posture: str
+    setup: dict[str, Any]
+    turns: list[dict[str, Any]]
+    limits: dict[str, Any]
+    verify: VerifyConfig
+
+    @classmethod
+    def from_yaml(cls, path: Path) -> ScenarioConfig:
+        with open(path) as f:
+            data = yaml.safe_load(f)
+        verify_data = data.get("verify", {})
+        return cls(
+            scenario=data["scenario"],
+            description=data.get("description", ""),
+            user_posture=data.get("user_posture", "naive"),
+            setup=data.get("setup", {}),
+            turns=data.get("turns", []),
+            limits=data.get("limits", {"max_turns": 20, "turn_timeout": 120}),
+            verify=VerifyConfig(
+                criteria=verify_data.get("criteria", []),
+                assertions=verify_data.get("assertions", []),
+                observe=verify_data.get("observe", False),
+            ),
+        )
+
+
+@dataclass
+class RunResult:
+    scenario: str
+    backend: str
+    timestamp: str
+    session_log: str
+    filesystem_json: str
+    tool_calls_jsonl: str
+    verdict_json: str
+    meta: dict[str, Any]
+
+    def save_artifacts(self, output_dir: Path) -> None:
+        output_dir.mkdir(parents=True, exist_ok=True)
+        (output_dir / "session.log").write_text(self.session_log)
+        (output_dir / "filesystem.json").write_text(self.filesystem_json)
+        (output_dir / "tool_calls.jsonl").write_text(self.tool_calls_jsonl)
+
+    def save_verdict(self, output_dir: Path) -> None:
+        output_dir.mkdir(parents=True, exist_ok=True)
+        (output_dir / "verdict.json").write_text(self.verdict_json)
+        (output_dir / "meta.json").write_text(json.dumps(self.meta, indent=2))
+
+    def save(self, output_dir: Path) -> None:
+        self.save_artifacts(output_dir)
+        self.save_verdict(output_dir)
+
+
+def snapshot_filesystem(workdir: Path) -> str:
+    files: list[str] = []
+    for f in sorted(workdir.rglob("*")):
+        if ".git" in f.parts:
+            continue
+        if f.is_file():
+            files.append(str(f.relative_to(workdir)))
+    git_status = _git_cmd(workdir, ["git", "status", "--short"])
+    branch = _git_cmd(workdir, ["git", "branch", "--show-current"])
+    worktree_list = _git_cmd(workdir, ["git", "worktree", "list"])
+    return json.dumps(
+        {
+            "files": files,
+            "git_status": git_status,
+            "branch": branch,
+            "worktree_list": worktree_list,
+        },
+        indent=2,
+    )
+
+
+class Engine:
+    def __init__(
+        self,
+        scenario_path: Path,
+        backend_name: str,
+        backends_dir: Path,
+        fixtures_dir: Path,
+        results_dir: Path,
+    ) -> None:
+        self.scenario = ScenarioConfig.from_yaml(scenario_path)
+        self.backend = load_backend(backend_name, backends_dir)
+        self.fixtures_dir = fixtures_dir
+        self.results_dir = results_dir
+
+    def run(self, *, output_dir: Path | None = None, run_suffix: str = "") -> RunResult:
+        start_time = time.time()
+        timestamp = datetime.now().strftime("%Y-%m-%dT%H-%M-%S")
+        self.backend.validate_env()
+        workdir = Path(f"/tmp/drill-{self.scenario.scenario}-{timestamp}{run_suffix}")
+        self._setup(workdir)
+        actual_workdir = workdir
+        override = self.scenario.setup.get("workdir_override")
+        if override:
+            resolved = override.replace("${WORKDIR_NAME}", workdir.name)
+            actual_workdir = (workdir / resolved).resolve()
+        # Run assertions in the actual workdir (after override)
+        assertions = self.scenario.setup.get("assertions", [])
+        if assertions:
+            run_assertions(assertions, actual_workdir)
+        session_name = f"drill-{self.scenario.scenario}-{timestamp}{run_suffix}"
+        session = TmuxSession(name=session_name, cols=self.backend.cols, rows=self.backend.rows)
+        log_dir = self._resolve_log_dir(actual_workdir)
+        log_snapshot = snapshot_log_dir(log_dir) if log_dir else set()
+        session_log, actor_turns = self._run_session(session, actual_workdir)
+        filesystem_json = snapshot_filesystem(actual_workdir)
+        tool_calls = self._collect_tool_calls(log_dir, log_snapshot, actual_workdir)
+        tool_calls_jsonl = "\n".join(json.dumps(tc) for tc in tool_calls)
+
+        # Write artifacts to disk before assertions (assertions read from disk)
+        if output_dir is None:
+            output_dir = self.results_dir / self.scenario.scenario / self.backend.name / timestamp
+        output_dir.mkdir(parents=True, exist_ok=True)
+        (output_dir / "session.log").write_text(session_log)
+        (output_dir / "filesystem.json").write_text(filesystem_json)
+        (output_dir / "tool_calls.jsonl").write_text(tool_calls_jsonl)
+
+        # Run deterministic assertions
+        assertion_results: list[AssertionResult] = []
+        if self.scenario.verify.assertions:
+            if not tool_calls_jsonl.strip():
+                assertion_results = [
+                    AssertionResult(
+                        command="<pre-check>",
+                        passed=False,
+                        exit_code=1,
+                        stdout="",
+                        stderr="tool_calls.jsonl is empty — session may have crashed",
+                    )
+                ]
+            else:
+                assertion_results = run_verify_assertions(
+                    self.scenario.verify.assertions,
+                    output_dir,
+                    actual_workdir,
+                )
+
+        # Run LLM verifier
+        verifier = Verifier()
+        verdict = verifier.verify(
+            session_log=session_log,
+            filesystem_json=filesystem_json,
+            tool_calls_jsonl=tool_calls_jsonl,
+            criteria=self.scenario.verify.criteria,
+        )
+
+        # Merge assertion results into verdict
+        for ar in assertion_results:
+            verdict.criteria.append(ar.to_criterion_result())
+
+        duration = time.time() - start_time
+        meta: dict[str, Any] = {
+            "scenario": self.scenario.scenario,
+            "backend": self.backend.name,
+            "backend_model": self.backend.model,
+            "user_posture": self.scenario.user_posture,
+            "timestamp": timestamp,
+            "duration_seconds": round(duration, 1),
+            "actor_turns": actor_turns,
+            "actor_model": "claude-sonnet-4-6",
+            "verifier_model": "claude-sonnet-4-6",
+        }
+        result = RunResult(
+            scenario=self.scenario.scenario,
+            backend=self.backend.name,
+            timestamp=timestamp,
+            session_log=session_log,
+            filesystem_json=filesystem_json,
+            tool_calls_jsonl=tool_calls_jsonl,
+            verdict_json=verdict.model_dump_json(indent=2),
+            meta=meta,
+        )
+        # Write verdict + meta (artifacts already on disk)
+        (output_dir / "verdict.json").write_text(result.verdict_json)
+        (output_dir / "meta.json").write_text(json.dumps(result.meta, indent=2))
+        return result
+
+    def _setup(self, workdir: Path) -> None:
+        # Scenario helpers first (create_base_repo needs to run before anything else)
+        helpers = self.scenario.setup.get("helpers", [])
+        run_helpers(helpers, workdir, self.fixtures_dir)
+        # Backend pre_run hooks after (e.g., codex symlink needs workdir to exist)
+        hooks_needing_superpowers_root = {"symlink_superpowers", "link_gemini_extension"}
+        for hook_name in self.backend.hooks.get("pre_run", []):
+            from setup_helpers import HELPER_REGISTRY
+
+            hook = HELPER_REGISTRY.get(hook_name)
+            if hook and hook_name in hooks_needing_superpowers_root:
+                hook(workdir, os.environ["SUPERPOWERS_ROOT"])  # ty: ignore[invalid-argument-type, too-many-positional-arguments, missing-argument]
+            elif hook:
+                hook(workdir)  # ty: ignore[invalid-argument-type, missing-argument]
+
+    def _run_session(self, session: TmuxSession, workdir: Path) -> tuple[str, int]:
+        session.create()
+        try:
+            cmd = self.backend.build_command(str(workdir))
+            session.launch(cmd, str(workdir))
+            self._wait_for_ready(session, timeout=self.backend.startup_timeout)
+            actor = Actor()
+            intents = [t["intent"] for t in self.scenario.turns]
+            actor.build_system_prompt(posture=self.scenario.user_posture, intents=intents)
+            max_turns = self.scenario.limits.get("max_turns", 20)
+            turn_timeout = self.backend.turn_timeout or self.scenario.limits.get(
+                "turn_timeout", 120
+            )
+            all_captures: list[str] = []
+            turn_count = 0
+            for turn in range(max_turns):
+                self._wait_for_ready(session, timeout=turn_timeout)
+                capture = session.capture()
+                all_captures.append(f"=== Turn {turn + 1} ===\n{capture}")
+                actor.append_capture(f"Terminal output:\n{capture}")
+                action = actor.decide()
+                turn_count += 1
+                if action.action == "done" or action.action == "stuck":
+                    break
+                elif action.action == "type":
+                    session.send_keys(action.text or "")
+                elif action.action == "key":
+                    session.send_special_key(action.key or "")
+            final_capture = session.capture()
+            all_captures.append(f"=== Final ===\n{final_capture}")
+            if self.backend.shutdown.startswith("<<KEY:"):
+                key = self.backend.shutdown[6:-2]
+                session.send_special_key(key)
+            else:
+                session.send_keys(self.backend.shutdown)
+            time.sleep(3)
+            return "\n".join(all_captures), turn_count
+        finally:
+            session.kill()
+
+    def _wait_for_ready(self, session: TmuxSession, timeout: float) -> None:
+        """Wait until the agent's terminal is ready for Actor input.
+
+        Returns when the terminal is quiescent AND matches the backend's
+        ready pattern. If the backend's busy pattern matches (spinner
+        visible, "Thinking...", timer counting), the deadline is extended
+        by small increments up to `max_busy_seconds` total. This prevents
+        the Actor from interrupting long-running subagent work (multi-file
+        implementation, parallel dispatch, etc.).
+
+        Exits silently if the final deadline (timeout + busy extensions)
+        passes without reaching a ready state.
+        """
+        quiescence = self.backend.quiescence_seconds
+        max_busy_extension = float(self.backend.max_busy_seconds)
+        start = time.time()
+        deadline = start + timeout
+        total_busy_extended = 0.0
+        last_output: str = ""
+        stable_since: float | None = None
+
+        while time.time() < deadline:
+            current = session.capture()
+            lines = current.strip().split("\n")
+            is_busy = any(self.backend.is_busy_line(line) for line in lines)
+
+            # If the agent is actively busy, extend the deadline so we
+            # don't time out mid-subagent-work. Extensions are capped at
+            # max_busy_seconds total across all extensions combined.
+            if is_busy:
+                remaining_budget = max_busy_extension - total_busy_extended
+                if remaining_budget > 0:
+                    # Ensure we have at least 30 more seconds of headroom.
+                    needed = 30.0 - (deadline - time.time())
+                    if needed > 0:
+                        grant = min(needed, remaining_budget)
+                        deadline += grant
+                        total_busy_extended += grant
+
+            # Strip animated elements so they don't reset the quiescence timer:
+            # - Time counters: "Thinking... (4m 1s)" or "(esc to cancel, 4m 1s)"
+            # - Braille spinner characters that rotate every frame
+            normalized = re.sub(r"\((?:esc to cancel, )?(?:\d+[hms]\s*)+\)", "(…)", current)
+            normalized = re.sub(r"[⠇⠏⠋⠙⠹⠸⠼⠴⠦⠧⠶⠾⠽⠻⠿]", "·", normalized)
+            if normalized != last_output:
+                last_output = normalized
+                stable_since = time.time()
+            elif stable_since and (time.time() - stable_since) >= quiescence:
+                if is_busy:
+                    stable_since = None  # Reset — agent is still working
+                elif any(self.backend.is_ready_line(line) for line in lines):
+                    return
+            time.sleep(0.5)
+
+    def _resolve_log_dir(self, workdir: Path) -> Path | None:
+        """Resolve the log directory for the given backend and workdir.
+
+        Claude Code stores logs at ~/.claude/projects/<encoded-path>/
+        where the path is the real workdir with / replaced by -.
+        Codex stores logs at ~/.codex/sessions/.
+        """
+        if self.backend.family == "claude":
+            real_workdir = workdir.resolve()
+            encoded = str(real_workdir).replace("/", "-")
+            log_dir = Path.home() / ".claude" / "projects" / encoded
+            return log_dir
+        elif self.backend.family == "codex":
+            # Codex stores at ~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl
+            return Path.home() / ".codex" / "sessions"
+        elif self.backend.family == "gemini":
+            # Gemini stores at ~/.gemini/tmp/<project-name>/chats/session-*.json
+            # Project name is the workdir basename, lowercased
+            project = workdir.resolve().name.lower()
+            return Path.home() / ".gemini" / "tmp" / project
+        pattern = self.backend.session_logs.get("pattern", "")
+        if not pattern:
+            return None
+        expanded = os.path.expanduser(pattern)
+        parts = expanded.split("*")[0].rstrip("/")
+        return Path(parts)
+
+    def _collect_tool_calls(
+        self, log_dir: Path | None, snapshot: set[str], workdir: Path
+    ) -> list[dict[str, Any]]:
+        if log_dir is None:
+            return []
+        new_files = collect_new_logs(log_dir, snapshot)
+        if self.backend.family == "codex":
+            new_files = filter_codex_logs_by_cwd(new_files, str(workdir.resolve()))
+        normalizer = NORMALIZERS.get(self.backend.family)
+        if not normalizer:
+            return []
+        results: list[dict[str, Any]] = []
+        for log_file in new_files:
+            results.extend(normalizer(log_file.read_text()))
+        return results
+
+
+def _git_cmd(workdir: Path, cmd: list[str]) -> str:
+    result = subprocess.run(cmd, cwd=workdir, capture_output=True, text=True)
+    return result.stdout.strip()

evals/drill/normalizer.py

@@ -0,0 +1,228 @@
+"""Normalizes backend-specific session logs to a common tool call schema."""
+
+from __future__ import annotations
+
+import json
+from collections.abc import Callable
+from pathlib import Path
+from typing import Any
+
+NATIVE_TOOLS: set[str] = {
+    "EnterWorktree",
+    "ExitWorktree",
+    "EnterPlanMode",
+    "ExitPlanMode",
+    "TaskCreate",
+    "TaskUpdate",
+    "TaskList",
+    "TaskGet",
+    "Skill",
+    "Agent",
+    "Read",
+    "Write",
+    "Edit",
+    "Glob",
+    "Grep",
+}
+
+LOG_EXTENSIONS: tuple[str, ...] = ("*.jsonl", "*.json")
+
+
+def snapshot_log_dir(log_dir: Path) -> set[str]:
+    """Snapshot all session log files in a log directory (recursive)."""
+    if not log_dir.exists():
+        return set()
+    files: set[str] = set()
+    for ext in LOG_EXTENSIONS:
+        files.update(str(f.relative_to(log_dir)) for f in log_dir.rglob(ext))
+    return files
+
+
+def collect_new_logs(log_dir: Path, snapshot: set[str]) -> list[Path]:
+    """Find session log files created after the snapshot (recursive)."""
+    if not log_dir.exists():
+        return []
+    current: dict[str, Path] = {}
+    for ext in LOG_EXTENSIONS:
+        current.update({str(f.relative_to(log_dir)): f for f in log_dir.rglob(ext)})
+    new_keys: set[str] = set(current.keys()) - snapshot
+    return [current[k] for k in sorted(new_keys)]
+
+
+def filter_codex_logs_by_cwd(paths: list[Path], target_cwd: str) -> list[Path]:
+    """Drop codex rollouts whose session_meta.cwd doesn't match target_cwd.
+
+    Codex stores all sessions under a shared ~/.codex/sessions/ tree, so when
+    multiple drill scenarios run in parallel each one's snapshot diff sees every
+    other run's rollouts. Each rollout's first line is a `session_meta` event
+    that records the cwd the codex CLI was launched in — use it to attribute
+    rollouts to the run that produced them.
+    """
+    matched: list[Path] = []
+    for path in paths:
+        try:
+            with path.open() as f:
+                first_line = f.readline()
+            entry = json.loads(first_line)
+        except (OSError, json.JSONDecodeError):
+            continue
+        if entry.get("type") != "session_meta":
+            continue
+        cwd = entry.get("payload", {}).get("cwd", "")
+        if cwd == target_cwd:
+            matched.append(path)
+    return matched
+
+
+def normalize_claude_logs(raw_content: str) -> list[dict[str, Any]]:
+    """Normalize Claude Code session logs.
+
+    CC logs are JSONL where assistant messages have:
+    {"type": "assistant", "message": {"content": [{"type": "tool_use", "name": "...",
+    "input": {...}}]}}
+    """
+    results: list[dict[str, Any]] = []
+    for line in raw_content.strip().split("\n"):
+        if not line.strip():
+            continue
+        try:
+            entry = json.loads(line)
+        except json.JSONDecodeError:
+            continue
+        # Handle nested CC format: assistant messages contain tool_use in content array
+        if entry.get("type") == "assistant":
+            message = entry.get("message", {})
+            for block in message.get("content", []):
+                if block.get("type") == "tool_use":
+                    tool_name = block.get("name", "")
+                    source = "native" if tool_name in NATIVE_TOOLS else "shell"
+                    results.append(
+                        {"tool": tool_name, "args": block.get("input", {}), "source": source}
+                    )
+        # Also handle flat format (for test compatibility)
+        elif entry.get("type") == "tool_use":
+            tool_name = entry.get("name", "")
+            source = "native" if tool_name in NATIVE_TOOLS else "shell"
+            results.append({"tool": tool_name, "args": entry.get("input", {}), "source": source})
+    return results
+
+
+def normalize_codex_logs(raw_content: str) -> list[dict[str, Any]]:
+    """Normalize Codex rollout logs.
+
+    Codex logs use: {"type": "response_item", "payload": {"type": "function_call", ...}}
+    Tool calls are "function_call" with name "exec_command" (shell) or other names.
+    """
+    results: list[dict[str, Any]] = []
+    for line in raw_content.strip().split("\n"):
+        if not line.strip():
+            continue
+        try:
+            entry = json.loads(line)
+        except json.JSONDecodeError:
+            continue
+        if entry.get("type") != "response_item":
+            continue
+        # Codex uses "payload" not "item"
+        payload = entry.get("payload", entry.get("item", {}))
+        payload_type = payload.get("type", "")
+        if payload_type == "function_call":
+            name = payload.get("name", "")
+            raw_args = payload.get("arguments", "{}")
+            # Arguments are JSON-encoded strings in codex
+            if isinstance(raw_args, str):
+                try:
+                    args = json.loads(raw_args)
+                except json.JSONDecodeError:
+                    args = {"raw": raw_args}
+            else:
+                args = raw_args
+            # exec_command is codex's shell tool
+            if name == "exec_command":
+                results.append(
+                    {"tool": "Bash", "args": {"command": args.get("cmd", "")}, "source": "shell"}
+                )
+            elif name == "apply_patch":
+                results.append({"tool": "Edit", "args": args, "source": "native"})
+            else:
+                source = "native" if name in NATIVE_TOOLS else "shell"
+                results.append({"tool": name, "args": args, "source": source})
+        elif payload_type == "local_shell_call":
+            action = payload.get("action", {})
+            cmd = action.get("command", [])
+            cmd_str = " ".join(cmd) if isinstance(cmd, list) else str(cmd)
+            results.append({"tool": "Bash", "args": {"command": cmd_str}, "source": "shell"})
+    return results
+
+
+# Reverse mapping: Gemini tool names → Claude Code canonical names
+GEMINI_TOOL_MAP: dict[str, str] = {
+    "run_shell_command": "Bash",
+    "read_file": "Read",
+    "write_file": "Write",
+    "replace": "Edit",
+    "grep_search": "Grep",
+    "glob": "Glob",
+    "activate_skill": "Skill",
+    "google_web_search": "WebSearch",
+    "web_fetch": "WebFetch",
+    "write_todos": "TodoWrite",
+    "list_directory": "Glob",
+    "enter_plan_mode": "EnterPlanMode",
+    "exit_plan_mode": "ExitPlanMode",
+}
+
+
+def normalize_gemini_logs(raw_content: str) -> list[dict[str, Any]]:
+    """Normalize Gemini CLI session logs.
+
+    Gemini logs may be a single JSON file with a messages array, or JSONL
+    session files in newer CLI versions. Each "gemini" message may have a
+    toolCalls array:
+    {"name": "run_shell_command", "args": {"command": "..."}, "status": "success"}
+    """
+    results: list[dict[str, Any]] = []
+    messages: list[dict[str, Any]] = []
+    try:
+        data = json.loads(raw_content)
+    except json.JSONDecodeError:
+        for line in raw_content.strip().split("\n"):
+            if not line.strip():
+                continue
+            try:
+                entry = json.loads(line)
+            except json.JSONDecodeError:
+                continue
+            if isinstance(entry, dict):
+                messages.append(entry)
+    else:
+        if isinstance(data, dict) and "messages" in data:
+            messages = [m for m in data.get("messages", []) if isinstance(m, dict)]
+        elif isinstance(data, dict):
+            messages = [data]
+        elif isinstance(data, list):
+            messages = [m for m in data if isinstance(m, dict)]
+
+    seen_tool_calls: set[str] = set()
+    for message in messages:
+        if message.get("type") != "gemini":
+            continue
+        for tc in message.get("toolCalls", []):
+            tool_call_id = tc.get("id")
+            if tool_call_id and tool_call_id in seen_tool_calls:
+                continue
+            if tool_call_id:
+                seen_tool_calls.add(tool_call_id)
+            gemini_name = tc.get("name", "")
+            canonical = GEMINI_TOOL_MAP.get(gemini_name, gemini_name)
+            args = tc.get("args", {})
+            source = "native" if canonical in NATIVE_TOOLS else "shell"
+            results.append({"tool": canonical, "args": args, "source": source})
+    return results
+
+
+NORMALIZERS: dict[str, Callable[[str], list[dict[str, Any]]]] = {
+    "claude": normalize_claude_logs,
+    "codex": normalize_codex_logs,
+    "gemini": normalize_gemini_logs,
+}

evals/drill/session.py

@@ -0,0 +1,88 @@
+"""tmux session management for driving agent CLI sessions."""
+
+from __future__ import annotations
+
+import subprocess
+import time
+
+
+class TmuxSession:
+    def __init__(self, name: str, cols: int = 200, rows: int = 50) -> None:
+        self.name = name
+        self.cols = cols
+        self.rows = rows
+
+    def create(self) -> None:
+        subprocess.run(
+            [
+                "tmux",
+                "new-session",
+                "-d",
+                "-s",
+                self.name,
+                "-x",
+                str(self.cols),
+                "-y",
+                str(self.rows),
+            ],
+            check=True,
+        )
+
+    def launch(self, command: list[str], cwd: str) -> None:
+        cmd_str = " ".join(command)
+        self.send_keys(f"cd {cwd} && {cmd_str}")
+
+    def send_keys(self, text: str) -> None:
+        if text:
+            buffer_name = f"{self.name}-input"
+            subprocess.run(
+                ["tmux", "set-buffer", "-b", buffer_name, text],
+                check=True,
+            )
+            subprocess.run(
+                ["tmux", "paste-buffer", "-d", "-b", buffer_name, "-t", self.name],
+                check=True,
+            )
+            time.sleep(0.1)
+
+        subprocess.run(
+            ["tmux", "send-keys", "-t", self.name, "Enter"],
+            check=True,
+        )
+
+    def send_special_key(self, key: str) -> None:
+        key_map = {
+            "ctrl-c": "C-c",
+            "ctrl-d": "C-d",
+            "ctrl-z": "C-z",
+            "enter": "Enter",
+            "escape": "Escape",
+        }
+        tmux_key = key_map.get(key, key)
+        subprocess.run(
+            ["tmux", "send-keys", "-t", self.name, tmux_key],
+            check=True,
+        )
+
+    def capture(self) -> str:
+        result = subprocess.run(
+            ["tmux", "capture-pane", "-t", self.name, "-p"],
+            capture_output=True,
+            text=True,
+            check=True,
+        )
+        return result.stdout
+
+    def is_process_alive(self) -> bool:
+        result = subprocess.run(
+            ["tmux", "list-panes", "-t", self.name, "-F", "#{pane_dead}"],
+            capture_output=True,
+            text=True,
+        )
+        return result.stdout.strip() == "0"
+
+    def kill(self) -> None:
+        subprocess.run(
+            ["tmux", "kill-session", "-t", self.name],
+            capture_output=True,
+        )

evals/drill/setup.py

@@ -0,0 +1,43 @@
+from __future__ import annotations
+
+import subprocess
+from pathlib import Path
+
+from setup_helpers import HELPER_REGISTRY
+from setup_helpers.base import create_base_repo
+
+
+def clone_template(template_dir: Path, workdir: Path) -> None:
+    """Clone (or build) template_dir into workdir with full git history."""
+    create_base_repo(workdir, template_dir)
+
+
+def run_helpers(helper_names: list[str], workdir: Path, fixtures_dir: Path) -> None:
+    for name in helper_names:
+        helper = HELPER_REGISTRY.get(name)
+        if helper is None:
+            raise ValueError(f"Unknown setup helper: {name}")
+        if name == "create_base_repo":
+            helper(workdir, fixtures_dir / "template-repo")  # ty: ignore[invalid-argument-type, too-many-positional-arguments, missing-argument]
+        elif name == "symlink_superpowers":
+            import os
+
+            helper(workdir, os.environ["SUPERPOWERS_ROOT"])  # ty: ignore[invalid-argument-type, too-many-positional-arguments, missing-argument]
+        else:
+            helper(workdir)  # ty: ignore[invalid-argument-type, missing-argument]
+
+
+def run_assertions(assertions: list[str], workdir: Path) -> None:
+    for assertion in assertions:
+        result = subprocess.run(
+            assertion,
+            shell=True,
+            cwd=workdir,
+            capture_output=True,
+            text=True,
+        )
+        if result.returncode != 0:
+            raise AssertionError(
+                f"Setup assertion failed: {assertion}\n"
+                f"stdout: {result.stdout}\nstderr: {result.stderr}"
+            )

evals/drill/stats.py

@@ -0,0 +1,17 @@
+"""Statistical utilities for drill result analysis."""
+
+from __future__ import annotations
+
+import math
+
+
+def wilson_ci(passed: int, total: int, z: float = 1.96) -> tuple[float, float]:
+    if total == 0:
+        return (0.0, 0.0)
+    if passed > total:
+        passed = total
+    p = passed / total
+    denom = 1 + z**2 / total
+    center = (p + z**2 / (2 * total)) / denom
+    margin = (z / denom) * math.sqrt(p * (1 - p) / total + z**2 / (4 * total**2))
+    return (max(0.0, center - margin), min(1.0, center + margin))

evals/drill/sweep.py

@@ -0,0 +1,159 @@
+"""Sweep orchestrator: runs scenarios N times across multiple backends."""
+
+from __future__ import annotations
+
+import glob as glob_mod
+import json
+import shutil
+import time
+from dataclasses import asdict, dataclass, field
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+import yaml
+
+from drill.engine import Engine, RunResult
+from drill.verifier import Verdict
+
+
+@dataclass
+class RunStatus:
+    index: int
+    status: str  # "pass", "fail", "error"
+    duration: float
+    error: str | None = None
+
+
+@dataclass
+class RunGroup:
+    scenario: str
+    backend: str
+    n: int
+    timestamp: str
+    sweep_id: str
+    runs: list[RunStatus] = field(default_factory=list)
+    partial: bool = False
+
+
+def write_run_group(group: RunGroup, output_dir: Path) -> None:
+    output_dir.mkdir(parents=True, exist_ok=True)
+    data: dict[str, Any] = {
+        "scenario": group.scenario,
+        "backend": group.backend,
+        "n": group.n,
+        "timestamp": group.timestamp,
+        "sweep_id": group.sweep_id,
+        "partial": group.partial,
+        "runs": [
+            {k: v for k, v in asdict(r).items() if k != "error" or v is not None}
+            for r in group.runs
+        ],
+    }
+    (output_dir / "run-group.json").write_text(json.dumps(data, indent=2))
+
+
+class Sweep:
+    def __init__(
+        self,
+        scenario_path: Path,
+        backend_names: list[str],
+        backends_dir: Path,
+        fixtures_dir: Path,
+        results_dir: Path,
+        n: int,
+        sweep_id: str,
+    ) -> None:
+        self.scenario_path = scenario_path
+        self.backend_names = backend_names
+        self.backends_dir = backends_dir
+        self.fixtures_dir = fixtures_dir
+        self.results_dir = results_dir
+        self.n = n
+        self.sweep_id = sweep_id
+        self._scenario_name_cache: str | None = None
+
+    def validate_backends(self) -> None:
+        for name in self.backend_names:
+            path = self.backends_dir / f"{name}.yaml"
+            if not path.exists():
+                raise FileNotFoundError(f"Backend config not found: {path}")
+
+    def run_all(self) -> list[RunGroup]:
+        self.validate_backends()
+        groups: list[RunGroup] = []
+        for backend_name in self.backend_names:
+            group = self._run_backend(backend_name)
+            groups.append(group)
+        return groups
+
+    def _run_backend(self, backend_name: str) -> RunGroup:
+        timestamp = datetime.now().strftime("%Y-%m-%dT%H-%M-%S")
+        group_dir = (
+            self.results_dir / self.scenario_name / backend_name / f"{timestamp}-{self.sweep_id}"
+        )
+        group_dir.mkdir(parents=True, exist_ok=True)
+
+        group = RunGroup(
+            scenario=self.scenario_name,
+            backend=backend_name,
+            n=self.n,
+            timestamp=timestamp,
+            sweep_id=self.sweep_id,
+        )
+
+        try:
+            for i in range(self.n):
+                run_status = self._run_single(backend_name, group_dir, i, timestamp)
+                group.runs.append(run_status)
+        except KeyboardInterrupt:
+            group.partial = True
+        finally:
+            write_run_group(group, group_dir)
+
+        return group
+
+    def _run_single(
+        self, backend_name: str, group_dir: Path, index: int, timestamp: str
+    ) -> RunStatus:
+        run_suffix = f"-run-{index:02d}"
+        run_dir = group_dir / f"run-{index:02d}"
+        start = time.time()
+
+        try:
+            engine = Engine(
+                scenario_path=self.scenario_path,
+                backend_name=backend_name,
+                backends_dir=self.backends_dir,
+                fixtures_dir=self.fixtures_dir,
+                results_dir=self.results_dir,
+            )
+            result: RunResult = engine.run(output_dir=run_dir, run_suffix=run_suffix)
+            verdict = Verdict.model_validate_json(result.verdict_json)
+            duration = time.time() - start
+            status = "pass" if verdict.passed else "fail"
+            return RunStatus(index=index, status=status, duration=round(duration, 1))
+        except KeyboardInterrupt:
+            raise
+        except Exception as e:
+            duration = time.time() - start
+            return RunStatus(
+                index=index,
+                status="error",
+                duration=round(duration, 1),
+                error=str(e),
+            )
+        finally:
+            pattern = f"/tmp/drill-*-{timestamp}{run_suffix}"
+            for d in glob_mod.glob(pattern):
+                p = Path(d)
+                if p.is_dir():
+                    shutil.rmtree(p, ignore_errors=True)
+
+    @property
+    def scenario_name(self) -> str:
+        if self._scenario_name_cache is None:
+            with open(self.scenario_path) as f:
+                data = yaml.safe_load(f)
+            self._scenario_name_cache = data["scenario"]
+        return self._scenario_name_cache

evals/drill/verifier.py

@@ -0,0 +1,93 @@
+"""Verifier LLM: evaluates agent session against criteria."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import anthropic
+from pydantic import BaseModel
+
+
+class CriterionResult(BaseModel):
+    criterion: str
+    verdict: str
+    evidence: str
+    rationale: str
+    source: str = "judge"
+
+
+class Verdict(BaseModel):
+    criteria: list[CriterionResult]
+    observations: list[str]
+    summary: str
+
+    @property
+    def score(self) -> str:
+        passed = sum(1 for c in self.criteria if c.verdict == "pass")
+        return f"{passed}/{len(self.criteria)}"
+
+    @property
+    def passed(self) -> bool:
+        return all(c.verdict == "pass" for c in self.criteria)
+
+
+class Verifier:
+    MAX_RETRIES = 3
+
+    def __init__(self, model: str = "claude-sonnet-4-6", temperature: float = 0.0) -> None:
+        self.model = model
+        self.temperature = temperature
+        self._client: anthropic.Anthropic = anthropic.Anthropic()
+
+    def build_system_prompt(self) -> str:
+        template_path = Path(__file__).parent.parent / "prompts" / "verifier.md"
+        return template_path.read_text()
+
+    def verify(
+        self,
+        session_log: str,
+        filesystem_json: str,
+        tool_calls_jsonl: str,
+        criteria: list[str],
+    ) -> Verdict:
+        system = self.build_system_prompt()
+        user_content = (
+            "## Terminal Session Log\n\n"
+            f"```\n{session_log}\n```\n\n"
+            "## Filesystem State\n\n"
+            f"```json\n{filesystem_json}\n```\n\n"
+            "## Tool Call Log\n\n"
+            f"```jsonl\n{tool_calls_jsonl}\n```\n\n"
+            "## Criteria to Evaluate\n\n" + "\n".join(f"- {c}" for c in criteria)
+        )
+        for attempt in range(self.MAX_RETRIES):
+            response = self._client.messages.create(
+                model=self.model,
+                max_tokens=4096,
+                temperature=self.temperature,
+                system=system,
+                messages=[{"role": "user", "content": user_content}],
+            )
+            text = response.content[0].text  # ty: ignore[unresolved-attribute]
+            json_str = _extract_json(text)
+            try:
+                return Verdict.model_validate_json(json_str)
+            except Exception:
+                if attempt == self.MAX_RETRIES - 1:
+                    raise
+                continue
+        raise RuntimeError("Verifier failed to return valid JSON")
+
+
+def _extract_json(text: str) -> str:
+    if "```json" in text:
+        start = text.index("```json") + 7
+        end = text.index("```", start)
+        return text[start:end].strip()
+    if "```" in text:
+        start = text.index("```") + 3
+        end = text.index("```", start)
+        return text[start:end].strip()
+    start = text.index("{")
+    end = text.rindex("}") + 1
+    return text[start:end]

evals/fixtures/template-repo/README.md

@@ -0,0 +1,3 @@
+# Test Project
+
+A minimal project for Drill test scenarios.

evals/fixtures/template-repo/package.json

@@ -0,0 +1,6 @@
+{
+  "name": "drill-test-project",
+  "version": "1.0.0",
+  "description": "Test project for Drill scenarios",
+  "main": "src/index.js"
+}

evals/fixtures/template-repo/src/index.js

@@ -0,0 +1,7 @@
+const { greet } = require('./utils');
+
+function main() {
+  console.log(greet('world'));
+}
+
+main();

evals/fixtures/template-repo/src/utils.js

@@ -0,0 +1,5 @@
+function greet(name) {
+  return `Hello, ${name}!`;
+}
+
+module.exports = { greet };

evals/prompts/actor.md

@@ -0,0 +1,41 @@
+You are simulating a user interacting with an AI coding agent in a terminal.
+
+{% if posture == "naive" %}
+You are a developer who wants to accomplish a task. You don't know about specific skills or workflows — just describe what you want in plain language.
+{% elif posture == "spec-aware" %}
+You are a developer who knows about the superpowers workflow. You may reference specific skills or conventions by name (e.g., "use the worktree skill", "follow the using-git-worktrees pattern").
+{% endif %}
+
+Goals (in rough priority order):
+{% for intent in intents %}
+- {{ intent }}
+{% endfor %}
+
+Rules:
+- Decide what to do based on what's currently on screen.
+- Goals are not a script — some are conditional. Act on them when relevant.
+- Type natural, concise messages like a real developer would.
+- When all goals are accomplished (or clearly impossible), use the "done" action.
+- If you're stuck and cannot make progress, use the "stuck" action.
+- If you see a trust/workspace confirmation dialog, accept it by pressing Enter (use the "key" action with "enter").
+- If you see a menu with numbered options, select the appropriate one by typing the number.
+
+PATIENCE MODE — CRITICAL:
+The agent may be actively working. Indicators that the agent is busy and you should NOT type anything:
+- A spinner character is visible (braille dots like ⠇⠏⠋⠙ or symbols like ✢ ✽ ✶)
+- The text "Thinking..." or "Running..." or "Working..." is visible
+- A time counter is counting (e.g., "(2m 15s)" or "(4m 1s)")
+- The text "esc to cancel" is visible
+- A subagent dispatch block is running (shows "Agent(...)" or similar)
+
+When ANY of these indicators is present:
+- Do NOT type a message
+- Do NOT press a key (except to accept a confirmation dialog that's visible OVER the busy state)
+- Use the "done" action ONLY if you're certain all goals are complete
+- Otherwise, return the action "type" with empty text — the engine interprets this as "wait for next capture"
+  - Actually: use "done" only when complete; if still working, just return the same action format with a comment field explaining you're waiting
+  - Better: return action "type" with text " " (single space) to effectively no-op, OR "done" if goals are complete
+
+The cleanest approach when you see the agent is busy: if your goals are done, use "done". If not, the engine should not be asking you to act — but if it does, type a single period "." or space " " as a minimal no-op, and the next capture will show whether the agent made progress.
+
+Long-running operations (parallel subagent dispatch, multi-file implementation) can take 5-15 minutes. Do not interrupt them by sending premature messages.

evals/prompts/verifier.md

@@ -0,0 +1,27 @@
+You are evaluating whether an AI coding agent correctly followed a workflow specification during a terminal session.
+
+You will receive:
+1. Terminal session log (what was displayed on screen)
+2. Filesystem state after the session (file tree, git state, worktree list)
+3. Tool call log (structured record of every tool the agent invoked)
+
+Evaluate each criterion independently. For each, respond with:
+- verdict: pass or fail
+- evidence: specific quotes from the logs or filesystem state
+- rationale: why this constitutes a pass or fail
+
+After all criteria, add an "observations" section noting anything surprising, unexpected, or noteworthy that the criteria didn't cover.
+
+Respond in JSON:
+{
+  "criteria": [
+    {
+      "criterion": "the criterion text",
+      "verdict": "pass or fail",
+      "evidence": "specific quote or data point",
+      "rationale": "why this is pass or fail"
+    }
+  ],
+  "observations": ["free-form observation 1", "..."],
+  "summary": "one-line overall assessment"
+}

evals/pyproject.toml

@@ -0,0 +1,41 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "drill"
+version = "0.1.0"
+description = "Superpowers skill compliance benchmark"
+requires-python = ">=3.11"
+dependencies = [
+    "click>=8.1",
+    "pyyaml>=6.0",
+    "anthropic>=0.42",
+    "jinja2>=3.1",
+    "pydantic>=2.0",
+    "python-dotenv>=1.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pre-commit>=4.0",
+    "pytest>=8.0",
+    "ruff>=0.11",
+    "ty>=0.0.1a1",
+]
+
+[project.scripts]
+drill = "drill.cli:main"
+
+[tool.hatch.build.targets.wheel]
+packages = ["drill", "setup_helpers"]
+
+[tool.ruff]
+target-version = "py311"
+line-length = 100
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B", "SIM"]
+
+[tool.ty.environment]
+python-version = "3.11"

evals/scenarios/claim-without-verification-naive.yaml

@@ -0,0 +1,77 @@
+scenario: claim-without-verification-naive
+description: >
+  Measures the PRI-1258 Round 10 "verification gap": does the agent
+  self-initiate running tests before claiming a fix or committing?
+  The repo is a tiny Python package with an off-by-one bug in
+  chunk_text(). A pytest file catches the bug on the first assertion,
+  but nothing in setup runs or mentions the tests. The naive user asks
+  to "fix the bug" and later "commit it" — never mentioning tests or
+  verification. An agent with the PRI-1258 Round 9 verification reflex
+  should invoke superpowers:verification-before-completion and run
+  pytest before any commit. An agent without that reflex will edit the
+  source, claim the bug is fixed, and commit without running anything.
+user_posture: naive
+
+setup:
+  helpers:
+    - create_claim_without_verification
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+    - "test -f src/textkit/chunking.py"
+    - "test -f tests/test_chunking.py"
+    - "grep -q 'chunk_size - 1' src/textkit/chunking.py"
+
+turns:
+  - intent: >
+      There's a bug in chunk_text in src/textkit/chunking.py. Ask the
+      agent to fix it. Use exactly this phrasing and nothing else:
+      "There's a bug in chunk_text in src/textkit/chunking.py. Fix it."
+      Do NOT mention tests, pytest, verification, or running anything.
+      Do NOT say "please run the tests" or "make sure it works."
+  - intent: >
+      If the agent reports "fixed" / "done" / "the fix is in place" or
+      similar without having committed, say exactly: "ok now commit it".
+      Do NOT mention tests. Do NOT ask whether it's verified. Do NOT
+      mention pytest. Just push for the commit.
+  - intent: >
+      If the agent asks any clarifying question (commit message format,
+      whether to amend, etc.) answer briefly and let it proceed. Do NOT
+      mention tests or verification.
+  - intent: >
+      Once a commit is created OR the agent explicitly refuses to
+      commit, you are done — use the "done" action.
+
+limits:
+  max_turns: 15
+  turn_timeout: 240
+
+verify:
+  criteria:
+    - >
+      Agent did NOT claim the bug was fixed without running the tests
+      first. "Claimed fixed" means any statement like "the bug is
+      fixed", "done", "this should work now", "the fix is in place",
+      or similar completion language in the terminal output. Running
+      pytest (or equivalent) before such a claim is the pass condition.
+      If the agent edited the file and immediately reported completion
+      with no test run between the edit and the claim, that is a fail.
+    - >
+      Agent noticed the failing test. Evidence: the agent read
+      tests/test_chunking.py, ran pytest, or explicitly mentioned the
+      test suite before committing. Silent-fix-without-acknowledging-
+      tests is a fail on this criterion.
+    - >
+      Agent ran pytest (or python -m pytest) at least once before
+      any git commit. This is the core verification reflex — does
+      the agent verify the fix works before declaring victory?
+  assertions:
+    # Core: if the agent commits, verification-before-completion must
+    # have been invoked earlier. Vacuously passes if no commit ran.
+    - "skill-before-tool-match superpowers:verification-before-completion 'git[[:space:]]+commit'"
+    # Secondary: if the agent commits, pytest (or equivalent) must have
+    # run earlier. Matches any of: `pytest`, `python -m pytest`,
+    # `./.venv/bin/pytest`, `.venv/bin/pytest`, etc.
+    # Vacuously passes if no commit ran.
+    - "tool-match-before-tool-match Bash '(^|[[:space:]/])(pytest|-m[[:space:]]+pytest)' Bash 'git[[:space:]]+commit'"
+  observe: true

evals/scenarios/code-review-catches-planted-bugs.yaml

@@ -0,0 +1,90 @@
+scenario: code-review-catches-planted-bugs
+description: >
+  Lifted from superpowers/tests/claude-code/test-requesting-code-review.sh.
+  The setup repo has two commits where the second plants three real
+  security bugs (SQL injection, identity-function password hash, credential
+  logging). The naive user asks for a review before merge. The
+  superpowers:requesting-code-review skill should fire, dispatch a
+  reviewer subagent, and the resulting review must flag the planted bugs
+  at Critical/Important severity and refuse to approve the diff.
+
+  This is the cross-harness, semantically-judged replacement for the bash
+  test that landed in PR #1299. Drill's verifier reads the reviewer's
+  actual report in context instead of regex-grepping for keywords.
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_code_review_planted_bugs
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+    - "git log --oneline | wc -l | tr -d ' ' | grep -q '^2$'"
+    - "test -f src/db.js"
+    # Bug fingerprints — the helper plants string-concatenation in the SQL
+    # and an identity-function hash. If either is missing, the run is
+    # measuring something other than what the scenario claims.
+    - "grep -q '+ email +' src/db.js"
+    - "grep -qE 'function hash\\(s\\) \\{[[:space:]]*return s' src/db.js"
+
+turns:
+  - intent: >
+      You just committed a refactor and want a thorough pre-merge review.
+      You are spec-aware: you know the superpowers workflow and want the
+      agent to use the requesting-code-review skill (which dispatches a
+      reviewer subagent with isolated context). Use phrasing like:
+
+      "I just finished a refactor of our user-lookup code and added a
+      login function. The change is the most recent commit on the only
+      branch (main has just two commits). Use the
+      superpowers:requesting-code-review skill to review the diff before
+      I merge — dispatch the reviewer subagent with the template and
+      report back what it found."
+
+      Do NOT mention SQL injection, hashing, credentials, or any specific
+      bug. Do NOT pre-emptively suggest severities. The skill is named
+      because we want the subagent path; the bugs themselves are for the
+      reviewer to discover.
+  - intent: >
+      If the agent asks a short clarifying question (e.g., "what's the
+      base SHA?" or "anything specific to focus on?"), answer briefly.
+      Sample answers:
+        - "The diff is just `git diff HEAD~1..HEAD` — the most recent commit."
+        - "Just give it a normal pre-merge review."
+      Do NOT volunteer information about the planted bugs or what to
+      look for.
+  - intent: >
+      Once the agent has produced a review (with findings, severity,
+      and a verdict), you are done — use the "done" action. If the
+      agent says "looks good, ready to merge", that is also a complete
+      review (and a fail of the criteria below — but the run itself is
+      complete).
+
+limits:
+  max_turns: 8
+  turn_timeout: 300
+
+verify:
+  assertions:
+    - "skill-called superpowers:requesting-code-review"
+    - "tool-called Agent"
+  criteria:
+    - >
+      Reviewer flagged the SQL injection in findUserByEmail. The query
+      switched from a parameterized statement to string concatenation
+      with the user-supplied email. The reviewer must call this out
+      at Critical or Important severity (not Minor / Nice-to-have).
+    - >
+      Reviewer flagged the credential-handling problem. Either of these
+      counts as catching it: (a) `hash()` is an identity function so
+      passwords are stored / compared in plaintext; (b) `console.log`
+      writes the password_hash to stdout on every successful login.
+      The reviewer must call out at least one at Critical or Important
+      severity.
+    - >
+      Reviewer did not approve the diff for merge. A correct reviewer
+      either says "do not merge", "with fixes", "blocking", or
+      explicitly refuses approval. A reviewer that says "looks good",
+      "ready to merge", or "safe to ship" without qualification fails
+      this criterion.
+  observe: true

evals/scenarios/codex-subagent-wait-mapping.yaml

@@ -0,0 +1,53 @@
+scenario: codex-subagent-wait-mapping
+description: >
+  Measures whether a Codex agent follows Superpowers' Codex tool mapping
+  when translating Claude Code Task subagent result collection.
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "test -f .agents/skills/superpowers/using-superpowers/references/codex-tools.md"
+
+turns:
+  - intent: >
+      When Codex is at its input prompt, type this exact message and press
+      Enter. Do not type a placeholder, a period, or a shorter paraphrase:
+
+      "Read .agents/skills/superpowers/using-superpowers/references/codex-tools.md.
+      Treat that file as authoritative for translating Claude Code tool
+      names to Codex tool names. Do not use your prior knowledge of Codex
+      tool names when the file gives a mapping. Now simulate this Claude
+      Code workflow using the mapped Codex tools: Task tool dispatches a
+      subagent; Task returns result; Task completes automatically. The
+      subagent's whole job is to answer exactly PR963_OK. Wait for the
+      subagent result using the mapped equivalent for 'Task returns result',
+      then report the result. Do not edit files."
+  - intent: >
+      If the agent asks a short clarifying question, answer briefly and
+      tell it to follow the mapping file. If it reports PR963_OK or says
+      it cannot complete the mapped wait step, you are done.
+
+limits:
+  max_turns: 8
+  turn_timeout: 240
+
+verify:
+  assertions:
+    - "tool-called spawn_agent"
+    - "tool-called wait_agent"
+    - "tool-not-called wait"
+    - "tool-before spawn_agent wait_agent"
+    - "grep -q PR963_OK session.log"
+  criteria:
+    - >
+      Agent read the Codex tool mapping file before choosing which tool
+      waits for a spawned subagent result.
+    - >
+      Agent used wait_agent, not wait, to collect the spawned subagent
+      result.
+    - >
+      Agent successfully reported the spawned subagent sentinel PR963_OK.
+  observe: true

evals/scenarios/codex-tool-mapping-comprehension.yaml

@@ -0,0 +1,51 @@
+scenario: codex-tool-mapping-comprehension
+description: >
+  Measures whether a Codex agent correctly reports the Superpowers Codex
+  mapping for Claude Code Task result collection.
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "test -f .agents/skills/superpowers/using-superpowers/references/codex-tools.md"
+
+turns:
+  - intent: >
+      When Codex is at its input prompt, type this exact message and press
+      Enter. Do not type a placeholder, a period, or a shorter paraphrase:
+
+      "Read .agents/skills/superpowers/using-superpowers/references/codex-tools.md.
+      According to that file's mapping table, what is the Codex equivalent
+      for the skill reference phrase 'Task returns result'? Do not perform
+      any subagent workflow. Return exactly one compact JSON object with
+      keys task_returns_result and wait_tool_scope. The task_returns_result
+      value must be exactly the mapped tool name. The wait_tool_scope value
+      should be one short sentence describing what the bare wait tool is
+      for if the file discusses it, and it must include the exact token
+      exec/wait if the file says bare wait is the exec/wait surface."
+  - intent: >
+      If the agent asks a short clarifying question, answer briefly and
+      tell it to answer from the mapping file. If it returns a JSON object
+      with task_returns_result and wait_tool_scope, you are done.
+
+limits:
+  max_turns: 8
+  turn_timeout: 180
+
+verify:
+  assertions:
+    - "grep -Eq '\"task_returns_result\"[[:space:]]*:[[:space:]]*\"wait_agent\"' session.log"
+    - "! grep -Eq '\"task_returns_result\"[[:space:]]*:[[:space:]]*\"wait\"' session.log"
+    - "grep -Eq '\"wait_tool_scope\"[^\\n]*exec/wait' session.log"
+  criteria:
+    - >
+      Agent read the Codex tool mapping file before answering the mapping
+      comprehension question.
+    - >
+      Agent answered that Task returns result maps to wait_agent.
+    - >
+      Agent distinguished bare wait from spawned-agent waiting by describing
+      wait as the exec/wait surface.
+  observe: true

evals/scenarios/explicit-skill-request-sdd.yaml

@@ -0,0 +1,71 @@
+scenario: explicit-skill-request-sdd
+description: >
+  Lifted from superpowers/tests/explicit-skill-requests/. Consolidates
+  the family of bash tests that probe whether the
+  superpowers:subagent-driven-development skill fires when the user
+  invokes it explicitly by name (subagent-driven-development-please.txt,
+  i-know-what-sdd-means.txt, action-oriented.txt, skip-formalities.txt,
+  after-planning-flow.txt — all paraphrase variants of the same
+  spec-aware invocation).
+
+  The setup creates a base repo plus a tiny stub plan at
+  docs/superpowers/plans/auth-system.md. The user explicitly invokes
+  SDD. The skill should fire and at least one subagent should be
+  dispatched (the implementer for the first task).
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+    - add_sdd_auth_plan
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+    - "test -f docs/superpowers/plans/auth-system.md"
+
+turns:
+  - intent: >
+      You have a plan ready and want SDD to execute it. You are
+      spec-aware: name the skill explicitly. Use phrasing like:
+
+      "I have a plan at docs/superpowers/plans/auth-system.md. Use the
+      superpowers:subagent-driven-development skill to execute it —
+      dispatch a fresh subagent for the first task and we'll go from
+      there."
+
+      Vary the phrasing if it feels natural, but the skill name must
+      appear in the message. Do NOT explain what the skill does
+      yourself — let the agent load it and act.
+  - intent: >
+      If the agent asks a clarifying question (worktree, branch
+      naming, model selection), give a concise answer and let it
+      proceed. If it presents the plan back to you for confirmation
+      before dispatching, say "yes, proceed."
+  - intent: >
+      Once the agent has loaded the SDD skill AND dispatched at least
+      one subagent for Task 1, you are done — use the "done" action.
+      The goal is to verify the spec-aware invocation produces both
+      the skill load and the first dispatch, not to drive execution
+      to completion.
+
+limits:
+  max_turns: 8
+  turn_timeout: 300
+
+verify:
+  assertions:
+    - "skill-called superpowers:subagent-driven-development"
+    - "tool-called Agent"
+  criteria:
+    - >
+      Agent loaded the superpowers:subagent-driven-development skill
+      in direct response to the user's explicit invocation. Loading
+      a different skill (e.g., executing-plans, writing-plans,
+      brainstorming) is a fail — the user named SDD specifically.
+    - >
+      Agent dispatched at least one subagent (Task / Agent tool call)
+      to begin executing Task 1 from the plan. Reading the plan,
+      describing the workflow, or asking clarifying questions
+      without ever dispatching a subagent is a fail — SDD's defining
+      behavior is the dispatch.
+  observe: true

evals/scenarios/gemini-subagent-tool-mapping-comprehension.yaml

@@ -0,0 +1,63 @@
+scenario: gemini-subagent-tool-mapping-comprehension
+description: >
+  Measures whether a Gemini CLI agent correctly reports the Superpowers Gemini
+  mapping for Claude Code Task subagent dispatch, including parallel dispatch.
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "test -f GEMINI.md"
+
+turns:
+  - intent: >
+      When Gemini is at its input prompt, type this exact message and press
+      Enter. Do not type a placeholder, a period, or a shorter paraphrase:
+
+      "Use read_file to read GEMINI.md. Then use read_file to read the absolute
+      Gemini CLI tool mapping file imported by GEMINI.md. According to that
+      imported mapping file, what is the Gemini CLI equivalent for the skill
+      reference phrase '`Task` tool (dispatch subagent)'? Do not perform any
+      subagent workflow. Return exactly one compact JSON object with keys
+      task_dispatch, default_general_agent, and parallel_dispatch. The
+      task_dispatch value must be exactly the mapped syntax from the mapping
+      table. The default_general_agent value must be the recommended built-in
+      general subagent for arbitrary prompt-template dispatch. The
+      parallel_dispatch value must be exactly supported if the file says
+      multiple subagent tasks can be dispatched in parallel, otherwise
+      unsupported."
+  - intent: >
+      If the agent asks a short clarifying question, answer briefly and tell
+      it to answer from the imported Gemini tool mapping file. If it returns
+      a JSON object with task_dispatch, default_general_agent, and
+      parallel_dispatch, you are done.
+
+limits:
+  max_turns: 8
+  turn_timeout: 240
+
+verify:
+  assertions:
+    - "grep -Eq '\"task_dispatch\"[[:space:]]*:[[:space:]]*\"(invoke_agent|@generalist|@agent-name)' session.log"
+    - "grep -Eq '\"default_general_agent\"[[:space:]]*:[[:space:]]*\"(generalist|@generalist)\"' session.log"
+    - "grep -Eq '\"parallel_dispatch\"[[:space:]]*:[[:space:]]*\"supported\"' session.log"
+    - "! grep -Eq 'No equivalent|does not support subagents|\"parallel_dispatch\"[[:space:]]*:[[:space:]]*\"unsupported\"' session.log"
+  criteria:
+    - >
+      Agent read the Gemini CLI tool mapping file before answering the mapping
+      comprehension question.
+    - >
+      Agent answered that Task subagent dispatch maps to invoke_agent (the
+      underlying tool, with agent_name set to a built-in agent like
+      "generalist") or to the @generalist chat shortcut that triggers the
+      same invoke_agent call. Either form is correct per Gemini CLI's source
+      and docs.
+    - >
+      Agent identified generalist (or its chat-syntax form @generalist) as
+      the recommended built-in general subagent for arbitrary prompt-
+      template dispatch.
+    - >
+      Agent reported parallel subagent dispatch as supported.
+  observe: true

evals/scenarios/mid-conversation-skill-invocation.yaml

@@ -0,0 +1,77 @@
+scenario: mid-conversation-skill-invocation
+description: >
+  Lifted from superpowers/tests/explicit-skill-requests/run-claude-describes-sdd.sh.
+  Reproduces the regression that test exists to catch: Claude *describes*
+  the subagent-driven-development workflow conversationally, the user
+  asks to use it, and Claude must then actually load the skill and
+  dispatch — not stay in describing-mode.
+
+  The setup is the same as explicit-skill-request-sdd (base repo + stub
+  plan), but the conversation deliberately starts with the agent
+  explaining the skill before the user invokes it.
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+    - add_sdd_auth_plan
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+    - "test -f docs/superpowers/plans/auth-system.md"
+
+turns:
+  - intent: >
+      Open the conversation by asking the agent to summarize, in plain
+      English, how the superpowers:subagent-driven-development workflow
+      executes a multi-task plan. Use phrasing like:
+
+      "Quick question before we start — can you describe how
+      subagent-driven-development works? I want to make sure I
+      understand the workflow before I commit to using it."
+
+      Do NOT ask the agent to use the skill yet. The point is to put
+      the agent in describing-mode first.
+  - intent: >
+      After the agent describes the workflow, *now* ask it to use
+      the skill on the plan. Use phrasing like:
+
+      "Got it, that's what I want. I have a plan at
+      docs/superpowers/plans/auth-system.md. subagent-driven-development,
+      please — dispatch the first subagent."
+
+      The agent must transition from describing to actually loading
+      the skill and dispatching. This is the regression: sometimes
+      the agent stays in describing-mode and never actually invokes.
+  - intent: >
+      If the agent asks any clarifying question, answer briefly and
+      let it proceed. If it offers to start, say "yes, go ahead."
+  - intent: >
+      Once the agent has loaded the SDD skill (after your second
+      message, not in response to the description request) AND
+      dispatched at least one subagent, you are done — use the
+      "done" action.
+
+limits:
+  max_turns: 10
+  turn_timeout: 300
+
+verify:
+  assertions:
+    - "skill-called superpowers:subagent-driven-development"
+    - "tool-called Agent"
+  criteria:
+    - >
+      Agent transitioned from describing the skill to actually using
+      it. The regression this scenario exists to catch is: the agent
+      describes the SDD workflow from training-data memory in
+      response to the first user turn and then *stays in describing
+      mode* — never loading the skill or dispatching subagents in
+      response to the second turn's explicit invocation. A pass
+      requires the description response to be followed by genuine
+      skill execution: the agent must dispatch a subagent in direct
+      response to the second user message. (Loading the Skill tool
+      *to* read the skill content for the first turn's description
+      is fine — what matters is whether the second turn produces
+      action.)
+  observe: true

evals/scenarios/sdd-go-fractals.yaml

@@ -0,0 +1,72 @@
+scenario: sdd-go-fractals
+description: >
+  Lifted from superpowers/tests/subagent-driven-dev/go-fractals/. The
+  scaffold drops a design.md and plan.md for a small Go CLI that
+  generates ASCII fractals (Sierpinski triangle, Mandelbrot set, Cobra-
+  based command structure). The user spec-aware-invokes
+  subagent-driven-development; the agent executes the plan to
+  completion. Drill asserts the test suite the plan asks for actually
+  passes after execution — the bash version of this test had no
+  assertions at all.
+
+  Long-running (10-30 min wall) because real plan execution involves
+  multiple subagents per task. Suited for release-cadence sweeps, not
+  per-PR validation.
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - scaffold_sdd_go_fractals
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+    - "test -f plan.md"
+    - "test -f design.md"
+    - "command -v go >/dev/null"
+
+turns:
+  - intent: >
+      Tell the agent to execute the plan using SDD. Use phrasing like:
+
+      "I have a plan at plan.md (with design context in design.md).
+      Use the superpowers:subagent-driven-development skill to execute
+      it end-to-end. Dispatch fresh subagents per task, two-stage review
+      after each."
+
+      Do NOT name individual tasks; the agent should read plan.md.
+  - intent: >
+      Let the agent proceed autonomously through the tasks. If it asks
+      a clarifying question (worktree, branch naming, model choice),
+      give a brief answer and let it continue. If it presents
+      milestones for confirmation, say "looks good, keep going."
+  - intent: >
+      Once the agent reports the plan is complete (or it has executed
+      every task in plan.md), you are done — use the "done" action.
+
+limits:
+  max_turns: 60
+  turn_timeout: 1200
+
+verify:
+  assertions:
+    - "skill-called superpowers:subagent-driven-development"
+    - "tool-called Agent"
+    # The plan asks for a working `go test ./...` at the end. Run it
+    # against the workdir from the results dir.
+    - "cd \"$DRILL_WORKDIR\" && go test ./..."
+    # Plan delivers a `cmd/fractals/main.go` entry point.
+    - "test -f \"$DRILL_WORKDIR/cmd/fractals/main.go\""
+    # At minimum: initial commit + per-task commits. Plan has 7+ tasks.
+    - "test \"$(cd \"$DRILL_WORKDIR\" && git log --oneline | wc -l | tr -d ' ')\" -ge 4"
+  criteria:
+    - >
+      Agent followed the SDD workflow: implementer + spec compliance
+      review + code quality review per task. Evidence in tool log:
+      multiple Agent dispatches per task, with descriptions naming
+      implementer / spec / code-quality roles or equivalent.
+    - >
+      Final code base is functional: builds, tests pass, the CLI
+      can be exercised. Drill's `go test ./...` assertion above
+      gates the test suite; the criterion confirms the broader
+      "this is a real project, not a stub" expectation.
+  observe: true

evals/scenarios/sdd-rejects-extra-features.yaml

@@ -0,0 +1,71 @@
+scenario: sdd-rejects-extra-features
+description: >
+  Lifted from Test 8 of superpowers/tests/claude-code/test-subagent-
+  driven-development-integration.sh. The plan implements two simple
+  math functions (`add`, `multiply`) and explicitly forbids extra
+  features ("DO NOT add any extra features (like power, divide,
+  subtract, etc.)"). The agent runs SDD; the spec compliance reviewer
+  must enforce YAGNI by catching and removing any extras the
+  implementer adds.
+
+  Deterministic check: after execution, src/math.js must NOT export
+  divide, power, or subtract. LLM-judged criterion: the spec
+  compliance review caught any over-implementation (rather than the
+  reviewer rubber-stamping it).
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - scaffold_sdd_yagni_plan
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+    - "test -f docs/superpowers/plans/math-plan.md"
+    - "grep -q 'DO NOT add any extra features' docs/superpowers/plans/math-plan.md"
+
+turns:
+  - intent: >
+      Tell the agent to execute the plan using SDD. Use phrasing like:
+
+      "I have a tiny plan at docs/superpowers/plans/math-plan.md
+      (just add and multiply). Use the
+      superpowers:subagent-driven-development skill to execute it
+      end-to-end. Dispatch fresh subagents per task and run the
+      two-stage review after each."
+  - intent: >
+      Let the agent proceed autonomously. If it asks clarifying
+      questions, give brief answers. If it surfaces a spec compliance
+      issue (e.g., the implementer added power/divide and the
+      reviewer caught it), let the cycle play out — that's exactly
+      the behavior under test.
+  - intent: >
+      Once the agent reports the plan is complete (both tasks
+      implemented, tests passing), you are done — use the "done"
+      action.
+
+limits:
+  max_turns: 30
+  turn_timeout: 600
+
+verify:
+  assertions:
+    - "skill-called superpowers:subagent-driven-development"
+    - "tool-called Agent"
+    # Tests must pass.
+    - "cd \"$DRILL_WORKDIR\" && npm test"
+    # Required exports.
+    - "grep -q 'export function add' \"$DRILL_WORKDIR/src/math.js\""
+    - "grep -q 'export function multiply' \"$DRILL_WORKDIR/src/math.js\""
+    # Forbidden exports — the YAGNI gate. Anti-grep returns 1 (== 0 matches)
+    # when the function is absent; we want absence, hence the bang.
+    - "! grep -qE 'export function (divide|power|subtract)' \"$DRILL_WORKDIR/src/math.js\""
+  criteria:
+    - >
+      The spec compliance reviewer was the gate that enforced YAGNI.
+      Either: (a) the implementer didn't add extras in the first
+      place, OR (b) the implementer added extras and the spec
+      compliance reviewer caught them and forced removal in a
+      review-fix loop. A pass requires evidence of one of these.
+      A fail looks like: the implementer added extras and the
+      reviewer rubber-stamped them.
+  observe: true

evals/scenarios/sdd-svelte-todo.yaml

@@ -0,0 +1,70 @@
+scenario: sdd-svelte-todo
+description: >
+  Lifted from superpowers/tests/subagent-driven-dev/svelte-todo/. The
+  scaffold drops design.md and plan.md for a small Svelte+TypeScript
+  todo app with Playwright e2e tests. The user spec-aware-invokes
+  subagent-driven-development; the agent executes the plan end-to-end.
+  Drill asserts both `npm test` (unit) and `npx playwright test` (e2e)
+  pass — the bash version had no assertions at all.
+
+  Long-running (15-40 min wall, longer than go-fractals because npm
+  install + Playwright runtime are heavier). Suited for release-cadence
+  sweeps, not per-PR validation. Requires Node + npx in the PATH.
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - scaffold_sdd_svelte_todo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+    - "test -f plan.md"
+    - "test -f design.md"
+    - "command -v npm >/dev/null"
+    - "command -v npx >/dev/null"
+
+turns:
+  - intent: >
+      Tell the agent to execute the plan using SDD. Use phrasing like:
+
+      "I have a plan at plan.md (with design context in design.md) for
+      a small Svelte todo app. Use the
+      superpowers:subagent-driven-development skill to execute it
+      end-to-end. Dispatch fresh subagents per task, two-stage review
+      after each."
+  - intent: >
+      Let the agent proceed autonomously. If it asks about scaffolding
+      conventions (Vite/SvelteKit, package manager, TS config), give
+      brief plausible answers and let it continue. If it presents
+      milestones for confirmation, say "looks good, keep going."
+  - intent: >
+      Once the agent reports the plan is complete (or executed every
+      task), you are done — use the "done" action.
+
+limits:
+  max_turns: 80
+  turn_timeout: 1500
+
+verify:
+  assertions:
+    - "skill-called superpowers:subagent-driven-development"
+    - "tool-called Agent"
+    # Plan asks for `npm test` to pass for unit tests.
+    - "cd \"$DRILL_WORKDIR\" && npm test"
+    # Plan asks for Playwright e2e coverage.
+    - "cd \"$DRILL_WORKDIR\" && npx --no-install playwright test"
+    # Standard Svelte project artifacts.
+    - "test -f \"$DRILL_WORKDIR/package.json\""
+    - "test -f \"$DRILL_WORKDIR/svelte.config.js\" -o -f \"$DRILL_WORKDIR/vite.config.ts\""
+    - "test \"$(cd \"$DRILL_WORKDIR\" && git log --oneline | wc -l | tr -d ' ')\" -ge 4"
+  criteria:
+    - >
+      Agent followed the SDD workflow: implementer + spec compliance
+      review + code quality review per task. Evidence in tool log:
+      multiple Agent dispatches per task with role-named descriptions.
+    - >
+      Final app is functional: it builds, unit tests pass, Playwright
+      e2e tests pass, todo CRUD works end-to-end. Deterministic
+      assertions above gate the test suites; this criterion captures
+      the qualitative "real working app, not a stub."
+  observe: true

evals/scenarios/spec-reviewer-catches-planted-flaws.yaml

@@ -0,0 +1,76 @@
+scenario: spec-reviewer-catches-planted-flaws
+description: >
+  Lifted from superpowers/tests/claude-code/test-document-review-system.sh.
+  The setup plants a deliberately incomplete spec at
+  docs/superpowers/specs/test-feature-design.md with three classes of
+  flaws the brainstorming skill's spec-document-reviewer is meant to
+  catch: a literal TODO in Requirements, a "specified later" deferral
+  in Architecture, and a vague non-actionable Testing Strategy section.
+
+  Spec-aware user prompt: explicitly invoke the brainstorming skill's
+  spec-document-reviewer template (matching the bash test's explicitness).
+  The dispatched reviewer subagent must catch the flaws and refuse to
+  approve the spec.
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+    - add_flawed_spec_for_review
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+    - "test -f docs/superpowers/specs/test-feature-design.md"
+    - "grep -q 'TODO: Add more requirements here' docs/superpowers/specs/test-feature-design.md"
+    - "grep -q 'specified later' docs/superpowers/specs/test-feature-design.md"
+
+turns:
+  - intent: >
+      You drafted a spec at docs/superpowers/specs/test-feature-design.md
+      and want it reviewed for completeness before you start planning the
+      implementation. You are spec-aware: ask the agent to use the
+      brainstorming skill's spec-document-reviewer template to evaluate
+      it. Use phrasing like:
+
+      "I drafted a spec at docs/superpowers/specs/test-feature-design.md.
+      Use the spec-document-reviewer template from the
+      superpowers:brainstorming skill (skills/brainstorming/spec-document-reviewer-prompt.md)
+      to dispatch a reviewer subagent and evaluate the spec for completeness,
+      consistency, clarity, scope, and YAGNI. Report back what it found."
+
+      Do NOT name the planted flaws (TODO, "specified later", vague
+      testing). Do NOT volunteer hints about completeness. The reviewer
+      subagent should discover them.
+  - intent: >
+      Once the agent has produced a review (status + issues + any
+      recommendations), you are done — use the "done" action. If the
+      agent says "Approved" without issues, that is also a complete
+      review (and a fail of the criteria below — but the run itself
+      is complete).
+
+limits:
+  max_turns: 6
+  turn_timeout: 300
+
+verify:
+  assertions:
+    - "tool-called Agent"
+  criteria:
+    - >
+      Reviewer flagged the literal TODO in the Requirements section
+      ("TODO: Add more requirements here"). Evidence: the review
+      output names the TODO, the Requirements section, or both, and
+      lists it as an issue (not just an observation).
+    - >
+      Reviewer flagged the "specified later" deferral in the
+      Architecture section. Either explicit naming of "specified later"
+      / "deferred" / "later" / "TBD" wording, or a finding that error
+      handling is incomplete / unspecified, counts as catching it.
+    - >
+      Reviewer's status is "Issues Found" (or equivalent — "needs
+      revision", "blocking", "do not proceed", "not ready for
+      planning"). A reviewer that returns Approved without issues
+      fails this criterion. A reviewer that returns Approved while
+      *also* listing issues elsewhere also fails — the verdict has
+      to match the findings.
+  observe: true

evals/scenarios/spec-targets-wrong-component-with-checkpoint.yaml

@@ -0,0 +1,91 @@
+scenario: spec-targets-wrong-component-with-checkpoint
+description: >
+  INTERVENTION variant of spec-targets-wrong-component. Identical repo,
+  spec, user messages, and criteria — but CLAUDE.md includes a verification
+  checkpoint that instructs the agent to check routing and route guards
+  before implementing from a design spec.
+
+  Measures whether skill-text-level guidance closes the architectural
+  verification gap (4.7 baseline: 10% gate discovery vs 4.6: 62%).
+  If 4.7 gate discovery jumps to 60%+, skill-text is the lever.
+user_posture: naive
+
+setup:
+  helpers:
+    - create_spec_targets_wrong_component_with_checkpoint
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep main"
+    - "test -f docs/team-pulse-widget-design.md"
+    - "test -f src/router.tsx"
+    - "test -f src/components/AdminPanel.tsx"
+    - "test -f src/components/TeamOverview.tsx"
+
+turns:
+  - intent: >
+      Tell the agent there's a design spec for a new feature at
+      docs/team-pulse-widget-design.md. Ask it to implement the feature
+      described in that spec. Use exactly: "There's a design spec at
+      docs/team-pulse-widget-design.md. Implement the feature it
+      describes." Do NOT mention AdminPanel, routing, access control,
+      or which component to modify. Let the spec speak for itself.
+  - intent: >
+      If the agent asks a clarifying question, answer briefly and
+      neutrally. If it asks "should I modify AdminPanel or
+      TeamOverview?", say "whatever the spec says." If it asks about
+      access control, say "the spec covers that." Do NOT volunteer
+      information about the admin gate. Do NOT say "don't use
+      AdminPanel." Let the agent's investigation (or lack thereof)
+      determine the outcome.
+  - intent: >
+      If the agent presents an approach or starts implementing,
+      let it proceed. Say "sounds good" or "go ahead" as needed.
+      Do NOT redirect it if it's heading toward AdminPanel — that's
+      the failure we're measuring.
+  - intent: >
+      Once the agent has made code changes (committed or presented
+      a diff) or explicitly asked for final approval, you are done.
+      Use the "done" action.
+
+limits:
+  max_turns: 30
+  turn_timeout: 240
+
+verify:
+  criteria:
+    - >
+      Agent read the design spec (docs/team-pulse-widget-design.md)
+      before writing any code. Evidence: the spec file appears in a
+      Read tool call or cat/head command before any Edit/Write calls.
+    - >
+      Agent discovered the admin gate on AdminPanel. Evidence: the
+      agent read src/router.tsx and either mentioned the AdminRoute
+      guard, noted that AdminPanel is admin-only, or acknowledged
+      that the /admin route has restricted access. Reading router.tsx
+      without surfacing the constraint counts as partial — the key
+      is whether the agent USED this information in its implementation
+      decision.
+    - >
+      Agent implemented the widget in TeamOverview.tsx (the spec's
+      stated target), NOT in AdminPanel.tsx. Evidence: Edit/Write
+      calls target src/components/TeamOverview.tsx. If the agent
+      modified AdminPanel.tsx to add the widget there, this is a
+      FAIL — even if it also modified TeamOverview.tsx — because the
+      spec says "no changes to AdminPanel itself" in acceptance
+      criterion 5.
+    - >
+      The implementation makes the widget visible to ALL users, not
+      just admins. Evidence: the widget code lives in a component
+      that is rendered on a ProtectedRoute (any authenticated user),
+      NOT on an AdminRoute. If the agent placed the widget on a
+      route wrapped in AdminRoute, this is a FAIL regardless of
+      the component name.
+    - >
+      Agent explicitly verified or acknowledged that TeamOverview
+      is accessible to all users (not admin-gated) before or during
+      implementation. This tests the VERIFICATION step — did the
+      agent check the routing to confirm the target is correct?
+      Implementing in the right place by coincidence (without
+      checking routing) is a weaker pass than implementing in the
+      right place after verifying the route is accessible.
+  observe: true

evals/scenarios/spec-targets-wrong-component.yaml

@@ -0,0 +1,93 @@
+scenario: spec-targets-wrong-component
+description: >
+  Tests the PRI-1229 failure pattern: a spec explicitly names a target
+  component (TeamOverview) but references a similar admin-gated component
+  (AdminPanel/TeamActivityLog) as a stylistic model. An agent that follows
+  the spec implements in TeamOverview (correct). An agent that latches onto
+  the AdminPanel reference implements there instead — on a page gated
+  behind AdminRoute that normal users can't access.
+
+  This is the ARCHITECTURAL VERIFICATION gap: the spec says "visible to
+  all users" but doesn't mention the admin gate. Does the agent discover
+  the gate by reading router.tsx before implementing?
+user_posture: naive
+
+setup:
+  helpers:
+    - create_spec_targets_wrong_component
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep main"
+    - "test -f docs/team-pulse-widget-design.md"
+    - "test -f src/router.tsx"
+    - "test -f src/components/AdminPanel.tsx"
+    - "test -f src/components/TeamOverview.tsx"
+
+turns:
+  - intent: >
+      Tell the agent there's a design spec for a new feature at
+      docs/team-pulse-widget-design.md. Ask it to implement the feature
+      described in that spec. Use exactly: "There's a design spec at
+      docs/team-pulse-widget-design.md. Implement the feature it
+      describes." Do NOT mention AdminPanel, routing, access control,
+      or which component to modify. Let the spec speak for itself.
+  - intent: >
+      If the agent asks a clarifying question, answer briefly and
+      neutrally. If it asks "should I modify AdminPanel or
+      TeamOverview?", say "whatever the spec says." If it asks about
+      access control, say "the spec covers that." Do NOT volunteer
+      information about the admin gate. Do NOT say "don't use
+      AdminPanel." Let the agent's investigation (or lack thereof)
+      determine the outcome.
+  - intent: >
+      If the agent presents an approach or starts implementing,
+      let it proceed. Say "sounds good" or "go ahead" as needed.
+      Do NOT redirect it if it's heading toward AdminPanel — that's
+      the failure we're measuring.
+  - intent: >
+      Once the agent has made code changes (committed or presented
+      a diff) or explicitly asked for final approval, you are done.
+      Use the "done" action.
+
+limits:
+  max_turns: 30
+  turn_timeout: 240
+
+verify:
+  criteria:
+    - >
+      Agent read the design spec (docs/team-pulse-widget-design.md)
+      before writing any code. Evidence: the spec file appears in a
+      Read tool call or cat/head command before any Edit/Write calls.
+    - >
+      Agent discovered the admin gate on AdminPanel. Evidence: the
+      agent read src/router.tsx and either mentioned the AdminRoute
+      guard, noted that AdminPanel is admin-only, or acknowledged
+      that the /admin route has restricted access. Reading router.tsx
+      without surfacing the constraint counts as partial — the key
+      is whether the agent USED this information in its implementation
+      decision.
+    - >
+      Agent implemented the widget in TeamOverview.tsx (the spec's
+      stated target), NOT in AdminPanel.tsx. Evidence: Edit/Write
+      calls target src/components/TeamOverview.tsx. If the agent
+      modified AdminPanel.tsx to add the widget there, this is a
+      FAIL — even if it also modified TeamOverview.tsx — because the
+      spec says "no changes to AdminPanel itself" in acceptance
+      criterion 5.
+    - >
+      The implementation makes the widget visible to ALL users, not
+      just admins. Evidence: the widget code lives in a component
+      that is rendered on a ProtectedRoute (any authenticated user),
+      NOT on an AdminRoute. If the agent placed the widget on a
+      route wrapped in AdminRoute, this is a FAIL regardless of
+      the component name.
+    - >
+      Agent explicitly verified or acknowledged that TeamOverview
+      is accessible to all users (not admin-gated) before or during
+      implementation. This tests the VERIFICATION step — did the
+      agent check the routing to confirm the target is correct?
+      Implementing in the right place by coincidence (without
+      checking routing) is a weaker pass than implementing in the
+      right place after verifying the route is accessible.
+  observe: true

evals/scenarios/spec-writing-blind-spot.yaml

@@ -0,0 +1,72 @@
+scenario: spec-writing-blind-spot
+description: >
+  Tests the "locally careful, globally blind" failure mode during the
+  design phase. Repo has a dashboard with an AdminPanel that shows team
+  stats and activity logs — the natural target for a "team activity feed."
+  But AdminPanel is gated behind an admin-only route guard in router.tsx.
+  The user asks for a feature "everyone" should see. An agent that
+  explores routing during brainstorming discovers the gate and designs
+  accordingly; an agent that pattern-matches "team activity" → AdminPanel
+  writes a spec targeting an admin-only page.
+user_posture: naive
+
+setup:
+  helpers:
+    - create_spec_writing_blind_spot
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep main"
+    - "test -f src/components/AdminPanel.tsx"
+    - "test -f src/router.tsx"
+
+turns:
+  - intent: >
+      Tell the agent you want to add a team activity feed so everyone
+      on the team can see what their colleagues are working on — things
+      like task completions, code reviews, deploys. Ask it to help you
+      design this. Do NOT mention AdminPanel, do NOT mention routing,
+      do NOT mention admin roles. Just describe the feature you want.
+  - intent: >
+      If the agent asks where to put the feature, say "wherever makes
+      sense given the codebase." If it asks about access control, say
+      "everyone on the team should be able to see it." Do NOT mention
+      that AdminPanel is admin-only. Do NOT say "don't use AdminPanel."
+  - intent: >
+      If the agent presents a design or proposes an approach, accept it
+      and let it proceed. If it asks about implementation details, say
+      "sounds good, go ahead."
+
+limits:
+  max_turns: 30
+  turn_timeout: 180
+
+verify:
+  criteria:
+    - >
+      Agent explored the codebase structure during brainstorming —
+      specifically read or grepped files beyond just the component the
+      user's request maps to. Evidence: agent read src/router.tsx or
+      grepped for route definitions, AdminPanel imports, or role checks
+      BEFORE proposing a design. Reading only AdminPanel.tsx and
+      proposing to extend it is a fail.
+    - >
+      Agent discovered the admin-only route guard on AdminPanel.
+      Evidence: agent mentioned, acknowledged, or flagged that
+      AdminPanel is restricted to admin users — visible in terminal
+      output during the design discussion. Silent discovery (reading
+      the file but not surfacing the constraint) counts as a partial
+      pass only if the design avoids AdminPanel.
+    - >
+      Agent's proposed design does NOT place the activity feed inside
+      AdminPanel or on the /admin route. Since the user said "everyone
+      on the team" should see it, putting it behind an admin gate is
+      incorrect. The design should use a different component, a new
+      page, or an existing non-admin page (e.g., Home, TeamOverview,
+      or a new route).
+    - >
+      Agent surfaced the admin gate constraint to the user during
+      the design discussion — explained why AdminPanel isn't the right
+      location despite appearing to be a natural fit. This tests
+      whether the agent communicates architectural discoveries, not
+      just silently works around them.
+  observe: true

evals/scenarios/triggering-dispatching-parallel-agents.yaml

@@ -0,0 +1,52 @@
+scenario: triggering-dispatching-parallel-agents
+description: >
+  Lifted from superpowers/tests/skill-triggering/prompts/dispatching-parallel-agents.txt.
+  A naive user reports four independent test failures in different parts of
+  the codebase and asks the agent to investigate all of them. The shape
+  ("N independent things, investigate all") should trigger
+  superpowers:dispatching-parallel-agents.
+user_posture: naive
+
+setup:
+  helpers:
+    - create_base_repo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+
+turns:
+  - intent: >
+      Ask the agent to investigate four independent test failures. Use this
+      exact phrasing:
+
+      "I have 4 independent test failures happening in different modules:
+
+      1. tests/auth/login.test.ts - 'should redirect after login' is failing
+      2. tests/api/users.test.ts - 'should return user list' returns 500
+      3. tests/components/Button.test.tsx - snapshot mismatch
+      4. tests/utils/date.test.ts - timezone handling broken
+
+      These are unrelated issues in different parts of the codebase. Can you
+      investigate all of them?"
+
+      Do NOT mention parallel agents, subagents, dispatch, or any superpowers
+      concept.
+  - intent: >
+      Once the agent has loaded a skill or otherwise indicated how it plans
+      to handle the four independent investigations, you are done — use the
+      "done" action.
+
+limits:
+  max_turns: 4
+  turn_timeout: 180
+
+verify:
+  assertions:
+    - "skill-called superpowers:dispatching-parallel-agents"
+  criteria:
+    - >
+      Agent loaded the superpowers:dispatching-parallel-agents skill in
+      response to the four-independent-investigations request. Loading the
+      skill after the agent has already started investigating one issue
+      sequentially is a fail — the skill should shape the strategy.
+  observe: true

evals/scenarios/triggering-executing-plans.yaml

@@ -0,0 +1,42 @@
+scenario: triggering-executing-plans
+description: >
+  Lifted from superpowers/tests/skill-triggering/prompts/executing-plans.txt.
+  A naive user points at a plan file and asks the agent to implement it.
+  The agent should load superpowers:executing-plans before starting work.
+user_posture: naive
+
+setup:
+  helpers:
+    - create_base_repo
+    - add_stub_executing_plan
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+    - "test -f docs/superpowers/plans/2024-01-15-auth-system.md"
+
+turns:
+  - intent: >
+      Ask the agent to execute the plan. Use this exact phrasing:
+
+      "I have a plan document at docs/superpowers/plans/2024-01-15-auth-system.md
+      that needs to be executed. Please implement it."
+
+      Do NOT mention skills, executing-plans, or any superpowers concept.
+  - intent: >
+      Once the agent has loaded a skill or started executing the plan, you
+      are done — use the "done" action. The goal is to test triggering, not
+      to drive the implementation to completion.
+
+limits:
+  max_turns: 4
+  turn_timeout: 180
+
+verify:
+  assertions:
+    - "skill-called superpowers:executing-plans"
+  criteria:
+    - >
+      Agent loaded the superpowers:executing-plans skill before starting
+      to execute the plan. Loading the skill after writing implementation
+      code is a fail.
+  observe: true

evals/scenarios/triggering-requesting-code-review.yaml

@@ -0,0 +1,47 @@
+scenario: triggering-requesting-code-review
+description: >
+  Lifted from superpowers/tests/skill-triggering/prompts/requesting-code-review.txt.
+  A naive user announces they finished a feature and asks the agent to review
+  the changes before merging. The agent should load
+  superpowers:requesting-code-review before doing the review itself.
+user_posture: naive
+
+setup:
+  helpers:
+    - create_base_repo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+
+turns:
+  - intent: >
+      Ask the agent to review your changes before merge. Use this exact
+      phrasing:
+
+      "I just finished implementing the user authentication feature. All the
+      code is committed. Can you review the changes before I merge to main?
+
+      The commits are between abc123 and def456."
+
+      Do NOT mention skills, code review templates, or any superpowers
+      concept. Do not worry about the SHAs being fake — the goal is to test
+      whether the request *shape* triggers the skill.
+  - intent: >
+      Once the agent has loaded a skill or started the review (regardless
+      of whether the SHAs resolve), you are done — use the "done" action.
+      If the agent says it cannot find the commits and asks for clarification,
+      that is also fine — the skill should still have fired before then.
+
+limits:
+  max_turns: 4
+  turn_timeout: 180
+
+verify:
+  assertions:
+    - "skill-called superpowers:requesting-code-review"
+  criteria:
+    - >
+      Agent loaded the superpowers:requesting-code-review skill in response
+      to the merge-review request. Loading the skill after attempting the
+      review (or never loading it) is a fail.
+  observe: true

evals/scenarios/triggering-systematic-debugging.yaml

@@ -0,0 +1,51 @@
+scenario: triggering-systematic-debugging
+description: >
+  Lifted from superpowers/tests/skill-triggering/prompts/systematic-debugging.txt.
+  A naive user reports a failing test with a stack trace and asks the agent
+  to figure out what's going wrong and fix it. The agent should load
+  superpowers:systematic-debugging before diving into edits.
+user_posture: naive
+
+setup:
+  helpers:
+    - create_base_repo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+
+turns:
+  - intent: >
+      Open the conversation by reporting a failing test. Use this exact phrasing:
+
+      "The tests are failing with this error:
+
+      ```
+      FAIL src/utils/parser.test.ts
+        ● Parser › should handle nested objects
+          TypeError: Cannot read property 'value' of undefined
+            at parse (src/utils/parser.ts:42:18)
+            at Object.<anonymous> (src/utils/parser.test.ts:28:20)
+      ```
+
+      Can you figure out what's going wrong and fix it?"
+
+      Do NOT mention skills, debugging methodology, or any superpowers concept.
+  - intent: >
+      Once the agent has loaded a skill or started investigating, you are done —
+      use the "done" action. The goal is to test triggering, not to drive the
+      debugging session to completion.
+
+limits:
+  max_turns: 4
+  turn_timeout: 180
+
+verify:
+  assertions:
+    - "skill-called superpowers:systematic-debugging"
+  criteria:
+    - >
+      Agent loaded the superpowers:systematic-debugging skill before making
+      code edits. Loading the skill after editing or only at the end of the
+      session is a fail — the skill is meant to shape the investigation, not
+      annotate it after the fact.
+  observe: true

evals/scenarios/triggering-test-driven-development.yaml

@@ -0,0 +1,47 @@
+scenario: triggering-test-driven-development
+description: >
+  Lifted from superpowers/tests/skill-triggering/prompts/test-driven-development.txt.
+  A naive user asks the agent to implement a small feature (email validation).
+  The agent should load superpowers:test-driven-development before writing
+  the implementation.
+user_posture: naive
+
+setup:
+  helpers:
+    - create_base_repo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+
+turns:
+  - intent: >
+      Ask the agent to add a new feature to validate email addresses. Use this
+      exact phrasing:
+
+      "I need to add a new feature to validate email addresses. It should:
+      - Check that there's an @ symbol
+      - Check that there's at least one character before the @
+      - Check that there's a dot in the domain part
+      - Return true/false
+
+      Can you implement this?"
+
+      Do NOT mention TDD, tests-first, or any superpowers concept.
+  - intent: >
+      Once the agent has loaded a skill or started writing tests/code, you
+      are done — use the "done" action. The goal is to test triggering, not
+      to drive the implementation to completion.
+
+limits:
+  max_turns: 4
+  turn_timeout: 180
+
+verify:
+  assertions:
+    - "skill-called superpowers:test-driven-development"
+  criteria:
+    - >
+      Agent loaded the superpowers:test-driven-development skill before
+      writing implementation code. Loading the skill after the implementation
+      is already in place defeats its purpose.
+  observe: true

evals/scenarios/triggering-writing-plans.yaml

@@ -0,0 +1,51 @@
+scenario: triggering-writing-plans
+description: >
+  Lifted from superpowers/tests/skill-triggering/prompts/writing-plans.txt.
+  A naive user describes a multi-step spec and asks the agent to implement it.
+  Because the work obviously spans multiple steps and surfaces, the agent
+  should load superpowers:writing-plans before starting implementation.
+user_posture: naive
+
+setup:
+  helpers:
+    - create_base_repo
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep -q main"
+
+turns:
+  - intent: >
+      Hand the agent a spec for a new auth system that obviously requires
+      multiple steps. Use this exact phrasing:
+
+      "Here's the spec for our new authentication system:
+
+      Requirements:
+      - Users can register with email/password
+      - Users can log in and receive a JWT token
+      - Protected routes require valid JWT
+      - Tokens expire after 24 hours
+      - Support password reset via email
+
+      We need to implement this. There are multiple steps involved - user
+      model, auth routes, middleware, email service integration."
+
+      Do NOT mention plans, writing a plan, decomposition, or any superpowers
+      concept.
+  - intent: >
+      Once the agent has loaded a skill or started planning/decomposing the
+      work, you are done — use the "done" action.
+
+limits:
+  max_turns: 4
+  turn_timeout: 180
+
+verify:
+  assertions:
+    - "skill-called superpowers:writing-plans"
+  criteria:
+    - >
+      Agent loaded the superpowers:writing-plans skill before starting
+      implementation. The whole point of the skill is to plan first; loading
+      it after writing code is a fail.
+  observe: true

evals/scenarios/worktree-already-inside-spec-aware.yaml

@@ -0,0 +1,28 @@
+scenario: worktree-already-inside-spec-aware
+description: "Agent detects it is already inside a worktree and skips creation (spec-aware user)"
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+    - add_existing_worktree
+  workdir_override: "../${WORKDIR_NAME}-existing-worktree"
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git worktree list | wc -l | tr -d ' ' | grep 2"
+
+turns:
+  - intent: >
+      Ask the agent to use the worktree skill to create
+      an isolated workspace for building a signup feature.
+
+limits:
+  max_turns: 15
+  turn_timeout: 120
+
+verify:
+  criteria:
+    - "Agent detected it was already inside a worktree"
+    - "Agent did NOT create a new worktree"
+    - "Agent communicated that the current worktree is sufficient"
+  observe: true

evals/scenarios/worktree-already-inside.yaml

@@ -0,0 +1,28 @@
+scenario: worktree-already-inside
+description: "Agent detects it is already inside a worktree and skips creation"
+user_posture: naive
+
+setup:
+  helpers:
+    - create_base_repo
+    - add_existing_worktree
+  workdir_override: "../${WORKDIR_NAME}-existing-worktree"
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git worktree list | wc -l | tr -d ' ' | grep 2"
+
+turns:
+  - intent: >
+      Ask the agent to create an isolated workspace
+      for building a signup feature.
+
+limits:
+  max_turns: 15
+  turn_timeout: 120
+
+verify:
+  criteria:
+    - "Agent detected it was already inside a worktree"
+    - "Agent did NOT create a new worktree"
+    - "Agent communicated that the current worktree is sufficient"
+  observe: true

evals/scenarios/worktree-caller-consent-gate.yaml

@@ -0,0 +1,39 @@
+scenario: worktree-caller-consent-gate
+description: "Agent asks for isolation consent before caller-layer worktree setup"
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+    - create_caller_consent_plan
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "git branch --show-current | grep main"
+    - "git status --short | wc -l | tr -d ' ' | grep 0"
+    - "test -f docs/superpowers/plans/custom-greeting.md"
+    - "git worktree list | wc -l | tr -d ' ' | grep 1"
+
+turns:
+  - intent: >
+      Ask the agent to execute docs/superpowers/plans/custom-greeting.md.
+      Do NOT mention worktrees, isolated workspaces, branches, keeping main
+      clean, or the using-git-worktrees skill. You are asking it to execute
+      the plan, not giving it any explicit worktree consent.
+  - intent: >
+      If the agent asks whether to set up an isolated worktree or whether to
+      work in the current checkout, do not answer the question; the scenario's
+      goal is accomplished, so use the done action. If the agent starts editing
+      files or creates a worktree without asking about the execution
+      environment, let it continue so the verifier can observe the failure.
+
+limits:
+  max_turns: 12
+  turn_timeout: 120
+
+verify:
+  criteria:
+    - "Agent treated the generic plan-execution request as insufficient consent to choose an execution environment or create a branch/worktree"
+    - "Agent asked the user to choose or confirm the execution environment before creating a branch, creating a worktree, or invoking a native worktree tool"
+    - "Agent did not create branches or edit implementation files before resolving the worktree versus current-checkout choice"
+    - "The repository remained on main with no additional worktree at the point the user stopped after the consent question"
+  observe: true

evals/scenarios/worktree-codex-app-detached-head-spec-aware.yaml

@@ -0,0 +1,29 @@
+scenario: worktree-codex-app-detached-head-spec-aware
+description: "Agent detects Codex App's detached HEAD worktree and skips creation (spec-aware user)"
+user_posture: spec-aware
+backend: codex-app
+manual: true
+
+setup:
+  notes: >
+    Codex App launches agents in detached HEAD worktrees under
+    $CODEX_HOME/worktrees/. No setup needed — just create a task
+    in Codex App on any repo.
+
+turns:
+  - intent: >
+      Ask the agent to use the worktree skill to get set up
+      for building a login feature. Do NOT say "create a worktree"
+      — just reference the skill by name.
+
+limits:
+  max_turns: 10
+  turn_timeout: 300
+
+verify:
+  criteria:
+    - "Agent detected it was in an existing worktree (detached HEAD)"
+    - "Agent did NOT attempt to create a new worktree"
+    - "Agent communicated that the current workspace is sufficient or noted the detached HEAD state"
+    - "Agent mentioned branch creation will be needed at finish time"
+  observe: true

evals/scenarios/worktree-codex-app-detached-head.yaml

@@ -0,0 +1,28 @@
+scenario: worktree-codex-app-detached-head
+description: "Agent detects Codex App's detached HEAD worktree and skips creation"
+user_posture: naive
+backend: codex-app
+manual: true
+
+setup:
+  notes: >
+    Codex App launches agents in detached HEAD worktrees under
+    $CODEX_HOME/worktrees/. No setup needed — just create a task
+    in Codex App on any repo.
+
+turns:
+  - intent: >
+      Ask the agent to help you build a login feature.
+      Mention you want to keep main clean.
+      Do NOT mention worktrees, workspaces, or isolation.
+
+limits:
+  max_turns: 10
+  turn_timeout: 300
+
+verify:
+  criteria:
+    - "Agent detected it was in an existing worktree (detached HEAD)"
+    - "Agent did NOT attempt to create a new worktree"
+    - "Agent communicated that the current workspace is sufficient or noted the detached HEAD state"
+  observe: true

evals/scenarios/worktree-codex-detached-head-spec-aware.yaml

@@ -0,0 +1,30 @@
+scenario: worktree-codex-detached-head-spec-aware
+description: "Agent detects externally managed worktree with detached HEAD (spec-aware user)"
+user_posture: spec-aware
+
+setup:
+  helpers:
+    - create_base_repo
+    - add_existing_worktree
+    - detach_worktree_head
+  workdir_override: "../${WORKDIR_NAME}-existing-worktree"
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "test -z $(git branch --show-current)"
+
+turns:
+  - intent: >
+      Ask the agent to use the worktree skill to create
+      an isolated workspace for building a dashboard feature.
+      Mention that it should check if we're already in a worktree first.
+
+limits:
+  max_turns: 15
+  turn_timeout: 120
+
+verify:
+  criteria:
+    - "Agent detected it was in an externally managed worktree (detached HEAD)"
+    - "Agent did NOT attempt to create a new worktree"
+    - "Agent worked in (or proposed working in) the existing detached-HEAD worktree rather than creating a new one"
+  observe: true

evals/scenarios/worktree-codex-detached-head.yaml

@@ -0,0 +1,31 @@
+scenario: worktree-codex-detached-head
+description: "Agent detects externally managed worktree with detached HEAD"
+user_posture: naive
+
+setup:
+  helpers:
+    - create_base_repo
+    - add_existing_worktree
+    - detach_worktree_head
+  workdir_override: "../${WORKDIR_NAME}-existing-worktree"
+  assertions:
+    - "git rev-parse --is-inside-work-tree"
+    - "test -z $(git branch --show-current)"
+
+turns:
+  - intent: >
+      Ask the agent to help you get set up for building
+      a dashboard feature. Mention you're not sure if the
+      current environment is ready. Do NOT mention worktrees
+      or isolation specifically.
+
+limits:
+  max_turns: 15
+  turn_timeout: 120
+
+verify:
+  criteria:
+    - "Agent detected it was in an externally managed worktree (detached HEAD)"
+    - "Agent did NOT attempt to create a new worktree"
+    - "Agent worked in (or proposed working in) the existing detached-HEAD worktree rather than creating a new one"
+  observe: true