mirror of
https://github.com/block/buzz.git
synced 2026-08-18 06:50:31 +02:00
Michael Neale
eea537f6be
P5 could never pass: install_fake_shell returned the bin directory, but resolve_bash requires BUZZ_SHELL to name a file, so it silently fell back to real bash and the marker was never written. The fake shell also matched `-lc` while shell_flag() emits `-c` on Unix. Return the executable, accept both flags, and append rather than overwrite so a probing model does not erase the recorded send. P5 stops at the shell boundary, so it cannot show a reply ever lands. Add P6: the agent runs the genuine `buzz messages send` (dev-mcp's multicall shim, real buzz-cli) against a running relay, and the harness then queries the relay itself — the agent's prose is not evidence. Infra-gated on BUZZ_E2E_RELAY_URL / _PRIVATE_KEY / _CHANNEL; skipped when unset so P1-P5 still run without Postgres and Redis. Also drive the agent with the real base_prompt.md instead of a terse hand-written string. A short prompt is not a proxy for production: tool adherence in small models depends on it, so testing with 56 characters where Buzz injects ~3.2k tokens measures the harness, not Buzz. Signed-off-by: Michael Neale <michael.neale@gmail.com>