mirror of
https://github.com/block/buzz.git
synced 2026-08-18 06:50:31 +02:00
261c460761
## Problem
Two failure modes from the `tb21-glm52-crusoe-1` benchmark run (GLM-5.2
solo, TB2.1) wedged or killed 13 of 89 trials without the model being at
fault:
1. **Conversation poisoning on text-only endpoints.** Crusoe's
serverless `crusoeai/GLM-5.2-NVFP4` rejects any request whose history
contains an image with `400: ... is not a multimodal model`. The
recovery machinery for exactly this case already exists —
`AgentError::UnsupportedImageInput` → `replace_unsupported_images()`
strips the image blocks, marks the tool result as an error, substitutes
a text placeholder, and continues the turn. But classification only
matched OpenRouter's 404 body (`no endpoints found that support image
input`) and was only consulted on the 404 arms. The Crusoe 400 fell
through to terminal `AgentError::Llm`: the image stayed in history,
every subsequent call failed identically, buzz-acp rode its 10-retry
ladder (~40 min), and the trial idled to budget death. Measured blast
radius: **8 trials wedged, 12.7h aggregate idle-after-poison.**
2. **Bounded agent rounds in benchmark trials.** The harness default
`DEFAULT_MAX_AGENT_ROUNDS = 32` ended solo trials mid-work when turns
rotated (thinking-heavy models hit max_tokens rotation fast; 4 trials
died this way). Benchmark trials already have a wall-clock budget as the
real limit — the round cap only converts recoverable rotation into trial
death.
## Fix
- `is_unsupported_image_input_error()` also matches the verbatim `is not
a multimodal model` body. Matcher stays deliberately tight (same
doctrine as `is_context_length_error`): misclassifying a generic 400 as
recoverable would mutate history for an error that removing images
cannot fix.
- Both status ladders — shared `post()` and `openrouter_post()` —
consult it on their 400 arms and return the typed
`UnsupportedImageInput` (OpenAI-compatible providers report this as 400;
a BYOK/passthrough upstream can surface the provider's own 400 through
OpenRouter).
- Harness `DEFAULT_MAX_AGENT_ROUNDS` → `0` (unbounded —
`BUZZ_AGENT_MAX_ROUNDS=0` is the agent config's documented unbounded
value). Per-agent `budget.max_calls` in manifests still overrides.
## Acceptance
- A 400 with the image-rejection body reaches the existing image-strip
recovery path instead of wedging the session — asserted through
`complete()` (covers the return path into the convergence mapper) and at
the `openrouter_post` terminal, both proving single-attempt (a
deterministic capability rejection must never be retried).
- Ordinary 400s stay terminal `AgentError::Llm` (existing negative tests
unchanged).
- Benchmark trials run unbounded rounds by default; python tests updated
for 0-is-legal with a negative arm at -1.
## Verification
- `cargo test -p buzz-agent`: 427 + 18 + 20 + 15 + 8 + 1 + 48 passed, 0
failed (full package, 3 consecutive clean runs)
- `cargo clippy -p buzz-agent --all-targets`, `cargo fmt --check`: clean
- `uv run --extra dev pytest tests/` in harbor-buzz-orchestra: 35 passed
- Pre-push hooks (full workspace rust-tests + desktop-tauri-checks)
green on rustc 1.95.0 at head b0438602
Signed-off-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>