mirror of
https://github.com/block/buzz.git
synced 2026-08-18 06:50:31 +02:00
5bf78671f4
## Problem
A provider can return HTTP 200 with a **truncated JSON body** — cleanly
closed connection, correct framing, content cut off mid-value. Both LLM
HTTP loops treated this as a terminal error on the first attempt:
`AgentError::Llm("json: EOF while parsing a value")`, surfaced as code
-32000 at the ACP boundary, killing the agent turn before it produced
anything.
Observed live in a tb2.1 bench trial (write-compressor, tb21-twins-1):
deepseek via OpenRouter returned a truncated body, the agent died
mid-prompt with 0 turns completed, and the trial scored 0 on a provider
hiccup.
Meanwhile the same loops already retry timeouts, 429s, 5xxs, 499s, and
mid-body stream stalls — a truncated-but-complete body was the one
transient upstream fault that fell through to terminal.
## Fix
In both `post()` and `openrouter_post()`
(`crates/buzz-agent/src/llm.rs`): when the fully-received success body
fails `serde_json::from_slice`, `continue` the **existing** retry loop
instead of returning terminal — same `MAX_RETRIES` (3) bound, same
`backoff_with_jitter`. On exhaustion, the error goes through
`terminal_llm_error` so it carries cumulative duration + attempt count
like every other retried failure (previously the `json:` error carried
neither).
`post_anthropic` routes through `post()`, so
Anthropic/OpenAI/Databricks/mesh and OpenRouter are all covered.
## Why this cannot re-run a tool call
Hard requirement: tool calls are not idempotent, and this change must
not introduce any possibility of replaying one.
1. **The retry lives inside the HTTP POST helper, below the parse
boundary.** Tool calls are only ever extracted from a *successfully
parsed* response value
(`parse_openai`/`parse_anthropic`/`parse_responses`, all downstream of
these helpers' `Ok` return). A malformed body never parses, therefore no
tool call was ever extracted from it, therefore nothing downstream of it
ever dispatched.
2. **What is re-sent is the completion request itself** — the identical
`body_bytes` captured once at function entry. Sending a completion
request executes no tools; it asks the model for the next message.
3. **Same safety class as existing behavior.** The loop already re-sends
this identical request on 429/5xx/timeout/stream-stall; this adds one
more transient-fault arm to the same loop with the same bytes.
## Tests
Three new tests mirroring the existing 499/dropped-connection fixtures
(raw `TcpListener` stubs):
- `post_retries_malformed_json_body_and_succeeds` — truncated 200 body
on attempt 1, valid JSON on attempt 2; asserts success and **exactly 2**
server-side requests
- `post_exhausts_retries_on_persistent_malformed_json` —
always-truncated body; asserts exactly `MAX_RETRIES` attempts and a
terminal error carrying `json:` + cumulative/attempt context
- `openrouter_post_retries_malformed_json_body_and_succeeds` — same
recovery through OpenRouter's separate loop
Full `cargo test -p buzz-agent` green at e7a5d7bb (430 lib + all
integration targets, 0 failures); `cargo fmt` + `clippy --all-targets`
clean.
Originating conversation: buzz-benchmarking channel, thread 397a992d.
Signed-off-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>