mirror of
https://github.com/block/buzz.git
synced 2026-08-18 06:50:31 +02:00
Will PflegerandGitHub
346ae8cadc
Non-streaming LLM calls (`"stream": false`) through slow model/provider combinations routinely take longer than the fixed `BUZZ_AGENT_LLM_TIMEOUT_SECS` window (default 240 s) to return their first response byte. The retry loop then re-ran the identical 240 s bet three times, failed the turn, and the ACP harness requeued the whole turn from scratch: agents spent 30+ minutes producing nothing while every attempt died at the same wall. And because the LLM path only logged WARN lines on failure, a healthy-but-slow call was indistinguishable from a wedged one. ### Timeout handling - **Per-attempt escalation**: the per-request budget doubles after each timeout failure (`base × 2^n`, capped at `max(1200 s, base)` — `escalated_timeout()` in `llm.rs`), shared by the main `post()` loop and `openrouter_post()`. Non-timeout retryables (429/5xx/connect) do not escalate. A call that needs six minutes now succeeds on a later attempt instead of never. - **Per-request total timeouts**: enforcement moved from the client-level `read_timeout` to `RequestBuilder::timeout()` on each LLM request, so escalated budgets aren't silently floored by the shared client and each attempt's bound covers connect through body completion. Timeout error messages were updated to match the new semantics and still point at `BUZZ_AGENT_LLM_TIMEOUT_SECS`. ### Observability - One INFO line per completed LLM call: model, provider, `duration_ms`, `input_tokens`, `cached_input_tokens`, `output_tokens`. Slowness and prompt-cache effectiveness are now visible in harness logs without waiting for a failure, and `None` vs `0` token reports stay distinguishable. Handoff summarization calls log the same line with duration only. - The agent main loop wraps the call in a `session_id` tracing span so each line is attributable to a session. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com>