mirror of
https://github.com/block/buzz.git
synced 2026-08-18 06:50:31 +02:00
wpfleger/desktop-admin-surface
59
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
72d56e7bd3 |
fix(agent): raise output limit and allow 3 recoveries (#5475)
## Summary Raise the built-in output and recovery defaults so long-running agents have more room to finish useful work instead of terminating after repeated 32,768-token reasoning-only responses. - Raise `BUZZ_AGENT_MAX_OUTPUT_TOKENS` from 32,768 to 65,536 - Raise the finite output-truncation recovery allowance from 2 to 3 via `BUZZ_AGENT_MAX_TOKEN_RECOVERIES`; `0` still disables recovery - Strengthen the recovery prompt so the model stops prolonged reasoning, uses tools immediately, and builds scripts or artifacts in small verifiable steps - Preserve the safety invariant that incomplete truncated tool calls are discarded and never executed - Keep proactive handoff independently at 90% of `BUZZ_AGENT_MAX_CONTEXT_TOKENS` (180,000 tokens with the 200,000 default), regardless of the output allowance - Add request-loop and configuration regressions for exact-N recovery, disabled recovery, successful tool-first recovery, discarded truncated calls, and finite round bounds `BUZZ_AGENT_MAX_OUTPUT_TOKENS` remains an explicit per-agent deployment setting. Operators should configure it at or below the served model's output limit; this PR does not perform live provider capability discovery or automatic clamping. **Risk:** Medium — this increases the default request size and permits one additional recovery attempt by default. Recovery remains finite and bounded by `BUZZ_AGENT_MAX_ROUNDS`. Deployments whose served model rejects 65,536 output tokens must set a lower per-agent value. Current output limits - model - output token max - DeepSeek V4 Flash - 384,000 tokens - Qwen 3.8 (Max) - 131,072 tokens - GLM 5.2 - 131,072 tokens - GPT 5.6 - 128,000 tokens - Claude Opus 5 - 128,000 tokens - Gemini 3.6 Flash - 65,536 tokens - Kimi K3 (Moonshot)- 131,072 tokens ### Related issue None found. Originating benchmark analysis: `buzz://message?channel=c3252dd2-0142-4e01-88c7-a2183c3960a5&id=91e991aab5fd49094583c3937477f6c12db57a41d86edf7fd4745d0d57d10017` ### Testing - `cargo fmt --all -- --check` - `cargo test -p buzz-agent` — 595 passed, 0 failed, 0 ignored at `bd6de557b367850f50325bafdd3c046131942bef` - `cargo clippy -p buzz-agent --all-targets -- -D warnings` - Previously failing `cancelled_turn_with_usage_emits_notification_before_response` passed alone and in the full rerun - Push hooks passed: organization guard, branch skew, Rust tests, and Desktop Tauri checks ### Update — 2026-08-11 Per review feedback, the recovery default is 3. The OpenRouter live `/models` output-cap discovery, cache, request clamp, and related tests/documentation were removed. Per-agent output configuration is now the sole output-cap mechanism. Proactive handoff and its pre-usage byte fallback now depend only on 90% of `BUZZ_AGENT_MAX_CONTEXT_TOKENS`; with the 200,000 default, the handoff threshold is 180,000 regardless of `BUZZ_AGENT_MAX_OUTPUT_TOKENS`. Generated with Brainy Bumble ### Targeted validation — 2026-08-11 Ran the exact PR binary once on each of the 11 benchmark tasks causally affected by the previous 32,768-token ceiling, using OpenRouter with `deepseek/deepseek-v4-flash-0731` pinned to Fireworks and maximum reasoning effort. Relay-429 collection failures were excluded and rerun at concurrency 2. - **6/11 passed:** `circuit-fibsqrt`, `feal-linear-cryptanalysis`, `model-extraction-relu-logits`, `path-tracing`, `schemelike-metacircular-eval`, and `sqlite-db-truncate` - **5/11 reached the benchmark deadline:** `adaptive-rejection-sampler`, `dna-assembly`, `path-tracing-reverse`, `regex-chess`, and `write-compressor` - `regex-chess` reached exactly 65,536 output tokens, triggered one output-limit recovery, and then reached the deadline. This directly confirms that the larger ceiling and recovery path were active, but not that recovery guarantees completion. For context, ten of these tasks were 0/5 in the historical baseline; `sqlite-db-truncate`, the clean control, was 4/5. This is targeted one-attempt-per-task validation rather than a statistically powered comparison. The result should not be attributed solely to the recovery default of 3: this PR also raises the output ceiling and strengthens recovery behavior, and OpenRouter routing conditions may differ from the historical direct-Fireworks runs. Generated with Brainy Bumble --------- Signed-off-by: Atish Patel <atish@squareup.com> Signed-off-by: Brainy Bumble <0ed7657b57c0e8a9f5288390dd6c8d5d0a3a06abe9b01b9006814f52077d6cdf@buzz.block.builderlab.xyz> Signed-off-by: Hardworking Honey <c5c455215c2506cb8ba776518cec804af62d3a0526e32d496a22072e395042b9@buzz.block.builderlab.xyz> Co-authored-by: Brainy Bumble <0ed7657b57c0e8a9f5288390dd6c8d5d0a3a06abe9b01b9006814f52077d6cdf@buzz.block.builderlab.xyz> Co-authored-by: Hardworking Honey <c5c455215c2506cb8ba776518cec804af62d3a0526e32d496a22072e395042b9@buzz.block.builderlab.xyz> |
||
|
|
240cdd3ea1 |
chore: mesh upgrade, clean up legacy special case code, simplify model selection for mesh (#5289)
Shared compute now has exactly two model choices: MeshLLM's virtual `mesh` model, or a model you name. Buzz picks between them in one place, and buzz-agent no longer knows meshes exist. ## What changed - **MeshLLM v0.74.0 → v0.75.1.** v0.75.0 added `degrade_to_single_model`, so a `model=mesh` request is answered by one served model when there is no committee to form, instead of failing. v0.75.1 adds Mesh-LLM#1196, which skips stale pre-0.75 runtime cache entries rather than aborting startup on them — without it, anyone who had run mesh on 0.73/0.74 could not start. - **Deleted the client-side mesh catalog probe.** buzz-agent used to poll `/v1/models` (5s TTL, 30s cooldown, two-observation debounce) to decide whether `mesh` was safe to send. MeshLLM now decides per request, so the polling, its hysteresis, and its 503 fallback are gone. - **One mapping point.** `relay_mesh_wire_model()` turns the stored value into a wire name: `auto` becomes `mesh`, a named model passes through. The spawn env, the ACP harness, and the readiness probe all use it, so they cannot disagree — previously `BUZZ_ACP_MODEL` and the probe both said `auto`, a name the mesh does not advertise. - **Removed the `nostr-relay-pool` advisory exception.** #5404 allowed RUSTSEC-2026-0243 "after mesh-llm migrates to nostr-sdk >= 0.45". v0.75.1 does, so the retired crate is gone from both lockfiles and the exception would only mask a future advisory for it. - **Deleted `scripts/ensure-mesh-native-runtime.sh`** and its six justfile call sites. It built llama.cpp from source into the runtime cache; the app already downloads the signed release runtime itself, and CI never called it. ## Why it is better **−639 lines of Rust.** Availability is decided by the node that knows the answer, per request, instead of by a client cache that could be stale for up to 30 seconds. A second worker joining now takes effect on the next request rather than after two confirming probes. ## Behaviour change A 503 on an explicit `mesh` request takes the ordinary transport retry under the same model instead of failing over to a second one — there is no second model to fail over to now. MoA repairs partial committee results internally before it reaches that point. ## Validation `crates/buzz-relay/examples/mesh_agent_e2e.rs` now sends `mesh` where it previously sent `auto` or the physical model id, so no leg was covering what Buzz actually puts on the wire. 4/4 on gemma-4-E4B, gemma-4-26B-A4B, and Qwen3-8B — including a real ACP tool call through `mesh` into buzz-dev-mcp, asserted by reading the written file back off disk. Hand-tested in the desktop app on both gemma-4 sizes: picked Auto, agent logged `model_id=mesh`, replied in channel. ## Not covered A committee that forms and then loses a worker returns 502, and that needs two workers to reproduce — not testable on one machine. --------- Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com> |
||
|
|
5e4d0fe925 |
fix(buzz-agent): harden Databricks OAuth token cache and callback (#5534)
Hardens the Databricks PKCE OAuth code in `crates/buzz-agent/src/auth.rs`. Two fixes. ## Token cache is owner-only across its whole lifecycle, and race-safe The PKCE cache holds both the access and refresh tokens, but `save()` wrote it with a bare `fs::write` + `fs::rename`. Under a `022` umask the file landed world-readable, and the fixed `*.json.tmp` temp name races across concurrent savers sharing `$HOME` — one writer's `rename` can fail on another's half-written temp. **On write**, `write_private_cache()` creates a temp file with owner-only permissions from the moment it exists — mode `0o600` on Unix via `OpenOptions::mode` — writes and fsyncs it, then renames over the destination. The rename swaps the inode wholesale, so a pre-existing cache file with loose permissions is *replaced* by the new private inode rather than inheriting its mode. `unique_suffix()` (getrandom, timestamp fallback) gives each write a distinct temp name, and a drop guard removes the temp on any failure path. **On load**, owner-only is enforced as a cache lifecycle invariant, not just a write-path property. A world-readable cache left by an older buzz-agent was previously read straight into memory and returned on the fresh cache-hit path without ever invoking `save()`, so a token file with no advertised expiry could stay exposed indefinitely. `read_cache()` now funnels every load — initial and cross-process re-reads — through `read_private_cache()`, which on Unix opens with `O_NOFOLLOW` (kernel-level symlink refusal, no stat/open TOCTOU), requires a regular file, and `fchmod`s the pinned handle to `0o600` when any group/other bit is set. A cache that cannot be secured is treated as absent, so callers fail closed to a fresh flow rather than trusting an exposed file. ## OAuth callback no longer reflects untrusted input The localhost callback embedded the untrusted `error` query param straight into the HTML response — an XSS sink on the redirect page — and routed that same raw value into the error string that reaches the logs. `callback_outcome()` is now a pure function returning `(result, static_page)`: the browser always sees a fixed literal page that embeds no request parameter, and failure detail travels only through the result channel. `sanitize_callback_detail()` strips control characters (CR/LF log-line injection) and caps length before that detail enters the error string bound for the logs. ## Deferred: Windows owner-only ACLs Windows owner-only protection is out of scope for this change. The goose-parity route (`CreateFileW` with an owner-only SDDL `D:P(A;;FA;;;OW)`) requires `unsafe` FFI, which this crate's `#![forbid(unsafe_code)]` prohibits; reconciling that conflict is a separate decision. Both platform seams — `create_private_temp_file` (write) and `read_private_cache` (load) — have a `#[cfg(not(unix))]` branch that relies on the default per-user ACLs and is the drop-in point if Windows protection is added later. No new dependency and no `unsafe` are introduced here. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <41ea58f1e64c243627e8acde7c89be667052ee6e17d8f021c1195be4324ebf04@buzz.block.builderlab.xyz> |
||
|
|
5e4c05f90b |
feat(desktop): NIP-AM agent-usage backend — P2 emission/transport/archive + P4a aggregation/D6 (#4000)
## What Implements Phases 2 and 4a of the Usage v2 plan (plan events `d0268cd0`/`0e95b035`), extending the archive backend to emit, transport, archive, and aggregate both cache categories and billing identity fail-closed. ### P2 — emission, transport, archive **Tri-state accumulators** (`Unseen`/`Exact`/`Unknown`) for cache-read and cache-write in `buzz-agent` turn and session state. Absent field = Unknown (never zero) through the full pipeline. No `unwrap_or(0)` on the cache path. Both cache folds are gated on usage-bearing responses (same gate as the total-state and identity folds) — a response with no usage at all must not poison either accumulator. **Overflow-aware input token parsing and accumulation** — closed end-to-end from parse through wire to ACP: - `sum_usage()` returns `SumUsageResult` (`Exact(u64)` | `Overflow`) — checked arithmetic, never clamps. `anthropic_input_tokens()` returns `Option<SumUsageResult>` since it sums three fields (`input_tokens + cache_read_input_tokens + cache_creation_input_tokens`) that can collectively overflow. Single-field callers (`prompt_tokens`, `completion_tokens`, etc.) convert via `.into_exact()` — their single-field sums cannot overflow. - `LlmResponse.input_tokens_overflowed: bool` propagates the parse-layer signal into the run loop. When set, `input_tokens` is `None` (clamped value discarded), the context-gate baseline (`last_request_input_tokens`) is frozen at its prior reading, and `turn_input_tokens` is poisoned to `TurnIOState::Poisoned` before any emission — including mid-turn `emit_usage_update` calls. A dedicated enum on `LlmResponse.input_tokens` would ripple into ~20 existing test assertions on `r.input_tokens == Some(...)`; the bool flag confines the change to the two call sites that check it. - `TurnIOState` (`Unseen`/`Exact`/`Poisoned`) for input and output: per-round fold uses `checked_add`; overflow poisons permanently at turn and session level, no healing. Absence does not poison (pass-2-cleared contract unchanged). Wire emission omits `accumulatedInputTokens`/`accumulatedOutputTokens` when poisoned — never null, never `u64::MAX`. ACP treats absent = publisher-poisoned: `delta_reliable: false`, null turn fields, null cumulative for that category; session cumulative stays unknown for all subsequent turns once poisoned. **Conditional wire emission** for `accumulatedCachedInputTokens` and new `accumulatedCacheWriteTokens`: fields are omitted when the cumulative is Unseen or Unknown. ACP `_goose/unstable/session/update` contract documented next to the payload with tests for all absence/zero variants. **`PricingIdentity` stamping (publisher-side)**: - `pricing_authority()`: canonical parsed-URL endpoint comparison against the official allowlist — HTTPS only, exact allowlisted host (lookalike-safe), default port (omitted or explicit :443), required API base path, rejects userinfo/query/fragment/path-prefix lookalikes. - Model: the actually-requested `request_model` after mesh/auto resolution (not `effective_model_str`). - Turn discipline: identity retained only while ALL usage in the current turn carries one identical proven identity; any mismatch, unproven-usage-bearing response, or unpaired cumulative snapshot poisons to absent; a later matching notification does not heal a mixed turn. **ACP `UsageTracker` identity fold**: per-in-flight-turn tri-state identity accumulator replacing last-update-wins. Any absent identity on a token-advancing notification or exact mismatch poisons to absent; poison survives later updates; reset in `begin_turn()`/`take()`; reset also when a request fails (baseline cleared so preflight gate cannot stay frozen sub-threshold on retries). **M3 migration**: adds `turn_cache_write_tokens`, `cumulative_cache_write_tokens`, `pricing_authority`, `pricing_model`, `pricing_cache_class` to `agent_metric_index`. Additive, idempotent, guarded per-column by marker. M2 migration also guarded per-column (turn and cumulative cache-read columns checked and added independently; marker commits only after both are present). Fresh-DB schema includes all columns. **First-turn baselines**: `seed_zero_baseline` seeds `last_input: Some(0)`, `last_output: Some(0)`, `last_cached_input: Some(0)`, `last_cache_write: Some(0)`, and `last_total: Some(0)` — all have the known-zero-at-spawn argument. Absent fields from incoming snapshots still produce unknown (tri-state unchanged). Sessions buzz-acp did not spawn (no seed) remain fail-closed on turn one. **`ReportedUsage` TS mirror**: `cacheReadTokens`, `cacheWriteTokens`, `freshInputTokens` added to `tauriArchive.ts` as `UsageField` members, field-for-field with the Rust struct. ### P4a — aggregation layer **Extended S-1 ladder** to cache-read and cache-write via the same `ladder_token` path as the existing token fields. **`freshInputTokens` derivation**: checked arithmetic, fail-closed — absent cache fields produce Unknown (not zero), overflow and `cacheRead+cacheWrite > input` both produce `incomplete: true`. Aggregated as a `UsageField`. **D6 comparator**: `sort_value()` = provider total when known, else `input+output` when both known, else `None` (unknown-last). Replaces the prior total-only comparator for both agent-level and model-level sort. Ships a pinned test vector that the TS render layer (P5) must match. ## Test coverage - `buzz-agent`: 440 lib + 15 integration (golden_transcripts) — includes 13 new `cache_total_state_tests`; 14 new `turn_io_state_tests`; 3 new `sum_usage_*` tests (exact single-field, exact two-field, overflow signals correctly); 3 new `parse_anthropic_*` tests (overflow flag set + value cleared, normal sum no flag, absent usage no flag); end-to-end golden transcript drives real subprocess with Anthropic-shaped `input_tokens: u64::MAX, cache_read: 1` response and asserts `accumulatedInputTokens` absent from the emitted `usage_update` — no logic duplication; 3 wire pin tests; 4 `fold_pricing_identity_*` tests; `pricing_authority()` explicit-:443 acceptance - `buzz-acp`: 700 tests (691 lib + 9 integration) — 4 new usage tests (absent input → unreliable+null; absent output → unreliable+null; goose-shaped both present unchanged; poison mid-session); 3 ACP behavior tests; 7 pool lifecycle tests - Desktop (Rust): 2259+ tests — 14 new P4a pinned tests; 2 M3 round-trip tests; 1 serde key-shape test; 2 M2 partial-schema migration tests; first-turn cache round-trip test ## Related PRs - P1 NIP-AM spec: [#4632](https://github.com/block/buzz/pull/4632) - P3 pricing table: [#4629](https://github.com/block/buzz/pull/4629) - UI (P5): [#4001](https://github.com/block/buzz/pull/4001) --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
|
5bf78671f4 |
fix(agent): retry LLM completion on malformed 2xx JSON body (#5351)
## Problem
A provider can return HTTP 200 with a **truncated JSON body** — cleanly
closed connection, correct framing, content cut off mid-value. Both LLM
HTTP loops treated this as a terminal error on the first attempt:
`AgentError::Llm("json: EOF while parsing a value")`, surfaced as code
-32000 at the ACP boundary, killing the agent turn before it produced
anything.
Observed live in a tb2.1 bench trial (write-compressor, tb21-twins-1):
deepseek via OpenRouter returned a truncated body, the agent died
mid-prompt with 0 turns completed, and the trial scored 0 on a provider
hiccup.
Meanwhile the same loops already retry timeouts, 429s, 5xxs, 499s, and
mid-body stream stalls — a truncated-but-complete body was the one
transient upstream fault that fell through to terminal.
## Fix
In both `post()` and `openrouter_post()`
(`crates/buzz-agent/src/llm.rs`): when the fully-received success body
fails `serde_json::from_slice`, `continue` the **existing** retry loop
instead of returning terminal — same `MAX_RETRIES` (3) bound, same
`backoff_with_jitter`. On exhaustion, the error goes through
`terminal_llm_error` so it carries cumulative duration + attempt count
like every other retried failure (previously the `json:` error carried
neither).
`post_anthropic` routes through `post()`, so
Anthropic/OpenAI/Databricks/mesh and OpenRouter are all covered.
## Why this cannot re-run a tool call
Hard requirement: tool calls are not idempotent, and this change must
not introduce any possibility of replaying one.
1. **The retry lives inside the HTTP POST helper, below the parse
boundary.** Tool calls are only ever extracted from a *successfully
parsed* response value
(`parse_openai`/`parse_anthropic`/`parse_responses`, all downstream of
these helpers' `Ok` return). A malformed body never parses, therefore no
tool call was ever extracted from it, therefore nothing downstream of it
ever dispatched.
2. **What is re-sent is the completion request itself** — the identical
`body_bytes` captured once at function entry. Sending a completion
request executes no tools; it asks the model for the next message.
3. **Same safety class as existing behavior.** The loop already re-sends
this identical request on 429/5xx/timeout/stream-stall; this adds one
more transient-fault arm to the same loop with the same bytes.
## Tests
Three new tests mirroring the existing 499/dropped-connection fixtures
(raw `TcpListener` stubs):
- `post_retries_malformed_json_body_and_succeeds` — truncated 200 body
on attempt 1, valid JSON on attempt 2; asserts success and **exactly 2**
server-side requests
- `post_exhausts_retries_on_persistent_malformed_json` —
always-truncated body; asserts exactly `MAX_RETRIES` attempts and a
terminal error carrying `json:` + cumulative/attempt context
- `openrouter_post_retries_malformed_json_body_and_succeeds` — same
recovery through OpenRouter's separate loop
Full `cargo test -p buzz-agent` green at
|
||
|
|
261c460761 |
fix(buzz-agent): recover from 400-shaped image rejections; unbound benchmark agent rounds (#5318)
## Problem
Two failure modes from the `tb21-glm52-crusoe-1` benchmark run (GLM-5.2
solo, TB2.1) wedged or killed 13 of 89 trials without the model being at
fault:
1. **Conversation poisoning on text-only endpoints.** Crusoe's
serverless `crusoeai/GLM-5.2-NVFP4` rejects any request whose history
contains an image with `400: ... is not a multimodal model`. The
recovery machinery for exactly this case already exists —
`AgentError::UnsupportedImageInput` → `replace_unsupported_images()`
strips the image blocks, marks the tool result as an error, substitutes
a text placeholder, and continues the turn. But classification only
matched OpenRouter's 404 body (`no endpoints found that support image
input`) and was only consulted on the 404 arms. The Crusoe 400 fell
through to terminal `AgentError::Llm`: the image stayed in history,
every subsequent call failed identically, buzz-acp rode its 10-retry
ladder (~40 min), and the trial idled to budget death. Measured blast
radius: **8 trials wedged, 12.7h aggregate idle-after-poison.**
2. **Bounded agent rounds in benchmark trials.** The harness default
`DEFAULT_MAX_AGENT_ROUNDS = 32` ended solo trials mid-work when turns
rotated (thinking-heavy models hit max_tokens rotation fast; 4 trials
died this way). Benchmark trials already have a wall-clock budget as the
real limit — the round cap only converts recoverable rotation into trial
death.
## Fix
- `is_unsupported_image_input_error()` also matches the verbatim `is not
a multimodal model` body. Matcher stays deliberately tight (same
doctrine as `is_context_length_error`): misclassifying a generic 400 as
recoverable would mutate history for an error that removing images
cannot fix.
- Both status ladders — shared `post()` and `openrouter_post()` —
consult it on their 400 arms and return the typed
`UnsupportedImageInput` (OpenAI-compatible providers report this as 400;
a BYOK/passthrough upstream can surface the provider's own 400 through
OpenRouter).
- Harness `DEFAULT_MAX_AGENT_ROUNDS` → `0` (unbounded —
`BUZZ_AGENT_MAX_ROUNDS=0` is the agent config's documented unbounded
value). Per-agent `budget.max_calls` in manifests still overrides.
## Acceptance
- A 400 with the image-rejection body reaches the existing image-strip
recovery path instead of wedging the session — asserted through
`complete()` (covers the return path into the convergence mapper) and at
the `openrouter_post` terminal, both proving single-attempt (a
deterministic capability rejection must never be retried).
- Ordinary 400s stay terminal `AgentError::Llm` (existing negative tests
unchanged).
- Benchmark trials run unbounded rounds by default; python tests updated
for 0-is-legal with a negative arm at -1.
## Verification
- `cargo test -p buzz-agent`: 427 + 18 + 20 + 15 + 8 + 1 + 48 passed, 0
failed (full package, 3 consecutive clean runs)
- `cargo clippy -p buzz-agent --all-targets`, `cargo fmt --check`: clean
- `uv run --extra dev pytest tests/` in harbor-buzz-orchestra: 35 passed
- Pre-push hooks (full workspace rust-tests + desktop-tauri-checks)
green on rustc 1.95.0 at head
|
||
|
|
c7b663680a |
fix(buzz-agent): budget summarizer reasoning separately so it cannot starve the handoff summary (#5248)
## Problem
The handoff summarizer sends `max_tokens: 8192`
(`HANDOFF_MAX_OUTPUT_TOKENS`) with no reasoning budget separation. On
reasoning models, thinking tokens count against that cap: the model can
spend the entire budget reasoning, length-stop with empty `content`, and
`summarize()` — which only reads `content` — reports an empty summary.
The handoff then degrades to lossy history truncation.
Observed on deepseek-v4-flash during a terminal-bench 2.1 run
(tb21-solo-3, 89 tasks): **13 consecutive handoff attempts across 5
trials failed exactly this way** (`handoff returned empty summary;
truncating`), each burning ~3 minutes of full-cap reasoning, before a
stochastically-short reasoning run finally fit. circuit-fibsqrt alone: 5
failures, 5 truncations, then success on attempt 6. video-processing
failed its task by one frame after 3 context truncations.
## Fix
`openrouter_summary_body` now grants reasoning its own equal-sized
budget and excludes it from the response:
- `reasoning.max_tokens = max_output_tokens` — thinking gets a dedicated
budget instead of competing with the summary text
- `reasoning.exclude = true` — reasoning is never in the response body;
`summarize()` only reads `content`
- `max_tokens = max_output_tokens * 2` — the total cap covers both
budgets, so the text budget the caller asked for is actually available
for text
Non-reasoning endpoints ignore the `reasoning` object. Deliberately not
paired with `provider.require_parameters`, for the reasons documented at
`apply_openrouter_mutations` (it hard-404s valid model ids).
The prior test
`openrouter_summary_carries_neither_reasoning_nor_provider` asserted
`reasoning` absent from the summary body — that assertion guarded
against *effort-based* reasoning leaking in from config (the body is
built independently of `cfg`, which is still true and still tested:
`reasoning.effort` stays unset). Replaced with
`openrouter_summary_budgets_reasoning_separately_and_carries_no_provider`.
## Verification
- `cargo test -p buzz-agent`: 422 unit + 110 integration tests pass at
|
||
|
|
2b873cf208 |
Recover from max-token response truncation (#5223)
## Summary - treat provider `max_tokens` as an interrupted assistant response and continue the same turn with actionable feedback - discard tool calls from truncated responses, including malformed partial arguments, so they are neither executed nor replayed with invalid tool-result pairing - bound recovery to two retries while preserving normal finite `max_rounds` accounting ## Verification - `cargo fmt --all -- --check` - `cargo test -p buzz-agent` (422 unit tests plus all package integration/doc suites passed) - `cargo clippy -p buzz-agent --all-targets -- -D warnings` ## Notes The pre-push repository-wide hook also ran. Its Rust tests passed (2,270 passed, 14 ignored), but its `buzz-db` unit-test build was blocked because local rustc 1.89 is below sqlx 0.9's rustc 1.94 requirement. The affected package suite above is green on the exact pushed commit. Originating Buzz channel: `c3252dd2-0142-4e01-88c7-a2183c3960a5` Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> |
||
|
|
742e8d1197 |
fix(buzz-agent): Responses reasoning summary, Anthropic display:summarized, ACP v2 messageId (#5195)
Three pre-existing gaps in the buzz-agent observer feed fixed together
per Will's ruling ("all 3 in the current PR"):
1. **OpenAI/DBv2-GPT route** — `responses_body` never requested
`reasoning.summary`; GPT-family models billed thinking tokens but
returned `summary: []`.
2. **Anthropic/DBv2-Claude route** — `anthropic_thinking_config()` never
sent `thinking.display`; newest Claude models (Opus 5, Sonnet 5, Fable
5, Mythos 5, Opus 4.7/4.8, Mythos Preview) default to
`display:"omitted"`, returning thinking blocks with an empty `thinking`
field — observer rendered nothing.
3. **ACP v2 compliance** — buzz-agent negotiates ACP v2 but emitted
`agent_thought_chunk` and `agent_message_chunk` without `messageId`,
which ACP v2's `ContentChunk` requires (`messageId` + `content` both
required at schema head `d13d1baa`).
## Changes
**`crates/buzz-agent/src/config.rs`**
- New `ThinkingSummary` enum (`Auto`/`Concise`/`Detailed`) with
`BUZZ_AGENT_THINKING_SUMMARY` env var (default `Auto`); mirrors
`BUZZ_AGENT_THINKING_EFFORT` pattern
- `anthropic_thinking_config()` now emits `"display": "summarized"` in
both the adaptive shape and the manual-budget shape whenever thinking is
enabled
- Rewrote `is_adaptive_thinking_model` and `anthropic_thinking_config`
doc comments to match Anthropic's exact three-way per-model terminology
(doc:
https://platform.claude.com/docs/en/build-with-claude/thinking-troubleshooting#supported-models):
- Opus 4.6/4.7/4.8, Sonnet 4.6: **Off** — thinking OFF by default;
`type:"adaptive"` required to enable
- Opus 5, Sonnet 5: **On** — thinking on by default, can be disabled; we
still send `type:"adaptive"` to activate `output_config.effort`
- Fable 5, Mythos 5, Mythos Preview: **Always on** — thinking cannot be
disabled; we still send `type:"adaptive"` to activate
`output_config.effort`
**`crates/buzz-agent/src/llm.rs`**
- `responses_body` emits `reasoning.summary` alongside
`reasoning.effort` when effort is set (gated — no bare
`reasoning:{summary}` without effort)
- Covers both the pure-OpenAI Responses path and the DBv2 GPT-family
Responses path
**`crates/buzz-agent/src/agent.rs`**
- `agent_thought_chunk` carries `"messageId":
format!("{run_id}-thought-{round}")`
- `agent_message_chunk` carries `"messageId":
format!("{run_id}-message-{round}")`
- The two IDs are distinct (thought and assistant are two logical
messages per the ACP v2 Message ID RFD)
- `run_id` is a fresh random token per `session/prompt` invocation so
IDs are session-unique across multiple prompts
**`crates/buzz-agent/src/lib.rs`**
- `run_id` plumbed into `RunCtx` (was already generated in `run_prompt`,
just not threaded through)
**`crates/buzz-agent/tests/golden_transcripts.rs`**
- `test_acp_v2_chunks_carry_message_id` — negotiates v2, drives two
consecutive `session/prompt` calls, asserts: both chunk types carry
non-empty `messageId`; thought and message IDs are **distinct**; IDs do
**not** recur across the two prompts in the same ACP session
**`desktop/src-tauri/src/managed_agents/env_vars.rs`**
- `BUZZ_AGENT_THINKING_SUMMARY` added to `is_safe_to_reveal` allowlist
**`desktop/src-tauri/src/commands/agent_config_tests.rs`**
- Tests for `BUZZ_AGENT_THINKING_SUMMARY` allowlist entry
(case-insensitive)
## Tests added
- `parse_thinking_summary_round_trips_all_values`
- `parse_thinking_summary_unset_and_empty_yield_auto`
- `parse_thinking_summary_is_case_insensitive`
- `parse_thinking_summary_rejects_unknown_value`
- `thinking_summary_as_str_mapping`
- `responses_body_summary_present_iff_effort_set`
- `responses_body_emits_configured_summary_mode`
- `responses_body_concise_summary_mode`
- `anthropic_thinking_config_adaptive_emits_display_summarized`
- `anthropic_thinking_config_manual_budget_emits_display_summarized`
- `test_acp_v2_chunks_carry_message_id` (integration test — two-prompt
cross-session case)
## Notes
- **DBv2 gateway parity for `display`**: unverified — the DBv2 Claude
route proxies Anthropic Messages shape, but whether the gateway passes
`thinking.display` through is not confirmed. Flagged here rather than
blocking on it.
- buzz-acp and Desktop TS are unchanged — they already parse `messageId`
as optional and will pick it up from the wire automatically.
- Chat Completions and OpenRouter paths: untouched.
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
|
||
|
Will PflegerandGitHub
|
346ae8cadc |
fix(buzz-agent): escalate LLM timeouts per retry and log per-call latency (#5130)
Non-streaming LLM calls (`"stream": false`) through slow model/provider combinations routinely take longer than the fixed `BUZZ_AGENT_LLM_TIMEOUT_SECS` window (default 240 s) to return their first response byte. The retry loop then re-ran the identical 240 s bet three times, failed the turn, and the ACP harness requeued the whole turn from scratch: agents spent 30+ minutes producing nothing while every attempt died at the same wall. And because the LLM path only logged WARN lines on failure, a healthy-but-slow call was indistinguishable from a wedged one. ### Timeout handling - **Per-attempt escalation**: the per-request budget doubles after each timeout failure (`base × 2^n`, capped at `max(1200 s, base)` — `escalated_timeout()` in `llm.rs`), shared by the main `post()` loop and `openrouter_post()`. Non-timeout retryables (429/5xx/connect) do not escalate. A call that needs six minutes now succeeds on a later attempt instead of never. - **Per-request total timeouts**: enforcement moved from the client-level `read_timeout` to `RequestBuilder::timeout()` on each LLM request, so escalated budgets aren't silently floored by the shared client and each attempt's bound covers connect through body completion. Timeout error messages were updated to match the new semantics and still point at `BUZZ_AGENT_LLM_TIMEOUT_SECS`. ### Observability - One INFO line per completed LLM call: model, provider, `duration_ms`, `input_tokens`, `cached_input_tokens`, `output_tokens`. Slowness and prompt-cache effectiveness are now visible in harness logs without waiting for a failure, and `None` vs `0` token reports stay distinguishable. Handoff summarization calls log the same line with duration only. - The agent main loop wraps the call in a `session_id` tracing span so each line is attributable to a session. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> |
||
|
Kalvin CandGitHub
|
c293b3cd40 |
fix(agent): resolve oauth cache home cross-platform (#5151)
## Summary - resolve the OAuth cache home directory with the repository's platform-aware `dirs` convention - preserve the existing `.config/buzz-agent/oauth` cache layout on macOS and Linux - make the cache-path regression assertion portable across path separators ## Problem `buzz-agent` read only `$HOME` when constructing the OAuth token cache path. Packaged Windows processes do not guarantee that Unix variable, so OAuth source construction failed with `oauth cache: $HOME not set` even though Windows had a valid user profile. ## Validation Independent reviewers validated exact commit `836d820483b141b7291170cb33535ac7cb49b2eb` on Windows/MSVC with `HOME` unset and `USERPROFILE` present: - `cargo +1.94.1 clippy -p buzz-agent --all-targets --locked -- -D warnings` - `cargo +1.94.1 fmt --all -- --check` - `git diff --check f53bbd1..836d820` - `auth::` tests: 11/11 passed with `HOME` unset - full package lib target: 396 passed / 2 failed; identical-base controls classified both as pre-existing Windows failures The changed regression fails on the base with `$HOME not set` and passes on this branch. `dirs 6.0.0` was already locked by other workspace crates; the lockfile change adds only the `buzz-agent` dependency edge. Signed-off-by: Kalvin C <kalvinnchau@users.noreply.github.com> |
||
|
|
bd2fdf4a2f |
fix(buzz-agent): classify read timeouts distinctly in LLM error messages (#4959)
## Problem
When `buzz-agent` exhausts retries on a stalled LLM call, the error
message reads:
```
transport: error sending request for url (...) (cumulative 721s, 3 attempts)
```
That text is reqwest's generic pre-response failure string — identical
whether the cause is a TLS abort, a reset connection, or a
`read_timeout` fire. An operator reading the log cannot tell whether
something broke or whether the LLM generation legitimately took longer
than the configured timeout.
## Root cause (probe-confirmed)
A live probe against `goose-claude-fable-5` with a 900s client timeout
completed in **370s** — well past the default
`BUZZ_AGENT_LLM_TIMEOUT_SECS=240`. Extended-thinking models emit zero
bytes on non-streaming calls until generation is complete, so reqwest's
`read_timeout` fires on byte-silence regardless of whether the server is
healthy. The 46× exact-721s stall signatures in production logs (3 ×
240s + backoff) are deterministic self-inflicted timeouts, not network
faults.
## Fix
### Pure classifier over `{is_connect, llm_timeout, phase}`
A new `timeout_message(is_connect: bool, llm_timeout: Duration, phase:
TimeoutPhase)` pure function produces factual messages with the
configured duration value embedded verbatim. Two thin wrappers
(`classify_transport_error`, `classify_body_read_error`) extract the
reqwest flags and delegate. The duration reaches the classifiers through
a new `read_timeout: Duration` parameter on `post()` and
`openrouter_post()`; callers pass `cfg.llm_timeout`.
### Messages emitted
| Case | Message |
|---|---|
| Connect-phase timeout (`is_connect && is_timeout`) | `connect timeout:
no connection established within 10s` |
| Transport read-timeout | `read timeout: no response bytes received
within 240s (consider raising BUZZ_AGENT_LLM_TIMEOUT_SECS)` |
| Body-read timeout | `read timeout: no further response bytes received
within 240s (consider raising BUZZ_AGENT_LLM_TIMEOUT_SECS)` |
| Non-timeout | `transport: {reqwest text}` / `body read: {reqwest
text}` (unchanged) |
`LLM_CONNECT_TIMEOUT` is now a named `const` (was inline
`from_secs(10)`).
**Out of scope by explicit decision:** streaming support, changes to
`MAX_RETRIES` or backoff.
## Files changed
- `crates/buzz-agent/src/llm.rs` — `timeout_message` pure fn +
`TimeoutPhase` enum + `LLM_CONNECT_TIMEOUT` const; two classifier
wrappers updated; `post()` and `openrouter_post()` gain `read_timeout`
param; tests replaced.
## Tests
`cargo test -p buzz-agent`: **397 passed, 0 failed** at `294ce5897`.
**Pure-function tests (no network):**
- `timeout_message_connect_true_shows_connect_timeout` —
`is_connect=true` → connect-flavored text with 10s value; both phases
checked
- `timeout_message_transport_phase_shows_read_timeout_and_duration` —
transport phase includes 240s and config knob
- `timeout_message_body_read_phase_says_no_further_bytes_and_duration` —
body phase says "no further", shows 300s
- `timeout_message_duration_is_not_hardcoded` — 600s supplied → 600s in
output, not 240s
**Loopback reqwest integration tests:**
- `classify_transport_error_read_timeout_is_loopback_verified` — TCP
connect succeeds, server sends no bytes; verifies reqwest sets
`is_timeout && !is_connect` and message contains 50ms value
- `classify_transport_error_non_timeout_preserves_reqwest_text` —
controlled accept-then-close on an owned loopback listener → non-timeout
error; asserts exact `transport: {err}` output equality
- `classify_body_read_error_timeout_says_no_further_bytes` — loopback
server sends headers + 4 bytes of a declared-1024-byte body, then holds;
verifies `is_timeout`, "no further", 100ms value, config knob
No test performs egress beyond loopback (`127.0.0.1`). The TEST-NET-3
dial is deleted.
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
|
||
|
|
ed4b3e7afa |
fix(buzz-agent): recover from context-window 400s instead of sticking (#4946)
Provider `context_length_exceeded` 400s permanently wedged agent sessions: the turn errored, the oversized history persisted in the in-memory session, and the usage baseline stayed frozen at the last successful sub-threshold reading (failed requests report no usage), so the preflight handoff gate never fired again — every later prompt failed identically until an agent restart. The byte-truncation fallback never intervened because it is a request-body limiter (`estimated_bytes`), not a context-window defence; at context-window scale it is a measured no-op. This adds the reactive recovery path: - **Typed classification.** `AgentError::LlmContextExceeded` is classified at both non-success provider terminals — the shared `post()` (Anthropic, OpenAI, Databricks) and `openrouter_post()` — on `status == 400` plus a context-window body match, so ordinary 400s stay terminal. - **Forced handoff.** A context-400 forces a summarize-handoff that bypasses `should_handoff()` and `BUZZ_AGENT_MAX_HANDOFFS`, bounded by its own per-turn budget (`MAX_CONTEXT_RECOVERIES_PER_RUN = 3`). - **Shrink ladder.** The summarize prompt budget halves from the observed rejected history size — not from `max_context_tokens`, the number the provider just contradicted — rung to rung, with a 4096-byte floor. A summarize call rejected for the same reason takes the next rung instead of re-sticking. At the floor (overflow dominated by unshrinkable frame: system prompt, tool schemas, live prompt) recovery is refused and the provider error surfaces clearly instead of self-healing. - **Baseline reset.** The stale usage baseline is cleared when a request fails, so the preflight gate cannot stay frozen sub-threshold on retries. Named behavior changes: 1. **Anthropic and OpenRouter errors now carry the `(model)` stamp.** Provider arms return their `Result` into the central error mapper instead of early-returning past it, making the code match its documented single-convergence contract at that mapper. 2. **`max_rounds` now counts completions the loop acts on.** A request rejected with a context-400 that is then successfully recovered refunds its round before the retry, paired 1:1 with a consumed recovery rung, so the round cap is neither weakened nor able to drop a recovered turn unanswered. Related: #4805 — the complementary proactive fix (per-session handoff-cap kill switch that let sessions grow to the provider wall). #4805 prevents reaching the wall; this PR recovers at it. --------- Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
|
6df7eba24d |
fix(buzz-agent): scope handoff cap per turn, not per session lifetime (#4805)
`BUZZ_AGENT_MAX_HANDOFFS` compared against the session-cumulative `handoff_count` (persisted across prompts). After N handoffs a long-lived session hit the cap permanently: `maybe_handoff()` returned `Skipped` on every subsequent prompt, the 16 MiB byte-truncation fallback never bound before a 1M-token provider wall, and the session wedged on the first 400 with no recovery path. Thufir's session log shows 8 days of cap-forced truncation before the first `context_length_exceeded` 400. The fix replaces the session-level cap comparison with a local `handoff_attempts` counter constructed at the start of `run()` and passed into `maybe_handoff()`. The counter resets on every `session/prompt` turn so `BUZZ_AGENT_MAX_HANDOFFS` caps compaction loops within a single turn while allowing unbounded compactions across a session's lifetime. The session-cumulative `handoff_count` is retained for log context only and is not reset. Steer-driven rounds share the per-turn budget automatically since steers inject into the running `run()` loop, not a new call. - Move `handoff_attempts` increment to before `summarize()` so failed, empty, and cancelled summarize calls each consume one budget slot — the cap cannot be bypassed by a repeatedly-failing summarizer - Upgrade cap-forced `Skipped` from `INFO` to `WARN`; add structured fields for `session_id`, attempt count, projected tokens, and threshold so the cap→wall pairing is attributable per session - Document `max_handoffs` in `config.rs` as a per-`session/prompt`-turn bound - Three new behavioral regression tests: per-turn reset proven across two separate turns; within-turn cap proven via multi-round tool-call turn; failed summarize proven to burn the attempt budget Note: this is the proactive half of the context-window fix. The reactive `context_length_exceeded` 400 recovery path is owned by Sami's branch (`buzz-ctxfix-sami`, Tyler's crew); this PR is intended to land after that one. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
|
8a7eb8d3d7 |
fix(agent): recover from unsupported image input instead of poisoning the turn (#4896)
## Problem
`buzz-dev-mcp` advertises `view_image` to every agent regardless of
whether the session's model accepts images. When a text-only model (e.g.
DeepSeek V4 Flash) takes the bait, the image lands in session history
and every subsequent LLM request 404s with `No endpoints found that
support image input`. The error was classified as `LlmModelNotFound` and
propagated fatally out of the turn loop — history stays poisoned,
buzz-acp retries the batch with exponential backoff, and the session
burns its entire clock doing no work. In a recent trial run, **all 57
trials that called `view_image` on a text-only model died this way; none
recovered.**
## Fix
Capability-gating the advertised tool isn't reliable — there is no
image-capability metadata at the agent layer across providers. Instead,
recover at the turn loop:
- **Typed error**: new `AgentError::UnsupportedImageInput`, classified
narrowly on the exact provider phrase `No endpoints found that support
image input` on both the generic 404 path and OpenRouter's 404 path.
Unknown-model 404s and OpenRouter parameter-routing 404s keep their
existing classifications. No deterministic retry.
- **In-turn recovery**: on this error, `RunCtx::run` strips every image
block from history — keeping the tool result (and therefore
tool-call/result pairing) intact — marks the result `is_error`, appends
actionable model-facing guidance ("The current model does not support
image input. The image was removed from conversation history so this
turn can continue. Use a text-based inspection tool…"), and continues
the same turn. Base64 never replays again.
- **Loop guard**: recovery only fires when at least one image was
removed; if the provider says "image" and history has none, the error
propagates as before.
## Tests
- Unit: phrase classification (typed, not retried; unknown-model 404
unaffected), idempotent image-to-error history mutation preserving call
IDs and text.
- End-to-end (`fake_llm.rs` + `fake_mcp.rs`): tool call → MCP image
result → 404 unsupported-image → same-turn recovery. Captured requests
prove round 2 carried the image, round 3 replays no image, carries the
guidance text, preserves pairing, and ends `end_turn`.
- Loop guard: typed unsupported-image error with **no** image in history
fails after exactly one provider request instead of spinning —
mutation-testing showed deleting the `removed == 0` guard survived the
suite, and `max_rounds` defaults to unlimited in production, so this
branch needed direct coverage.
Verified at `a210305019b33d5f56677b4c82bab79e4ac52d24`: `cargo test -p
buzz-agent` (full package, 381 unit + all integration suites) green;
`clippy --all-targets -D warnings` green; `fmt --check` green; pre-push
hooks (rust-tests, desktop-tauri-checks, branch-skew) green.
**Scope of the classification guarantee**: the classifier runs in the
shared `post()` (which Anthropic and OpenAI paths route through) and in
`openrouter_post()` — i.e., every 404 path in `llm.rs`. It only runs on
404 responses; providers that reject images with a different status
(e.g. a 400) are out of scope for this PR — see the review-comment
discussion for why broadening the phrase list alone would not cover
them.
Authored by Wren, loop-guard test by Sami, reviewed by Eva.
---------
Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Signed-off-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
|
||
|
|
4a2305170e |
fix: reauthenticate databricks model discovery (#4008)
## Summary - preserve Databricks catalog 401 responses as authentication failures and retry discovery exactly once after silently refreshing the rejected bearer - preserve runtime OAuth recovery: when discovery has no usable OAuth credential, `session/new` succeeds with only the trimmed configured model so the first `session/prompt` can run the existing browser PKCE flow - reject a rejected configured `DATABRICKS_TOKEN` with actionable, non-interactive guidance; static credentials cannot recover through PKCE - use the configured-model fallback for non-auth discovery failures without caching failed or fallback catalogs, so later sessions retry discovery - keep known Databricks v2 models only for authenticated empty-catalog responses and mark their provenance - resolve discovery before MCP spawn or session registration, preventing failed discovery from leaking resources or consuming session capacity - permit serialized interactive PKCE only from the explicit saved-agent model picker; passive draft discovery never opens a browser ## Runtime flow 1. OAuth discovery attempts cached credentials and silent refresh without opening a browser. 2. If no usable OAuth bearer exists, `session/new` advertises only the configured model and succeeds. 3. The first `session/prompt` uses `TokenSource::bearer()`, which may launch browser PKCE. 4. A later session retries discovery and caches only the authenticated catalog. ## Regression coverage - rejected-but-locally-fresh OAuth bearer performs one refresh and one catalog retry - OAuth mode with no cached token allows `session/new` and returns exactly the trimmed configured model - the OAuth fallback is not cached; a later authenticated session retries discovery and caches the returned catalog - rejected static tokens still reject `session/new` - failed discovery does not consume the sole session slot or spawn the supplied MCP process - Desktop interactive/passive auth intent, static-token redaction, and authenticated empty-catalog provenance ## Verification - `cargo test -p buzz-agent` - `cargo test --manifest-path desktop/src-tauri/Cargo.toml --lib commands::agent_models` - `cargo clippy --manifest-path desktop/src-tauri/Cargo.toml --all-targets -- -D warnings` - `cargo fmt --all -- --check` - `git diff --check` - full pre-push hooks ## Review Adversarial review found and drove fixes for session/MCP resource leakage, duplicate concurrent PKCE flows, sensitive error propagation, incorrect 403 reauthentication, missing discovery-level coverage, passive browser launch, and the Desktop file-size ratchet. The final follow-up preserves the existing prompt-time OAuth flow while retaining static-token rejection and pre-allocation discovery ordering. --------- Signed-off-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> Co-authored-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> |
||
|
|
09c86c56e5 |
fix: report agent usage per provider round, not once per turn (#4545)
## The bug buzz-agent emitted its `usage_update` notification in exactly one place: after `ctx.run()` returned. Until that moment a turn's token counters lived only in the prompt task's stack frame. **A turn killed mid-flight reported nothing at all** — the provider had already billed every round it completed, and no consumer ever saw any of it. That is not a corner case for anything that ends a turn on a clock. It is the normal case for a long-horizon benchmark run that relaunches its agent between phases. ## How big Measured against a provider's own billing ledger over one run's window: | | provider ledger | what we recorded | |---|---|---| | the relaunched lead seat | $485 / 348M tok | $98.99 / 90.3M tok | | the two seats that were not relaunched | $29.90 / 856M | $25.81 / 765M — reconciles | 97% of that run's usage rows came back all zeros, against 1–4% for comparable runs that never relaunch. In one 450-phase trial exactly 7 phases recorded any usage — and each of those carries 177k–437k input tokens, a whole session's worth landing in the one phase that happened to end gracefully. Worth being precise about what was *not* wrong, since both were plausible and both were checked: - **Not pricing.** The rates were verified against the provider's endpoints API and match what we charge. - **Not a truncation bug.** The usage files were intact and internally consistent. The tokens were never captured in the first place. ## The fix The run loop now emits a session-cumulative `usage_update` after every usage-bearing provider response, so an interrupted turn has reported everything but its single in-flight request. - **Emitting more than once per turn is already part of the contract.** buzz-acp's `UsageTracker` advances its committed baseline only at publish time, and goose behaves the same way — which is why the tracker was written to tolerate it. - **The turn-start session baseline is snapshotted into `RunCtx`** so the mid-turn figure stays *session*-cumulative. A turn-local number would be discarded by a high-water-mark consumer and lose the turn entirely; there is a test for exactly that. - **Snapshot by value, not a session handle.** The loop reports once per round, and taking the sessions lock on each would serialise concurrent sessions behind one another's provider round-trips. Nothing else advances those counters while the turn holds `busy`, so it cannot go stale. - **One shared `wire::usage_update_payload`** for both call sites, so the mid-turn and end-of-turn shapes cannot drift. A drift there would present as tokens silently vanishing, which is the failure this reporting exists to prevent. ## Why not a SIGTERM handler That was the obvious shape and it does not work. At signal time the counters are not sitting anywhere a handler could reach — they are in the turn's stack frame, and the value the handler would need has not been folded into the session yet. Making usage durable *during* the turn is what actually fixes it; once it is, a handler adds nothing beyond the in-flight request, whose cost is unknown until its response lands. ## Tests - `usage_is_reported_after_each_round_not_only_at_turn_end` — two rounds; asserts the **first** notification carries round 1's counts alone, proving it went out before round 2 returned. - `mid_turn_usage_includes_earlier_turns` — a mid-turn report must be session-cumulative, not turn-local. buzz-agent 18/18 on the `fake_llm` suite, 382 unit. `cargo fmt` / `clippy` / `cargo check --workspace --all-targets` clean. ## Scope Agent-side only, against `main`. The matching harness change — settling usage on the timeout path, which was skipped on the reasoning that an incomplete turn has nothing to flush — is **#4553**, against the benchmark branch, since that harness does not exist on `main`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Atish Patel <atish@squareup.com> Co-authored-by: Claude Code <noreply@anthropic.com> |
||
|
|
081f805d5e |
feat(agent): optional reply guard reminds a silent turn to publish (#3763)
## Why
A Buzz agent's assistant text and reasoning are never shown to anyone —
only what it posts through the CLI. A turn that runs fifteen tool calls
and never publishes is a silent failure: the requester waits on a result
that was produced and thrown away.
This adds an optional reminder at the end-of-turn gate, off by default.
Tyler asked for it in buzz-mesh; plan iterated to **9.5/10 with @Wren**
(Minimalness 9.7, Elegance 9.5, Correctness 9.3).
## What
`BUZZ_AGENT_REQUIRE_REPLY=1` (default off, per-agent opt-in). A turn
about to end with no recognized attempt to post gets a reminder and is
rerolled. **At most two, then the turn ends regardless** — the guard
catches accidental omission, it does not compel speech. The reminder
text explicitly licenses silence so it cannot fight the base prompt's
"silence is usually correct."
**This is not a new MCP hook.** `RunCtx::run` *is* the turn, so the two
per-turn locals need no plumbing, and every tool call already passes
through it with arguments visible. The objection is appended at the
existing `_Stop` gate and rides `push_hook_outputs_as_tool_results`, so
the model receives it as a lower-trust tool result with `{hook, server,
text}` attribution. No new trust path, no new lifecycle event, no
dev-mcp or CLI protocol change.
Earlier revisions of this plan needed four crates (a `_UserPromptSubmit`
hook, a marker file, a `buzz-cli` change, dev-mcp state). Tyler pointed
out the agent already knows both facts; that deleted all of it. Net
runtime change is ~35 lines in `agent.rs` + ~4 in `config.rs`.
### Recognition contract
A registered non-hook tool whose qualified name ends in `__shell`, whose
`command` argument contains `messages send` or `reactions add`.
- **The `__` separator is exact, not approximate.** Given `has()` +
`!is_hook()`, `ends_with("__shell")` is *provably equivalent* to a bare
name of `shell`: registration forbids `__` in server and bare names
(`mcp.rs:227,268`) and qnames are `{server}__{bare}`, so a trailing
`__shell` could only straddle the separator if the bare name began with
`_` — which `is_hook` excludes. Without the separator, `powershell` and
`noshell` would match.
- **Reads the structured `command` field**, not serialized arguments, so
a `description` that quotes a send cannot disarm the guard, and a
non-string `command` is rejected rather than coerced.
- **Detects an attempt, not a successful publish.** A failed send
already returns non-zero exit and error JSON — louder than this
reminder. The variable is named `buzz_reply_call_seen` so the code can't
pretend otherwise.
- **Checked after the per-turn tool-call cap**, since a discarded call
never ran.
- `messages send` also covers `messages send-diff`. Reactions count
because the base prompt directs agents to react rather than post a bare
acknowledgement.
**Known limits, both deliberate and documented:** a command assembled at
runtime (`$CMD`) or hidden in a wrapper script is missed; text that
merely quotes a send (`echo "buzz messages send"`) matches. Missing a
real post is the expensive direction and substring matching is the
forgiving one there. Neither edge is pinned by a test, so the matcher
stays free to improve.
### Budget
Reminders share `BUZZ_AGENT_STOP_MAX_REJECTIONS`, the existing outer cap
on every end-turn objection. Default 3 fits both; at 1 only one fits; at
0 the guard is off with the hooks. A round carrying both a hook
objection and a reminder costs one rejection and delivers both texts. An
independent budget would either violate that bound or need a second
arbitration rule.
## Prior art
- **#3467** (closed) built the same detector one layer up in `buzz-acp`
for a different remedy. None of its symbols are on main — this borrows
its permission to be coarse, but reads structured data that ACP didn't
have.
- **#3648** (open) detects turns with *no output at all*; a turn with
fifteen tool calls and no post counts as output there, so it does not
cover this case.
- **#3741** (merged) is mesh-only.
## Testing
**14 new tests.** 4 unit tests on the matcher; 10 integration tests
through the ACP wire harness: off by default, `=0` still off, opted-in
silent → exactly 2 reminders then `end_turn`, registered `fake__shell`
send → 0 reminders, hallucinated `fake__shell` → still reminded, publish
call truncated past the 64-call cap → still reminded, budget 1 → 1
reminder, budget 0 → off, combined `_Stop` hook objection + reminder →
one round both texts and after 2 reminders the hook objection continues
alone, unparseable `=true` → startup error naming the key.
**10 mutation checks, each breaking a specific named test** — neutralize
the nag cap, stop sharing the budget, neutralize `buzz_reply_call_seen`,
drop `has`/`is_hook`, ignore the flag, drop the `__`, drop `reactions
add`, read serialized args, move detection before truncation.
`tests/bin/fake_mcp.rs` gains `FAKE_MCP_SHELL_TOOL=1`: it previously
exposed no tool with a bare name of `shell`, so the satisfied-guard path
was untestable.
Full `cargo test -p buzz-agent` green at 9e0ae1f04; clippy `-D warnings`
and `cargo fmt --check` clean.
**Unrelated flake found:**
`cancelled_turn_with_usage_emits_notification_before_response`
(`tests/fake_llm.rs`) is timing-sensitive. Under 10 loaded cores it
fails **2/20 on this branch and 1/20 at unmodified
`origin/main@02be413b8`** — pre-existing, not caused by this change
(which is inert without the env var). Flagging so it isn't misattributed
to the next PR that's open when CI hits it.
## Docs
`crates/buzz-agent/README.md` is the primary home (env var, recognition
contract, limits, budget interaction). `docs/MCP_DRIVEN_HOOKS.md` gets a
short cross-reference explaining this is *not* a hook — otherwise
readers hunt for a `_ReplyGuard` tool that doesn't exist.
---------
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
Signed-off-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
Co-authored-by: Dawn (sprout agent) <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
|
||
|
|
ab55fee818 |
feat: add first-class OpenRouter provider support (#1975)
## Summary
First-class `Provider::OpenRouter` support joining the existing
anthropic/openai/databricks providers. Reuses the Chat Completions path
with targeted mutations for OpenRouter's routing contract.
**Core (`crates/buzz-agent`):**
- `Provider::OpenRouter` enum variant with `OPENROUTER_API_KEY`,
`BUZZ_AGENT_MODEL` → `OPENROUTER_MODEL` fallback, `OPENROUTER_BASE_URL`
env convention
- Body mutator: `reasoning: {effort}` when effort is configured, and
`max_completion_tokens` translated to OpenRouter's `max_tokens`
spelling; no `provider.require_parameters` filter (it routes only to
endpoints advertising every parameter in the body, which hard-404s a
valid model id); summaries get neither. `openai_body` is always called
with `effort=None` on the OpenRouter path — the `reasoning` object is
added by the mutator directly, so `reasoning_effort` is structurally
absent.
- Attribution headers: `HTTP-Referer: https://github.com/block/buzz`,
`X-OpenRouter-Title: Buzz`
- Error-inside-200 check in shared `parse_openai` (`finish_reason ==
"error"`)
- 401 auth handling: static API keys (`refresh_now` returns the same
token) fail terminal immediately with one wire request; PKCE/minting
sources get one retry with the fresh token.
- Status+`error_type` retry matrix (4-arm collapsed form): 429 (honor
`Retry-After`), 502 (retry), 503/`provider_overloaded` (honor
`Retry-After`), everything else including untyped 503 (bounded retries →
actionable routing message). 499 included matching shared `post()`
(#2175) for turn-timeout stall surfacing. Terminal failures wrapped in
`terminal_llm_error` for duration+attempt-count context.
- `anthropic/*` `cache_control` injection (model-gated, mixed-content
safe)
- Provider-agnostic `reasoning_details` opaque round-trip on
`HistoryItem::Assistant` for tool-call continuations — captured verbatim
in `parse_openai_with_reasoning_details`, replayed verbatim in
`openai_body`, byte-accounting charged. `provider_extra` passthrough
from `make_tool_call` composes independently.
**Desktop:**
- Readiness arms checking `OPENROUTER_API_KEY` + `OPENROUTER_MODEL`
- Model discovery via `{OPENROUTER_BASE_URL}/models` filtered on
`supported_parameters` contains `tools`
- Picker entry, credential config, effort table 3-file sync
**`desktop/src/features/agents/AGENTS.md`: no rules changed** — the
scoped rule requiring an explicit note is satisfied here.
Implements the gate-cleared plan from
`PLANS/OPENROUTER_PROVIDER_PLAN.md` (rev 3).
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
|
||
|
|
f95fdc1a10 |
feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (#3593)
## What Wires genuine provider-reported `total_tokens` through the full buzz-agent → buzz-acp publish chain so kind-44200 events carry real per-turn and cumulative totals for OpenAI-backed models, while preserving all existing behaviour for Anthropic and external harnesses (goose, claude-code). ## Why Live prod data showed 0 of 1,934 archived reports carry `totalTokens`. Both hardcoded `total_tokens: None` in `pool.rs` and the absent field in `buzz-agent`'s parser are root causes. This is the backend half of a two-track fix; the display-fallback half lands in [#2035](https://github.com/block/buzz/pull/2035). ## Changes **`crates/buzz-agent/src/types.rs`** - Added `total_tokens: Option<u64>` to `LlmResponse` with an explicit doc comment that NIP-AM forbids deriving it. - Added `TurnTotalState` enum (`Unseen | Exact(u64) | Unknown`) with `fold()` and `exact_value()` — the tri-state accumulator that distinguishes not-yet-observed from permanently poisoned. **`crates/buzz-agent/src/llm.rs`** - `parse_responses` and `parse_openai`: read `usage.total_tokens` from OpenAI Chat Completions (including Databricks routes) and the Responses API via `sum_usage`. - Anthropic: explicit `total_tokens: None` — no genuine total available; NIP-AM forbids summing categories. **`crates/buzz-agent/src/agent.rs`** - Added `turn_total_state: &'a mut TurnTotalState` to `RunCtx`. - Fold `response.total_tokens` into the accumulator after each usage-bearing response; non-usage-bearing responses (keepalive/stream frames) do not poison. **`crates/buzz-agent/src/lib.rs`** - Added `accumulated_total_state: TurnTotalState` to `Session` (default `Unseen`). - Per-turn state passed to `RunCtx`, folded into session cumulative after each turn. - Emits `accumulatedTotalTokens` in `usage_update` only when cumulative is `Exact(n)`. **`crates/buzz-acp/src/usage.rs`** - Added `accumulated_total_tokens: Option<u64>` (serde default) to `UsageUpdatePayload` — optional for goose compat. - Added `last_total: Option<u64>` to `SessionState`. - Added `turn_total_tokens` and `cumulative_total_tokens` to `TurnUsage` (field-local — never affect `delta_reliable`). - Derive turn-total delta only when prev and current are both `Some` and monotonic; absence, decrease, or no baseline leaves only the total delta null without touching input/output reliability. **`crates/buzz-acp/src/pool.rs`** - Replaced both hardcoded `total_tokens: None` in `publish_agent_turn_metric` with `usage.turn_total_tokens` and `usage.cumulative_total_tokens`. ## Tests 20 new tests across the four touched files: | File | Tests | |------|-------| | `types.rs` | `TurnTotalState` fold, accumulation, exact_value, default (7 tests) | | `llm.rs` | Chat present/absent, Responses present/absent, Anthropic always-None (5 tests) | | `usage.rs` | First turn no baseline, second-turn delta, cumulative decrease (field-local), current absent, goose-shaped deserialization, baseline absent (6 tests) | | `pool.rs` | Exact turn+cumulative mapping, null totals never derived (2 tests) | `cargo test -p buzz-acp -p buzz-agent` — all passing, 0 failures. ## Scope Boundary: `crates/buzz-agent/**` + `crates/buzz-acp/**` only. Desktop unchanged. `costUsd` explicitly out of scope. Related: [#2035](https://github.com/block/buzz/pull/2035) --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw <41ea58f1e64c243627e8acde7c89be667052ee6e17d8f021c1195be4324ebf04@buzz.block.builderlab.xyz> |
||
|
|
4a1ebf25c7 |
feat(agent): make Gemini and MLflow-route models usable through databricks_v2 (#3569)
## Summary
Makes Gemini — and every other non-Claude, non-GPT-5 model on the
Databricks MLflow route (`databricks_v2`) — usable in an agent loop.
These are the non-`benchmarks/` changes from
`benchmark/harness-accounting-and-solo`, lifted onto a clean base off
`main` so they can land independently while the harness work continues.
Two defects made these models unusable, one fatal and one silent. Both
live only in `openai_body` / `parse_openai`, which is the
least-exercised of the three `databricks_v2` sub-routes — the
`luna`/`sol` conditions run the Responses route and the `opus`
conditions run the Anthropic route, so **this change is inert for every
model already in use** and only lights up the MLflow path.
## Why a third route at all
`databricks_v2_route_for_model` buckets by model family: `claude*` →
Anthropic Messages, `gpt-5`/code-names → OpenAI Responses, **everything
else → MLflow chat-completions**. Gemini, Qwen, gpt-oss, and friends all
fall through to that third pair — and both bugs below live only there.
## D1 — dropped thought signatures (fatal)
Gemini returns a `thoughtSignature` on every tool call and **requires it
echoed back**. `openai_body` reserialized each call as `{id, type,
function}` only, dropping the field, so the next request 400'd:
```
HTTP 400 Function call is missing a thought_signature in functionCall parts.
```
For a coding agent this fires on the **first** tool call, so the model
never completes a single turn.
**Position is load-bearing.** A four-shape replay probe against the live
gateway established that the signature must sit as a *sibling* of
`function` — nesting it inside `function{}` fails with the *same* 400 as
omitting it. A fix that "preserves the field" without preserving its
position passes a unit test and still 400s.
The fix: `ToolCall` gains `provider_extra: Map<String, Value>`.
`parse_openai` captures every top-level wire key except the three we
model (`id`, `type`, `function`); `openai_body` re-emits them beside
`function`. Keeping *whatever we did not model*, rather than naming
`thoughtSignature`, means the next provider with an opaque per-call
token needs no change here. The Responses and Anthropic replay shapes
are fully modelled, so they pass `Default::default()` and stay
**byte-identical** to before.
### D1b — duplicate tool-call ids (same root cause)
Gemini returns the **function name** as the id, so two parallel calls to
one function arrive sharing an id — and that id is what pairs a
`role:"tool"` result back to its call, making two results
indistinguishable. `dedupe_provider_ids` suffixes collisions
(`get_weather`, `get_weather-2`). Safe because both halves of the
pairing (the assistant `tool_calls[].id` and the result's
`tool_call_id`) are re-emitted from this same value; the provider never
sees its original id again.
## D2 — block-array content discarded (silent, worse than a crash)
`parse_openai` read `content` with `as_str()`, which returns `""` for
anything that isn't a JSON string. Gemini (and Qwen35, gpt-oss) send an
array of typed blocks:
```json
"content": [
{"type": "reasoning", "summary": [{"type": "summary_text", "text": "…"}]},
{"type": "text", "text": "391"}
]
```
So the model answered and the answer was thrown away — no error, no
warning, just a turn that looked like the model had said nothing. On a
benchmark this reads as "Gemini is bad at the task" rather than "buzz
dropped the reply."
`openai_content_parts` now accepts either shape — string as before, or a
block array where `text` blocks concatenate into text and `reasoning`
blocks into reasoning (Gemini nests the prose one level down under
`summary`). Message-level `reasoning_content` / `reasoning` still win
when present, so DeepSeek and vLLM-style hosts are unchanged; block
reasoning is the last fallback.
## Also: a turn-start log line (`buzz-acp` `pool.rs`)
Small, independent observability change that also rides in the
non-benchmark delta: `run_prompt_task` now emits a `pool::prompt` "turn
starting" line, labelled by the same `prompt_label` helper as
`log_stop_reason`, so a log reads as start/stop pairs. An unpaired start
is the only durable evidence that a turn was entered and never returned
— without it, a stalled agent and an agent nobody woke leave identical
(zero-completion) logs.
## Interaction with #3538
#3538 (already merged) rewrote `databricks_v2_route_for_model` to route
by boundary-aware model-family segments. That change and this one touch
**different functions** in `llm.rs` — routing vs. body/parse — and
compose cleanly; the family routing decides *which* pair runs, and this
fixes the MLflow pair it can now select.
## Testing
- `cargo fmt --all -- --check`, `cargo clippy -p buzz-agent -p buzz-acp
--all-targets -- -D warnings` — clean.
- `cargo test -p buzz-agent -p buzz-acp` — all green (304 + 632 lib
tests plus integration suites, 0 failures). Five new tests cover:
block-array text extraction, plain-string regression, passthrough
capture (and non-duplication of the modelled keys), replay position
(`thoughtSignature` beside `function`, not inside it), and id
de-duplication.
- Wire evidence: the four-shape replay table and the reasoning-effort
probe were run against `block-lakehouse-staging` (recorded in the design
doc).
## Relationship to the benchmark branch
The full design write-up (four-shape replay table, position-matters
analysis, effort verification, and open pricing item) lives in
`docs/08-gemini-provider-fixes.md` on
`benchmark/harness-accounting-and-solo`. The benchmark manifests and
endpoint-config entries that exercise these models are separable and
stay on that branch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Atish Patel <atish@squareup.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
|
6438dedf83 |
feat(agent): route Claude/GPT model families to their native gateway wire (#3538)
## Summary Databricks v2 chooses the gateway wire format — OpenAI Responses, Anthropic Messages, or MLflow chat — purely from substrings in the endpoint name. There is no family field on the endpoint to key off, so the substring set *is* the routing contract. The matcher only recognised `gpt-5`/`gpt5` and `claude`, which makes correct billing depend on every Claude endpoint happening to be named with the literal string "claude". ## Why this matters Getting a Claude model onto the Anthropic Messages route is exactly what lets buzz attach the `cache_control` breakpoint (the fix in #3463). If a Claude endpoint's catalog name omits "claude" — an alias, a bare `opus-5`, a `goose-opus-5` — it silently falls through to the MLflow (OpenAI-wire) path, where Anthropic prompt caching is **structurally impossible**. The result is the same failure #3463 fixed: 0% cache reads, the full ~10x read discount lost, and no error — a naming convention quietly holding up a billing-correctness invariant. ## What changed `databricks_v2_route_for_model` (`crates/buzz-agent/src/llm.rs`) now matches broader, case-insensitive marker sets: - **Claude → Anthropic Messages:** `claude`, `opus`, `sonnet`, `haiku`, `mythos`, `fable` — the Claude family names and release code names, so a Claude endpoint reaches the cache-capable route regardless of how it's named. - **GPT → OpenAI Responses:** the `gpt` family (now `gpt` on its own, not just `gpt-5`) plus the GPT-5 launch code names `sol`, `luna`, `terra`. OpenAI markers are evaluated first, preserving the prior `gpt-5`-first precedence for any name that could carry both. Names matching neither set still fall through to the MLflow chat route. ## Testing - `cargo fmt`, `cargo clippy -p buzz-agent --all-targets -- -D warnings` — clean. - `cargo test -p buzz-agent` — all green (299 lib + integration suites, 0 failures). The `databricks_v2_routes_by_model_family` test was expanded to cover each new marker, the GPT-5 code names, case-insensitivity, and the unchanged MLflow fallback (including `gemini`). ## Relationship to #3463 #3463 taught the Anthropic path to request caching; this makes sure Claude models actually land on that path. Follow-up still open: surfacing `cache_creation_input_tokens` end-to-end so a persistent `reads == 0 && writes == 0` reveals a disabled cache regardless of which wire a model takes — happy to do that next. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Atish Patel <atish@squareup.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
c405ad1d4b |
feat(agent): fix Anthropic prompt caching with Databricks (+ MCP proxy/TLS passthrough) (#3463)
> On 8 tasks matched by name across the two runs, cost fell $8.36 → $1.77 (4.71×) and wall-clock 12,423 s → 1,085 s (11.45×). ## Summary Two independent, self-contained fixes to `buzz-agent`/`buzz-acp`, split out of the benchmark branch so they can land while the harness work continues: 1. **Request and surface Anthropic prompt caching.** buzz never sent a `cache_control` breakpoint, so on the Databricks Anthropic route `cache_read_input_tokens` was **structurally always 0** and the ~10× cache-read discount was never claimed. This teaches `anthropic_body()` to mark the cacheable prefix, and plumbs the cache split end-to-end so accounting can price it. 2. **Pass proxy + TLS-trust env into MCP tool subprocesses**, so agent tools on a proxy-only host stop reporting a live network as offline. ## Why the caching gap matters The Anthropic Messages API does **not** cache unless the request carries a `cache_control` breakpoint, and the Databricks AI Gateway — a third-party proxy in front of the model, in the same category as Bedrock/Vertex — does **not** auto-cache (only the first-party Anthropic API and Claude-on-AWS do zero-config caching). So every request was billed cold. Measured live against the Databricks gateway (`databricks-claude-opus-5`, 2026-07-28), the same call with and without a single `cache_control` marker: | Run | `input_tokens` | `cache_creation` | `cache_read` | latency | |---|---|---|---|---| | No `cache_control`, two byte-identical calls | 121,625 | 0 | **0** | ~9.3 s | | With one marker — cold (write) | 4 | 121,625 | 0 | 9.3 s | | With one marker — warm (read) | 4 | 0 | **121,625** | **4.5 s** | One marker moved 121,625 tokens from full-price input to a 0.1× cache read and roughly halved latency (a clean, isolated ~2.07× prefill speedup on this single-threaded microbenchmark). The gateway honours `cache_control`; buzz simply never sent it. At fleet scale this was a real budget item. Across matched Terminal-Bench solo sweeps (89 tasks, `-n 20`, before the fix), the two OpenAI-route models independently landed at ~86–87% cache reads — the expected shape for an agentic loop, where system + tools + append-only history repeat every turn — while the Anthropic route returned a hard 0% on every receipt: | Condition | Route | Input tokens | Cache reads | Cost | Cost if uncached | Discount | |---|---|---|---|---|---|---| | luna (`gpt-5-6`) | OpenAI | 20,320,818 | **17.7M (87.0%)** | $6.96 | $22.87 | **3.28×** | | sol (`gpt-5-6`) | OpenAI | 22,312,290 | **19.2M (85.9%)** | $37.07 | $123.35 | **3.33×** | | opus (`claude-opus-5`) | Anthropic | 12,459,822 | **0 (0.0%)** | $81.31 | $81.31 | **1.00×** | Applying luna's measured 87% read rate to the opus token counts at list prices (`input $5/M`, `cached_input $0.5/M`, `output $25/M`) puts the opus run at **~$32.53 vs the $81.31 actually paid — a ~60% overspend on those 49 trials (~$89 on a full sweep)**. That is an upper bound (it prices every cached token at the 0.1× read rate and ignores the 1.25× write premium), and the opus discount is structurally smaller than luna/sol's because opus emits ~3.5× more uncacheable output per trial, which sets a floor on what caching can recover. There is also a plausible **second-order effect**: Databricks appears to meter its per-minute rate limit on *uncached* input tokens, so the missing cache also cost rate-limit headroom — the opus endpoint lost 63% of its trials to fatal 429s while running alone at one-third of a GPT endpoint's raw throughput. This is a hypothesis, not a proven mechanism (the only zero-cache condition is also the only Anthropic endpoint), but it is the reading that explains the throttling with one rule instead of two. ## Post-fix results (provisional — first trials of an in-flight re-run) On 8 tasks matched by name across the two runs, cost fell **$8.36 → $1.77 (4.71×)** and wall-clock **12,423 s → 1,085 s (11.45×)**. | Metric | before (`4a955a858`) | after (`3bef1f6a`) | |---|---|---| | Cache reads as % of input | **0.0%** | **78.7%** (still climbing toward the ~86% steady state) | | `cost_usd_no_cache_discount / cost_usd` | **1.00×** | **2.18×** (tracking the projected ~2.5×) | | Trials with a fatal 429 (same `-n 20`) | **63%** | **15–19%** | To be clear about attribution: **~2× of that is the clean prefill saving from caching itself**; the rest is second-order — cached requests burn far less rate-limit budget, so they stall less and redo less destroyed work. The 11.45× is a system-level result specific to this throttled workspace, not a caching benchmark. Quality held (7/8 solved in each run). A controlled low-`-n` A/B (neither arm hitting a 429), which the `BUZZ_AGENT_PROMPT_CACHING` opt-out exists to enable, is still owed before this becomes a published claim. ## What changed ### 1. Request caching (`llm.rs`, `config.rs`) `anthropic_body()` emits ephemeral `cache_control` breakpoints, gated by `BUZZ_AGENT_PROMPT_CACHING` (**default on**, `=0` to opt out): - **Static prefix** — marker on the `system` block. Prefix order is `tools → system → messages`, so this single marker caches **tools + system** together. Byte-identical on every turn of a run, and survives a context handoff (system/tools come from cfg/mcp, not `self.history`). - **Rolling tail + leapfrog** — marker on the last block of the last **two** messages. The append-only history re-reads the prior turn's prefix from cache; marking two messages (not one) keeps consecutive breakpoints inside Anthropic's **20-block lookback window** even as tool parallelism rises, avoiding a silent full-price miss. An empty system prompt stays a bare string (Anthropic rejects empty text blocks), and below-threshold prefixes are silently not cached, so the flag is safe on by default. ### 2. Surface the cache split end-to-end — the plumbing (`types.rs`, `llm.rs`, `agent.rs`, `lib.rs`, `usage.rs`, `acp.rs`) This is the part that makes gaps like the one above **visible** instead of silent. A consumer that prices all of `input_tokens` at the full rate can't tell a route that's caching from one that isn't — the total looks right either way. So: - `LlmResponse` gains `cached_input_tokens` (a **subset** of `input_tokens`, never an addition); `parse_anthropic` / `parse_openai` / `parse_responses` each populate it. - A `usage_first()` helper reads the cache count wherever a provider hides it — flat `cache_read_input_tokens` (Anthropic), `prompt_tokens_details.cached_tokens` (OpenAI chat), `input_tokens_details.cached_tokens` (Responses) — taking the **first present value, never a sum**. Reading only flat keys is exactly why the OpenAI route's nested `cached_tokens` had *also* been going unclaimed: `prompt_tokens` is already inclusive, so the total looked correct while the discount silently went unreported. - The per-turn/per-session accumulators and the goose `usage_update` payload now carry `accumulatedCachedInputTokens`; `buzz-acp` deserializes it (`serde` default `0` for goose, which doesn't send it) and logs `cached=<n>`. ### 3. Fix a Databricks MLflow-route double-count (`llm.rs`) The Databricks MLflow route reports the flat Anthropic-spelled `cache_read_input_tokens` *alongside* an already-inclusive `prompt_tokens`, so the old code summed them and nearly doubled the count — inflating both the context-budget gate and cost. `openai_chat_input_tokens()` now reads `prompt_tokens` alone. Verified on a live `databricks-glm-5-2` response where `prompt_tokens + completion == total` proves inclusivity. (Anthropic's native route genuinely *excludes* the cache fields and is still summed — the two never collide, because `claude*` models route to the Anthropic path.) ### 4. Proxy + TLS-trust passthrough into MCP tools (`mcp.rs`) — independent fix `buzz-agent` `env_clear()`s each MCP child, and the allowlist carried no proxy/TLS vars. On a proxy-only host that doesn't degrade the tools, it **blinds** them: apt, curl, pip, git connect directly, the egress firewall resets the socket, and the agent reports "Connection reset by peer" — indistinguishable from a genuinely offline task. Adds both spellings of `HTTP(S)_PROXY`/`NO_PROXY`/`ALL_PROXY` (curl/git read lowercase; Go/Python read uppercase; libcurl ignores uppercase `HTTP_PROXY`) plus `SSL_CERT_FILE`/`SSL_CERT_DIR` for TLS-terminating proxies that present their own CA. ## Testing - `cargo fmt --all -- --check`, `cargo clippy -p buzz-agent -p buzz-acp --all-targets -- -D warnings` — clean. - `cargo test -p buzz-agent -p buzz-acp` — **all green** (632 + 299 lib tests plus integration suites, 0 failures). New tests cover: the three breakpoints and the disabled/empty-system/single-message edge cases; nested-vs-flat cache parsing for all three routes; the Databricks inclusive-`prompt_tokens` fix; wire deserialization of `accumulatedCachedInputTokens`; and the proxy/TLS passthrough allowlist. - Pre-push lefthook suite green (branch-skew, rust-tests, test, desktop-check/test/tauri). ## Relationship to the benchmark branch These are the non-`benchmarks/` changes from `benchmark/harness-accounting-and-solo`, lifted onto a clean base off `main` so they can merge independently. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Atish Patel <atish@squareup.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
90e058ebf6 |
feat: add explicit entry for claude-opus-5 in model config (#2831)
Fixes #2787 - Added `claude-opus-5` to `config.rs` model classification and adaptive effort helpers. - Updated fixture test configurations to cover `claude-opus-5`. - Verified with `cargo test` and JS unit tests. Signed-off-by: Apurva Shaw <apurvashaw@Apurvas-MacBook-Air.local> Co-authored-by: Apurva Shaw <apurvashaw@Apurvas-MacBook-Air.local> |
||
|
|
18eef633d8 |
feat(git): use agent display name as git author name (#3040)
Agent commits were authored by a raw 63-character npub, which makes `git log`, `git blame`, and GitHub's author column effectively unreadable. This uses the agent's display name for `user.name` instead, while leaving the pubkey where it does real work. ## What changes `build_git_env` in `crates/buzz-dev-mcp/src/shim.rs` now reads `BUZZ_ACP_DISPLAY_NAME`, sanitizes it, and uses the result as `user.name`. When the variable is absent or unusable it falls back to `info.npub` — byte-identical to today's behavior. `user.email`, `user.signingkey`, and the whole credential/signing block are untouched. The pubkey is what NIP-98 auth, NIP-GS signing, and contributor matching key on, and it stays in the email verbatim. `crates/buzz-acp/src/lib.rs` forwards the variable into the dev-mcp server's declared env, mirroring the existing `BUZZ_AUTH_TAG` block. It reads `std::env::var` directly rather than going through `Config`, so the variable is picked up whenever the process has it. `crates/buzz-agent/src/mcp.rs` adds one `PASSTHROUGH_ENV` entry so ACP clients that spawn `buzz-agent` without declaring the variable on the wire still propagate it. ## Why a dedicated variable `BUZZ_ACP_DISPLAY_NAME` is its own contract rather than a reuse of the ACP session title. Commits outlive sessions: a session title is per-session UI chrome and may be composed downstream into `Agent · #channel`, and if that composed form ever reached the env var, git attribution would change silently with no test able to catch it. Git identity gets a variable whose contract is "bare agent display name, never channel-qualified." Nothing writes it yet — a one-line Desktop write lands as a follow-up. Until then `std::env::var` returns `Err`, the npub fallback fires, and behavior is byte-for-byte current `main`. ## Sanitizing Strip control characters, Unicode format characters, and angle brackets; collapse whitespace runs, trim, cap at 80 characters (by `chars()`, so a multi-byte name is never split mid-UTF-8). Angle brackets go because git drops them silently rather than erroring: `Duncan <evil@x.com>` renders as `Duncan evil@x.com <hex@relay>`. It forges nothing, but it reads as though it might. The empty result also has to cover more than literal emptiness. git's `ident.c` treats a set of characters as "crud" — stripped from both ends, and fatal when a name is *nothing but* those characters: ``` $ git -c user.name=';;' commit -m t fatal: name consists only of disallowed characters: ;; ``` Verified against git 2.54.0 by committing with each ASCII byte 32..=126 as the entire `user.name`: exactly space, `"`, `'`, `,`, `:`, `;`, `<`, `>`, `\` abort, plus all control characters (the predicate is `c <= 32`). `.` is not crud in this version, despite older lore. Names that merely *contain* crud are fine — `O'Brien` and `Smith, Jr.` both commit cleanly — so the check is "at least one non-crud character survives," not "no crud present." Without it, a display name of `;;` or `""` would abort every commit that agent makes. ## Unicode format characters `char::is_control` covers only category `Cc`. Category `Cf` — zero-width spaces and joiners, bidi embedding and override marks, invisible math operators, tag characters — is neither control, nor whitespace, nor git crud, so those characters survived every one of the checks above. A display name of nothing but U+200B ZERO WIDTH SPACE therefore satisfied "at least one non-crud character survives" and git accepted the commit with a visually blank author: ``` # pre-fix, BUZZ_ACP_DISPLAY_NAME set to two U+200B $ git log -1 --format='%an' | xxd -p e2808be2808b0a ``` Embedded marks were the other half: a trailing U+202E RIGHT-TO-LEFT OVERRIDE reorders everything after it, so a stored author line renders as something other than what it stores — the same confusion class the angle-bracket filtering exists to prevent. `is_unicode_format` rejects the whole `Cf` category rather than the known-bad marks, because the boundary that matters is "invisible or reorders text", not "the codepoint someone thought of". The 21 ranges come from the UCD's `DerivedGeneralCategory.txt` (17.0.0), cross-checked against Python's `unicodedata` (16.0.0); both yield exactly the same set. They are inlined as a `matches!` rather than pulling in a Unicode-tables crate for one predicate, and a test asserts both endpoints of every range plus the codepoints immediately outside them — including U+2065, which sits inside the U+2060 block but is unassigned rather than `Cf`. Filtering happens inside the existing per-word filter, so a format-only name collapses to empty and falls out through the same `None` → npub path as a crud-only name. No new fallback logic. And because filtering precedes truncation, invisible padding cannot eat the 80-character budget. ## NUL is handled one layer up An interior NUL is a sibling constraint that cannot be fixed here: it makes `Command::env` fail the entire spawn before this code runs, so it has to die at the writer. #3028 establishes that pattern for the session title in `resolve_session_title` via `filter(|c| !c.is_control())`, and the Desktop follow-up that writes `BUZZ_ACP_DISPLAY_NAME` inherits it. The shim sanitizer is a second line of defense for values that arrive from somewhere other than Desktop. ## Verified end to end Driving the real `buzz-dev-mcp` binary over stdio MCP and committing inside its shimmed environment: ``` # BUZZ_ACP_DISPLAY_NAME="Duncan Idaho" Duncan Idaho <dcfd242e...0f95@buzz.block.builderlab.xyz> verify_exit=0 # BUZZ_ACP_DISPLAY_NAME unset npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e...0f95@buzz.block.builderlab.xyz> verify_exit=0 # BUZZ_ACP_DISPLAY_NAME=";;" (crud-only; would otherwise be fatal) npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e...0f95@buzz.block.builderlab.xyz> verify_exit=0 # BUZZ_ACP_DISPLAY_NAME=U+200B U+200B (format-only; would otherwise be blank) npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e...0f95@buzz.block.builderlab.xyz> verify_exit=0 # BUZZ_ACP_DISPLAY_NAME="Duncan" + U+202E (bidi override stripped) Duncan <dcfd242e...0f95@buzz.block.builderlab.xyz> verify_exit=0 # BUZZ_ACP_DISPLAY_NAME="Dun" + U+200B + "can" (zero-width removed, word not split) Duncan <dcfd242e...0f95@buzz.block.builderlab.xyz> verify_exit=0 ``` Signature verification passes in every case — the signing identity is unchanged. `Related: #3028` — it establishes the Desktop-side env plumbing this builds beside; the one-line Desktop follow-up that writes `BUZZ_ACP_DISPLAY_NAME` alongside the session title ships after it merges. Not a dependency: with the variable absent, `std::env::var` returns `Err` and the npub fallback keeps current behavior exactly. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
|
16d4ec335e |
feat(desktop): use collective mesh routing for Auto (#2825)
Uses MeshLLM built-in `mesh` collective intelligence / Mixture of Agents when Buzz Auto sees two or more distinct physical models. With zero or one model, Auto remains ordinary `auto`. This is an alternative way to improve tool responses, accuracy, and resistance to hallucination when a high-latency distributed mesh contains diverse models. Models and members may come and go: collective routing enables only after stable capacity, drops on confirmed contraction, and can recover later. Mesh-specific failures retry once through ordinary Auto. This update also pins MeshLLM to a v0.73.1-compatible backport of [MeshLLM #1074](https://github.com/Mesh-LLM/mesh-llm/pull/1074), so client-only Buzz nodes cannot enter model election or download a remote provider model. Buzz preserves the selected local sharing model and switches an existing client to sharing across a controlled app restart, retaining one runtime and one `:9337` / `:3131` pair per machine. Validation: - Full local `just ci` passes on the cleaned branch. - MeshLLM host-runtime suite: 1,568 passed, 0 failed; strict Clippy passes. - Buzz desktop Tauri suite with `mesh-llm`: 1,721 passed, 0 failed; strict feature Clippy passes. - Playwright covers client-to-share using the saved local model and no destructive stop. - Packaged two-machine testing proved single-model routing, dual-model collective routing, tool-markup fallback, and runtime reuse. - Packaged client-only recheck routed a real Mini Buzz turn through M5 while Mini stayed `is_client=true`, `is_host=false`, hosted no models, and created no Gemma cache. Builds on the recovery work merged in #2823; this PR does not duplicate it. --------- Signed-off-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> |
||
|
|
8eb6e3eb60 |
fix(agents): run live Databricks discovery instead of the fallback list (#2890)
## Problem
The Databricks model dropdown offers a handful of stale models — and
there's no way to tell that list apart from the real one. The AI Gateway
exposes **66** chat/embedding endpoints on `block-lakehouse-production`,
but the picker was showing a short list that includes models the gateway
no longer serves and embedding endpoints that can't chat at all.
Three independent defects, all on the discovery path:
**1. Live discovery never ran for agents with no saved provider.**
`get_agent_models` gates every in-process discovery attempt on the
provider (`is_openai_compatible_provider` / `is_anthropic_provider` /
`is_databricks_provider`), reading it straight from `record.provider`.
That field is `null` for every agent record created before provider
persistence — and for any agent that inherits its provider from the
build. So all three gates saw `None`, no HTTP discovery ran, and the
request fell through to the `buzz-acp models` subprocess. On the
Databricks path that subprocess returns `discovery_failure_fallback` —
the small hardcoded `DATABRICKS_V2_KNOWN_MODELS` catalog — which the
frontend renders exactly like a live catalog. An internal DMG that bakes
`BUZZ_AGENT_PROVIDER=databricks_v2` and a `DATABRICKS_HOST` still got
the fallback.
**2. The fallback list couldn't represent the running model.**
When discovery genuinely fails, the picker should at minimum be able to
show what the agent is actually configured with. For `DatabricksV2` it
couldn't: the fallback returned only the hardcoded slate, so a model
like `databricks-gpt-5-5` wasn't selectable in its own picker.
**3. Embedding endpoints were offered as chat models.**
`databricks-bge-large-en` was selectable (visible in the dialog today).
The v2 endpoints payload carries no `task` or `state` field, so there is
nothing to filter on but the name.
## Changes
- **`effective_discovery_provider`** (new,
`desktop/src-tauri/src/commands/agent_models_env.rs`) — an explicit
provider (saved record value, or the create/edit dialog's current form
value) still always wins; when there is none, discovery falls back to
the runtime's own provider env var (`GOOSE_PROVIDER`,
`BUZZ_AGENT_PROVIDER`, …) read off the merged env, which by that point
already carries the baked build floor and the process env. Wired into
both `get_agent_models` and `discover_agent_models`.
`SavedAgentModelDiscoveryConfig` now carries `provider_env_var` from
`known_acp_runtime`, so each runtime reads *its own* key rather than a
shared guess.
- The relay-mesh branches in `discover_agent_models` deliberately keep
using `input.provider`: those key off a deliberate provider selection,
never a baked default.
- **Asserted vs inferred matters for missing credentials.** The OpenAI
and Anthropic gates error on a missing API key, while the Databricks
gate falls through; an inferred provider hitting the first two would
have replaced a working subprocess catalog with `config:
ANTHROPIC_API_KEY required` (`export GOOSE_PROVIDER=anthropic` is
goose's documented way to pick a provider, and it keeps the key in its
own keyring). So `effective_discovery_provider` returns a
`DiscoveryProvider` that remembers how the value was resolved, and
`required_env` only reports a missing credential for an asserted
provider. A wrong guess declines and lets the subprocess answer.
- **`is_chat_capable_endpoint`** (new,
`crates/buzz-agent/src/catalog.rs`) — applied in
`parse_v2_endpoints_page`. Drops `*embedding*` and segment-matched `bge`
/ `gte` endpoints; keeps everything unrecognised (fail-open, so a new
model family is never hidden). Segment matching is why it's `split('-')`
and not `contains`: a substring check would swallow legitimate names.
- **`discovery_failure_fallback`** for `Provider::DatabricksV2` now
leads with the configured model (deduped against the known slate,
blank-tolerant), so a failed discovery still yields a picker that can
show the running model. The configured model is trimmed once up front —
`resolve_model` doesn't trim, so a padded `DATABRICKS_MODEL` used to
slip past the dedupe and appear twice.
- **`sort_v2_endpoints_newest_first`** (new, second commit) — the
catalog is now ordered newest-first on each endpoint's
`created_timestamp`, ties broken by name. Previously Buzz sorted
nothing, so the gateway's own order reached the picker: it pages in two
phases (Databricks-managed, then workspace-created — the page token
decodes to `{"phase":"user"}`), each alphabetical, which buried
`databricks-claude-opus-5` 8th behind five older Claude endpoints and
`goose-claude-opus-5` — the newest endpoint in the catalog — 55th of 63.
Sorting in `fetch_v2_models` means both discovery paths inherit it with
no wire or type changes, and the combobox filter preserves incoming
order. Endpoints with an absent or unparseable timestamp sort last
rather than first, so a wire-shape change degrades to "unordered at the
bottom" instead of "shuffled to the top".
- The name tiebreak is load-bearing: eleven managed endpoints share one
placeholder timestamp (`1699610000000`), so without it their relative
order would vary between runs. That placeholder is also not always
accurate — a few genuinely recent endpoints
(`databricks-kimi-k2-7-code`, `databricks-llama-4-maverick`) land at the
bottom with the 2023 batch. The gateway offers nothing better to sort
on.
- Env/provider lookup helpers moved out of `agent_models.rs` into
`agent_models_env.rs`. This keeps the command module under the file-size
limit **without ratcheting the override up** — the existing 1079 entry
is untouched (file is now 1066 lines).
## Verification
Live against `block-lakehouse-production`, release build:
```
BUZZ_ACP_AGENT_COMMAND=$PWD/target/release/buzz-agent \
BUZZ_AGENT_PROVIDER=databricks_v2 \
DATABRICKS_HOST=https://block-lakehouse-production.cloud.databricks.com \
DATABRICKS_MODEL=databricks-gpt-5-5 \
./target/release/buzz-acp models --json
```
- before: 66 endpoints, including `databricks-bge-large-en`,
`databricks-gte-large-en`, `databricks-qwen3-embedding-0-6b`
- after: **63** endpoints, `[.models[] | select(.id |
test("embedding|-bge-|-gte-"))]` → `[]`
Top of the list after the sort commit:
```
goose-claude-opus-5 2026-07-24
databricks-claude-opus-5 2026-07-23
databricks-gemini-3-6-flash 2026-07-20
databricks-gemini-3-5-flash-lite 2026-07-20
databricks-inkling 2026-07-14
```
Tests: 15 new (8 in `catalog.rs` — including the two-wire-shape
timestamp parse, the sort's tiebreak/no-timestamp cases, and the
padded-model dedupe — and 7 plus one assertion in
`agent_models_tests.rs`, 3 of them covering the asserted/inferred
credential split), two existing tests updated. `just check`, `just
test-unit`, and `just desktop-tauri-test` all pass (1636 desktop-tauri
tests, 274 buzz-agent lib tests).
Not run locally: the Docker-backed integration suite (`just test`) —
this diff touches neither `buzz-relay`, `buzz-db`, nor `buzz-auth`.
## Follow-ups (deliberately out of scope)
Two inference-path defects found while investigating, both reproduced
live against the gateway and both independent of discovery:
1. **Gemini thought signatures are dropped.** The gateway returns a bare
`thoughtSignature` on tool calls; the external-model serving endpoints
return it nested as `extra_content.google.thought_signature`. Neither
shape is round-tripped, so multi-turn tool use on `databricks-gemini-*`
fails with a 400 on the second turn.
2. **Array-shaped `content` is silently discarded.** Some models return
OpenAI `content` as a block array rather than a string; `parse_openai`'s
`str_field` returns `None` and the text is dropped.
The legacy `serving-endpoints` path does not work around either one, and
costs reasoning support on the GPT-5 family.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
|
5afa16157a |
fix(desktop): suppress Windows console flashes and reject WSL bash alias (#2587)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
|
cd6b573f8d |
fix(buzz-acp,buzz-agent): surface stall duration and fate (#2204)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |
||
|
Will PflegerandGitHub
|
0f86a608b9 | fix(timeout): unified turn-timeout fix — cap inheritance, steer renewal, activity-aware requeue, LLM stall surfacing (#2175) | ||
|
thomaspblockandGitHub
|
7f769b2439 |
fix(desktop): align inbox layout and draft details (#2143)
Approved |
||
|
|
ca8324d2cf |
Log handoff token counts (#1954)
Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> |
||
|
morgmartandGitHub
|
54e174b5e1 | Chat-first agent creation via portable buzz CLI drafts (#1878) | ||
|
|
020ac7f405 |
fix(desktop): resolve Git Bash for Windows shell tool via PATH/git/registry fallback (#1821)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |
||
|
|
e34e0974a1 |
fix(buzz-agent): support max effort for gpt-5.6 family (#1806)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |
||
|
|
77c08365e4 |
feat(desktop): global agent config defaults with readiness/spawn parity (#1448)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |
||
|
Will PflegerandGitHub
|
9b244ff22b | fix(agents): surface provider errors structurally instead of raw dumps (#1653) | ||
|
Will PflegerandGitHub
|
3c6c0d4478 | feat(acp,desktop): identify and reap stale agent harness processes (#1582) | ||
|
|
08e707eb57 |
feat(acp,buzz-agent): thread model name through NIP-AM kind 44200 emit path (#1564)
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |
||
|
|
71265ca361 |
feat(nips,relay,acp): NIP-AM durable encrypted agent turn metrics (kind 44200) (#1441)
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1fgdl5qqnh3k3f2xkqrvt7cujalhm623x4s7fdjdj5yrtp5fzjl9qrjpucw <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@sprout-oss.stage.blox.sqprod.co> |
||
|
|
4a09510429 |
fix(agent): make stop-hook rejection budget per-prompt, fix stale hook docs (#1503)
Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Brain <21994759fc7a6fa6b965551d35cfd7897d262f2495467f2d78694ddcfa6a5c7e@sprout-oss.stage.blox.sqprod.co> |
||
|
|
1c297d2f26 |
fix(agent): honor stop hook retry budget (#1501)
Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Pinky <44b8e82baa6e0e254e0208d68f335c283c94e7b78dd1fa10d5a49d3f13dd0435@sprout-oss.stage.blox.sqprod.co> |
||
|
|
228122fddc |
Add agent catalog modal (#1302)
Signed-off-by: Taylor Ho <taylorkmho@gmail.com> Co-authored-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Taylor Ho <taylorkmho@gmail.com> |
||
|
|
3e282a2418 |
feat(buzz-agent): config parity — thinking effort, model switching, normalized token limits (#1470)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |
||
|
|
d9c4e4aa7f |
fix: agent reliability — no restart on channel-add, visible dead-letter notice (#1468)
Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Brain <21994759fc7a6fa6b965551d35cfd7897d262f2495467f2d78694ddcfa6a5c7e@sprout-oss.stage.blox.sqprod.co> |
||
|
|
9f2a11b338 |
feat(buzz-agent): emit agent_thought_chunk for reasoning content (#1436)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> |
||
|
|
eacbbe880a |
feat(desktop): provider-agnostic model selection + databricks discovery (#1307)
Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1fgdl5qqnh3k3f2xkqrvt7cujalhm623x4s7fdjdj5yrtp5fzjl9qrjpucw <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@sprout-oss.stage.blox.sqprod.co> |
||
|
Kalvin CandGitHub
|
15a73aa27f | feat(buzz-agent): add databricks_v2 provider for AI Gateway v2 (#1311) | ||
|
|
e567491a19 |
feat(buzz-acp): steering as the default mid-turn mention delivery (#1160)
Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1fgdl5qqnh3k3f2xkqrvt7cujalhm623x4s7fdjdj5yrtp5fzjl9qrjpucw <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@sprout-oss.stage.blox.sqprod.co> |
||
|
|
744c77bc2d |
fix(buzz-agent): charge images a token-equivalent for the handoff gate (#1332)
Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> |