mirror of
https://github.com/block/buzz.git
synced 2026-08-18 06:50:31 +02:00
Atish PatelandClaude Opus 5
4c931d8e8e
An OpenRouter model id is a family of upstream deployments, not one deployment. `deepseek-v4-flash-0731` is served by nine upstreams spanning fp4 to fp8 quantization, 262K to 1M context and a 1.6x price spread; `kimi-k3` by ten spanning 1.5x. Unpinned, consecutive requests land on different ones, so a benchmark cell measures a mixture whose composition moves with provider load rather than a fixed condition. It also decides whether prompt caching happens at all. Measured today against a fixed 21.8K-token prefix, three calls each: deepseek, unpinned cached 0, 0, 21760 (GMICloud, Cloudflare x2) deepseek gmicloud/fp8 cached 21760 x3 $0.00059/call deepseek siliconflow/fp8 cached 0 x3 $0.00306/call kimi, unpinned cached 0, 0, 20480 (Fireworks, Moonshot x2) kimi moonshotai/mxfp4 cached 20480 x3 $0.00715 vs $0.06245 That is a 5-9x swing in input cost for identical work, decided by routing luck. The `/endpoints` metadata is no guide: it advertises `supports_implicit_caching: false` for every endpoint measured caching above, and `true` only for one that is not routable on this account. So: `OPENROUTER_PROVIDER_ORDER` (comma-separated, accepting both the bare `moonshotai` slug and the `gmicloud/fp8` slug/quantization form) sets `provider.order`. It is strictly opt-in -- unset leaves the body byte identical, which is what keeps the existing "no body shape adds a provider routing filter" assertions honest. Paired with `allow_fallbacks: false`, because `order` alone is only a preference: OpenRouter still serves from elsewhere when the named upstream is busy, which is exactly the silent mid-run condition change this exists to prevent. A pin that quietly falls back is not a pin. This is not `require_parameters` -- that filter screens on advertised parameters and 404s otherwise-valid model ids, which is why the reasoning block nearby stays away from it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Atish Patel <atish@squareup.com>