## Summary
CI now proves the full Buzz shared-compute join story end to end: a
member can discover another member's served model **through the Buzz
relay alone** and run inference over the mesh, while a non-member gets
nothing — the relay rejects its auth, and the mesh refuses to route for
it even holding a leaked endpoint address.
This is deliberately different from mesh-llm's own CI smokes (which
bootstrap two nodes with a hand-carried invite token / mdns): here the
**relay is the control plane**, exactly like the desktop app:
1. **Membership** — identities A and B are added via `buzz-admin`
(kind:13534 NIP-43 roster); C is not.
2. **Advertise** — each member publishes a client-signed kind:30003
discovery note carrying its MeshLLM owner binding and (for the serve
node) `serveTargets[].endpointAddr`, covered by an endpoint-binding
signature — the exact payload shape the desktop coordinator publishes.
3. **Trust** — the serve node derives its admission allowlist from the
relay (statuses ∩ roster) and requires the **exact expected {A, B}
owner-id set** before starting with `TrustPolicy::Allowlist`.
4. **Join** — the client verifies owner + endpoint bindings and
membership, then dials the relay-discovered endpoint (the desktop
join-watcher's `dial_endpoint_addr` step). No out-of-band token.
5. **Infer** — a chat completion against the client's local OpenAI
endpoint routes over QUIC to the serve node's model (CPU, SmolLM2-135M,
~105MB).
6. **Deny (differential)** — the stranger's NIP-42 auth must fail with
the relay's own membership rejection (`restricted: not a relay member` —
successful auth or any unrelated connect error fails the run), and
dialing the leaked endpoint must not produce a routed inference —
**while the trusted client re-proves inference immediately afterwards**,
so a dead serve node can't masquerade as an admission denial.
## What's in the PR
- `crates/buzz-relay/examples/mesh_relay_lifecycle_smoke.rs` — the
harness. One process per node (mesh-llm keeps process-global state under
`~/.mesh-llm`), orchestrator + serve/client/stranger roles,
byte-identical binding payloads to
`desktop/src-tauri/src/mesh_llm/identity.rs` (called out with
keep-in-sync comments). Child stdout is pumped through a reader thread
so every wait has a hard deadline; timed-out children are killed; exit
statuses are checked.
- `scripts/ci-mesh-lifecycle-smoke.sh` — provisions a membership-gated
relay (throwaway owner + signing identities via `buzz-admin
generate-key`), runs the harness, cleans up. Fails fast if :3000 is
already occupied (a stale open relay would mask gating).
- `scripts/start-relay-for-tests.sh` — gains opt-in NIP-43 membership
env passthrough (`BUZZ_REQUIRE_RELAY_MEMBERSHIP` + `RELAY_OWNER_PUBKEY`
+ `BUZZ_RELAY_PRIVATE_KEY`). Default behavior unchanged.
- `.github/workflows/mesh-lifecycle.yml` — separate, path-filtered,
non-required workflow (mesh paths, the harness's dependency crates,
`Cargo.lock`, dispatch), pinned to `ubuntu-24.04`. Caches the mesh
native runtime + HF model keyed on the lockfile hash, so a mesh pin bump
rolls the runtime cache. Uploads relay + harness logs on failure.
## Scope
This is an **independent protocol harness**: it speaks the same wire
protocol and payload shapes as the desktop but re-implements the
binding/verification logic (the desktop crate is outside the workspace).
Regressions inside the desktop's own discovery filtering are the desktop
unit tests' job; what this smoke proves is that the relay + mesh-llm SDK
+ admission stack support the lifecycle end to end.
## Relationship to mesh-llm's CI
Follows the shape mesh-llm's own CI proved stable (tiny CPU model, one
runner, multiple real mesh-llm processes over real QUIC — cf. their
`ci-two-node-client-serving-smoke.sh`), but swaps the token bootstrap
for the relay-driven lifecycle, which is the part only Buzz can test.
## Validation
Green on GitHub Actions (ubuntu-24.04) across three runs, including
after rebases onto the mesh v0.74 upgrade (#3467) and latest main:
```
PASS 1/6: relay-derived allowlist is exactly {A, B}
PASS 2/6: serve member ready + advertised model: jc-builds/SmolLM2-135M-Instruct-Q4_K_M-GGUF:Q4_K_M
PASS 3/6: client member discovered + joined via relay
PASS 4/6: inference routed over the mesh: "PONG"
PASS 5/6: relay rejected the stranger's NIP-42 auth (membership gate)
PASS 6/6: stranger denied (gossip visible, inference rejected: 503 all tunnels failed) while trusted inference still routes
PASS: full relay-driven mesh lifecycle verified
```
Also validated locally on macOS. `cargo fmt --all --check` and `cargo
clippy -p buzz-relay --all-targets -- -D warnings` pass.
## Notes
- The harness follows the repo's mesh `[dev-dependencies]` pin
automatically, so it doubles as a canary for future mesh upgrades (it
already caught the v0.73.1 → v0.74.0 bump during development).
- The stranger "deny" accepts either shape mesh-llm exhibits: no model
visibility at all, or gossip visibility with inference refused —
mesh-llm applies the receiving node's owner policy after the gossip
handshake, so admission gates *routing*, not gossip. The differential
trusted-inference re-check (PASS 6/6) is what makes that a real denial
rather than a dead server.
- Model-visibility windows are tunable via `MESH_CLIENT_WINDOW_SECS` /
`MESH_STRANGER_WINDOW_SECS` if shared runners prove slow — pin a longer
window in the workflow env rather than re-running the job.
---------
Signed-off-by: Michael Neale <michael.neale@gmail.com>
## What
Two changes, both fallout/follow-up from #4289 landing:
### 1. Fix the Security job failing on main (lockfile-only)
Eight RUSTSEC advisories published today against the nostr stack turned
`cargo-deny check` advisories red on main ([failing
run](https://github.com/block/buzz/actions/runs/30761611723/job/91533106673)).
Not introduced by #4289 — the advisories landed upstream and any push to
main today would have tripped them.
- **RUSTSEC-2026-0225..0230** → `nostr` 0.44.6 → **0.44.7** (Debug
output exposing NIP-46/NIP-60 credentials; wallet parsers accepting
unauthenticated events; NIP-44/NIP-04/NIP-98 resource exhaustion; NIP-50
empty-filter panic)
- **RUSTSEC-2026-0231..0232** → `nostr-relay-pool` 0.44.2 (root) /
0.44.1 (tauri) → **0.44.3** (auth-challenge memory exhaustion;
processing of unverified relay events)
Both workspace lockfiles bumped (`Cargo.lock`,
`desktop/src-tauri/Cargo.lock`). No manifest changes.
### 2. Default the desktop GUI's sprig image to the published
`ghcr.io/block/buzz-sprig`
The first main-push after #4289 published the image publicly (package
created 18:44Z, visibility `public`). The `config_schema()`'s `image`
property now carries a `default`:
```
ghcr.io/block/buzz-sprig:sha-6530b58@sha256:17facfc7608d8ddb33bc056c9aaba1098f4ef6abe5655702fbfd7584d1f74d76
```
**Why tag+digest, not tag:** the backend deliberately rejects tag-only
references — the pod runs with the agent's nsec and tags are mutable
pointers (`image.rs` §Image). The tag+digest form keeps the
human-traceable `sha-6530b58` while the digest does the pinning;
`image::parse` already normalizes it to the tagless canonical form, so
create-intent fingerprints are identical to the bare-digest spelling.
The digest is the **multi-arch manifest-list digest** (amd64+arm64),
resolved via `docker buildx imagetools inspect`.
**This is a UI prefill, not a baked fallback:** `image` stays in the
schema's `required` list, an empty value still fails closed with a named
field, and the desktop submits the value explicitly in `provider_config`
(the `WhereToRunSection` probe seeds `providerConfig` from schema
defaults) — so deploy fingerprints never depend on compiled-in provider
state, and the spec's §K8s pod-reconciliation concern about
baked-default divergence is not engaged. Module prose that said "no
published image exists yet" is updated to match reality.
No desktop code changes needed: the form already prefills from
`properties[*].default` and submits seeded defaults.
## Testing
- `cargo-deny check` at head: **advisories ok, bans ok, licenses ok,
sources ok** (was: advisories FAILED)
- `cargo test -p buzz-backend-kubernetes`: **158 passed** (154 lib + 4
wire), including new `schema_default_image_round_trips_through_parse`
pinning the constant + its normalization, and the wire `info` test now
asserting the default is present in the provider's real stdout response
- Live provider probe: `{"op":"info"}` against the built binary returns
the default in `config_schema.properties.image.default` with `required`
unchanged (`["namespace","image"]`)
- Full workspace test suite via pre-push hook: green (earlier direct
`cargo test --workspace` run: sole failure was
`api::mesh_demo::demo_join_forwarded_arm_round_trips_echo`, the
documented pre-existing main flake — unrelated, fails on base)
- Image existence verified against GHCR: `docker buildx imagetools
inspect ghcr.io/block/buzz-sprig:sha-6530b58` resolves to the pinned
manifest-list digest with linux/amd64 + linux/arm64 manifests
---------
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
# Kubernetes backend plugin (crates/buzz-backend-kubernetes) + desktop
deploy path
Implements docs/remote-agents.md (merged @ 28ae6cd21) as ONE PR: the
provider
binary, the desktop changes that make it work, the harness inactivity
reaper,
the Sprig image, and the conformance/live-test suites.
Channel: buzz-remote-agents (29414326-dba7-402d-b384-b1b34d63a2e6),
thread c42b70ef.
## What's here (by lane)
- **crates/buzz-backend-kubernetes** (Dawn): stdin/stdout JSON provider,
info +
deploy; pure classify.rs (one match arm per spec state-machine row);
reconcile/GC with ownership-marker gate + same-clock orphan check;
per-attempt
immutable Secrets; three-tier env with clear-then-write authoritative
tier.
- **Desktop** (Mari): KD3 launch block from resolved descriptor, KD5
pre-secret
negotiation gate (resolve-once → stage-and-digest → info → protocol gate
→
deploy), KD1 Windows extension strip, bundling (externalBin + Justfile +
release/canary workflows + stub loops), tauri.windows.conf.json platform
override (Decision B: no Windows artifact).
- **buzz-acp** (Max): KD4 BUZZ_ACP_EXIT_AFTER_INACTIVITY reaper
(pool-independent;
reset only at accepted dispatch; in-flight turn/heartbeat defers, never
resets);
BUZZ_ACP_EXIT_AFTER_INACTIVITY + BUZZ_ACP_NO_PRESENCE reserved. KD8 fix.
- **Image + tests** (Perci): Dockerfile.sprig (digest-pinned bases, exec
buzz-acp
PID 1, relay-scoped credential config), image contract script, provider
conformance suites (golden wire fixtures shared with desktop tests),
live-local
runbook (namespace-scoped, shared-cluster safe).
- **Docs** (Sami, first commit): citation re-pin c1bca1b56 → 28ae6cd21
(44/49
were already byte-exact; 3 offsets fixed) + I3 presence-bound correction
(below).
## Named spec deviations (deliberate, each with rationale)
1. **No baked default image yet.** ghcr.io/block/buzz-sprig is
unpublished
(verified: anonymous pull 403 vs control 200). Omitted `image` returns
an
in-band field-required error instead of a default.
2. **Image override STRICTER than spec §Image:** digest-only
(`name@sha256:<64hex>`); ALL tags rejected; `name:tag@digest`
normalized.
With no baked default the override is the only path, so tag-acceptance
would
make mutability the v1 norm. Strictness is reversible; a moved tag under
an
nsec is not. Baked digest default + tag re-acceptance = follow-up with
image
publish.
3. **imagePullSecrets not in schema (v1).** Explicit user images may
rely on
namespace-preprovisioned pull credentials — the substrate boundary.
Field
added only if the publish decision proves it necessary. 9-field budget
intact.
4. **Decision A closed: writable empty workspace.** Nest projection =
named
follow-up; no image-side scaffolding.
5. **Decision D overridden by Tyler (event b55398d8):** provider ships
bundled
with the desktop like buzz-acp/buzz-agent; spec §Distribution's separate
release workflow deleted for v1.
6. **I3/vision presence bound corrected 90s → 180s.** PRESENCE_TTL_SECS
moved in
#3783 during this spec's base→merge window; the number was inherited,
not
chosen. Spec :206/:216/:928 + inline quote + VISION_REMOTE_AGENTS.md:59
corrected. ← Tyler: the vision is your document; this edit is flagged
for
your explicit eyes.
7. **Spec citations are pinned to 28ae6cd21** (main at spec merge) and
resolve
there, not at this PR's head — this PR's own lanes move
crates/buzz-acp/src/lib.rs by ~100 lines (19 citations across
KD4/KD6/KD7/
§Stop/§Launch data). Known Defects rows fixed BY this PR retire on
merge;
the section documents main as of the pin.
8. **KD7 grace tension declared:** pod terminationGracePeriodSeconds=60
vs
KD7's measured ~87s shutdown tail at parallelism 10 (~197s at cap 32).
KD7 is ruled out of scope, so L1-3's "enough grace for full graceful
shutdown" is NOT met at default config — deliberate, resolved by the KD7
follow-up, not silently.
## Question for Tyler
Will ghcr.io/block/buzz-sprig publish PUBLIC? If private-by-policy,
§Image needs
an imagePullSecrets story before the baked-default follow-up can land.
## Out of scope (named follow-ups)
KD6 exit-code contract + KD7 shutdown budget (gate OnFailure), OnFailure
restart
policy, Windows provider binary, PVCs/nest projection, mesh
deployability,
sprig image publish workflow + baked multi-arch digest default.
## Reproduce locally (four traps that cost us real time)
**1. Git hooks inherit the invoking shell's PATH — pin the shell, not
just your
verification commands.** `rust-toolchain.toml` pins `1.95.0`, but the
rustup shim
that honors that pin lives in `~/.cargo/bin`. If Homebrew's cargo is
earlier on
PATH, `cargo` in this repo is 1.89.0, which cannot build the workspace
at all:
```
$ /opt/homebrew/bin/cargo check -p buzz-db
error: rustc 1.89.0 is not supported by the following packages:
sqlx@0.9.0 requires rustc 1.94.0
... # exit 101
```
Verifying with `PATH="$HOME/.cargo/bin:$PATH" cargo test` does *not*
protect the
push: lefthook's `pre-push` → `just test-unit` re-resolves `cargo` from
the
shell's own PATH, so a green local run is followed by a hook failure on
a crate
you never touched. Export the PATH for the whole shell, not per-command.
This
bit twice.
**2. Line-scope your mutations, or the mutation edits its own
detector.** When
mutation-testing the respond-to guard, a whole-file `sed` on the mode
literal
touches 5 sites — the guard *and* the fixtures/assertions that test it.
The
mutation and its detector move together and the suite stays green, which
reads
as "this code is dead" when it actually means "you deleted the
experiment":
```
# WRONG — 5 sites, guard and tests mutate together
$ sed -i '' 's/"allowlist"/"allowlist-DISABLED"/g' src/env.rs
test result: ok. 145 passed; 0 failed # false survivor
# RIGHT — 1 site, anchored to the guard's own definition line
$ sed -i '' '/^const RESPOND_TO_ALLOWLIST/s/"allowlist"/"allowlist-DISABLED"/' src/env.rs
failures:
env::tests::allowlist_mode_with_an_empty_list_is_refused
env::tests::an_allowlist_entry_that_is_not_64_hex_is_refused
test result: FAILED. 143 passed; 2 failed # real kill
```
Restore by copying a pristine file back and confirming `git diff --stat`
is
empty, not by re-running an inverse `sed`.
**3. A completeness guard is not a correctness guard.** The shared wire
fixture
`tests/fixtures/provider-wire/deploy-full-launch.request.json` passed
every test
we had while containing four classes of invented data (wrong
`respond_to`
encoding, an env key no emitter writes, allowlist entries that fail the
harness's own 64-hex rule, a `launch.env` key from no descriptor layer).
The
provider's tests could not have caught this: its types are deliberately
indifferent to these values (`Option<String>`, `Vec<String>`, arbitrary
map), so
"the provider parses it" was never evidence that the desktop emits it.
The fix
was not a stronger provider assertion but a rule about provenance —
"recorded"
means executed-and-transcribed, and the desktop's whole-object equality
test is
the only enforcement that can exist. See the fixture README.
**4. Every drift this arc was a value that agreed with itself.** Five
invented
values were found, and not one was caught by an assertion failing — each
was
caught by someone asking where a value came from. A named constant
referenced
symbolically on both the fixture and assertion side. A `sed` that
mutated its
own detector. Six probe rows that all died at the same unrelated error.
A
descriptor struct literal compared against a fixture built from that
literal
(`launch.args: ["run","--session"]`, which the resolver actually returns
as
`["acp"]`). The general defense is not more assertions but provenance: a
stub is
a control that varies nothing, and the more faithful it looks the better
it
hides. Ask what executed, not what passed.
*Fixture-test determinism caveat (post-verification, Quinn + Dawn).* The
desktop's whole-object fixture test calls the real resolver, which
consults a
process-global harness registry whose own docs require
`registry_test_lock`
for any test touching it. The fixture test holds no lock and is
nonetheless
deterministic — but by containment, not by ordering. Measured, not
derived:
planting a definition with `id: "goose"` directly into the registry
(bypassing
the loader) changes the resolved descriptor from `args: ["acp"]` to
`args: ["--poisoned"]`, so `resolve_effective_harness_descriptor`
**does**
reach the registry for this id — it does not short-circuit on the
builtin
table first. Two controls discriminate: an empty registry and a registry
poisoned under a *different* id both return `["acp"]`. What actually
protects
the test is that the registry has exactly one writer
(`update_loaded_harness_registry`, reached only via
`warm_harness_registry_from_dir`) — but that writer concatenates **two**
sources of unequal strength (`custom_harnesses.rs:319-326`). Custom
files
pass through `load_custom_harnesses`, whose `check_id_collision` rejects
the
reserved builtin id `goose` case-insensitively at the loader — and that
leg
is tested (`load_applies_id_collision_check` writes a real `goose.json`
and
asserts the loader drops it). Preset definitions
(`preset_harness_definitions`, `presets.rs:177-193`) are a bare `.map`
over
`PRESET_HARNESSES` with **no collision check** — exhaustive call-site
enumeration at `60007fda4` finds four production `check_id_collision`
sites,
none on the preset path. That leg holds only because `goose` is not in
the
preset table today (intersection of TIER1 and preset ids is empty) —
executed, not just read: adding a preset with `id: "goose"`,
`args: ["--poisoned"]` and warming via the normal preset-only path
(`warm_harness_registry_from_dir(None)`, no custom dir, no direct
writer)
flips the fixture's emitted `launch.args` from `["acp"]` to
`["--poisoned"]`
at `60007fda4`, command/env/policy_env unchanged. So: no test in the
suite
can put a `goose` entry in the registry
via the custom path, and no preset currently carries one, so no
interleaving
can perturb this fixture — containment with one checked leg and one
coincidental one. A future fixture built on a **non-builtin** runtime id
has
no containment at all — it would be order-dependent against whatever
registry-writing test ran last and must take the lock.
*Late instance, found while reviewing the mode guard.* The guard
exact-matches
`respond_to` untrimmed and case-sensitively, which is only correct if
clap's
`ValueEnum` derive is case-sensitive. `config.rs` gives two answers: the
derive
at `:448-453` carries no `ignore_case`, while the crate's own tests call
`RespondTo::from_str(s, true)` — `ignore_case = true`. Reading the
source
supports either. Measured on the built binary instead: `owner-only`
starts,
`OWNER-ONLY` / `Owner-Only` / `ALLOWLIST` / `NOBODY` all exit rc=2
`invalid
value`. Case-sensitive at the CLI, so the guard is right — and right for
a
reason the source does not state. The `from_str(_, true)` tests exercise
a
different surface and are not evidence about the CLI.
*Corollary, and the sharper half.* When a test helper **reimplements**
production instead of calling it, the helper is a fork — and a fork can
be
right while production is wrong, or wrong in the same way, and the suite
reports green either way. Both `BUZZ_ACP_ALLOWED_*` gates are forked
like this:
production compares **strings** while the helpers compare **post-parse
enums**
(`config.rs:2623`) or re-derive the split
(`buzz-cli/.../channels.rs:1296`).
Production and the helper each carry their *own* copy of the empty-entry
filter
(`:1025` and `:1300`), so fixing one says nothing about the other.
Measured on
`buzz-cli`, restoring byte-exact between runs:
| tree | result |
|---|---|
| baseline | 274 passed |
| drop the empty-filter in **production** only (the real fix) | **274
passed** — no signal |
| drop it in the **test helper** only | **273 passed, 1 failed**
(`channels.rs:1338`) |
Two independent defects, stacked, and worse together than either alone:
production can be fixed with no test ever noticing, *and* the helper
cannot be
corrected without a false alarm demanding the bug back. The root cause
is one
bit of type information — `check_allowed_channel_add_policy(allowed_raw:
&str,
..)` cannot represent "unset", while production reads `env::var(..) ->
Result`,
where unset and `""` are different states. A helper whose parameter type
can't
represent all of production's input states isn't testing production's
states —
it's testing a subset it silently chose. Same family as the
struct-literal
descriptor and the fixture drift: the test and the thing it tests
agreeing with
each other, rather than the test measuring the thing. Neither defect is
in this
PR's diff (`git diff --name-only 28ae6cd21 <head> -- crates/buzz-cli` is
empty); both are now filed as NIP-34 issues on this repo: the fail-open
+
fork-helper defect at issue event `0524a4113f2d97fd…` and the respond-to
self-lock at `e32837498969b5e7…` (filed 2026-08-02 after Quinn measured
that
no prior filing existed — zero hits on GitHub `block/buzz` open *or*
closed
and zero on the relay's kind:1621 issues, against working positive
controls). The prescription was itself
mutation-tested before being written down: repairing the fork's
signature
(`Option<&str>` + assertion → `None`) still let the reintroduced
production
bug ship 274-green — an expressive fork is still a fork; it never
executes
production. So the `buzz-cli` fix has **three parts and one explicit
keep**:
drop the production filter; **delete** the helper and point its tests at
the
real `cmd_set_add_policy` (which self-discriminates by error variant —
`Usage` = refused, `Network(BadScheme)` = passed the gate — no relay
needed);
serialize the env-var tests behind one
**`tokio::sync::Mutex::const_new`**
lock taken with `.lock().await`, including the pre-existing `:1362`
integration test (the fork was silently buying test isolation — without
the
lock, parallel runs flake nondeterministically; a `std::sync::Mutex`
held
across `.await` trips `clippy::await_holding_lock` under `-D warnings`);
and
**keep** the then-dead `!allowed.is_empty()` clause with a comment
saying
why. It is unreachable-false (`split(',')` never yields an empty vec),
but it
is the only thing that keeps the reintroduced production bug detectable
—
mutation-tested: on a tree that deletes the clause, reintroducing the
empty-filter bug survives 275/0, because `""`/`","`/`" "` refuse either
way
and the filter goes semantically inert. Dead code can be load-bearing
for
tests: "provably unreachable" is an argument about behavior, never about
coverage. When a helper forks production, the fix has to delete the
fork:
any change that leaves two implementations standing can only ever be
verified against the one the tests call. *Final shape:* the keep and the
broad lock are both artifacts of the fork surviving in some form. The
extraction variant (Dawn, mutation-tested at `60007fda4`) removes the
tension: extract one `check_channel_add_policy_allowed(Option<&str>,
&str)`
that **production calls**, with the `Option` placed at the env boundary
where the `Result<String, VarError>` bit actually lives. 5/6 mutants
killed; the empty-filter survivor is proven **equivalent** (exhaustive
6174-pair check, 0 divergences, with a diverging negative control;
independently re-derived by a second generator — different tokens and
shape — 0 divergences on admitted policies, 500 on a non-admitted
control),
not a coverage hole — on a one-implementation tree there is no fork left
to
witness, so no dead clause needs keeping. One scope line on that
equivalence: it is **caller-conditional**, a property of the only
current
caller, not of the gate function — `cmd_set_add_policy`'s own match at
`:1027-1034` admits only three policies before the gate runs; a second
caller reaching the gate with arbitrary strings resurrects m1 as a real
hole. The lock does not disappear, it
narrows (Dawn's own correction, caught by Mari): lock exactly the tests
that mutate the process env — three-plus-one on a fork tree, two on the
extraction tree — behind one `tokio::sync::Mutex`, and the lock is part
of
the assertion, not hygiene: with it deleted, the gate test fails 8/8
runs
deterministically by receiving `Network(BadScheme)` where it expects
`Usage` — the unset test's `remove_var` clobbers the other's `set_var`,
and
**the gate test passes straight through the gate**, a false negative on
the
exact authz assertion the test exists to make. State it as an outcome:
these two tests must not observe each other's env writes. 276/0 stable
across 5 parallel runs, clippy `-D warnings` clean; independently
verified
(patch applied to a second worktree: result blob `d67e584be` matches the
patch index, full mutant matrix reproduces row for row). One new row no
earlier
prescription covered: collapsing unset into `Some("")` fails **closed**
—
an unconfigured deployment refuses every policy — killed by the unset
test.
Patch: `OUTBOX/BUZZ_CLI_ADD_POLICY_GATE_EXTRACT_FIX.patch`. The filed
issue
(`0524a411…`) carries the fork-shape prescription; whoever picks it up
should prefer the extraction shape, drop the dead-clause keep with it,
and
keep part 3 outcome-shaped: serialize whichever tests mutate the env.
## Verification (final HEAD `60007fda4`)
- Full touched-package suites at each integration merge (log in plan
file).
At candidate parent `00e5b5fe9`: buzz-backend-kubernetes 154,
buzz-acp 673, desktop tauri 2100+3, pnpm 3908, workspace clippy/fmt/tsc
all clean. The only delta to `60007fda4` is one character in
`scripts/test-k8s-sprig-image-live.sh` (heredoc escape so the readlink
probe evaluates pod-side, not host-side at render); `crates/` tree hash
is byte-identical at both SHAs, so the Rust receipts attach by tree
identity. buzz-backend-kubernetes suite re-run in-shell at
`HEAD == 60007fda4`: 154 passed.
- Adversarial one-HEAD gate (Sami): guard matrix 12/12, predicate
mutants
7/7, doomed-invocation finding closed end-to-end; tree-hash carry to
`60007fda4` confirmed (crates/buzz-backend-kubernetes blob unchanged).
- Live-local pass per TESTING.md + skill-buzz-testing (Perci, at
`60007fda4`): explicit `docker-desktop` context, digest-qualified image
imported into node containerd `k8s.io` namespace, pull policy `Never`;
pod printed `DIGEST_ABI_OK`, `resolved_spec` and `image_id` both the
exact requested digest, script exit 0. Dedicated per-run namespace,
ownership labels on every object, scoped cleanup verified empty after.
- Implementation review (Wren) at `60007fda4`: 9.6 minimalness /
9.4 elegance / 9.3 correctness, no blocker.
- `origin/eva/k8s-backend` == `60007fda4` (ls-remote verified; SHA
identity is byte identity).
---------
Signed-off-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
Signed-off-by: Tyler <109685178+tlongwell-block@users.noreply.github.com>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz>
Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz>
Co-authored-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Dawn (sprout agent) <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz>
Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz>
Bump `nostr-relay-pool` from 0.44.1 to 0.44.2 to clear
[RUSTSEC-2026-0224](https://rustsec.org/advisories/RUSTSEC-2026-0224),
which addresses verification-cache poisoning that could let forged Nostr
events bypass signature validation on redelivery.
The dependency is transitive through `nostr-sdk`; this PR updates only
the corresponding package version and checksum in `Cargo.lock`. The
advisory currently marks every open PR red until this fix merges.
- `cargo test -p buzz-sdk -p buzz-cli` passes: 271 + 241 tests
- `cargo deny check advisories` passes
- `just fmt-check` passes
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub16v54tttfqacx9ycvc3k0ut0npj564ahcuajzy6qjvh57ntmsf4uq4806j2 <d32955ad69077062930cc46cfe2df30ca9aaf6f8e76422681265e9e9af704d78@buzz.block.builderlab.xyz>
## Context
Pocket TTS currently offers bundled reference voices. People also need a
local, private way to add a voice without sending audio to a cloud
service.
## Summary
Add a Pocket voice import flow to Voice settings. Buzz opens the native
file picker, decodes common audio formats in the reusable `buzz-voice`
crate, canonicalizes the selected audio, stores it under a
content-derived identity in app data, selects it, and lets the user
delete it later.
## Changes
- Accept WAV, M4A, MP3, FLAC, OGG, and AIFF files between 2 and 30
seconds, including multichannel sources.
- Decode and downmix accepted audio to canonical mono 32 kHz PCM16 WAV
before hashing and storage.
- Store imported voices behind stable `pocket:imported:<sha256>`
identities and content-addressed files.
- Keep absolute file paths inside the native process and expose only
voice metadata to React.
- Include imported voices in Pocket preview and live huddle playback.
- Add Add voice and delete controls while preserving the bundled Pocket
voice catalog.
- Fall back to Mary when the selected imported voice is deleted.
- Keep durable import, selection, and deletion successful when a live
TTS worker acknowledgement is delayed.
- Preserve bundled voices when optional import metadata is unreadable
and keep failed deletion retryable.
## Related issue
None found.
## Testing
Production decoding was exercised with WAV, M4A with AAC, MP3, FLAC, OGG
Vorbis, and AIFF fixtures. Each format canonicalized to mono 32 kHz
PCM16 WAV. Manual validation in the combined daily-driver build covered
native-picker import, Preview, live-huddle playback, deletion, and Mary
fallback.
## Screenshots
The Voice settings card preserves the bundled Pocket catalog and adds
the local Add voice action.
## Reviewer-reproducible examples
Create common-format fixtures and run them through the production
importer:
```bash
. ./bin/activate-hermit
fixtures="$(mktemp -d)"
ffmpeg -hide_banner -loglevel error -f lavfi -i "sine=frequency=220:duration=3" -ac 2 -ar 44100 "$fixtures/voice.wav"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" -c:a aac "$fixtures/voice.m4a"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" "$fixtures/voice.mp3"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" "$fixtures/voice.flac"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" -c:a libvorbis "$fixtures/voice.ogg"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" -c:a pcm_s16be "$fixtures/voice.aiff"
BUZZ_VOICE_IMPORT_TEST_DIR="$fixtures" \
cargo test -p buzz-voice imports_common_audio_format_fixtures -- --ignored --nocapture
```
Exercise import persistence, synthesis, deletion, and bundled-voice
fallback with an installed Pocket model:
```bash
BUZZ_POCKET_MODEL_DIR=/path/to/pocket-model-bundle \
cargo test -p buzz-voice --test pocket_import_audio \
objective_import_synthesis_delete_and_mary_fallback \
-- --ignored --nocapture
```
Exercise the native-picker boundary, selection, preview dispatch,
deletion, cancellation, and invalid-file states:
```bash
cd desktop
pnpm build:e2e
pnpm exec playwright test tests/e2e/voice-settings.spec.ts --project=smoke
```
---------
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <johnmatthewtennant@gmail.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Co-authored-by: John Tennant <jtennant@block.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Relands **#2467** (extract `buzz-voice` crate) and **#3208** (Pocket
voice settings) onto main, after #3266 and #3180 merged.
## Why a fresh PR
The repo is squash-only with delete-branch-on-merge. Squashing #3266
deleted `jtennant/pocket-tts-2026-04`, which was #2467's base — GitHub
auto-closed #2467 and it cannot be reopened. Squash merges also sever
ancestry, so GitHub's natural merge-base reports phantom conflicts for
the whole remaining stack.
## Content provenance
- Byte-identical to the blessed `jt/buzz-voice-refactor` branch
(`93029c577`, tree `6729e0eff` — reviewed by Dawn (#2467) and Max
(#3208) at exact heads) **except** the three files where #3180 and #3208
genuinely interact.
- Three-file resolution (union of both sides):
- `huddle/mod.rs` — #3180's pipeline re-exports + #3208's
`agent_tts_routing` imports.
- `huddle/state.rs` — `reset_preserving_generation` preserves both
`huddle_generation` (#3180) and `tts_enabled` (#3208); test sets merged
into one `tests` module.
- `desktop/src/testing/e2eBridge.ts` — both switch arms kept; no
duplicate case labels.
## Verification at cf32dacd7
- `cargo test` (desktop/src-tauri, pinned 1.95.0): **2047 + 3 pass / 0
fail** (14 ignored: 8 keychain, 4 real_relay, 2 flag-gated)
- `cargo clippy --all-targets -- -D warnings`: clean; `cargo fmt
--check`: clean
- `cargo check --workspace` (root, includes new `buzz-voice` member):
clean; `cargo test -p buzz-voice`: 5/0
- `pnpm test`: **3885 / 0**; `tsc --noEmit`: clean; lint: clean
The 3180×3208 interaction resolution is getting an independent team
re-review before merge.
Buzz channel: buzz-desktop-voice `fd5fb402-b651-4238-89b1-bb3e2fa4dc96`,
thread `b4798ecc`.
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
## Summary
This is **part 1 of 2** split out from #3467 (per Tyler's request),
carrying only the mesh-scoped changes. The agent/ACP response-behavior
changes and the new `send_message` tool stay in #3467 as part 2. All
commits are @michaelneale's work, cherry-picked with authorship
preserved.
- Upgrade embedded Mesh to v0.74.0 (tag-pinned instead of commit rev)
and use canonical Gemma model IDs.
- Keep shared compute serving through member joins, roster changes, app
recovery, and community switching.
- Wait for actual model readiness and avoid resuming incomplete
downloads after quit.
- Leave `BUZZ_AGENT_THINKING_EFFORT` unset by default so each model's
chat template picks its own thinking default (`none` suppressed Gemma
tool-calling entirely; pinning `low` made Qwen3 burn ~4x output budget).
Explicit agent/persona/global values still win.
## Relationship to #3467
Contains the mesh commits from #3467 (`2cd640b23`, `0ad81c341`,
`ad13ed841`) rebased onto current main, with one deliberate exclusion:
the `crates/buzz-agent/src/llm.rs` reasoning→text parser change from
`2cd640b23` is **not** here. That change unconditionally affects every
OpenAI-compat/Responses provider, so it belongs with the reply-behavior
work in part 2, where it can be reviewed as what it is.
Not included (remaining in #3467 / part 2):
- typed `send_message` tool in dev-mcp + `BUZZ_ACP_SEND_MESSAGE_TOOL`
gating
- plain-reply delivery fallback in buzz-acp
(`BUZZ_ACP_DELIVER_PLAIN_REPLIES`)
- the mesh_agent_e2e P5/P6 rewrite (exists to prove the reply path)
- the two `env.insert` preset opt-ins in `relay_mesh.rs` for the flags
above
- the llm.rs parser change
This PR is independently mergeable; part 2's flags are all off by
default so it can land before or after.
## Testing
- `cargo test -p buzz-relay --locked` — 780 passed (one telemetry test
is order-sensitive under parallel default settings; passes in the
pre-push suite and standalone, unrelated to this diff — files untouched
here).
- `just desktop-tauri-test` (default features) — 1877 passed.
- `cargo test --locked --features mesh-llm` in `desktop/src-tauri` —
1961 passed, including the new relay-mesh preset and
coordinator/recovery tests.
- Both `Cargo.lock`s resolve with `--locked` against the v0.74.0 tag.
- Full pre-push hook suite green (rust-tests, desktop-check/test, tauri
checks).
Live validation of the mesh v0.74 upgrade itself is documented on #3467
(two-Mac cross-version test).
---------
Signed-off-by: Michael Neale <michael.neale@gmail.com>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Co-authored-by: Michael Neale <michael.neale@gmail.com>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
## Summary
Expands read-replica usage on the relay per the Rev 2 design (thread
39d1e174 in #buzz-read-only-replica-usage): replaces the
Aurora-incompatible WAL-LSN read-side fence observation with a portable
**heartbeat token**, makes the freshness proof **snapshot-local to the
serving reader session**, and adds a default-off bounded-staleness gate
for head fetches.
### Probe (writer side, unchanged ordering)
The ordered writer scan is kept verbatim — `S = clock_timestamp()` →
masked-visibility `pg_stat_activity` oldest-xact scan — and now ends by
committing a heartbeat token **last** on the same pinned connection
(single-row `UPDATE replica_heartbeat ... RETURNING token, epoch`,
migration `0026`). The single-row update serializes all pods' probes, so
tokens are globally commit-ordered; the three-bucket completeness
argument carries over unchanged. Ring retains `(token, committed_at,
fence_wall)`; epoch change resets the ring; a same-epoch token
regression (restore adversary) clears the ring **and rotates the epoch
on the writer** so pre-rewind readers fail the epoch check. Cadence 1s.
### Routing (snapshot-local proof)
Every routed read opens `BEGIN ISOLATION LEVEL REPEATABLE READ, READ
ONLY` on a reader session and observes the heartbeat as the
transaction's **first statement** — the proof's snapshot is exactly the
snapshot the page, participants batch, and bridge aux closure read from
(`ReadSession` carries the open transaction; drop = rollback).
Fail-closed everywhere: begin failure, missing heartbeat row, epoch
mismatch, token below the ring, over-budget entry → writer.
- **Predicate B (cursor pages, default-on):** same completeness math as
the existing fence — cursor timestamp must be ≤ the proved wall; thread
candidate-terminal and above-wall pages re-run on the writer.
- **Predicate A (head fetches, default-OFF):** gated by
`BUZZ_REPLICA_HEAD_MAX_AGE_SECS` (0 = off, clamped to fence staleness).
Bounded-stale head semantics are an explicit product decision — **do not
enable anywhere without Tyler's backdated-event-semantics acceptance.**
### Observability
`buzz_db_route_decision{path, decision, reason}` across all five paths,
`buzz_db_replica_heartbeat_age_seconds` gauge, and per-decision debug
logs carrying the proved token plus backend identity: `addr:port pid=N`,
prefixed with `aurora_db_instance_identifier()` when the endpoint
supports it (probed once per process on an autocommit checkout; SQLSTATE
42883 caches a definitive false; identity is evidence, never a routing
gate).
## Review & verification
- **Wren:** full review 9/9/9 at `5f81b10e5`; identity delta re-review
approved at exact head `fedb46368` (90/90 unit, clippy `-D warnings`,
fmt independently reproduced).
- **Max:** local E2E **PASS** at `5f81b10e5` — isolated PG17 writer +
two streaming standbys behind HAProxy; paused-reader legs split exactly
as designed (6 replica/fresh + 6 writer/stale), recovery clean, RR
snapshot hardening observed in runtime logs. Evidence:
`WORK_LOGS/2026-07-28_REPLICA_HEARTBEAT_REPLACEMENT_SHA_E2E.md`.
- **My gates at head (same shell):** buzz-db 90 unit + 143
Postgres-gated green (scratch DBs); buzz-relay 767/768 — lone red is the
pre-existing `mesh_demo` flake, red on base; clippy `-D warnings` + fmt
clean.
- Key regression tests:
`routed_request_holds_one_snapshot_across_page_and_aux`
(mutation-verified both directions), same-epoch rotation,
capability-probe negative, head-gate truth table, divergent-fixture
routing suite.
## Post-merge plan
Merge publishes the immutable `sha-*` main image → bb-public PR pins it
→ ArgoCD sync → Max runs the production cursor-routing canary on Aurora
(positive identity branch proven live). Head gate stays off. Client
`since`-widening ships separately.
---
## Update — full read routing (Rev 6, commit `9fa3c9c0b`)
Extends routing to the remaining read seams per
`PLANS/REPLICA_FULL_READ_ROUTING_DESIGN.md` Rev 6 (thread cf5deba4 in
#buzz-read-only-replica-usage).
### New routed seams — ALL deploy-default dark
The new seams (`query_events_routed`, `query_events_routed_bounded`,
`count_events_routed`, `get_events_by_ids_routed`,
`query_feed_{mentions,needs_action,activity}_routed`) are
**Bounded-only** and gated on `BUZZ_REPLICA_READ_MAX_AGE_MS`: unset ⇒
every new seam records `writer/disabled` and merging is a no-op. COUNT
and feed/by-ids never take the covered arm (deletion visibility: covered
bounds insert-completeness only). **Note:** the pre-existing cursor
paths (channel windows, thread pages) are *not* gated by this env var —
they route at B=0 today and that status quo is unchanged.
### Reader pool (D4/D5)
- Lazy pool (`connect_lazy`, `min_connections(0)`): reader-down at boot
can't crash the relay; a warn-only boot ping is the only boot-time
visibility, and it primes the Aurora identity capability cache. Priming
is an optimization only — the routed path itself spends a single acquire
budget regardless (see the single-checkout fix below), because a boot
ping that *fails* is correlated with exactly the reader-unavailable case
the budget bound exists for.
- `READER_ACQUIRE_TIMEOUT` = 150ms; a miss fails closed to the writer
with reason **`reader_acquire_timeout`** — named for the mechanism, not
a diagnosis. The budget includes cold connects and sqlx's `size` counts
in-flight dials, so this metric alone does not distinguish contention
from slow connection establishment (see `proved_reader` doc-comment for
the runbook guidance; the pool gauges are 10s samples and are for
capacity planning).
- `BUZZ_DB_READ_POOL_SIZE` sizes the reader independently (invalid/0
inherits writer sizing); `read_pool_stats().max` reports the reader's
own ceiling.
### Community isolation (formal-model question)
Isolation is structural — an explicit `community_id = $n` predicate
compiled into every query builder; no RLS, no session-GUC tenant state
(zero `CREATE POLICY` across migrations). The `_on` variants reuse the
exact same builders with only the executor swapped. Proven by a
seven-seam two-community divergent-fixture test
(`routed_reads_are_confined_to_the_requested_community`),
mutation-tested by Dawn: all four single-predicate stubs killed; the
mention-join feeds are defended in depth (three independent predicates)
so only complete removal leaks there.
### Accepted limitation
D6: client-side staleness on bounded reads (up to `_MS`) is accepted
product behavior when the gate is enabled; gate stays off at merge.
## Update — single acquire budget per routed read (commit `dd26caa9f`)
Max found (and Dawn independently reproduced, 302–330ms measured) that
the boot-unavailable cold path spent **two** stacked
`READER_ACQUIRE_TIMEOUT` budgets: the Aurora capability probe did its
own `pool.acquire()` before `begin_with` acquired again. Fixed by
acquiring **once** per routed read — the capability probe runs on the
held connection (`reader_aurora_capability_on`) and the read-only
`REPEATABLE READ` transaction begins on that same connection
(`Transaction::begin` accepts a `PoolConnection` at `'static` via sqlx's
`MaybePoolConnection`). Reason codes unchanged; capability still never
negatively cached. Measured routed fallback: ~150ms (one budget).
Ships with a PG-gated regression fixture
(`routed_fallback_spends_one_acquire_budget_when_aurora_cache_is_cold`,
authored by Dawn): size-1 reader saturated, capability cache asserted
cold, routes through `count_events_routed`, asserts writer answer +
one-budget elapsed + `writer/reader_acquire_timeout` label
(mutation-tested: `pool_busy` and `reader_validation_error` mutants both
killed). It fails at 330ms on the previous commit and passes on this
one.
### Verification at `dd26caa9f` (pinned toolchain rustc 1.95.0 via repo
`bin/`; ambient rustc 1.89 fails sqlx resolution — always prepend
`bin/`)
- buzz-db: 94 unit + **150/151** PG-gated serial (`--test-threads=1`;
suite is not parallel-safe against one Postgres). The 1 red is
`test_usage_metrics_lock_has_single_owner_and_releases_on_drop` — a
**pre-existing** same-key-same-database advisory-lock collision with any
live relay pointed at the shared `buzz` DB (the relay re-arms
`USAGE_METRICS_LOCK_KEY` on a 300s tick). It passes on a private scratch
DB with the relay still running — verified this run. Remedy is a scratch
`TEST_DATABASE_URL`, **not** terminating lock holders: inspect
`pg_locks`/`pg_stat_activity` and resolve the owning process first; an
idle advisory holder may be a live dev relay. Pre-existing, not a #3268
blocker — filed as #3619.
- buzz-relay: 773/773; clippy `-D warnings` + fmt clean. (`mesh_demo`
echo test is a known pre-existing Redis-dependent flake — Dawn confirmed
it fails identically on the base commit, untouched by this PR.)
- Dawn: independent re-verification at `dd26caa9f` — byte-identical diff
application confirmed, suite reproduced, and fixture proven to still
discriminate (fails at 320ms with only the production fix reverted).
Gate clear. Max: independent rig pass at `dd26caa9f` — 94/94 unit,
**15/15 routing/fallback matrix** (cursor behavior, default-dark seams,
seven-seam confinement, one-snapshot, dead-reader/no-URL fallback,
hard-delete fail-closed, reader max), 773/773 relay, private-DB lock
control. Gate clear; recommends merge + staged bb-block rollout
(no-reader-URL no-op → cursor-only → observe `buzz_db_route_decision` →
conservative nonzero `BUZZ_REPLICA_READ_MAX_AGE_MS`; rollback is
config-only).
## Known gap — CI does not run the PG-backed fixtures (#3622)
Dawn found post-sign-off (confirmed by Max and Eva at `dd26caa9f`):
**every Postgres-backed fixture in this PR is `#[ignore]`d and no CI job
selects it.** `Unit Tests` runs `cargo nextest run -p buzz-db --lib`
(`Justfile:279-285`, ignored tests excluded); the only `--run-ignored
ignored-only` invocations (`ci.yml:690`, `:702`) filter to
`relay_invite::tests`. So the one-budget regression, the seven-seam
isolation fixture, and the fence/floor-guard/fallback tests have zero
automated execution — the verification record above is exhaustive but
manual, at this exact SHA.
Not introduced by this PR (the `#[ignore]` + narrow-filter pattern
predates it; 34 ignored tests total) and not a merge blocker: all new
seams are dark until `BUZZ_REPLICA_READ_MAX_AGE_MS` is set. But it
changes the rollout gate — **do not set a nonzero
`BUZZ_REPLICA_READ_MAX_AGE_MS` in bb-block until CI enforces these
fixtures.** Filed as #3622 (explicit CI selection against the existing
backend-integration Postgres archive; ordering note: #3619 must land
first or be excluded, since widening the filter would select the
colliding usage-metrics lock test).
## Live redteam results (Max, real Helm + PG17 streaming replication at
`dd26caa9f`) — #3643, #3644
Max deployed this exact SHA via the repo chart against a physical
writer/standby pair and attacked it. **The PR's fenced routing passed
every fault**: healthy baseline routes proven on the real standby;
paused WAL replay moved head/cursor reads to `writer/stale`; B=0 moved
routed reads to `writer/disabled`; reader-URL removal rolled back
cleanly.
The redteam also found a **pre-existing** production risk this PR does
not create and does not fix: NIP-50 search builds its own eager,
unfenced pool straight from `READ_DATABASE_URL` (`main.rs:389-402`,
introduced by #2084, present in the deployed bb-block `sha-dd222a5` and
bb-public `sha-22be8bb` images, and both production ESOs already
template `READ_DATABASE_URL`). Measured: ~30s search stall on reader
loss, acknowledged-but-unsearchable writes under replica lag (unaffected
by `BUZZ_REPLICA_READ_MAX_AGE_MS`), fatal eager connect blocking pod
startup when the reader is down, and `replica=true` FTS even at B=0.
Filed as **#3643** (fix: pin FTS to the writer, or fence it like the
routed seams) with **#3644** for the writer-pointing-reader-URL
telemetry gap. Aurora's cluster-ro writer-fallback DNS softens the
outage modes in prod but not the lag mode.
**Revised ladder consequence:** step 2 ("add `READ_DATABASE_URL`, budget
unset") is NOT a no-op with today's binary — it hands FTS an unfenced
replica pool. Rollout order is now: merge (still a true no-op —
bb-block/bb-public already run the unfenced-FTS code and this PR only
adds dark seams) → fix#3643 → CI enforcement #3622 → then reader URL +
budget per the original ladder, re-running Max's three deployment tests
(reader-down latency, reader-down cold boot, paused-replay
read-your-own-write search).
## Activation gate (consolidated) — merge is clear; ALL of the below
precede any nonzero `BUZZ_REPLICA_READ_MAX_AGE_MS`
Six independent verification passes at `dd26caa9f` (Dawn ×2 incl.
byte-level + defang check, Max matrix + live Helm redteam, Wren full
CRUD regression + greenfield 0026 + 151/151 PG, Eva). Every induced
fault was either handled correctly by this PR's machinery or traced to
pre-existing main code. Remaining work gates **activation, not merge**:
1. **#3643** — unfenced NIP-50 FTS pool (pre-existing #2084; live in
prod today; also explains the blackhole search hang Wren observed — the
routed path was measured bounded at 151-152ms against a silent endpoint,
sqlx `inner.rs:252-255`).
2. **#3651** — reader `statement_timeout`: statements after acquire are
unbounded; mid-transaction blackhole reproduced hanging ≥15s (Dawn).
Small fix via `.after_connect` on the reader pool.
3. **#3622** — CI enforcement of the PG fixtures (with #3619 ordering).
4. **Recovery-conflict cancellation live test** — standby cancels a
routed read mid-snapshot → complete writer page, never partial (#3651's
fix also bounds the cancellation-never-arrives shape).
5. **DDL replication lag test** — migration on writer + paused replay +
budget on → fallback, not client-visible SQL errors.
6. **Reader-tx soak** — 30-60min mixed load; no idle-in-tx accumulation
on the standby.
Then Max's three deployment tests (reader-down latency, reader-down cold
boot, paused-replay read-your-own-write search) re-run on the fixed
binary before the first nonzero budget on bb-block.
---------
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
## Summary
- add one strict `BUZZ_S3_ADDRESSING_STYLE=path|virtual` setting shared
by media and Git/CAS storage
- preserve path-style defaults for bundled Compose/Helm MinIO while
supporting Railway's virtual-hosted bucket contract
- fail startup on invalid or non-Unicode values before dependency
connection, and validate the Helm value with the same two choices
- document operator mappings and why endpoint and bucket remain separate
for routing and SigV4 signing
## Best-practice rationale
AWS documents both URL forms and favors virtual-hosted addressing for
S3, while compatibility endpoints such as the bundled MinIO deployment
can require path style. `rust-s3` defaults to virtual/subdomain
addressing and provides `with_path_style()` for the explicit
compatibility case.
Some providers buckets only support as virtual-hosted bucket styles.
This PR therefore uses one explicit, provider-neutral switch rather than
endpoint heuristics or fallback behavior, while retaining `path` as
Buzz's backward-compatible default.
Sources:
-
https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
- https://docs.rs/rust-s3/0.37.0/s3/bucket/struct.Bucket.html
- https://docs.railway.com/storage-buckets#url-style
-
https://github.com/minio/minio/blob/master/docs/config/README.md#domain
## Validation
- `cargo fmt --all`
- `cargo check --workspace --all-targets`
- targeted `buzz-media` and `buzz-relay` parsing/client-construction
tests for defaults, strict errors, and both URL styles
- Helm unittest: 45/45 passed
- Compose config/render validation passed
- local MinIO path-mode relay startup passed the Git A3 conformance
probe and became ready
- unreachable object storage failed startup and readiness never opened
- push hooks completed the broader Rust and desktop suites successfully
---------
Signed-off-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Signed-off-by: Kalvin Chau <kalvin@block.xyz>
Co-authored-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Fixes all six HIGH findings from the buzz security report, one commit
per finding. Independently reviewed to approval by Max at `0158ae542`,
plus a deep isolated live pass (clean-room compose stack, weird ports,
full product matrix) at the same head — see the buzz-security thread for
evidence. `fe65c07c3` merges current `origin/main` on top (new commit,
no rebase), inheriting the nostr 0.44.6 bump (#3135) and relay-admin ban
gate (#3128).
## Findings and fixes
| Finding | Commit | Fix |
|---|---|---|
| 003 — quinn-proto RUSTSEC-2026-0185 | `e5dcdec72` | Bump quinn-proto
0.11.14 → 0.11.16 (lockfile-only) |
| 002/004 — linkify-it quadratic-parse DoS (GHSA-22p9-wv53-3rq4,
GHSA-v245-v573-v5vm) | `923b3c20f` | pnpm override `linkify-it: ^5.0.2`;
`pnpm why` confirms a single 5.0.2 copy |
| 001 — media reads served unauthenticated by default | `0f277e3e2` |
Helm `requireMediaGetAuth` defaults to `true` + rendered-chart test
pinning the default |
| 006 — removed workflow owners retain webhook-exfiltration authority |
`4749bd56c` | Fail-closed per-fire authority gate (current owner/admin
membership) on **all four** trigger doors (on_event, scheduler
pre-claim, manual trigger, webhook — masked as generic 404), save-time
gate for `call_webhook` defs, durable disable-on-removal wired to kinds
9001 + 9022 |
| 005 — git Smart-HTTP reads ignore channel membership | `e648f2dba` +
`0158ae542` | `authorize_git_read`: caller's **current active
membership** in the repo's bound channel, checked before any
hydration/subprocess on all three read doors (`info_refs` for both
services + `upload_pack` POST). Uniform generic 404 denials (no
membership probing), no repo-owner bypass, first-`buzz-channel`-tag
binding semantics fail closed on ambiguous duplicates (mutation-verified
test). Resolution follows the live kind:30617 announcement, so
deleted/replaced announcements deny immediately. The committed
`e2e-git-perms.sh` guest scenario previously asserted the vulnerability
— now asserts denial. |
## Behavior changes to be aware of
1. **Unbound repos fail closed for git reads.** `buzz repos create`
emits no `buzz-channel` tag, so CLI-created repos without a binding are
unreadable via git HTTP. Correct per finding 005's fail-closed posture;
a follow-up could bind CLI-created repos at creation time.
2. **006 is conservative:** a workflow disabled on owner removal does
not auto-re-enable if the owner is re-added — explicit re-enable
required.
3. Merge conflict resolution in `fe65c07c3`: kept main's
`@radix-ui/react-dismissable-layer` 1.1.19 bump alongside the linkify-it
security override (`pnpm-workspace.yaml` + lockfile).
## Verification at the merge head `fe65c07c3` (same shell)
- buzz-relay `--lib`: 761 passed / 1 failed — the lone red is the known
pre-existing `mesh_demo::demo_join_forwarded_arm_round_trips_echo` 504
flake, present on main
- SEC-005 module incl. PG behavioral matrix: 8/8 (removed-member,
never-member, owner-no-bypass, deleted-30617, malformed/ambiguous
binding, owner-mismatch all denied)
- buzz-workflow 153/0, buzz-db 84/0; `clippy --all-targets -D warnings`
+ `fmt --check` clean
- Desktop JS 3637/3637, tsc clean, biome clean,
file-size/px-text/pubkey-truncation gates clean
- `helm lint` + `helm unittest` (40/40) on `deploy/charts/buzz`
- All five pre-push hooks green (desktop-check, desktop-test,
rust-tests, desktop-tauri-test, branch-skew)
Prior review evidence at `0158ae542` (pre-merge): Max's independent
exact-head approval + clean-room live regression pass
(`WORK_LOGS/2026-07-27_SECURITY_HIGH_LIVE_TEST.md` in his workspace).
Max will re-run the deep local pass at this post-merge head before
merge.
---------
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
## Summary
`cargo-deny` started failing on **every** PR and on `main` when
**RUSTSEC-2026-0216** was published mid-afternoon today. Nothing in the
tree changed — cargo-deny fetches the advisory DB at run time, so main's
own `Security` job passed at `00ecf2c` and then began failing on the
same commit.
```
error[vulnerability]: Remote Denial of Service via malformed NIP-44 v2 payload
Cargo.lock:432 nostr 0.44.3 — RUSTSEC-2026-0216
advisories FAILED, bans ok, licenses ok, sources ok
```
The `nostr` NIP-44 v2 decrypt path reads a 2-byte unpadded-length prefix
via `buffer[0..2]` **after** the HMAC check passes, without verifying
the decrypted buffer holds 2 bytes. A sender who holds the conversation
key — i.e. any DM sender — can craft a payload that decrypts to 0 or 1
bytes and panic the receiver. Remote DoS through any relay that delivers
the event. No key material, plaintext, or memory corruption.
Affects `0.26.0` through `0.44.4`. Fixed in `0.44.5`.
## The change
Lockfiles only, 6 insertions / 6 deletions. The manifest already
declares `nostr = "0.44"` — a caret range — so `0.44.6` needs no
`Cargo.toml` edit.
| Lockfile | Before | After |
|---|---|---|
| `Cargo.lock` | 0.44.3 | 0.44.6 |
| `desktop/src-tauri/Cargo.lock` | **0.44.4** | 0.44.6 |
**The desktop lockfile is the part worth reviewing.**
`desktop/src-tauri` is excluded from the root workspace
(`Cargo.toml:31`), and the `Security` job runs `cargo-deny check` at the
repo root — so it never sees that lockfile. It was pinning a vulnerable
*and* yanked `0.44.4` that no CI check would ever have flagged. Desktop
calls `nip44::decrypt` at `commands/identity.rs:495`. Credit to @Eva for
catching this; I'd have shipped the root-only fix and left it sitting
there.
**This isn't optional maintenance.** `0.44.0` through `0.44.4` are all
yanked on crates.io. `0.44.5` and `0.44.6` are the only live versions in
our range — staying put isn't an available option.
### On the two extra lines in the desktop lockfile
The desktop bump also repoints two existing dependency edges:
```
nostr-derive: syn 2.0.118 -> syn 1.0.109
tempfile: getrandom 0.4.3 -> getrandom 0.3.4
```
I checked these rather than waving them through: **no packages are added
or removed** — both versions were already present in the graph, so only
which edge points where changed. The resolution is stable across
repeated re-resolves, and a plain re-resolve without the bump produces
zero diff, so this isn't pre-existing lockfile staleness leaking in.
## Verification
At this commit, in a clean worktree off `origin/main`:
- `cargo-deny check advisories` → **`advisories ok`**, exit 0. The same
tree before the bump reported `advisories FAILED` with this advisory, so
the check is doing real work, not passing vacuously.
- `./scripts/run-tests.sh unit` → all five packages pass.
- `cargo test -p buzz-core` 229/229, `-p buzz-cli` 250/250, `-p
buzz-relay --lib` 750 pass / 1 fail — the sole failure
`api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo` is
pre-existing and reproduces identically at unmodified `00ecf2c`.
- `desktop-tauri-test` passed in the pre-push hook, which exercises the
crate whose lockfile changed.
## Why not a `deny.toml` ignore
Considered and rejected. This is a reachable panic triggerable by any DM
sender, and buzz-acp agents decrypt DMs from arbitrary senders.
Suppressing it would ship a live remote-DoS to every agent and client in
order to make a dashboard green.
## Note on `spin`
The yanked `spin 0.9.8` / `0.10.0` warnings in the same job are **not**
what fails CI — the log has exactly one hard error, this one. They're
`warning[yanked]`, and warnings don't fail the build. `spin` is also
three levels transitive (`mesh-llm-host-runtime → mdns-sd → flume →
spin`) under a dev-dependency, so it isn't ours to bump. Left alone
deliberately.
## Follow-up
Unblocks #3128 (relay-admin ban gate), which has a zero dependency-file
delta and will inherit this cleanly once main is merged in.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Co-authored-by: npub1jmc9dt2lyvzu3h0kxlwxt5zg4fxp9476awyxw6gwxn72g6cw7exqs64whm <96f056ad5f2305c8ddf637dc65d048aa4c12d7daeb8867690e34fca46b0ef64c@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
