Implements the ratified owner-side community-derivation contract for the
cross-pod huddle join path (PLANS/MESH_GREENFIELD_DESIGN.md § "Owner-Side
Community Derivation, FINAL 2026-07-08"). The mesh dispatcher callback is
community-agnostic — Hello and the fenced header carry no community — yet the
owner's Redis fence key is (community_id, session_id). Rather than a shared
community-of-session registry, the community rides the wire on the first
stateful frame and is self-verified by the fence.
Hello is now structural-only: authenticated sender == from, HuddleControl
Session role, owner_runtime_id == local. It admits nothing and touches no room,
so it is deliberately NOT Redis-fenced (community unknown at Hello time). The
prior accept-time validate() call is dropped.
RegisterPeer carries community_id (as a raw Uuid — CommunityId is deliberately
non-deserializable so it can never be minted from client input; this is a
server-to-server mesh frame and the owner reconstitutes the CommunityId via
from_uuid before fencing). The serve loop latches the community on first receipt
and rejects any later frame naming a different one (tenant-boundary guard).
Validate-before-admit: the Redis fence keyed by the asserted community must pass
before room.add_peer. A wrong community keys a lease the owner never wrote →
typed no_active_lease → nothing mutates. The per-frame `f != fenced` check stays
(a lease that moves mid-stream rejects subsequent frames); a fence rejection now
returns RegisterRejected(Fenced(..)) and keeps the stream alive, while a
non-fence validate error (Redis unreachable, decode) still tears it down —
classified via the existing FenceRejection::from_mesh_error. UnregisterPeer
needs no fence (touches only this stream's registered map).
accept_inbound / serve_control_loop drop their community_id parameter to match
the dispatcher callback shape (from, hello, stream). dial_remote_owner gains a
community_id param; the handler passes tenant.community().
Adds two handshake round-trip tests over the public MeshStream::new seam with an
in-memory channel-backed stream pair: one asserting PeerRegistered on a valid
fence, one asserting RegisterRejected(Fenced) on a fence failure with no peer
admitted. fmt + clippy clean; 12 join tests pass.
Scoped out (Perci's increment, Wren-gated): the renew loop + teardown on
Lost/NotOwner. The huddle-side reaction (room teardown + floor.forget) is a
future seam to be agreed in-thread before coding.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Second increment on the cross-pod huddle join path (owner-side acceptor +
non-owner dial + the handler seam). The first commit defined the protocol,
resolver, and fenced directory trait; this one makes them load-bearing on a
live audio connection.
Owner side (join.rs): HuddleControlAcceptor::accept_inbound validates
sender==authenticated-peer, Session role, HuddleControl profile, fences against
Redis, and confirms owner-is-local before serving a register/unregister control
loop. RegisterPeer -> room.add_peer (as a remote peer) + spawn_remote_peer_sink
(fans owner audio back to the registering pod as datagrams). Every control frame
is re-fenced, not just the Hello -- a lease that moves mid-stream rejects
subsequent registers. Peers a stream registers are tracked so a stream close
tears them all down; no leaked index slots.
Non-owner side (join.rs): dial_remote_owner opens the HuddleControl stream,
registers the client, and returns a RemoteHuddleSession carrying the
owner-assigned peer index. forward_media ships client Opus to the owner tagged
with that index; close() sends UnregisterPeer + Goodbye. The owner is the sole
fan-out authority and sole index allocator -- co-located clients hear each other
via the owner round-trip (deliver_prefixed's index-skip prevents self-echo).
Handler seam (handler.rs): mesh-off path is byte-identical to today, including
the huddle_audio_available=false guardrail. Mesh-on resolves the join at the
horizontal-scaling seam; the RemoteOwner branch dials the owner after the local
add_peer (so a rejection backs out cleanly before any peer sees a joined event)
and threads the session into recv_loop -- forward to owner vs broadcast_frame.
Owner-rejection maps to the same client-facing WS error codes a same-pod join
produces (remote_rejection_ws_error), never a silent media drop.
Extracted a pure media_datagram helper (unit-tested) and relaxed
spawn_remote_peer_sink to Arc<dyn RelayPeerTransport> (only caller).
Scoped out, honestly flagged in-code: cross-pod roster sync. Non-owner joined/
peers reflect only co-located peers today; the media + fence path is complete
and correct. Roster deltas over HuddleControl are the next increment.
Build + clippy clean; 501 relay tests pass (10 join tests).
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Adds the control plane for cross-pod huddle joins: the counterpart to the
media datagram fan-out in audio/mesh.rs. `resolve_join` decides which pod owns
a huddle (Redis fenced CAS lease is the arbiter; mesh membership is only a
hint) and returns a `JoinOutcome` the handler acts on — LocalOwner (admit
here) or RemoteOwner (register with the owning pod over HuddleControl).
Fencing is enforced on the join hop: a remote-owner outcome is validated
against the live lease via `validate_fenced_header` before routing, mapping
failures to typed MeshError variants (never media-drop semantics). The owner
re-validates on receipt — fencing at every hop.
Defines the HuddleControl payload schema (`HuddleControlMsg`, postcard) carried
opaquely in MeshStreamFrame::Data: RegisterPeer/PeerRegistered/RegisterRejected/
UnregisterPeer, with a fence-rejection taxonomy 1:1 with the media path's.
Dependency-injected via the `HuddleDirectory` trait (impl'd for
SessionDirectory) so the resolver is unit-tested without Redis. No touches to
state.rs/main.rs/routes — the handler.rs hook lands after the startup-wiring
lane adds `state.mesh()`. 8 tests; build + clippy clean.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Wren's review of 8b077fdb raised two correctness blockers, both fixed:
1. BUZZ_MESH now defaults OFF. Mesh forms only on explicit
BUZZ_MESH=on|true|1 — an image upgrade with untouched env binds no
UDP port and writes no Redis key (strict rollout no-regression).
Pinned by mesh_defaults_off_when_env_absent.
2. Ready-record acceptance is anchored to the deployment's relay
identity: MeshMembership::with_expected_relay_pubkey (set from the
relay signing key in boot_mesh) rejects seeds attested by any other
key — possession of some relay key is not authorization. Unanchored
membership is fail-closed (admits nothing). Rejections are counted
as foreign_relay_rejections in /_mesh.
Plus the single-slot inbound dispatcher (thread-agreed contract):
MeshInboundDispatcher in mesh_boot.rs implements InboundHandler,
installed once by boot_mesh; consumers register per-profile
entrypoints via MeshHandle.dispatcher (register_huddle_control /
register_reliable_stream / register_datagrams, first-registration
wins). HuddleControl/ReliableStream streams fan out by hello profile;
RealtimeMedia as a stream is rejected (datagram-only); traffic before
registration is logged and dropped (bounded boot-window race, fencing
makes retry safe). MeshStream::new and BoxFuture are now public so
consumer crates can stub streams/transports in tests.
cargo test -p buzz-relay-mesh: 32 passed; -p buzz-relay: 494 passed,
2 ignored; clippy both packages clean; fmt clean.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Baseline the mesh lanes redeploy onto: build image -> helm install
quickstart HA (3 replicas) -> 3/3 Ready. Repeatable via
deploy/local/build-and-deploy.sh; evidence dir per run.
Chart fixes for multi-replica quickstart:
- Bucket-init as a NORMAL resource (revision-suffixed), not a helm hook:
the relay's A3 S3 conformance probe is startup-fatal and does not
create the bucket, so a post-install hook deadlocks against --wait
(relay never Ready without the bucket) and a pre-install hook deadlocks
against MinIO (a normal resource). Running the Job alongside MinIO +
the Deployment lets its until-loop gate cleanly.
- Relay initContainer waits for the bucket before start, replacing the
transient CrashLoopBackOff with a deterministic gate.
Dockerfile: optional EXTRA_CA_CERTS + NPM_REGISTRY build-args for builds
behind a TLS-intercepting corporate proxy / registry mirror. No-op on
public CI (args unset).
HA values: replicaCount 3 + persistence.git.enabled=false (per-pod
emptyDir; the default single RWO PVC cannot multi-attach across pods).
Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co>
Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co>
MeshAudioRouter fences inbound media datagrams against a per-session
GenerationFloor (monotonic; stale generations dropped = the fence), then
delivers [peer_index][v2 hdr][Opus] to local room peers via Room's new
deliver_prefixed, skipping the author's own index. spawn_remote_peer_sink
models a cross-pod participant as an ordinary AudioPeer whose audio_tx sink
is drained onto the mesh, so Room stays transport-agnostic and broadcast_frame
is unchanged. Zero client-wire change: peer_index is relay-added routing
metadata, always the first payload byte in both directions.
Datagram half only; the HuddleControl remote-peer-registration stream path is
wired from handler.rs in a following change. 6 unit tests cover fence
monotonicity, per-session isolation, forget/reset, and drop-on-stale/empty.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Wires buzz-relay-mesh into buzz-relay: workspace + crate dependency, and
Config.mesh (buzz_relay_mesh::MeshConfig) resolved from env.
- BUZZ_MESH: on by default; 'off'/'false'/'0' is the incident kill
switch — the relay must behave exactly like a single-instance deploy.
- BUZZ_MESH_BIND_ADDR: UDP bind for the iroh endpoint, default
0.0.0.0:3478 (excluded from istio sidecar capture in k8s).
- registry_refresh fixed at 15s (ready-registry heartbeat; expiry 3x).
No behavior change yet: nothing constructs a mesh from this config —
AppState handle, /_mesh mount, and startup/drain wiring follow.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Adds NoActiveLease, OwnerMismatch, and FutureGeneration alongside
StaleGeneration so every fence-visible reject is a typed variant, never
a generic Transport(..) — Wren's chaos-gate ruling: live kill-9 /
partition / replay evidence must be unambiguous, and the harness
counters (mesh_fence_rejections_total{reason=...}) key off these.
Purely additive, non-serialized: the wire-level fence signal remains
GoodbyeReason::StaleGeneration in wire.rs — zero wire-shape change.
Field naming per Wren: NoActiveLease carries frame_owner_runtime_id
(the frame's *claimed* owner; no current owner exists by definition).
Consumed by the session-directory lane's validate_fenced_header().
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Wren's contract-review blocker: the chart shares one BUZZ_RELAY_PRIVATE_KEY
Secret across all pods of a release, and that key is secp256k1 while iroh
endpoint ids are ed25519 — so 'runtime id = relay signing pubkey' would give
every pod the same runtime id (registry/lease collapse) and doesn't even
type-check against iroh. RuntimeId is now the ed25519 pubkey of a mesh
keypair generated fresh at process start: boot-unique by construction, and
identical to the iroh endpoint id so transport auth and ownership identity
stay one value. Deployment-identity binding moves out-of-band: the ready
registry record carries a relay-key-signed attestation of the runtime pubkey
(membership lane); peers accept connections only from attested endpoint ids.
Doc-only change; no wire shape or test change.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Doc-only: MeshDatagram.payload for realtime media is [peer_index: u8][client
frame] — peer_index is relay routing metadata allocated solely by the owner
pod; the media ciphertext stays NIP-44 end-to-end between clients. Ratified
in-thread (Dawn's design-lock post); no wire shape change.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
