Files
buzz/scripts/ci-mesh-lifecycle-smoke.sh
Michael NealeandGitHub 38bf642fcf ci: prove the relay-driven mesh lifecycle — discover, join, infer, deny — with real nodes (#3862)
## Summary

CI now proves the full Buzz shared-compute join story end to end: a
member can discover another member's served model **through the Buzz
relay alone** and run inference over the mesh, while a non-member gets
nothing — the relay rejects its auth, and the mesh refuses to route for
it even holding a leaked endpoint address.

This is deliberately different from mesh-llm's own CI smokes (which
bootstrap two nodes with a hand-carried invite token / mdns): here the
**relay is the control plane**, exactly like the desktop app:

1. **Membership** — identities A and B are added via `buzz-admin`
(kind:13534 NIP-43 roster); C is not.
2. **Advertise** — each member publishes a client-signed kind:30003
discovery note carrying its MeshLLM owner binding and (for the serve
node) `serveTargets[].endpointAddr`, covered by an endpoint-binding
signature — the exact payload shape the desktop coordinator publishes.
3. **Trust** — the serve node derives its admission allowlist from the
relay (statuses ∩ roster) and requires the **exact expected {A, B}
owner-id set** before starting with `TrustPolicy::Allowlist`.
4. **Join** — the client verifies owner + endpoint bindings and
membership, then dials the relay-discovered endpoint (the desktop
join-watcher's `dial_endpoint_addr` step). No out-of-band token.
5. **Infer** — a chat completion against the client's local OpenAI
endpoint routes over QUIC to the serve node's model (CPU, SmolLM2-135M,
~105MB).
6. **Deny (differential)** — the stranger's NIP-42 auth must fail with
the relay's own membership rejection (`restricted: not a relay member` —
successful auth or any unrelated connect error fails the run), and
dialing the leaked endpoint must not produce a routed inference —
**while the trusted client re-proves inference immediately afterwards**,
so a dead serve node can't masquerade as an admission denial.

## What's in the PR

- `crates/buzz-relay/examples/mesh_relay_lifecycle_smoke.rs` — the
harness. One process per node (mesh-llm keeps process-global state under
`~/.mesh-llm`), orchestrator + serve/client/stranger roles,
byte-identical binding payloads to
`desktop/src-tauri/src/mesh_llm/identity.rs` (called out with
keep-in-sync comments). Child stdout is pumped through a reader thread
so every wait has a hard deadline; timed-out children are killed; exit
statuses are checked.
- `scripts/ci-mesh-lifecycle-smoke.sh` — provisions a membership-gated
relay (throwaway owner + signing identities via `buzz-admin
generate-key`), runs the harness, cleans up. Fails fast if :3000 is
already occupied (a stale open relay would mask gating).
- `scripts/start-relay-for-tests.sh` — gains opt-in NIP-43 membership
env passthrough (`BUZZ_REQUIRE_RELAY_MEMBERSHIP` + `RELAY_OWNER_PUBKEY`
+ `BUZZ_RELAY_PRIVATE_KEY`). Default behavior unchanged.
- `.github/workflows/mesh-lifecycle.yml` — separate, path-filtered,
non-required workflow (mesh paths, the harness's dependency crates,
`Cargo.lock`, dispatch), pinned to `ubuntu-24.04`. Caches the mesh
native runtime + HF model keyed on the lockfile hash, so a mesh pin bump
rolls the runtime cache. Uploads relay + harness logs on failure.

## Scope

This is an **independent protocol harness**: it speaks the same wire
protocol and payload shapes as the desktop but re-implements the
binding/verification logic (the desktop crate is outside the workspace).
Regressions inside the desktop's own discovery filtering are the desktop
unit tests' job; what this smoke proves is that the relay + mesh-llm SDK
+ admission stack support the lifecycle end to end.

## Relationship to mesh-llm's CI

Follows the shape mesh-llm's own CI proved stable (tiny CPU model, one
runner, multiple real mesh-llm processes over real QUIC — cf. their
`ci-two-node-client-serving-smoke.sh`), but swaps the token bootstrap
for the relay-driven lifecycle, which is the part only Buzz can test.

## Validation

Green on GitHub Actions (ubuntu-24.04) across three runs, including
after rebases onto the mesh v0.74 upgrade (#3467) and latest main:

```
PASS 1/6: relay-derived allowlist is exactly {A, B}
PASS 2/6: serve member ready + advertised model: jc-builds/SmolLM2-135M-Instruct-Q4_K_M-GGUF:Q4_K_M
PASS 3/6: client member discovered + joined via relay
PASS 4/6: inference routed over the mesh: "PONG"
PASS 5/6: relay rejected the stranger's NIP-42 auth (membership gate)
PASS 6/6: stranger denied (gossip visible, inference rejected: 503 all tunnels failed) while trusted inference still routes
PASS: full relay-driven mesh lifecycle verified
```

Also validated locally on macOS. `cargo fmt --all --check` and `cargo
clippy -p buzz-relay --all-targets -- -D warnings` pass.

## Notes

- The harness follows the repo's mesh `[dev-dependencies]` pin
automatically, so it doubles as a canary for future mesh upgrades (it
already caught the v0.73.1 → v0.74.0 bump during development).
- The stranger "deny" accepts either shape mesh-llm exhibits: no model
visibility at all, or gossip visibility with inference refused —
mesh-llm applies the receiving node's owner policy after the gossip
handshake, so admission gates *routing*, not gossip. The differential
trusted-inference re-check (PASS 6/6) is what makes that a real denial
rather than a dead server.
- Model-visibility windows are tunable via `MESH_CLIENT_WINDOW_SECS` /
`MESH_STRANGER_WINDOW_SECS` if shared runners prove slow — pin a longer
window in the workflow env rather than re-running the job.

---------

Signed-off-by: Michael Neale <michael.neale@gmail.com>
2026-08-05 23:13:16 -04:00

114 lines
4.8 KiB
Bash
Executable File

#!/usr/bin/env bash
# =============================================================================
# ci-mesh-lifecycle-smoke.sh — relay-driven mesh lifecycle smoke
# =============================================================================
# Provisions a membership-gated buzz-relay and runs the full relay-driven
# mesh lifecycle harness (crates/buzz-relay/examples/mesh_relay_lifecycle_smoke.rs):
# membership → signed discovery notes → relay-derived allowlist → join →
# inference over QUIC → stranger denied.
#
# Mirrors the shape of mesh-llm's own CI smoke scripts (single runner, tiny
# CPU model, real multi-process mesh), but with the Buzz relay as the control
# plane instead of a hand-carried invite token.
#
# Usage:
# ./scripts/ci-mesh-lifecycle-smoke.sh [--profile <cargo-profile>] [--no-build]
#
# Env:
# MESH_SMOKE_MODEL Override the served model ref (default: SmolLM2-135M).
# =============================================================================
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
cd "${REPO_ROOT}"
CARGO_PROFILE="${CARGO_PROFILE:-ci}"
SKIP_BUILD=false
while [[ $# -gt 0 ]]; do
case "$1" in
--profile) CARGO_PROFILE="$2"; shift 2 ;;
--no-build) SKIP_BUILD=true; shift ;;
*) echo "Unknown option: $1" >&2; exit 1 ;;
esac
done
BLUE='\033[0;34m'
GREEN='\033[0;32m'
RED='\033[0;31m'
NC='\033[0m'
log() { echo -e "${BLUE}[mesh-lifecycle]${NC} $*"; }
ok() { echo -e "${GREEN}[mesh-lifecycle]${NC} $*"; }
err() { echo -e "${RED}[mesh-lifecycle]${NC} $*" >&2; }
# ── Build ─────────────────────────────────────────────────────────────────────
if [[ "${SKIP_BUILD}" == "true" ]]; then
log "Skipping build (--no-build)"
else
log "Building relay, admin CLI, and lifecycle harness (profile: ${CARGO_PROFILE})..."
cargo build --profile "${CARGO_PROFILE}" -p buzz-relay -p buzz-admin -p git-credential-nostr
cargo build --profile "${CARGO_PROFILE}" -p buzz-relay --example mesh_relay_lifecycle_smoke
fi
ADMIN_BIN="target/${CARGO_PROFILE}/buzz-admin"
HARNESS_BIN="target/${CARGO_PROFILE}/examples/mesh_relay_lifecycle_smoke"
for bin in "${ADMIN_BIN}" "${HARNESS_BIN}" "target/${CARGO_PROFILE}/buzz-relay"; do
if [[ ! -x "${bin}" ]]; then
err "Missing binary: ${bin}"
exit 1
fi
done
# ── Relay identity + membership gating ───────────────────────────────────────
# The relay owner and signing key are throwaway CI identities. RELAY_OWNER
# never participates in the mesh; it only satisfies the NIP-43 requirement
# that a membership-gated relay has an administrable owner.
log "Generating relay owner + signing identities..."
read_keys() { "${ADMIN_BIN}" generate-key 2>/dev/null; }
OWNER_OUT="$(read_keys)"
RELAY_OWNER_PUBKEY="$(echo "${OWNER_OUT}" | awk '/Public key:/ {print $3}')"
SIGNER_OUT="$(read_keys)"
BUZZ_RELAY_PRIVATE_KEY="$(echo "${SIGNER_OUT}" | awk '/Secret key:/ {print $3}')"
if [[ -z "${RELAY_OWNER_PUBKEY}" || -z "${BUZZ_RELAY_PRIVATE_KEY}" ]]; then
err "Failed to generate relay identities via buzz-admin generate-key"
exit 1
fi
export BUZZ_REQUIRE_RELAY_MEMBERSHIP=true
export RELAY_OWNER_PUBKEY
export BUZZ_RELAY_PRIVATE_KEY
# ── Start the membership-gated relay ─────────────────────────────────────────
# A stale relay on :3000 would pass the readiness poll while silently running
# WITHOUT membership gating — the stranger-denied assertion would then fail
# (or worse, an open relay would mask a real gating regression). Fail fast.
if lsof -nP -iTCP:3000 -sTCP:LISTEN >/dev/null 2>&1; then
err "Port 3000 is already in use — stop the existing relay first (its config would not be membership-gated)"
exit 1
fi
log "Starting membership-gated relay..."
CARGO_PROFILE="${CARGO_PROFILE}" ./scripts/start-relay-for-tests.sh --no-build
cleanup() {
log "Stopping relay..."
if [[ -f /tmp/buzz-relay.pid ]]; then
kill "$(cat /tmp/buzz-relay.pid)" 2>/dev/null || true
fi
# The harness kills its own children; sweep any stragglers from a hard fail.
pkill -f mesh_relay_lifecycle_smoke 2>/dev/null || true
}
trap cleanup EXIT
# ── Run the lifecycle harness ────────────────────────────────────────────────
log "Running relay-driven mesh lifecycle smoke..."
RELAY_URL=ws://localhost:3000 \
DATABASE_URL=postgres://buzz:buzz_dev@localhost:5432/buzz \
REDIS_URL=redis://localhost:6379 \
BUZZ_ADMIN_BIN="${ADMIN_BIN}" \
"${HARNESS_BIN}"
ok "Relay-driven mesh lifecycle smoke passed"