From 6530b58a61d4602d0a371100fedf80c5998b1e34 Mon Sep 17 00:00:00 2001 From: Tyler <109685178+tlongwell-block@users.noreply.github.com> Date: Sun, 2 Aug 2026 14:39:27 -0400 Subject: [PATCH] feat(k8s): Kubernetes backend plugin + desktop deploy path (#4289) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # Kubernetes backend plugin (crates/buzz-backend-kubernetes) + desktop deploy path Implements docs/remote-agents.md (merged @ 28ae6cd21) as ONE PR: the provider binary, the desktop changes that make it work, the harness inactivity reaper, the Sprig image, and the conformance/live-test suites. Channel: buzz-remote-agents (29414326-dba7-402d-b384-b1b34d63a2e6), thread c42b70ef. ## What's here (by lane) - **crates/buzz-backend-kubernetes** (Dawn): stdin/stdout JSON provider, info + deploy; pure classify.rs (one match arm per spec state-machine row); reconcile/GC with ownership-marker gate + same-clock orphan check; per-attempt immutable Secrets; three-tier env with clear-then-write authoritative tier. - **Desktop** (Mari): KD3 launch block from resolved descriptor, KD5 pre-secret negotiation gate (resolve-once → stage-and-digest → info → protocol gate → deploy), KD1 Windows extension strip, bundling (externalBin + Justfile + release/canary workflows + stub loops), tauri.windows.conf.json platform override (Decision B: no Windows artifact). - **buzz-acp** (Max): KD4 BUZZ_ACP_EXIT_AFTER_INACTIVITY reaper (pool-independent; reset only at accepted dispatch; in-flight turn/heartbeat defers, never resets); BUZZ_ACP_EXIT_AFTER_INACTIVITY + BUZZ_ACP_NO_PRESENCE reserved. KD8 fix. - **Image + tests** (Perci): Dockerfile.sprig (digest-pinned bases, exec buzz-acp PID 1, relay-scoped credential config), image contract script, provider conformance suites (golden wire fixtures shared with desktop tests), live-local runbook (namespace-scoped, shared-cluster safe). - **Docs** (Sami, first commit): citation re-pin c1bca1b56 → 28ae6cd21 (44/49 were already byte-exact; 3 offsets fixed) + I3 presence-bound correction (below). ## Named spec deviations (deliberate, each with rationale) 1. **No baked default image yet.** ghcr.io/block/buzz-sprig is unpublished (verified: anonymous pull 403 vs control 200). Omitted `image` returns an in-band field-required error instead of a default. 2. **Image override STRICTER than spec §Image:** digest-only (`name@sha256:<64hex>`); ALL tags rejected; `name:tag@digest` normalized. With no baked default the override is the only path, so tag-acceptance would make mutability the v1 norm. Strictness is reversible; a moved tag under an nsec is not. Baked digest default + tag re-acceptance = follow-up with image publish. 3. **imagePullSecrets not in schema (v1).** Explicit user images may rely on namespace-preprovisioned pull credentials — the substrate boundary. Field added only if the publish decision proves it necessary. 9-field budget intact. 4. **Decision A closed: writable empty workspace.** Nest projection = named follow-up; no image-side scaffolding. 5. **Decision D overridden by Tyler (event b55398d8):** provider ships bundled with the desktop like buzz-acp/buzz-agent; spec §Distribution's separate release workflow deleted for v1. 6. **I3/vision presence bound corrected 90s → 180s.** PRESENCE_TTL_SECS moved in #3783 during this spec's base→merge window; the number was inherited, not chosen. Spec :206/:216/:928 + inline quote + VISION_REMOTE_AGENTS.md:59 corrected. ← Tyler: the vision is your document; this edit is flagged for your explicit eyes. 7. **Spec citations are pinned to 28ae6cd21** (main at spec merge) and resolve there, not at this PR's head — this PR's own lanes move crates/buzz-acp/src/lib.rs by ~100 lines (19 citations across KD4/KD6/KD7/ §Stop/§Launch data). Known Defects rows fixed BY this PR retire on merge; the section documents main as of the pin. 8. **KD7 grace tension declared:** pod terminationGracePeriodSeconds=60 vs KD7's measured ~87s shutdown tail at parallelism 10 (~197s at cap 32). KD7 is ruled out of scope, so L1-3's "enough grace for full graceful shutdown" is NOT met at default config — deliberate, resolved by the KD7 follow-up, not silently. ## Question for Tyler Will ghcr.io/block/buzz-sprig publish PUBLIC? If private-by-policy, §Image needs an imagePullSecrets story before the baked-default follow-up can land. ## Out of scope (named follow-ups) KD6 exit-code contract + KD7 shutdown budget (gate OnFailure), OnFailure restart policy, Windows provider binary, PVCs/nest projection, mesh deployability, sprig image publish workflow + baked multi-arch digest default. ## Reproduce locally (four traps that cost us real time) **1. Git hooks inherit the invoking shell's PATH — pin the shell, not just your verification commands.** `rust-toolchain.toml` pins `1.95.0`, but the rustup shim that honors that pin lives in `~/.cargo/bin`. If Homebrew's cargo is earlier on PATH, `cargo` in this repo is 1.89.0, which cannot build the workspace at all: ``` $ /opt/homebrew/bin/cargo check -p buzz-db error: rustc 1.89.0 is not supported by the following packages: sqlx@0.9.0 requires rustc 1.94.0 ... # exit 101 ``` Verifying with `PATH="$HOME/.cargo/bin:$PATH" cargo test` does *not* protect the push: lefthook's `pre-push` → `just test-unit` re-resolves `cargo` from the shell's own PATH, so a green local run is followed by a hook failure on a crate you never touched. Export the PATH for the whole shell, not per-command. This bit twice. **2. Line-scope your mutations, or the mutation edits its own detector.** When mutation-testing the respond-to guard, a whole-file `sed` on the mode literal touches 5 sites — the guard *and* the fixtures/assertions that test it. The mutation and its detector move together and the suite stays green, which reads as "this code is dead" when it actually means "you deleted the experiment": ``` # WRONG — 5 sites, guard and tests mutate together $ sed -i '' 's/"allowlist"/"allowlist-DISABLED"/g' src/env.rs test result: ok. 145 passed; 0 failed # false survivor # RIGHT — 1 site, anchored to the guard's own definition line $ sed -i '' '/^const RESPOND_TO_ALLOWLIST/s/"allowlist"/"allowlist-DISABLED"/' src/env.rs failures: env::tests::allowlist_mode_with_an_empty_list_is_refused env::tests::an_allowlist_entry_that_is_not_64_hex_is_refused test result: FAILED. 143 passed; 2 failed # real kill ``` Restore by copying a pristine file back and confirming `git diff --stat` is empty, not by re-running an inverse `sed`. **3. A completeness guard is not a correctness guard.** The shared wire fixture `tests/fixtures/provider-wire/deploy-full-launch.request.json` passed every test we had while containing four classes of invented data (wrong `respond_to` encoding, an env key no emitter writes, allowlist entries that fail the harness's own 64-hex rule, a `launch.env` key from no descriptor layer). The provider's tests could not have caught this: its types are deliberately indifferent to these values (`Option`, `Vec`, arbitrary map), so "the provider parses it" was never evidence that the desktop emits it. The fix was not a stronger provider assertion but a rule about provenance — "recorded" means executed-and-transcribed, and the desktop's whole-object equality test is the only enforcement that can exist. See the fixture README. **4. Every drift this arc was a value that agreed with itself.** Five invented values were found, and not one was caught by an assertion failing — each was caught by someone asking where a value came from. A named constant referenced symbolically on both the fixture and assertion side. A `sed` that mutated its own detector. Six probe rows that all died at the same unrelated error. A descriptor struct literal compared against a fixture built from that literal (`launch.args: ["run","--session"]`, which the resolver actually returns as `["acp"]`). The general defense is not more assertions but provenance: a stub is a control that varies nothing, and the more faithful it looks the better it hides. Ask what executed, not what passed. *Fixture-test determinism caveat (post-verification, Quinn + Dawn).* The desktop's whole-object fixture test calls the real resolver, which consults a process-global harness registry whose own docs require `registry_test_lock` for any test touching it. The fixture test holds no lock and is nonetheless deterministic — but by containment, not by ordering. Measured, not derived: planting a definition with `id: "goose"` directly into the registry (bypassing the loader) changes the resolved descriptor from `args: ["acp"]` to `args: ["--poisoned"]`, so `resolve_effective_harness_descriptor` **does** reach the registry for this id — it does not short-circuit on the builtin table first. Two controls discriminate: an empty registry and a registry poisoned under a *different* id both return `["acp"]`. What actually protects the test is that the registry has exactly one writer (`update_loaded_harness_registry`, reached only via `warm_harness_registry_from_dir`) — but that writer concatenates **two** sources of unequal strength (`custom_harnesses.rs:319-326`). Custom files pass through `load_custom_harnesses`, whose `check_id_collision` rejects the reserved builtin id `goose` case-insensitively at the loader — and that leg is tested (`load_applies_id_collision_check` writes a real `goose.json` and asserts the loader drops it). Preset definitions (`preset_harness_definitions`, `presets.rs:177-193`) are a bare `.map` over `PRESET_HARNESSES` with **no collision check** — exhaustive call-site enumeration at `60007fda4` finds four production `check_id_collision` sites, none on the preset path. That leg holds only because `goose` is not in the preset table today (intersection of TIER1 and preset ids is empty) — executed, not just read: adding a preset with `id: "goose"`, `args: ["--poisoned"]` and warming via the normal preset-only path (`warm_harness_registry_from_dir(None)`, no custom dir, no direct writer) flips the fixture's emitted `launch.args` from `["acp"]` to `["--poisoned"]` at `60007fda4`, command/env/policy_env unchanged. So: no test in the suite can put a `goose` entry in the registry via the custom path, and no preset currently carries one, so no interleaving can perturb this fixture — containment with one checked leg and one coincidental one. A future fixture built on a **non-builtin** runtime id has no containment at all — it would be order-dependent against whatever registry-writing test ran last and must take the lock. *Late instance, found while reviewing the mode guard.* The guard exact-matches `respond_to` untrimmed and case-sensitively, which is only correct if clap's `ValueEnum` derive is case-sensitive. `config.rs` gives two answers: the derive at `:448-453` carries no `ignore_case`, while the crate's own tests call `RespondTo::from_str(s, true)` — `ignore_case = true`. Reading the source supports either. Measured on the built binary instead: `owner-only` starts, `OWNER-ONLY` / `Owner-Only` / `ALLOWLIST` / `NOBODY` all exit rc=2 `invalid value`. Case-sensitive at the CLI, so the guard is right — and right for a reason the source does not state. The `from_str(_, true)` tests exercise a different surface and are not evidence about the CLI. *Corollary, and the sharper half.* When a test helper **reimplements** production instead of calling it, the helper is a fork — and a fork can be right while production is wrong, or wrong in the same way, and the suite reports green either way. Both `BUZZ_ACP_ALLOWED_*` gates are forked like this: production compares **strings** while the helpers compare **post-parse enums** (`config.rs:2623`) or re-derive the split (`buzz-cli/.../channels.rs:1296`). Production and the helper each carry their *own* copy of the empty-entry filter (`:1025` and `:1300`), so fixing one says nothing about the other. Measured on `buzz-cli`, restoring byte-exact between runs: | tree | result | |---|---| | baseline | 274 passed | | drop the empty-filter in **production** only (the real fix) | **274 passed** — no signal | | drop it in the **test helper** only | **273 passed, 1 failed** (`channels.rs:1338`) | Two independent defects, stacked, and worse together than either alone: production can be fixed with no test ever noticing, *and* the helper cannot be corrected without a false alarm demanding the bug back. The root cause is one bit of type information — `check_allowed_channel_add_policy(allowed_raw: &str, ..)` cannot represent "unset", while production reads `env::var(..) -> Result`, where unset and `""` are different states. A helper whose parameter type can't represent all of production's input states isn't testing production's states — it's testing a subset it silently chose. Same family as the struct-literal descriptor and the fixture drift: the test and the thing it tests agreeing with each other, rather than the test measuring the thing. Neither defect is in this PR's diff (`git diff --name-only 28ae6cd21 -- crates/buzz-cli` is empty); both are now filed as NIP-34 issues on this repo: the fail-open + fork-helper defect at issue event `0524a4113f2d97fd…` and the respond-to self-lock at `e32837498969b5e7…` (filed 2026-08-02 after Quinn measured that no prior filing existed — zero hits on GitHub `block/buzz` open *or* closed and zero on the relay's kind:1621 issues, against working positive controls). The prescription was itself mutation-tested before being written down: repairing the fork's signature (`Option<&str>` + assertion → `None`) still let the reintroduced production bug ship 274-green — an expressive fork is still a fork; it never executes production. So the `buzz-cli` fix has **three parts and one explicit keep**: drop the production filter; **delete** the helper and point its tests at the real `cmd_set_add_policy` (which self-discriminates by error variant — `Usage` = refused, `Network(BadScheme)` = passed the gate — no relay needed); serialize the env-var tests behind one **`tokio::sync::Mutex::const_new`** lock taken with `.lock().await`, including the pre-existing `:1362` integration test (the fork was silently buying test isolation — without the lock, parallel runs flake nondeterministically; a `std::sync::Mutex` held across `.await` trips `clippy::await_holding_lock` under `-D warnings`); and **keep** the then-dead `!allowed.is_empty()` clause with a comment saying why. It is unreachable-false (`split(',')` never yields an empty vec), but it is the only thing that keeps the reintroduced production bug detectable — mutation-tested: on a tree that deletes the clause, reintroducing the empty-filter bug survives 275/0, because `""`/`","`/`" "` refuse either way and the filter goes semantically inert. Dead code can be load-bearing for tests: "provably unreachable" is an argument about behavior, never about coverage. When a helper forks production, the fix has to delete the fork: any change that leaves two implementations standing can only ever be verified against the one the tests call. *Final shape:* the keep and the broad lock are both artifacts of the fork surviving in some form. The extraction variant (Dawn, mutation-tested at `60007fda4`) removes the tension: extract one `check_channel_add_policy_allowed(Option<&str>, &str)` that **production calls**, with the `Option` placed at the env boundary where the `Result` bit actually lives. 5/6 mutants killed; the empty-filter survivor is proven **equivalent** (exhaustive 6174-pair check, 0 divergences, with a diverging negative control; independently re-derived by a second generator — different tokens and shape — 0 divergences on admitted policies, 500 on a non-admitted control), not a coverage hole — on a one-implementation tree there is no fork left to witness, so no dead clause needs keeping. One scope line on that equivalence: it is **caller-conditional**, a property of the only current caller, not of the gate function — `cmd_set_add_policy`'s own match at `:1027-1034` admits only three policies before the gate runs; a second caller reaching the gate with arbitrary strings resurrects m1 as a real hole. The lock does not disappear, it narrows (Dawn's own correction, caught by Mari): lock exactly the tests that mutate the process env — three-plus-one on a fork tree, two on the extraction tree — behind one `tokio::sync::Mutex`, and the lock is part of the assertion, not hygiene: with it deleted, the gate test fails 8/8 runs deterministically by receiving `Network(BadScheme)` where it expects `Usage` — the unset test's `remove_var` clobbers the other's `set_var`, and **the gate test passes straight through the gate**, a false negative on the exact authz assertion the test exists to make. State it as an outcome: these two tests must not observe each other's env writes. 276/0 stable across 5 parallel runs, clippy `-D warnings` clean; independently verified (patch applied to a second worktree: result blob `d67e584be` matches the patch index, full mutant matrix reproduces row for row). One new row no earlier prescription covered: collapsing unset into `Some("")` fails **closed** — an unconfigured deployment refuses every policy — killed by the unset test. Patch: `OUTBOX/BUZZ_CLI_ADD_POLICY_GATE_EXTRACT_FIX.patch`. The filed issue (`0524a411…`) carries the fork-shape prescription; whoever picks it up should prefer the extraction shape, drop the dead-clause keep with it, and keep part 3 outcome-shaped: serialize whichever tests mutate the env. ## Verification (final HEAD `60007fda4`) - Full touched-package suites at each integration merge (log in plan file). At candidate parent `00e5b5fe9`: buzz-backend-kubernetes 154, buzz-acp 673, desktop tauri 2100+3, pnpm 3908, workspace clippy/fmt/tsc all clean. The only delta to `60007fda4` is one character in `scripts/test-k8s-sprig-image-live.sh` (heredoc escape so the readlink probe evaluates pod-side, not host-side at render); `crates/` tree hash is byte-identical at both SHAs, so the Rust receipts attach by tree identity. buzz-backend-kubernetes suite re-run in-shell at `HEAD == 60007fda4`: 154 passed. - Adversarial one-HEAD gate (Sami): guard matrix 12/12, predicate mutants 7/7, doomed-invocation finding closed end-to-end; tree-hash carry to `60007fda4` confirmed (crates/buzz-backend-kubernetes blob unchanged). - Live-local pass per TESTING.md + skill-buzz-testing (Perci, at `60007fda4`): explicit `docker-desktop` context, digest-qualified image imported into node containerd `k8s.io` namespace, pull policy `Never`; pod printed `DIGEST_ABI_OK`, `resolved_spec` and `image_id` both the exact requested digest, script exit 0. Dedicated per-run namespace, ownership labels on every object, scoped cleanup verified empty after. - Implementation review (Wren) at `60007fda4`: 9.6 minimalness / 9.4 elegance / 9.3 correctness, no blocker. - `origin/eva/k8s-backend` == `60007fda4` (ls-remote verified; SHA identity is byte identity). --------- Signed-off-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta Signed-off-by: Tyler <109685178+tlongwell-block@users.noreply.github.com> Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com> Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz> Co-authored-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Co-authored-by: Dawn (sprout agent) Co-authored-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz> Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@buzz.block.builderlab.xyz> --- .../auto-tag-on-release-pr-merge.yml | 2 +- .github/workflows/benchmark-harbor.yml | 2 +- .github/workflows/ci.yml | 33 +- .github/workflows/docker.yml | 4 +- .github/workflows/helm-chart.yml | 6 +- .github/workflows/linux-canary.yml | 2 +- .github/workflows/release.yml | 18 +- .github/workflows/signed-macos-canary.yml | 2 +- .github/workflows/sprig-image.yml | 233 +++ .github/workflows/sprig.yml | 6 +- Cargo.lock | 149 ++ Cargo.toml | 8 + Dockerfile.sprig | 44 + Justfile | 26 +- VISION_REMOTE_AGENTS.md | 2 +- crates/buzz-acp/src/config.rs | 25 + crates/buzz-acp/src/lib.rs | 119 +- crates/buzz-acp/src/queue.rs | 5 + crates/buzz-backend-kubernetes/Cargo.toml | 36 + .../buzz-backend-kubernetes/src/classify.rs | 377 ++++ crates/buzz-backend-kubernetes/src/client.rs | 182 ++ crates/buzz-backend-kubernetes/src/cluster.rs | 427 +++++ crates/buzz-backend-kubernetes/src/config.rs | 443 +++++ crates/buzz-backend-kubernetes/src/env.rs | 732 ++++++++ crates/buzz-backend-kubernetes/src/gc.rs | 368 ++++ crates/buzz-backend-kubernetes/src/image.rs | 181 ++ crates/buzz-backend-kubernetes/src/intent.rs | 309 ++++ crates/buzz-backend-kubernetes/src/main.rs | 199 +++ crates/buzz-backend-kubernetes/src/naming.rs | 223 +++ crates/buzz-backend-kubernetes/src/observe.rs | 592 ++++++ crates/buzz-backend-kubernetes/src/pod.rs | 446 +++++ .../buzz-backend-kubernetes/src/reconcile.rs | 1585 +++++++++++++++++ crates/buzz-backend-kubernetes/src/wire.rs | 250 +++ .../tests/fixtures/provider-wire/README.md | 39 + .../deploy-full-launch.request.json | 52 + .../deploy-no-owner.request.json | 9 + .../deploy-no-owner.response.json | 1 + .../deploy-relay-mesh-padded.request.json | 11 + .../deploy-relay-mesh-padded.response.json | 1 + .../deploy-relay-mesh.request.json | 12 + .../deploy-relay-mesh.response.json | 1 + .../deploy-tag-image.request.json | 10 + .../deploy-tag-image.response.json | 1 + .../fixtures/provider-wire/info.request.json | 1 + .../tests/wire_fixtures.rs | 214 +++ desktop/scripts/build-release-config.mjs | 9 + desktop/src-tauri/Cargo.lock | 1 + desktop/src-tauri/Cargo.toml | 1 + desktop/src-tauri/src/commands/agents.rs | 2 +- .../src-tauri/src/commands/agents_deploy.rs | 246 ++- .../src-tauri/src/commands/agents_tests.rs | 138 +- .../src-tauri/src/managed_agents/backend.rs | 414 ++--- .../src/managed_agents/backend_tests.rs | 452 +++++ .../src-tauri/src/managed_agents/env_vars.rs | 27 +- .../src/managed_agents/env_vars/tests.rs | 9 + desktop/src-tauri/tauri.conf.json | 1 + desktop/src-tauri/tauri.windows.conf.json | 11 + .../agents/ui/ProviderConfigFields.test.mjs | 31 + .../agents/ui/ProviderConfigFields.tsx | 3 +- docs/remote-agents.md | 87 +- scripts/bundle-sidecars.sh | 8 +- scripts/run-tests.sh | 6 + scripts/sprig-entrypoint.sh | 16 + scripts/test-k8s-provider-release.sh | 78 + scripts/test-k8s-sprig-image-live.sh | 97 + scripts/test-sprig-image.sh | 43 + 66 files changed, 8633 insertions(+), 435 deletions(-) create mode 100644 .github/workflows/sprig-image.yml create mode 100644 Dockerfile.sprig create mode 100644 crates/buzz-backend-kubernetes/Cargo.toml create mode 100644 crates/buzz-backend-kubernetes/src/classify.rs create mode 100644 crates/buzz-backend-kubernetes/src/client.rs create mode 100644 crates/buzz-backend-kubernetes/src/cluster.rs create mode 100644 crates/buzz-backend-kubernetes/src/config.rs create mode 100644 crates/buzz-backend-kubernetes/src/env.rs create mode 100644 crates/buzz-backend-kubernetes/src/gc.rs create mode 100644 crates/buzz-backend-kubernetes/src/image.rs create mode 100644 crates/buzz-backend-kubernetes/src/intent.rs create mode 100644 crates/buzz-backend-kubernetes/src/main.rs create mode 100644 crates/buzz-backend-kubernetes/src/naming.rs create mode 100644 crates/buzz-backend-kubernetes/src/observe.rs create mode 100644 crates/buzz-backend-kubernetes/src/pod.rs create mode 100644 crates/buzz-backend-kubernetes/src/reconcile.rs create mode 100644 crates/buzz-backend-kubernetes/src/wire.rs create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/README.md create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-full-launch.request.json create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.request.json create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.response.json create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.request.json create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.response.json create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.request.json create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.response.json create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.request.json create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.response.json create mode 100644 crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/info.request.json create mode 100644 crates/buzz-backend-kubernetes/tests/wire_fixtures.rs create mode 100644 desktop/src-tauri/src/managed_agents/backend_tests.rs create mode 100644 desktop/src-tauri/tauri.windows.conf.json create mode 100644 desktop/src/features/agents/ui/ProviderConfigFields.test.mjs create mode 100755 scripts/sprig-entrypoint.sh create mode 100755 scripts/test-k8s-provider-release.sh create mode 100755 scripts/test-k8s-sprig-image-live.sh create mode 100755 scripts/test-sprig-image.sh diff --git a/.github/workflows/auto-tag-on-release-pr-merge.yml b/.github/workflows/auto-tag-on-release-pr-merge.yml index f31d4b835..3a090b3eb 100644 --- a/.github/workflows/auto-tag-on-release-pr-merge.yml +++ b/.github/workflows/auto-tag-on-release-pr-merge.yml @@ -45,7 +45,7 @@ jobs: github.event.pull_request.head.repo.full_name == github.repository runs-on: ubuntu-latest steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ github.event.pull_request.merge_commit_sha }} fetch-depth: 0 diff --git a/.github/workflows/benchmark-harbor.yml b/.github/workflows/benchmark-harbor.yml index 31efe933c..6024f0057 100644 --- a/.github/workflows/benchmark-harbor.yml +++ b/.github/workflows/benchmark-harbor.yml @@ -20,7 +20,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 15 steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 with: python-version: "3.12" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d392170f6..60507182d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -28,7 +28,7 @@ jobs: web: ${{ steps.filter.outputs.web }} mobile: ${{ steps.filter.outputs.mobile }} steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 2 - uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2 @@ -96,7 +96,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 with: @@ -117,7 +117,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: rui314/setup-mold@9c9c13bf4c3f1adef0cc596abc155580bcb04444 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 @@ -139,7 +139,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 2 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -235,7 +235,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - name: Get pnpm store directory id: pnpm-cache @@ -318,7 +318,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 # Reuse the relay binaries and backend test archive when none of their # inputs changed (desktop-only PRs hit this every time). The key covers @@ -391,7 +391,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - name: Start integration services run: | @@ -580,7 +580,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - name: Install cargo-nextest uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15 @@ -744,7 +744,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 with: @@ -785,7 +785,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 2 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -820,7 +820,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 2 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -881,7 +881,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - name: Dependency policy run: cargo-deny check @@ -893,7 +893,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - name: Check for dead API token references in client code run: | # Fail if dead API token patterns reappear in desktop, mobile, docs, or config. @@ -922,7 +922,7 @@ jobs: - x86_64-unknown-linux-musl - aarch64-unknown-linux-musl steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 with: @@ -959,7 +959,7 @@ jobs: env: TARGET: x86_64-pc-windows-msvc steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 # MSVC needs windows.h (aws-lc-sys et al.), so this runs on a real Windows # runner — hermit, used by the Linux jobs, does not provide MSVC. The # toolchain (1.95.0 + clippy via profile = default) comes from the @@ -1040,7 +1040,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 with: @@ -1054,6 +1054,7 @@ jobs: mkdir -p desktop/src-tauri/binaries touch "desktop/src-tauri/binaries/buzz-acp-$TARGET" touch "desktop/src-tauri/binaries/buzz-agent-$TARGET" + touch "desktop/src-tauri/binaries/buzz-backend-kubernetes-$TARGET" touch "desktop/src-tauri/binaries/buzz-dev-mcp-$TARGET" touch "desktop/src-tauri/binaries/git-credential-nostr-$TARGET" touch "desktop/src-tauri/binaries/buzz-$TARGET" diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index 52f21b28b..564cd74e9 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -101,7 +101,7 @@ jobs: steps: - name: Checkout - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 0 persist-credentials: false @@ -359,7 +359,7 @@ jobs: arch: arm64 steps: - name: Checkout - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 0 persist-credentials: false diff --git a/.github/workflows/helm-chart.yml b/.github/workflows/helm-chart.yml index e3d443d9f..7118d1670 100644 --- a/.github/workflows/helm-chart.yml +++ b/.github/workflows/helm-chart.yml @@ -59,7 +59,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1 with: # On chart-tag rescue dispatch, lint/render the tagged commit that the # publish job will package, not whatever `main` is when the dispatch @@ -119,7 +119,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1 with: fetch-depth: 0 @@ -166,7 +166,7 @@ jobs: packages: write # push the chart to GHCR steps: - name: Checkout - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1 with: # On the rescue dispatch, build the tagged commit (github.ref is # `main` there); on a tag push, the default ref is already the tag. diff --git a/.github/workflows/linux-canary.yml b/.github/workflows/linux-canary.yml index 980644337..166487877 100644 --- a/.github/workflows/linux-canary.yml +++ b/.github/workflows/linux-canary.yml @@ -166,7 +166,7 @@ jobs: - name: Build sidecars run: | - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh - name: Build Linux Tauri app diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 7d5f3fbf4..02011ad38 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -36,7 +36,7 @@ jobs: exit 1 fi - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 0 persist-credentials: false @@ -64,7 +64,7 @@ jobs: env: VERSION: ${{ needs.setup.outputs.version }} steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 @@ -91,7 +91,7 @@ jobs: - name: Build sidecars run: | - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh # Mesh rev derived from Cargo.lock (no lockstep edit on dep bump); cache key tracks it. @@ -278,7 +278,7 @@ jobs: VERSION: ${{ needs.setup.outputs.version }} TARGET: x86_64-apple-darwin steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 @@ -308,7 +308,7 @@ jobs: - name: Build sidecars run: | - cargo build --release --target "$TARGET" -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release --target "$TARGET" -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh "$TARGET" - name: Build unsigned Tauri app @@ -495,7 +495,7 @@ jobs: apt-get update apt-get install -y --no-install-recommends gh - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 @@ -563,7 +563,7 @@ jobs: - name: Build sidecars run: | - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh - name: Generate release config @@ -666,7 +666,7 @@ jobs: VERSION: ${{ needs.setup.outputs.version }} TARGET: x86_64-pc-windows-msvc steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 @@ -795,7 +795,7 @@ jobs: VERSION: ${{ needs.setup.outputs.version }} GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 diff --git a/.github/workflows/signed-macos-canary.yml b/.github/workflows/signed-macos-canary.yml index fb0656028..0a3a513ee 100644 --- a/.github/workflows/signed-macos-canary.yml +++ b/.github/workflows/signed-macos-canary.yml @@ -93,7 +93,7 @@ jobs: - name: Build sidecars run: | - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh # Mesh rev derived from Cargo.lock (no lockstep edit on dep bump); cache key tracks it. diff --git a/.github/workflows/sprig-image.yml b/.github/workflows/sprig-image.yml new file mode 100644 index 000000000..5d5e12ae0 --- /dev/null +++ b/.github/workflows/sprig-image.yml @@ -0,0 +1,233 @@ +name: Sprig image + +# Builds and publishes the public agent container image as +# ghcr.io/block/buzz-sprig — the digest-pinned box the Kubernetes backend +# deploys agents into (see Dockerfile.sprig and docs/remote-agents.md). +# +# Strategy mirrors docker.yml (the relay image): each architecture builds on +# its native runner, pushes to GHCR by digest, then a merge job stitches the +# per-arch digests into one multi-arch manifest and attests provenance. +# No QEMU emulation. +# +# Triggers: +# - push to main (paths-filtered) → :main + :sha-<7> +# - tag sprig-v* → semver family (shared with sprig.yml's +# binary release — one tag versions both) +# - pull_request (paths-filtered) → build only, no push +# - workflow_dispatch → manual publish at the current ref +# +# NOTE: the first push creates the GHCR package PRIVATE by default. An org +# admin must flip ghcr.io/block/buzz-sprig to public once (Package settings → +# Change visibility). Subsequent pushes keep the visibility. + +on: + push: + branches: [main] + tags: ["sprig-v[0-9]*"] + paths: + - "Dockerfile.sprig" + - "scripts/sprig-entrypoint.sh" + - ".github/workflows/sprig-image.yml" + - "Cargo.toml" + - "Cargo.lock" + - "rust-toolchain.toml" + - "crates/**" + pull_request: + paths: + - "Dockerfile.sprig" + - "scripts/sprig-entrypoint.sh" + - ".github/workflows/sprig-image.yml" + workflow_dispatch: {} + +concurrency: + group: sprig-image-${{ github.ref }} + cancel-in-progress: ${{ github.ref_type == 'branch' && github.event_name == 'pull_request' }} + +permissions: {} + +env: + # Single source of truth for the image name; override with the + # GHCR_SPRIG_IMAGE repo variable (same pattern as docker.yml). + IMAGE_NAME: ${{ vars.GHCR_SPRIG_IMAGE != '' && vars.GHCR_SPRIG_IMAGE || 'ghcr.io/block/buzz-sprig' }} + +jobs: + build: + name: Build (${{ matrix.platform }}) + runs-on: ${{ matrix.runner }} + timeout-minutes: 60 + permissions: + contents: read + packages: write + id-token: write + attestations: write + strategy: + fail-fast: false + matrix: + include: + - platform: linux/amd64 + runner: ubuntu-24.04 + arch: amd64 + - platform: linux/arm64 + runner: ubuntu-24.04-arm + arch: arm64 + + steps: + - name: Checkout + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 + with: + persist-credentials: false + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0 + with: + # Same OOM cap as docker.yml — Rust compiles blow the 7GB runner + # at buildkit's default parallelism of 4. + buildkitd-config-inline: | + [worker.oci] + max-parallelism = 2 + + - name: Log in to GHCR + if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository + uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0 + with: + registry: ghcr.io + username: ${{ github.repository_owner }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Extract metadata + id: meta + uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0 + with: + images: ${{ env.IMAGE_NAME }} + # match=^sprig-v(.*)$ strips the tag prefix for the semver parser, + # exactly as docker.yml does for relay-v. :latest comes from + # flavor.latest=auto — stable semver only, never main pushes. + tags: | + type=ref,event=branch + type=sha,prefix=sha-,format=short + type=semver,pattern={{version}},match=^sprig-v(.*)$ + type=semver,pattern={{major}}.{{minor}},match=^sprig-v(.*)$ + labels: | + org.opencontainers.image.title=Buzz Sprig + org.opencontainers.image.description=Agent runtime image for Buzz remote agents (buzz-acp multicall + git + curl) + org.opencontainers.image.licenses=Apache-2.0 + + - name: Build and push by digest + id: build + uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0 + with: + context: . + file: ./Dockerfile.sprig + platforms: ${{ matrix.platform }} + labels: ${{ steps.meta.outputs.labels }} + outputs: type=image,name=${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=${{ github.event_name != 'pull_request' }} + cache-from: | + type=registry,ref=${{ env.IMAGE_NAME }}-buildcache:${{ matrix.arch }} + cache-to: | + ${{ (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) && format('type=registry,ref={0}-buildcache:{1},mode=max,compression=zstd', env.IMAGE_NAME, matrix.arch) || '' }} + + - name: Export digest + if: github.event_name != 'pull_request' + env: + DIGEST: ${{ steps.build.outputs.digest }} + run: | + mkdir -p /tmp/digests + touch "/tmp/digests/${DIGEST#sha256:}" + + - name: Upload digest + if: github.event_name != 'pull_request' + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: sprig-digest-${{ matrix.arch }} + path: /tmp/digests/* + if-no-files-found: error + retention-days: 1 + + merge: + name: Merge multi-arch manifest + if: github.event_name != 'pull_request' + runs-on: ubuntu-24.04 + needs: build + timeout-minutes: 15 + permissions: + contents: read + packages: write + id-token: write + attestations: write + + steps: + - name: Download per-arch digests + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + path: /tmp/digests + pattern: sprig-digest-* + merge-multiple: true + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0 + + - name: Log in to GHCR + uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0 + with: + registry: ghcr.io + username: ${{ github.repository_owner }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Extract metadata + id: meta + uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0 + with: + images: ${{ env.IMAGE_NAME }} + # Must mirror the build job's tag matrix exactly (see docker.yml). + flavor: | + latest=auto + tags: | + type=ref,event=branch + type=sha,prefix=sha-,format=short + type=semver,pattern={{version}},match=^sprig-v(.*)$ + type=semver,pattern={{major}}.{{minor}},match=^sprig-v(.*)$ + + - name: Create and push manifest list + id: manifest + working-directory: /tmp/digests + env: + IMAGE_NAME: ${{ env.IMAGE_NAME }} + META_TAGS: ${{ steps.meta.outputs.tags }} + run: | + set -euo pipefail + tags=() + while IFS= read -r tag; do + [ -n "$tag" ] && tags+=("-t" "$tag") + done <<< "$META_TAGS" + + digests=() + for digest in *; do + digests+=("${IMAGE_NAME}@sha256:${digest}") + done + + docker buildx imagetools create "${tags[@]}" "${digests[@]}" + + first_tag=$(echo "$META_TAGS" | head -n1) + merged_digest=$(docker buildx imagetools inspect "$first_tag" \ + --format '{{json .Manifest}}' | jq -r '.digest') + echo "digest=${merged_digest}" >> "$GITHUB_OUTPUT" + + - name: Attest provenance for the merged image + # Verify with: gh attestation verify oci://ghcr.io/block/buzz-sprig: --owner block + uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1 + with: + subject-name: ${{ env.IMAGE_NAME }} + subject-digest: ${{ steps.manifest.outputs.digest }} + push-to-registry: true + + - name: Summary + env: + IMAGE_NAME: ${{ env.IMAGE_NAME }} + DIGEST: ${{ steps.manifest.outputs.digest }} + run: | + { + echo "### Sprig image published" + echo '```' + echo "${IMAGE_NAME}@${DIGEST}" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/sprig.yml b/.github/workflows/sprig.yml index b2dab3583..5e50808b3 100644 --- a/.github/workflows/sprig.yml +++ b/.github/workflows/sprig.yml @@ -42,7 +42,7 @@ jobs: - x86_64-unknown-linux-musl - aarch64-unknown-linux-musl steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -116,7 +116,7 @@ jobs: permissions: contents: write steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - name: Download all Sprig artifacts uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 @@ -154,7 +154,7 @@ jobs: permissions: contents: write steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - name: Download all Sprig artifacts uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 diff --git a/Cargo.lock b/Cargo.lock index fa02e17ce..62bcea0ca 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -915,6 +915,26 @@ dependencies = [ "uuid", ] +[[package]] +name = "buzz-backend-kubernetes" +version = "0.1.0" +dependencies = [ + "chrono", + "hex", + "http", + "http-body-util", + "k8s-openapi", + "kube", + "nostr", + "rand 0.10.1", + "rustls", + "serde", + "serde_json", + "sha2 0.11.0", + "tokio", + "tower", +] + [[package]] name = "buzz-cli" version = "0.1.0" @@ -3643,6 +3663,7 @@ dependencies = [ "http", "hyper", "hyper-util", + "log", "rustls", "rustls-native-certs", "tokio", @@ -4303,6 +4324,31 @@ dependencies = [ "ucd-trie", ] +[[package]] +name = "jsonpath-rust" +version = "0.7.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c00ae348f9f8fd2d09f82a98ca381c60df9e0820d8d79fce43e649b4dc3128b" +dependencies = [ + "pest", + "pest_derive", + "regex", + "serde_json", + "thiserror 2.0.18", +] + +[[package]] +name = "k8s-openapi" +version = "0.26.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06d9e5e61dd037cdc51da0d7e2b2be10f497478ea7e120d85dad632adb99882b" +dependencies = [ + "base64 0.22.1", + "chrono", + "serde", + "serde_json", +] + [[package]] name = "kasuari" version = "0.4.12" @@ -4348,6 +4394,70 @@ version = "0.4.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e037a2e1d8d5fdbd49b16a4ea09d5d6401c1f29eca5ff29d03d3824dba16256a" +[[package]] +name = "kube" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "48e7bb0b6a46502cc20e4575b6ff401af45cfea150b34ba272a3410b78aa014e" +dependencies = [ + "k8s-openapi", + "kube-client", + "kube-core", +] + +[[package]] +name = "kube-client" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4987d57a184d2b5294fdad3d7fc7f278899469d21a4da39a8f6ca16426567a36" +dependencies = [ + "base64 0.22.1", + "bytes", + "chrono", + "either", + "futures", + "home", + "http", + "http-body", + "http-body-util", + "hyper", + "hyper-rustls", + "hyper-timeout", + "hyper-util", + "jsonpath-rust", + "k8s-openapi", + "kube-core", + "pem", + "rustls", + "secrecy", + "serde", + "serde_json", + "serde_yaml", + "thiserror 2.0.18", + "tokio", + "tokio-util", + "tower", + "tower-http", + "tracing", +] + +[[package]] +name = "kube-core" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "914bbb770e7bb721a06e3538c0edd2babed46447d128f7c21caa68747060ee73" +dependencies = [ + "chrono", + "derive_more", + "form_urlencoded", + "http", + "k8s-openapi", + "serde", + "serde-value", + "serde_json", + "thiserror 2.0.18", +] + [[package]] name = "lab" version = "0.11.0" @@ -6265,6 +6375,15 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d" +[[package]] +name = "ordered-float" +version = "2.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68f19d67e5a2795c94e73e0bb1cc1a7edeb2e28efd39e2e1c9b7a40c1108b11c" +dependencies = [ + "num-traits", +] + [[package]] name = "ordered-float" version = "4.6.0" @@ -6445,6 +6564,16 @@ dependencies = [ "hmac 0.12.1", ] +[[package]] +name = "pem" +version = "3.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d30c53c26bc5b31a98cd02d20f25a7c8567146caf63ed593a9d87b2775291be" +dependencies = [ + "base64 0.22.1", + "serde_core", +] + [[package]] name = "pem-rfc7468" version = "1.0.0" @@ -8170,6 +8299,15 @@ dependencies = [ "cc", ] +[[package]] +name = "secrecy" +version = "0.10.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e891af845473308773346dc847b2c23ee78fe442e0472ac50e22a18a93d3ae5a" +dependencies = [ + "zeroize", +] + [[package]] name = "secret-service" version = "4.0.0" @@ -8269,6 +8407,16 @@ dependencies = [ "serde_derive", ] +[[package]] +name = "serde-value" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f3a1a3341211875ef120e117ea7fd5228530ae7e7036a779fdc9117be6b3282c" +dependencies = [ + "ordered-float 2.10.1", + "serde", +] + [[package]] name = "serde_bytes" version = "0.11.19" @@ -9869,6 +10017,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4cfcf7e2740e6fc6d4d688b4ef00650406bb94adf4731e43c096c3a19fe40840" dependencies = [ "async-compression", + "base64 0.22.1", "bitflags 2.13.0", "bytes", "futures-core", diff --git a/Cargo.toml b/Cargo.toml index 3268cfaf8..cc1dd0f9d 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -27,6 +27,7 @@ members = [ "crates/buzz-relay-mesh", "crates/buzz-dev-mcp", "crates/buzz-voice", + "crates/buzz-backend-kubernetes", "examples/countdown-bot", ] exclude = ["desktop/src-tauri"] @@ -58,6 +59,13 @@ sqlx = { version = "0.9", features = [ redis = { version = "1.0", features = ["tokio-comp", "connection-manager", "tokio-rustls-comp"] } deadpool-redis = { version = "0.23", features = ["rt_tokio_1"] } +# Kubernetes (buzz-backend-kubernetes provider). No `ring` feature here: the +# process-level CryptoProvider is installed explicitly at startup, matching +# buzz-cli/buzz-acp/buzz-admin/buzz-relay/buzz-dev-mcp — see the comment on the +# crate's own rustls dependency. +kube = { version = "2.0", default-features = false, features = ["client", "rustls-tls"] } +k8s-openapi = { version = "0.26", features = ["v1_31"] } + # Nostr nostr = { version = "0.44", features = ["nip44", "nip98"] } diff --git a/Dockerfile.sprig b/Dockerfile.sprig new file mode 100644 index 000000000..160e0b566 --- /dev/null +++ b/Dockerfile.sprig @@ -0,0 +1,44 @@ +# syntax=docker/dockerfile:1.7 +# Multi-arch is produced by building this file on native amd64 and arm64 runners. +# Keep both bases pinned to manifest-list digests so either architecture resolves +# to immutable source bytes. +FROM rust:1.95-alpine3.22@sha256:064dfc925d68d1a63f4fd2871bd7dc6e6ea56692989a487185855d62885d90aa AS builder + +RUN apk add --no-cache \ + build-base \ + cmake \ + git \ + musl-dev \ + openssl-dev \ + openssl-libs-static \ + perl \ + pkgconf \ + protoc +WORKDIR /build +COPY . . +RUN cargo build --locked --profile sprig -p sprig \ + && strip target/sprig/sprig + +FROM alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce + +RUN apk add --no-cache bash ca-certificates curl git \ + && adduser -D -h /home/agent agent \ + && install -d -o agent -g agent /workspace /home/agent \ + && git config --system gpg.format x509 \ + && git config --system gpg.x509.program /usr/local/bin/git-sign-nostr \ + && git config --system commit.gpgSign true \ + && git config --system tag.gpgSign true + +COPY --from=builder --chmod=0755 /build/target/sprig/sprig /usr/local/bin/sprig +COPY --chmod=0755 scripts/sprig-entrypoint.sh /usr/local/bin/sprig-entrypoint +RUN for name in \ + buzz-acp buzz-agent buzz-dev-mcp rg tree buzz \ + git-credential-nostr git-sign-nostr; do \ + ln -s sprig "/usr/local/bin/$name"; \ + done + +ENV HOME=/home/agent \ + PATH=/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/bin +WORKDIR /home/agent +USER agent +ENTRYPOINT ["/usr/local/bin/sprig-entrypoint"] diff --git a/Justfile b/Justfile index 64a1f36da..8dbe125a7 100644 --- a/Justfile +++ b/Justfile @@ -155,7 +155,11 @@ _ensure-sidecar-stubs: set -euo pipefail TARGET=$(rustc -vV | sed -n 's|host: ||p') mkdir -p desktop/src-tauri/binaries - for bin in buzz-acp buzz-agent buzz-dev-mcp git-credential-nostr buzz; do + SIDECARS=(buzz-acp buzz-agent buzz-dev-mcp git-credential-nostr buzz) + if [[ "$TARGET" != *windows* ]]; then + SIDECARS+=(buzz-backend-kubernetes) + fi + for bin in "${SIDECARS[@]}"; do touch "desktop/src-tauri/binaries/${bin}-${TARGET}" done @@ -236,6 +240,9 @@ desktop-release-build target="aarch64-apple-darwin": mkdir -p desktop/src-tauri/binaries touch "desktop/src-tauri/binaries/buzz-acp-$TARGET" touch "desktop/src-tauri/binaries/buzz-agent-$TARGET" + if [[ "$TARGET" != *windows* ]]; then + touch "desktop/src-tauri/binaries/buzz-backend-kubernetes-$TARGET" + fi touch "desktop/src-tauri/binaries/buzz-dev-mcp-$TARGET" touch "desktop/src-tauri/binaries/git-credential-nostr-$TARGET" touch "desktop/src-tauri/binaries/buzz-$TARGET" @@ -274,6 +281,7 @@ test: # Run unit tests only (no infra needed) test-unit: #!/usr/bin/env bash + set -euo pipefail if command -v cargo-nextest &>/dev/null; then cargo nextest run -p buzz-core -p buzz-auth --lib cargo nextest run -p buzz-voice --lib @@ -293,6 +301,12 @@ test-unit: # Gateway unit and black-box HTTP tests are infra-free. Postgres-backed # contract/race tests run in the dedicated CI job below. cargo nextest run -p buzz-push-gateway + # Kubernetes backend provider: the decision layers (state machine, GC + # planner, env precedence, naming, wire) are pure functions with a fake + # substrate, so they belong in the unit job. Enumerated explicitly + # because nothing in CI runs `cargo test --workspace` — workspace + # membership alone buys clippy/check, not a single executed test. + cargo nextest run -p buzz-backend-kubernetes else ./scripts/run-tests.sh unit fi @@ -430,7 +444,7 @@ dev *ARGS: bootstrap _ensure-sidecar-stubs _ensure-migrations fi done fi - cargo build -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr -p buzz-relay + cargo build -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr -p buzz-relay if [[ -n "{{mesh}}" ]]; then export MESH_LLM_NATIVE_RUNTIME_CACHE_DIR="$(./scripts/ensure-mesh-native-runtime.sh)" fi @@ -477,10 +491,10 @@ desktop-standalone *ARGS: _ensure-sidecar-stubs #!/usr/bin/env bash set -euo pipefail export PATH="{{justfile_directory()}}/bin:$PATH" - cargo build -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr + cargo build -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr TARGET=$(rustc -vV | sed -n 's|host: ||p') TARGET_DIR=$(cargo metadata --format-version 1 --no-deps | node -p "JSON.parse(require('fs').readFileSync(0, 'utf8')).target_directory") - for bin in buzz-acp buzz-agent buzz-dev-mcp git-credential-nostr buzz; do + for bin in buzz-acp buzz-agent buzz-backend-kubernetes buzz-dev-mcp git-credential-nostr buzz; do cp "${TARGET_DIR}/debug/${bin}" "desktop/src-tauri/binaries/${bin}-${TARGET}" chmod +x "desktop/src-tauri/binaries/${bin}-${TARGET}" done @@ -506,7 +520,7 @@ staging *ARGS: bootstrap _ensure-sidecar-stubs set -euo pipefail export PATH="{{justfile_directory()}}/bin:$PATH" pnpm install # unconditional: staging must always start with a clean dep tree - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr FEATURES=() if [[ -n "{{mesh}}" ]]; then FEATURES=(--features mesh-llm) @@ -533,7 +547,7 @@ production *ARGS: bootstrap _ensure-sidecar-stubs set -euo pipefail export PATH="{{justfile_directory()}}/bin:$PATH" pnpm install # unconditional: production must always start with a clean dep tree - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr FEATURES=() if [[ -n "{{mesh}}" ]]; then FEATURES=(--features mesh-llm) diff --git a/VISION_REMOTE_AGENTS.md b/VISION_REMOTE_AGENTS.md index b02d1bc92..4b187f355 100644 --- a/VISION_REMOTE_AGENTS.md +++ b/VISION_REMOTE_AGENTS.md @@ -56,7 +56,7 @@ Remote agents solve it from the inside. Because the desktop retains no substrate **The body's state is mortal.** Files, checkouts, half-finished working trees — gone with the body unless the substrate persists them. The agent survives; its scratch space doesn't. Durable knowledge belongs on the relay, and agents are built to put it there. -**Presence can lag the truth, but not for long.** If the substrate kills a body without ceremony, the presence dot can outlive the agent — by seconds if the connection drops cleanly, by at most about ninety if it doesn't. Presence is a lease the agent renews, not a flag it sets: a dead agent stops renewing and the relay forgets it. Ninety seconds of a wrong dot, never an indefinite one. +**Presence can lag the truth, but not for long.** If the substrate kills a body without ceremony, the presence dot can outlive the agent — by seconds if the connection drops cleanly, by at most about three minutes if it doesn't. Presence is a lease the agent renews, not a flag it sets: a dead agent stops renewing and the relay forgets it. A bounded wrong dot, never an indefinite one. **A running agent finishes on the configuration it started with.** New keys, new models, new settings take effect on the next body. And an instance that never got far enough to run — a body that failed to start — is the substrate operator's residue to clear, with the substrate's own tools. Editing an agent mid-sentence was never on the menu. diff --git a/crates/buzz-acp/src/config.rs b/crates/buzz-acp/src/config.rs index dab61be30..35aaec188 100644 --- a/crates/buzz-acp/src/config.rs +++ b/crates/buzz-acp/src/config.rs @@ -474,6 +474,11 @@ pub struct CliArgs { #[arg(long, env = "BUZZ_ACP_RELAY_OBSERVER", default_value_t = false)] pub relay_observer: bool, + /// Exit after this many seconds with no dispatched events and no turn in flight. + /// 0 disables inactivity self-termination. + #[arg(long, env = "BUZZ_ACP_EXIT_AFTER_INACTIVITY", default_value_t = 0)] + pub exit_after_inactivity: u64, + /// Connect and subscribe before starting the ACP/LLM subprocess pool. #[arg(long, env = "BUZZ_ACP_LAZY_POOL", default_value_t = false)] pub lazy_pool: bool, @@ -550,6 +555,8 @@ pub struct Config { pub has_generated_codex_config: bool, /// Whether to publish encrypted observer frames through the relay. pub relay_observer: bool, + /// Seconds without dispatched events before an idle harness exits. 0 = disabled. + pub exit_after_inactivity_secs: u64, /// Whether ACP/LLM subprocess initialization is deferred until accepted work arrives. pub lazy_pool: bool, /// Agent owner pubkey (hex). Used for `--respond-to=owner-only` gate. @@ -1098,6 +1105,7 @@ impl Config { persona_env_vars, has_generated_codex_config, relay_observer: args.relay_observer, + exit_after_inactivity_secs: args.exit_after_inactivity, lazy_pool: args.lazy_pool, agent_owner: args.agent_owner.map(|s| s.trim().to_ascii_lowercase()), no_base_prompt: args.no_base_prompt, @@ -1468,6 +1476,7 @@ mod tests { persona_env_vars: vec![], has_generated_codex_config: false, relay_observer: false, + exit_after_inactivity_secs: 0, lazy_pool: false, agent_owner: None, no_base_prompt: false, @@ -2167,6 +2176,22 @@ channels = "ALL" assert!(err.to_string().contains("turn liveness interval must be 0")); } + #[test] + fn inactivity_exit_defaults_disabled_and_accepts_cli_value() { + let key = "0".repeat(64); + let default = CliArgs::parse_from(["buzz-acp", "--private-key", &key]); + assert_eq!(default.exit_after_inactivity, 0); + + let configured = CliArgs::parse_from([ + "buzz-acp", + "--private-key", + &key, + "--exit-after-inactivity", + "120", + ]); + assert_eq!(configured.exit_after_inactivity, 120); + } + #[test] fn lazy_pool_defaults_off() { let key = "0".repeat(64); diff --git a/crates/buzz-acp/src/lib.rs b/crates/buzz-acp/src/lib.rs index 403512a32..811253e4a 100644 --- a/crates/buzz-acp/src/lib.rs +++ b/crates/buzz-acp/src/lib.rs @@ -1228,6 +1228,59 @@ impl Drop for RespawnGuard { // sync entry point — `std::env::set_var` is only safe before tokio spawns // worker threads (Rust 2024 edition safety requirement). +fn inactivity_expired( + last_activity: tokio::time::Instant, + now: tokio::time::Instant, + bound: Duration, + turn_in_flight: bool, +) -> bool { + !bound.is_zero() && !turn_in_flight && now.duration_since(last_activity) >= bound +} + +#[cfg(test)] +mod inactivity_tests { + use super::*; + + #[test] + fn zero_disables_expiry_and_in_flight_turns_defer_it() { + let started = tokio::time::Instant::now(); + let after_bound = started + Duration::from_secs(61); + + assert!(!inactivity_expired( + started, + after_bound, + Duration::ZERO, + false + )); + assert!(!inactivity_expired( + started, + after_bound, + Duration::from_secs(60), + true + )); + assert!(inactivity_expired( + started, + after_bound, + Duration::from_secs(60), + false + )); + } + + #[test] + fn dispatched_activity_restarts_the_inactivity_bound() { + let started = tokio::time::Instant::now(); + let dispatched = started + Duration::from_secs(50); + let checked = started + Duration::from_secs(61); + + assert!(!inactivity_expired( + dispatched, + checked, + Duration::from_secs(60), + false + )); + } +} + pub fn run() -> Result<()> { config::propagate_legacy_env_vars(); tokio_main() @@ -1601,6 +1654,21 @@ async fn tokio_main() -> Result<()> { let mut typing_channels: HashMap = HashMap::new(); let mut presence_task: Option> = None; + // Independent of pool readiness: a never-mentioned lazy agent must still + // self-terminate. The watch interval is capped so small configured bounds + // remain reasonably precise without waking long-lived agents frequently. + let inactivity_bound = Duration::from_secs(config.exit_after_inactivity_secs); + let mut last_activity = tokio::time::Instant::now(); + let mut inactivity_reaper = if inactivity_bound.is_zero() { + None + } else { + let interval = inactivity_bound.min(Duration::from_secs(30)); + Some(tokio::time::interval_at( + tokio::time::Instant::now() + interval, + interval, + )) + }; + // Runs at the TOP of every loop iteration via Instant check — cannot be // starved by the biased select. Slot refill spawns background tasks so // spawn_and_init never blocks the main loop. @@ -1774,7 +1842,9 @@ async fn tokio_main() -> Result<()> { // called on relay events or pool results, neither of which // arrive when the channel is silent. if queue.has_flushable_work() { - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -1810,7 +1880,9 @@ async fn tokio_main() -> Result<()> { // this, batches requeued during crash recovery sit idle until the // next relay event arrives — which can be minutes on quiet channels. if respawn_collected { - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -2258,7 +2330,7 @@ async fn tokio_main() -> Result<()> { } if pool_ready { for (channel_id, thread_tags) in - dispatch_pending(&mut pool, &mut queue, &ctx) + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) { typing_channels.insert(channel_id, thread_tags); } @@ -2275,6 +2347,27 @@ async fn tokio_main() -> Result<()> { } None } + _ = async { + match inactivity_reaper.as_mut() { + Some(timer) => timer.tick().await, + None => std::future::pending().await, + } + } => { + let _ = result_rx; + if inactivity_expired( + last_activity, + tokio::time::Instant::now(), + inactivity_bound, + queue.has_in_flight() || heartbeat_in_flight, + ) { + tracing::info!( + inactivity_seconds = config.exit_after_inactivity_secs, + "inactivity bound reached — exiting gracefully" + ); + let _ = shutdown_tx.send(()); + } + None + } _ = async { match heartbeat.as_mut() { Some(hb) => hb.tick().await, @@ -2287,7 +2380,7 @@ async fn tokio_main() -> Result<()> { } else if queue.has_flushable_work() { tracing::debug!("heartbeat_skipped_events"); for (channel_id, thread_tags) in - dispatch_pending(&mut pool, &mut queue, &ctx) + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) { typing_channels.insert(channel_id, thread_tags); } @@ -2385,7 +2478,9 @@ async fn tokio_main() -> Result<()> { { break; } - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -2408,7 +2503,9 @@ async fn tokio_main() -> Result<()> { tracing::error!("all agents dead — exiting"); break; } - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -2550,7 +2647,9 @@ async fn tokio_main() -> Result<()> { // tear down the in-flight task; on its completion the // queue drains. We still try here in case the in-flight // task has already returned. - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -2577,7 +2676,7 @@ async fn tokio_main() -> Result<()> { None, ); for (channel_id, thread_tags) in - dispatch_pending(&mut pool, &mut queue, &ctx) + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) { typing_channels.insert(channel_id, thread_tags); } @@ -2911,6 +3010,7 @@ fn dispatch_pending( pool: &mut AgentPool, queue: &mut EventQueue, ctx: &Arc, + last_activity: &mut tokio::time::Instant, ) -> Vec<(Uuid, ThreadTags)> { let mut dispatched_channels = Vec::new(); loop { @@ -2990,6 +3090,7 @@ fn dispatch_pending( }, ); dispatched_channels.push((channel_id, typing_scope)); + *last_activity = tokio::time::Instant::now(); } tracing::debug!( dispatched = dispatched_channels.len(), @@ -5031,6 +5132,7 @@ mod build_mcp_servers_tests { persona_env_vars: vec![], has_generated_codex_config: false, relay_observer: false, + exit_after_inactivity_secs: 0, lazy_pool: false, agent_owner: None, no_base_prompt: false, @@ -5252,6 +5354,7 @@ mod error_outcome_emission_tests { persona_env_vars: vec![], has_generated_codex_config: false, relay_observer: false, + exit_after_inactivity_secs: 0, lazy_pool: false, agent_owner: None, no_base_prompt: false, diff --git a/crates/buzz-acp/src/queue.rs b/crates/buzz-acp/src/queue.rs index 029bf86db..5c960de20 100644 --- a/crates/buzz-acp/src/queue.rs +++ b/crates/buzz-acp/src/queue.rs @@ -646,6 +646,11 @@ impl EventQueue { self.in_flight_channels.contains(&channel_id) } + /// Whether any channel currently has a turn in flight. + pub fn has_in_flight(&self) -> bool { + !self.in_flight_channels.is_empty() + } + // ── Goose-native steer withhold (side table) ────────────────────────── // // While a goose-native `_goose/unstable/session/steer` write is in flight diff --git a/crates/buzz-backend-kubernetes/Cargo.toml b/crates/buzz-backend-kubernetes/Cargo.toml new file mode 100644 index 000000000..1cd17030d --- /dev/null +++ b/crates/buzz-backend-kubernetes/Cargo.toml @@ -0,0 +1,36 @@ +[package] +name = "buzz-backend-kubernetes" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +description = "Kubernetes backend provider for Buzz remote agents (docs/remote-agents.md)" + +[[bin]] +name = "buzz-backend-kubernetes" +path = "src/main.rs" + +[dependencies] +kube = { workspace = true } +k8s-openapi = { workspace = true } +nostr = { workspace = true } +tokio = { workspace = true } +serde = { workspace = true } +serde_json = { workspace = true } +sha2 = { workspace = true } +hex = { workspace = true } +rand = { workspace = true } +chrono = { workspace = true } +http = "1" +http-body-util = "0.1" + +# Explicit rustls dep with the ring provider — required to install the +# process-level CryptoProvider at startup. Without it this binary panics on its +# first TLS connection to the apiserver: the release build compiles every +# sidecar in one cargo invocation (.github/workflows/release.yml), which unifies +# both ring and aws-lc-rs features and leaves rustls unable to auto-select a +# provider. Same dependency and reason as crates/buzz-cli/Cargo.toml. +rustls = { version = "0.23", default-features = false, features = ["ring", "std"] } + +[dev-dependencies] +tower = { workspace = true } diff --git a/crates/buzz-backend-kubernetes/src/classify.rs b/crates/buzz-backend-kubernetes/src/classify.rs new file mode 100644 index 000000000..e9cfaba40 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/classify.rs @@ -0,0 +1,377 @@ +//! The deploy state machine (spec §Deploy State Machine), as a pure function. +//! +//! `classify` maps a verified observation plus the desired create intent to +//! one [`Action`]. It performs no I/O, so every row of the spec's table is a +//! unit test with no cluster. `reconcile` executes actions and re-enters. +//! +//! Two invariants are structural rather than remembered: +//! +//! * [`Action::Delete`] carries the [`Fence`] from the exact observation that +//! authorized it. There is no way to build a delete without one, so a later +//! helper cannot re-read and silently substitute a fresher fence. +//! * The pull-failure classifier ([`PullFailure`]) reaches only +//! [`Action::Report`] and [`Action::Observe`]. It is absent from +//! `Action::Delete`'s type, so "reason strings are never deletion +//! authority" is enforced by the compiler. + +use crate::intent::Fingerprint; + +/// The compare-and-delete fence: UID + resourceVersion from the observation +/// that authorized the deletion. A failed precondition means the object +/// changed since the read — re-enter, never retry the delete. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Fence { + pub uid: String, + pub resource_version: String, +} + +/// Why a pod that never started looks permanently broken. Reporting only. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum PullFailure { + /// Registry auth: a 403/401 `ImagePullBackOff` retries forever without + /// ever succeeding, so "the pull retries" is false for this case. + Unauthorized, + /// The digest or repository does not exist at that registry. + ManifestUnknown, + /// The image has no variant for the node's architecture. + ArchMismatch, +} + +/// The container's startup state, already decoded from pod status. Decoding +/// happens at the edge so this module stays free of API types. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Startup { + /// `state.running` — the harness process is up. This, not pod phase, is + /// what "live" means. + Started, + /// Started once and reached a terminal phase (Succeeded/Failed). + Terminated, + /// Never started, and self-healing is plausible: unschedulable during + /// scale-from-zero, an image pull in progress, a transient + /// `CreateContainerConfigError` whose Secret exists. + NeverStartedRecoverable, + /// Never started, and the provider *verified* the cause — not a reason + /// string. Either the referenced Secret is confirmed absent by a + /// most-recent read, or the image reference is structurally invalid. + NeverStartedProvablyBroken, + /// Never started; the pull is failing in a way that will not self-heal. + /// Still recoverable in the *never delete* sense — this only changes what + /// we report and how long we wait. + NeverStartedPullFailing(PullFailure), +} + +/// A pod that passed identity and ownership verification: label-selected, +/// full-pubkey annotation equal to the derived pubkey, management marker +/// present. Constructing this type is the verification step's output, so an +/// unverified object cannot reach `classify` at all. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct VerifiedPod { + pub name: String, + pub fence: Fence, + /// Set once the apiserver accepts a delete. In Kubernetes there is no + /// `Terminating` phase — a pod being gracefully deleted stays in phase + /// `Running` for its whole grace period — so this must be checked + /// *before* startup state or the dying pod reads as the no-op row. + pub deletion_marked: bool, + pub startup: Startup, + /// The `buzz.block.xyz/create-intent` annotation as recorded at create. + /// `None` for a pod written before the annotation existed, which counts + /// as divergence. + pub recorded_intent: Option, +} + +/// What the reconciler should do next. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Action { + /// Create the pod, then wait for the harness container to start. + Create, + /// Compare-and-delete, poll for actual disappearance, then re-enter. + Delete { name: String, fence: Fence }, + /// Wait for a deletion already in flight, then re-enter. + AwaitDisappearance { name: String }, + /// Strict no-op: return this `agent_id`, mutate nothing. + NoOp { agent_id: String }, + /// Keep observing until started or the operation deadline expires; on + /// expiry report the latest condition. Never deletes, on this call or any + /// later one. + Observe { name: String }, + /// Surface an actionable condition immediately rather than burning the + /// deadline on a failure that will not self-heal. + Report { name: String, failure: PullFailure }, +} + +/// Apply the spec's ordered rules to one verified observation. +/// +/// `desired` is the freshly computed create intent; comparison is always +/// recorded-annotation vs freshly-computed, never a diff against the live pod +/// (admission defaulting would make every pod look divergent). +pub fn classify(observed: Option<&VerifiedPod>, desired: &Fingerprint) -> Action { + let Some(pod) = observed else { + // Row: no instance → create. First deploy, or after GC. + return Action::Create; + }; + + // Row: deletion-marked, ANY phase. Checked before startup state because + // there is no `Terminating` phase to match on — a gracefully deleting pod + // reports phase `Running` throughout its grace period, so testing startup + // first would mistake it for the live no-op row and return an id that + // evaporates. + if pod.deletion_marked { + return Action::AwaitDisappearance { + name: pod.name.clone(), + }; + } + + match &pod.startup { + // Row: live and started → strict no-op. Start must never kill a live + // agent mid-turn, whatever the fingerprint says. + Startup::Started => Action::NoOp { + agent_id: pod.name.clone(), + }, + + // Row: terminated → delete residue, then re-enter to create. This is + // the normal restart path — how a user revives a reaped agent. + Startup::Terminated => Action::Delete { + name: pod.name.clone(), + fence: pod.fence.clone(), + }, + + // Row: never started, provably non-recoverable → fenced replace. + // "Provably" means a verified absence or a structural defect, never a + // reason string. + Startup::NeverStartedProvablyBroken => Action::Delete { + name: pod.name.clone(), + fence: pod.fence.clone(), + }, + + // Inside the recoverable row: a pull that will not self-heal is + // reported immediately instead of consuming the 600s deadline. This + // changes reporting and wait behavior only — no delete authority. + Startup::NeverStartedPullFailing(failure) => Action::Report { + name: pod.name.clone(), + failure: *failure, + }, + + // Row: never started, recoverable — split on create-intent + // divergence. Divergence is evidence of a config change the user is + // waiting on, and it is the *only* thing that replaces a + // never-started pod. Pod age triggers nothing: any finite age + // threshold collides with Cluster Autoscaler's own pod-age delays, + // and delete-recreate resets exactly the age it keys on. + Startup::NeverStartedRecoverable => { + if pod.recorded_intent.as_ref() == Some(desired) { + Action::Observe { + name: pod.name.clone(), + } + } else { + Action::Delete { + name: pod.name.clone(), + fence: pod.fence.clone(), + } + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn fp(seed: &str) -> Fingerprint { + Fingerprint::for_test(seed) + } + + fn pod(startup: Startup, intent: Option) -> VerifiedPod { + VerifiedPod { + name: "buzz-agent-abc123def456".into(), + fence: Fence { + uid: "uid-1".into(), + resource_version: "rv-1".into(), + }, + deletion_marked: false, + startup, + recorded_intent: intent, + } + } + + #[test] + fn no_instance_creates() { + assert_eq!(classify(None, &fp("a")), Action::Create); + } + + #[test] + fn started_pod_is_strict_no_op() { + let p = pod(Startup::Started, Some(fp("a"))); + assert_eq!( + classify(Some(&p), &fp("a")), + Action::NoOp { + agent_id: p.name.clone() + } + ); + } + + /// The asymmetry the spec states plainly: an edit cannot reach a started + /// pod until it exits, but it *can* reach a never-started one — the + /// never-started pod is the one the user is editing because it did not + /// start. + #[test] + fn started_pod_no_ops_even_when_intent_diverges() { + let p = pod(Startup::Started, Some(fp("old"))); + assert_eq!( + classify(Some(&p), &fp("new")), + Action::NoOp { + agent_id: p.name.clone() + } + ); + } + + /// In Kubernetes a gracefully deleting pod stays in phase `Running`. If + /// the deletion mark were checked after startup state, this pod would + /// take the no-op row and `deploy` would return an id that evaporates. + #[test] + fn deletion_mark_beats_every_startup_state() { + for startup in [ + Startup::Started, + Startup::Terminated, + Startup::NeverStartedRecoverable, + Startup::NeverStartedProvablyBroken, + Startup::NeverStartedPullFailing(PullFailure::Unauthorized), + ] { + let mut p = pod(startup.clone(), Some(fp("a"))); + p.deletion_marked = true; + assert_eq!( + classify(Some(&p), &fp("a")), + Action::AwaitDisappearance { + name: p.name.clone() + }, + "deletion mark ignored for {startup:?}" + ); + } + } + + #[test] + fn terminated_pod_is_replaced() { + let p = pod(Startup::Terminated, Some(fp("a"))); + assert_eq!( + classify(Some(&p), &fp("a")), + Action::Delete { + name: p.name.clone(), + fence: p.fence.clone() + } + ); + } + + /// A never-started winner is repairable: pod exists, Secret confirmed + /// absent, container never started. A later deploy must delete-recreate + /// rather than no-op — the test that pins started-not-phase as the no-op + /// criterion. + #[test] + fn provably_broken_never_started_pod_is_replaced() { + let p = pod(Startup::NeverStartedProvablyBroken, Some(fp("a"))); + assert_eq!( + classify(Some(&p), &fp("a")), + Action::Delete { + name: p.name.clone(), + fence: p.fence.clone() + } + ); + } + + /// The anti-livelock rule: identical desired intent means *never* delete, + /// however long the pod has been pending. Age is not an input to this + /// function at all, which is the strongest way to say so. + #[test] + fn recoverable_with_matching_intent_only_observes() { + let p = pod(Startup::NeverStartedRecoverable, Some(fp("same"))); + assert_eq!( + classify(Some(&p), &fp("same")), + Action::Observe { + name: p.name.clone() + } + ); + } + + /// Repeated identical Starts can never delete anything — the same + /// classification, arbitrarily many times. + #[test] + fn repeated_identical_starts_never_delete() { + let p = pod(Startup::NeverStartedRecoverable, Some(fp("same"))); + for _ in 0..100 { + assert!(!matches!( + classify(Some(&p), &fp("same")), + Action::Delete { .. } + )); + } + } + + /// The wedge escape: the user corrected a resource request or image, so + /// the never-started pod is built from configuration they have since + /// changed. Without this row the edit could never materialize. + #[test] + fn recoverable_with_divergent_intent_is_replaced() { + let p = pod(Startup::NeverStartedRecoverable, Some(fp("old"))); + assert_eq!( + classify(Some(&p), &fp("new")), + Action::Delete { + name: p.name.clone(), + fence: p.fence.clone() + } + ); + } + + /// A pod predating the annotation has no recorded intent — that is + /// absence, which the spec groups with divergence. + #[test] + fn missing_recorded_intent_counts_as_divergence() { + let p = pod(Startup::NeverStartedRecoverable, None); + assert_eq!( + classify(Some(&p), &fp("any")), + Action::Delete { + name: p.name.clone(), + fence: p.fence.clone() + } + ); + } + + /// Permanent-looking pull failures report immediately instead of burning + /// 600s — and, critically, never delete. + #[test] + fn pull_failures_report_and_never_delete() { + for failure in [ + PullFailure::Unauthorized, + PullFailure::ManifestUnknown, + PullFailure::ArchMismatch, + ] { + let p = pod(Startup::NeverStartedPullFailing(failure), Some(fp("a"))); + // Divergent intent too — still no delete from this arm. + for desired in [fp("a"), fp("different")] { + assert_eq!( + classify(Some(&p), &desired), + Action::Report { + name: p.name.clone(), + failure + } + ); + } + } + } + + /// Every delete carries the fence from the observation that authorized + /// it. Exhaustive over the delete-producing states, so a future arm that + /// forgets is caught here rather than in a cluster. + #[test] + fn every_delete_carries_the_authorizing_fence() { + let states = [ + (Startup::Terminated, fp("a")), + (Startup::NeverStartedProvablyBroken, fp("a")), + (Startup::NeverStartedRecoverable, fp("divergent")), + ]; + for (startup, desired) in states { + let p = pod(startup, Some(fp("a"))); + match classify(Some(&p), &desired) { + Action::Delete { fence, .. } => assert_eq!(fence, p.fence), + other => panic!("expected Delete, got {other:?}"), + } + } + } +} diff --git a/crates/buzz-backend-kubernetes/src/client.rs b/crates/buzz-backend-kubernetes/src/client.rs new file mode 100644 index 000000000..0c3bed65b --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/client.rs @@ -0,0 +1,182 @@ +//! Cluster auth and client construction (spec §Cluster auth, +//! `docs/remote-agents.md:985-995`). +//! +//! Standard kubeconfig resolution (`$KUBECONFIG` → `~/.kube/config`). +//! `provider_config` carries `context` and `namespace` only — credentials +//! never transit config (I2, `:196-198`). + +use kube::config::{ExecConfig, KubeConfigOptions, Kubeconfig}; +use kube::{Client, Config}; +use std::path::{Path, PathBuf}; + +/// Directories prepended to `PATH` before the client is built. +/// +/// Kubeconfigs at Block near-universally authenticate through `exec` +/// credential plugins (`aws eks get-token`, `gke-gcloud-auth-plugin`) that +/// resolve via `PATH` — and this provider inherits a Finder-launched +/// desktop's minimal `PATH`, which contains none of the places those plugins +/// install to (`:989-994`). +const PATH_PREPEND: [&str; 2] = ["/opt/homebrew/bin", "/usr/local/bin"]; + +/// Compute the new `PATH` value: plugin directories first, inherited entries +/// after, in order. Pure so the ordering can be tested without mutating the +/// process's environment. +fn prepended_path(home: Option<&Path>, existing: &std::ffi::OsStr) -> Option { + let mut dirs: Vec = PATH_PREPEND.iter().map(PathBuf::from).collect(); + if let Some(home) = home { + dirs.push(home.join(".local/bin")); + } + // An empty inherited PATH splits into one empty entry, which POSIX + // resolves as the current directory — a place a credential plugin should + // never be looked up. Drop empties rather than propagate them. + dirs.extend(std::env::split_paths(existing).filter(|p| !p.as_os_str().is_empty())); + std::env::join_paths(dirs).ok() +} + +/// Prepend the plugin directories to this process's `PATH`. +/// +/// Modifies the provider's own environment, which is sound here: one process +/// per operation, called before any client or task exists, and the child +/// processes that read it are exactly the credential plugins this exists for. +fn prepend_plugin_path() { + let home = std::env::var_os("HOME"); + let existing = std::env::var_os("PATH").unwrap_or_default(); + if let Some(joined) = prepended_path(home.as_ref().map(Path::new), &existing) { + std::env::set_var("PATH", joined); + } +} + +/// Is `command` runnable — an executable on `PATH`, or an existing path? +fn resolves_on_path(command: &str) -> bool { + if command.contains(std::path::MAIN_SEPARATOR) { + return Path::new(command).is_file(); + } + std::env::var_os("PATH") + .map(|path| std::env::split_paths(&path).any(|dir| dir.join(command).is_file())) + .unwrap_or(false) +} + +/// The exec plugin the selected context authenticates with, if any. +/// +/// Read from the kubeconfig directly rather than from `Config`, which does not +/// expose it. A read failure yields `None`: this lookup exists only to improve +/// an error message, and must never be the thing that fails a deploy. +fn exec_plugin_for(context: Option<&str>) -> Option { + let kubeconfig = Kubeconfig::read().ok()?; + let context_name = context + .map(str::to_string) + .or_else(|| kubeconfig.current_context.clone())?; + let user_name = kubeconfig + .contexts + .iter() + .find(|c| c.name == context_name) + .and_then(|c| c.context.as_ref()) + .and_then(|c| c.user.clone())?; + kubeconfig + .auth_infos + .iter() + .find(|a| a.name == user_name) + .and_then(|a| a.auth_info.as_ref()) + .and_then(|a| a.exec.clone()) +} + +/// Turn a client-construction failure into an error a user can act on. +/// +/// When the context authenticates through an exec plugin that is not on +/// `PATH`, that is almost always the cause, and the actionable fact is the +/// plugin's name — not a kube-rs error chain (`:994-995`). +fn explain(context: Option<&str>, error: &kube::Error) -> String { + if let Some(command) = exec_plugin_for(context).and_then(|e| e.command) { + if !resolves_on_path(&command) { + return format!( + "kubeconfig context {} authenticates with the credential plugin \ + {command:?}, which is not on PATH. Install it or add its \ + directory to PATH, then try again.", + context.unwrap_or("(current)") + ); + } + } + format!( + "could not connect to the cluster using kubeconfig context {}: {error}", + context.unwrap_or("(current)") + ) +} + +/// Build a client for the selected context. +pub async fn connect(context: Option<&str>) -> Result { + prepend_plugin_path(); + + let options = KubeConfigOptions { + context: context.map(str::to_string), + ..Default::default() + }; + let config = Config::from_kubeconfig(&options).await.map_err(|e| { + // A named context that does not exist is a user typo, and the + // kube-rs message for it is already specific. + format!( + "could not load kubeconfig for context {}: {e}", + context.unwrap_or("(current)") + ) + })?; + + Client::try_from(config).map_err(|e| explain(context, &e)) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The three plugin directories must end up ahead of the inherited PATH, + /// or a Finder-launched desktop never finds `aws`/`gke-gcloud-auth-plugin`. + /// Tested on the pure computation: mutating the process PATH here would + /// race every other test in the binary. + #[test] + fn plugin_directories_are_prepended_in_order() { + let joined = prepended_path( + Some(Path::new("/tmp/fake-home")), + std::ffi::OsStr::new("/inherited/bin:/usr/bin"), + ) + .unwrap(); + let dirs: Vec = std::env::split_paths(&joined).collect(); + assert_eq!( + dirs, + [ + "/opt/homebrew/bin", + "/usr/local/bin", + "/tmp/fake-home/.local/bin", + "/inherited/bin", + "/usr/bin", + ] + .map(PathBuf::from) + ); + } + + /// No `HOME` is not a failure — the two absolute directories still apply. + #[test] + fn missing_home_still_prepends_the_absolute_directories() { + let joined = prepended_path(None, std::ffi::OsStr::new("/inherited/bin")).unwrap(); + let dirs: Vec = std::env::split_paths(&joined).collect(); + assert_eq!( + dirs, + ["/opt/homebrew/bin", "/usr/local/bin", "/inherited/bin"].map(PathBuf::from) + ); + } + + /// An empty inherited PATH must not produce an empty entry, which the + /// shell and `resolves_on_path` would both read as the cwd. + #[test] + fn empty_inherited_path_yields_no_empty_entry() { + let joined = prepended_path(None, std::ffi::OsStr::new("")).unwrap(); + let dirs: Vec = std::env::split_paths(&joined).collect(); + assert_eq!( + dirs, + ["/opt/homebrew/bin", "/usr/local/bin"].map(PathBuf::from) + ); + } + + #[test] + fn resolves_absolute_paths_directly() { + assert!(resolves_on_path("/bin/sh")); + assert!(!resolves_on_path("/nonexistent/plugin-binary")); + } +} diff --git a/crates/buzz-backend-kubernetes/src/cluster.rs b/crates/buzz-backend-kubernetes/src/cluster.rs new file mode 100644 index 000000000..755f41f6c --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/cluster.rs @@ -0,0 +1,427 @@ +//! The real [`Substrate`]: kube-rs against a live apiserver. +//! +//! Everything that *decides* lives in `classify`/`gc`; this module only +//! performs I/O and maps apiserver responses onto the trait's vocabulary. +//! Three mappings here are normative rather than incidental: +//! +//! * **409 is discriminated on `Status.reason`, never on the code.** A create +//! 409 is `AlreadyExists`; a delete 409 from a failed precondition is +//! `Conflict`. Branching on `code == 409` conflates a lost create race with +//! a stale fence and is the trap the spec names (`:780-794`). +//! * **Reads leave `resourceVersion` unset**, which is the quorum read. `"0"` +//! is the cache read, and a confirmed absence from a cache is proof of +//! nothing (`:761-769`). +//! * **Deletes never set `grace_period_seconds`**, so the object's own 60s +//! budget applies. Passing `0` is a force-kill that discards the shutdown +//! window the pod declares (`:1185-1189`). + +use crate::classify::Fence; +use crate::reconcile::{CreateOutcome, DeleteOutcome, Substrate}; +use chrono::{DateTime, Utc}; +use k8s_openapi::api::core::v1::{Namespace, Pod, Secret}; +use kube::api::{Api, DeleteParams, GetParams, ListParams, PostParams, Preconditions}; +use kube::core::ErrorResponse; +use kube::{Client, Resource}; +use std::time::{Duration, Instant}; + +/// `Status.reason` values we branch on. Spelled once so the two 409 arms are +/// visibly the same discriminator read two ways. +/// +/// These are wire strings `apimachinery` chooses, not names this crate picks: +/// `StatusReasonAlreadyExists`, `StatusReasonConflict`, `StatusReasonNotFound`, +/// and `StatusReasonForbidden` in `k8s.io/apimachinery/pkg/apis/meta/v1/types.go`. +/// kube-core types `ErrorResponse::reason` as a bare `String`, so there is no +/// upstream constant to bind to and the spelling is pinned by test instead. +const REASON_ALREADY_EXISTS: &str = "AlreadyExists"; +const REASON_CONFLICT: &str = "Conflict"; +const REASON_NOT_FOUND: &str = "NotFound"; +const REASON_FORBIDDEN: &str = "Forbidden"; + +/// The apiserver-backed substrate for one deploy operation. +pub struct Cluster { + client: Client, + namespace: String, + /// Start of *this operation*, for the deadline. Monotonic: the 600s budget + /// must not move when the wall clock does. + started: Instant, +} + +/// The typed API error underneath a `kube::Error`, if it is one. +fn api_error(error: &kube::Error) -> Option<&ErrorResponse> { + match error { + kube::Error::Api(response) => Some(response), + _ => None, + } +} + +/// Does this error carry the given `Status.reason`? +fn reason_is(error: &kube::Error, reason: &str) -> bool { + api_error(error).is_some_and(|e| e.reason == reason) +} + +impl Cluster { + pub fn new(client: Client, namespace: &str) -> Self { + Self { + client, + namespace: namespace.to_string(), + started: Instant::now(), + } + } + + fn pods(&self) -> Api { + Api::namespaced(self.client.clone(), &self.namespace) + } + + fn secrets(&self) -> Api { + Api::namespaced(self.client.clone(), &self.namespace) + } + + /// List an object kind through the raw client so the response's HTTP + /// `Date` header is reachable. + /// + /// `Api::list` returns only the decoded body, and the apiserver's clock is + /// the *only* clock the orphan-Secret age gate may use — a desktop's local + /// clock running fast computes every in-flight Secret as expired + /// (`:1321-1335`). So the list goes through `Client::send`, which hands + /// back the whole `http::Response`. + async fn list_with_date( + &self, + selector: &str, + ) -> Result<(Vec, Option>), String> + where + K: Resource + + Clone + + serde::de::DeserializeOwned + + std::fmt::Debug, + K::DynamicType: Default, + { + let dt = K::DynamicType::default(); + let url = K::url_path(&dt, Some(&self.namespace)); + // resourceVersion deliberately unset: quorum read. + let params = ListParams { + label_selector: Some(selector.to_string()), + ..Default::default() + }; + let request = kube::core::Request::new(url) + .list(¶ms) + .map_err(|e| format!("could not build a list request: {e}"))?; + let (parts, body) = request.into_parts(); + let response = self + .client + .send(http::Request::from_parts(parts, body.into())) + .await + .map_err(|e| format!("could not list {}: {e}", K::plural(&dt)))?; + + // Parsed before the body is consumed, and independently of it: a + // missing or malformed header is not a list failure, it just means the + // orphan sweep has no clock and skips. + let server_now = response + .headers() + .get(http::header::DATE) + .and_then(|v| v.to_str().ok()) + .and_then(|v| DateTime::parse_from_rfc2822(v).ok()) + .map(|v| v.with_timezone(&Utc)); + + let bytes = http_body_util::BodyExt::collect(response.into_body()) + .await + .map_err(|e| format!("could not read the {} list body: {e}", K::plural(&dt)))? + .to_bytes(); + let list: kube::core::ObjectList = serde_json::from_slice(&bytes) + .map_err(|e| format!("could not decode the {} list: {e}", K::plural(&dt)))?; + + Ok((list.items, server_now)) + } +} + +impl Substrate for Cluster { + async fn ensure_namespace(&self, namespace: &str) -> Result<(), String> { + let api: Api = Api::all(self.client.clone()); + if api + .get_opt(namespace) + .await + .map_err(|e| format!("could not check whether namespace {namespace} exists: {e}"))? + .is_some() + { + return Ok(()); + } + + let spec = Namespace { + metadata: kube::core::ObjectMeta { + name: Some(namespace.to_string()), + ..Default::default() + }, + ..Default::default() + }; + match api.create(&PostParams::default(), &spec).await { + Ok(_) => Ok(()), + // Someone else created it between our check and our create. That + // is the desired end state, not a failure. + Err(e) if reason_is(&e, REASON_ALREADY_EXISTS) => Ok(()), + // Namespace-create is frequently denied on shared clusters. Name + // the exact command an operator runs, and never silently fall back + // to `default` — deploying an agent into someone else's namespace + // is worse than refusing (`:1002-1005`). + Err(e) if reason_is(&e, REASON_FORBIDDEN) => Err(format!( + "not authorized to create namespace {namespace}. Ask a cluster \ + administrator to run `kubectl create namespace {namespace}`, \ + then try again." + )), + Err(e) => Err(format!("could not create namespace {namespace}: {e}")), + } + } + + async fn list_pods(&self, selector: &str) -> Result<(Vec, Option>), String> { + self.list_with_date::(selector).await + } + + async fn list_secrets(&self, selector: &str) -> Result, String> { + Ok(self.list_with_date::(selector).await?.0) + } + + async fn secret_exists(&self, name: &str) -> Result { + // `GetParams::default()` leaves resourceVersion unset — the quorum + // read this check requires to be proof of anything. + match self.secrets().get_with(name, &GetParams::default()).await { + Ok(_) => Ok(true), + Err(e) if reason_is(&e, REASON_NOT_FOUND) => Ok(false), + Err(e) => Err(format!("could not check whether secret {name} exists: {e}")), + } + } + + async fn create_secret(&self, secret: &Secret) -> Result<(), String> { + let name = secret.metadata.name.clone().unwrap_or_default(); + self.secrets() + .create(&PostParams::default(), secret) + .await + .map(|_| ()) + .map_err(|e| format!("could not create secret {name}: {e}")) + } + + async fn create_pod(&self, pod: &Pod) -> Result { + let name = pod.metadata.name.clone().unwrap_or_default(); + match self.pods().create(&PostParams::default(), pod).await { + Ok(_) => Ok(CreateOutcome::Created), + // The deterministic name is taken: a concurrent attempt won the + // election. Discriminated on the reason — a 409 whose reason is + // `Conflict` is a different condition and must not be read as a + // lost race. + Err(e) if reason_is(&e, REASON_ALREADY_EXISTS) => Ok(CreateOutcome::AlreadyExists), + Err(e) => Err(format!("could not create pod {name}: {e}")), + } + } + + async fn delete_pod(&self, name: &str, fence: &Fence) -> Result { + let params = DeleteParams { + preconditions: Some(Preconditions { + uid: Some(fence.uid.clone()), + resource_version: Some(fence.resource_version.clone()), + }), + // grace_period_seconds deliberately unset: the pod's own 60s + // budget applies. + ..Default::default() + }; + match self.pods().delete(name, ¶ms).await { + Ok(_) => Ok(DeleteOutcome::Accepted), + Err(e) if reason_is(&e, REASON_NOT_FOUND) => Ok(DeleteOutcome::NotFound), + // The object changed since the observation that authorized this + // delete. Same HTTP code as the create race above, different + // reason, different meaning. + Err(e) if reason_is(&e, REASON_CONFLICT) => Ok(DeleteOutcome::PreconditionFailed), + Err(e) => Err(format!("could not delete pod {name}: {e}")), + } + } + + async fn delete_secret(&self, name: &str) -> Result<(), String> { + match self.secrets().delete(name, &DeleteParams::default()).await { + Ok(_) => Ok(()), + // Already gone is the desired end state. + Err(e) if reason_is(&e, REASON_NOT_FOUND) => Ok(()), + Err(e) => Err(format!("could not delete secret {name}: {e}")), + } + } + + async fn get_pod(&self, name: &str) -> Result, String> { + match self.pods().get_with(name, &GetParams::default()).await { + Ok(pod) => Ok(Some(pod)), + Err(e) if reason_is(&e, REASON_NOT_FOUND) => Ok(None), + Err(e) => Err(format!("could not read pod {name}: {e}")), + } + } + + async fn sleep(&self, duration: Duration) { + tokio::time::sleep(duration).await; + } + + fn elapsed(&self) -> Duration { + self.started.elapsed() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use http::Response; + use kube::client::Body; + use std::sync::{Arc, Mutex}; + use tower::service_fn; + + fn list_response(date: Option<&str>) -> Response { + let mut response = Response::builder().status(200); + if let Some(date) = date { + response = response.header(http::header::DATE, date); + } + response + .body(Body::from( + serde_json::to_vec(&serde_json::json!({ + "apiVersion": "v1", + "kind": "PodList", + "metadata": {"resourceVersion": "17"}, + "items": [{ + "apiVersion": "v1", + "kind": "Pod", + "metadata": {"name": "sprig"}, + "spec": {"containers": [{"name": "agent", "image": "example.invalid/sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}]} + }] + })) + .unwrap(), + )) + .unwrap() + } + + async fn list_through_real_request_path( + date: Option<&'static str>, + ) -> (Vec, Option>, String) { + let observed_uri = Arc::new(Mutex::new(None)); + let service_uri = Arc::clone(&observed_uri); + let service = service_fn(move |request: http::Request| { + let service_uri = Arc::clone(&service_uri); + async move { + *service_uri.lock().unwrap() = Some(request.uri().to_string()); + Ok::<_, std::convert::Infallible>(list_response(date)) + } + }); + let cluster = Cluster::new(Client::new(service, "ignored"), "owned-ns"); + let result = cluster + .list_with_date::("app.kubernetes.io/managed-by=buzz-backend-kubernetes") + .await + .unwrap(); + let uri = observed_uri.lock().unwrap().take().unwrap(); + (result.0, result.1, uri) + } + + /// Exercise the shipped `Request` + `Client::send` seam. A fake + /// reconciler would not prove that kube-rs emits a quorum list request or + /// that the apiserver's clock survives body decoding. + #[tokio::test] + async fn list_with_date_uses_a_quorum_request_and_returns_the_server_clock() { + let (pods, server_now, uri) = + list_through_real_request_path(Some("Sun, 02 Aug 2026 04:00:00 GMT")).await; + + assert_eq!(pods.len(), 1, "fixture must contain one decoded pod"); + assert_eq!(pods[0].metadata.name.as_deref(), Some("sprig")); + assert!(uri.starts_with("/api/v1/namespaces/owned-ns/pods?")); + assert!( + uri.contains("labelSelector=app.kubernetes.io%2Fmanaged-by%3Dbuzz-backend-kubernetes") + ); + assert!( + !uri.contains("resourceVersion"), + "cache read leaked into {uri}" + ); + assert_eq!( + server_now.unwrap().to_rfc3339(), + "2026-08-02T04:00:00+00:00" + ); + } + + /// Header failure is deliberately not list failure: without a trustworthy + /// apiserver clock the orphan sweep skips, but normal reconciliation still + /// receives the decoded objects. + #[tokio::test] + async fn list_with_date_keeps_items_when_the_server_clock_is_unusable() { + for date in [Some("not a date"), None] { + let (pods, server_now, _) = list_through_real_request_path(date).await; + assert_eq!(pods.len(), 1, "fixture must contain one decoded pod"); + assert!(server_now.is_none(), "unexpected clock for {date:?}"); + } + } + + /// A typed apiserver error, as kube-rs surfaces it. + fn api(reason: &str, code: u16) -> kube::Error { + kube::Error::Api(ErrorResponse { + status: "Failure".into(), + message: String::new(), + reason: reason.into(), + code, + }) + } + + /// The one discriminator the whole file rests on, and the trap the spec + /// predicts: "an implementation that branches on the code alone will + /// eventually take the adoption path on a failed delete or vice versa" + /// (`:788-790`). + /// + /// Both of these are 409. Reading the *code* makes them identical; reading + /// `Status.reason` keeps a lost create race and a stale fence apart. The + /// mutation that must fail this test is `e.reason == …` → `e.code == 409`, + /// which no other test in the crate would catch — the fakes never produce + /// a real `kube::Error`. + #[test] + fn the_two_409s_are_never_conflated() { + let already_exists = api(REASON_ALREADY_EXISTS, 409); + let conflict = api(REASON_CONFLICT, 409); + + assert!(reason_is(&already_exists, REASON_ALREADY_EXISTS)); + assert!(reason_is(&conflict, REASON_CONFLICT)); + // The cross terms are the whole point. + assert!(!reason_is(&already_exists, REASON_CONFLICT)); + assert!(!reason_is(&conflict, REASON_ALREADY_EXISTS)); + } + + /// A transport-level failure is not an apiserver verdict. It must fall + /// through to the error arm rather than being read as any reason — a + /// connection reset silently classified as `NotFound` would report a pod + /// as confirmed-absent, which the classifier treats as proof. + #[test] + fn a_non_api_error_carries_no_reason() { + let transport = kube::Error::LinesCodecMaxLineLengthExceeded; + assert!(api_error(&transport).is_none()); + for reason in [ + REASON_ALREADY_EXISTS, + REASON_CONFLICT, + REASON_NOT_FOUND, + REASON_FORBIDDEN, + ] { + assert!(!reason_is(&transport, reason), "matched {reason}"); + } + } + + /// `reason` is `#[serde(default)]` in kube-core, so an apiserver that + /// omits it yields an empty string. That must match nothing rather than + /// matching an empty pattern by accident. + #[test] + fn an_absent_reason_matches_nothing() { + let bare = api("", 409); + assert!(!reason_is(&bare, REASON_ALREADY_EXISTS)); + assert!(!reason_is(&bare, REASON_CONFLICT)); + } + + /// The consts are the apiserver's spelling, asserted against literals + /// rather than against themselves. + /// + /// Every other test here references the consts symbolically on both sides + /// — fixture *and* assertion — which is true for any pair of distinct + /// values. That tests the discriminator is self-consistent, not that it is + /// correct: swapping the two 409 values inverts `AlreadyExists` and + /// `Conflict` at a real apiserver (`:788-790`'s failure, reached by + /// editing a string instead of a branch) with every other test still + /// green. These are `apimachinery`'s wire strings and kube-core exposes no + /// constant for them, so a literal is the only external anchor available. + /// Found by Quinn's mutation matrix; M2/M3/M4 survived without it. + #[test] + fn the_reason_consts_are_the_apiservers_spelling() { + assert_eq!(REASON_ALREADY_EXISTS, "AlreadyExists"); + assert_eq!(REASON_CONFLICT, "Conflict"); + assert_eq!(REASON_NOT_FOUND, "NotFound"); + assert_eq!(REASON_FORBIDDEN, "Forbidden"); + } +} diff --git a/crates/buzz-backend-kubernetes/src/config.rs b/crates/buzz-backend-kubernetes/src/config.rs new file mode 100644 index 000000000..39b68ce0f --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/config.rs @@ -0,0 +1,443 @@ +//! `provider_config` parsing and the `info` config schema +//! (spec §`provider_config` v1 fields, `docs/remote-agents.md:1384-1389`). +//! +//! Nine fields, all optional except `image` (v1 ships no baked default — +//! §Image). No credential field exists, by I2: cluster auth comes from ambient +//! kubeconfig resolution and nothing else (`:196-198`). + +use crate::image::{self, ImageRef}; + +/// Resource requests and limits (§Pod shape: 1cpu/2Gi → 2cpu/4Gi, all four +/// configurable — `cargo build` in an agent workspace makes 500m/1Gi +/// unrealistic). +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Resources { + pub cpu_request: String, + pub memory_request: String, + pub cpu_limit: String, + pub memory_limit: String, +} + +impl Default for Resources { + fn default() -> Self { + Self { + cpu_request: "1".into(), + memory_request: "2Gi".into(), + cpu_limit: "2".into(), + memory_limit: "4Gi".into(), + } + } +} + +/// Default inactivity budget: the I5 opt-in (§Auto-Stop). The config field and +/// `BUZZ_ACP_EXIT_AFTER_INACTIVITY` are one knob, not two. +pub const DEFAULT_INACTIVITY_SECONDS: u64 = 7200; + +/// Fixed nonzero UID/GID for the agent container (§Pod shape hardening). +pub const RUN_AS_UID: i64 = 10001; +pub const RUN_AS_GID: i64 = 10001; + +/// Writable workspace root; also `HOME` and the harness's cwd +/// (§Working directory). +pub const WORKSPACE_PATH: &str = "/home/agent"; + +/// `terminationGracePeriodSeconds` — a declared budget, not a derived sum +/// (§Pod shape). Kubernetes' default 30s would SIGKILL the harness mid-drain. +pub const TERMINATION_GRACE_SECONDS: i64 = 60; + +/// The only restart policy v1 ships. `OnFailure` is double-gated on the +/// harness exit-code contract *and* a crash-loop classification row the state +/// machine does not have (`:1121-1139`); until both land the provider refuses +/// the combination rather than shipping against an undefended convention. +pub const RESTART_POLICY: &str = "Never"; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ProviderConfig { + /// kubeconfig context; `None` uses the current context. + pub context: Option, + pub namespace: String, + pub image: ImageRef, + pub resources: Resources, + /// `None` when `inactivity_seconds` was 0 — refused in v1, see [`parse`]. + pub inactivity_seconds: Option, + pub service_account: Option, +} + +/// Read an optional non-empty string field. Rejects non-string scalars rather +/// than stringifying them, so a mistyped field is named at the boundary. +fn optional_string(cfg: &serde_json::Value, field: &str) -> Result, String> { + match cfg.get(field) { + None | Some(serde_json::Value::Null) => Ok(None), + Some(serde_json::Value::String(s)) if s.trim().is_empty() => Ok(None), + Some(serde_json::Value::String(s)) => Ok(Some(s.trim().to_string())), + Some(other) => Err(format!( + "provider_config.{field} must be a string, got {other}" + )), + } +} + +/// Read an optional unsigned integer. The desktop's form omits blank numeric +/// fields rather than sending `""`, but a hand-crafted payload may send a +/// numeric string — accept both, refuse anything else. +fn optional_u64(cfg: &serde_json::Value, field: &str) -> Result, String> { + match cfg.get(field) { + None | Some(serde_json::Value::Null) => Ok(None), + Some(serde_json::Value::Number(n)) => n.as_u64().map(Some).ok_or_else(|| { + format!("provider_config.{field} must be a non-negative integer, got {n}") + }), + Some(serde_json::Value::String(s)) if s.trim().is_empty() => Ok(None), + Some(serde_json::Value::String(s)) => s.trim().parse::().map(Some).map_err(|_| { + format!("provider_config.{field} must be a non-negative integer, got {s:?}") + }), + Some(other) => Err(format!( + "provider_config.{field} must be a non-negative integer, got {other}" + )), + } +} + +/// A Kubernetes namespace name: RFC 1123 label, ≤63 chars. Validated here so a +/// typo fails with a named field instead of an apiserver rejection partway +/// through a deploy. +fn valid_namespace(name: &str) -> bool { + !name.is_empty() + && name.len() <= 63 + && name.starts_with(|c: char| c.is_ascii_lowercase() || c.is_ascii_digit()) + && name.ends_with(|c: char| c.is_ascii_lowercase() || c.is_ascii_digit()) + && name + .chars() + .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-') +} + +pub fn parse(cfg: &serde_json::Value) -> Result { + if !cfg.is_object() && !cfg.is_null() { + return Err("provider_config must be a JSON object".to_string()); + } + + let namespace = optional_string(cfg, "namespace")?.ok_or_else(|| { + "provider_config.namespace is required: the info schema supplies a \ + generated default, so an empty value means the form was cleared" + .to_string() + })?; + if !valid_namespace(&namespace) { + return Err(format!( + "provider_config.namespace {namespace:?} is not a valid Kubernetes \ + namespace (lowercase alphanumerics and '-', ≤63 characters)" + )); + } + + let image = image::parse(optional_string(cfg, "image")?.unwrap_or_default().as_str())?; + + let defaults = Resources::default(); + let resources = Resources { + cpu_request: optional_string(cfg, "cpu_request")?.unwrap_or(defaults.cpu_request), + memory_request: optional_string(cfg, "memory_request")?.unwrap_or(defaults.memory_request), + cpu_limit: optional_string(cfg, "cpu_limit")?.unwrap_or(defaults.cpu_limit), + memory_limit: optional_string(cfg, "memory_limit")?.unwrap_or(defaults.memory_limit), + }; + + // `inactivity_seconds: 0` is a legal, blessed value in the spec (§Auto-Stop) + // meaning "no auto-stop" — but it selects `restartPolicy: OnFailure`, which + // §Pod shape forbids until the harness exit-code contract is pinned AND the + // state machine gains a crash-loop row. Refusing the *combination* is what + // the spec asks for; silently downgrading to `Never` would ship an + // indefinite agent that dies on its first crash. + let inactivity_seconds = match optional_u64(cfg, "inactivity_seconds")? { + None => Some(DEFAULT_INACTIVITY_SECONDS), + Some(0) => { + return Err( + "provider_config.inactivity_seconds: 0 (indefinite lifetime) is not \ + supported in this version: it requires restartPolicy OnFailure, \ + which is gated on the harness exit-code contract. Set a positive \ + number of seconds." + .to_string(), + ) + } + Some(n) => Some(n), + }; + + Ok(ProviderConfig { + context: optional_string(cfg, "context")?, + namespace, + image, + resources, + inactivity_seconds, + service_account: optional_string(cfg, "service_account")?, + }) +} + +/// A fresh `buzz-agents-` namespace default. +/// +/// Computed per `info` call, which is how "random default" is satisfied with +/// zero UI changes: the schema's `default` prefills the form (§K8s Namespace). +pub fn generated_namespace() -> String { + use rand::RngExt; + const ALPHABET: &[u8] = b"abcdefghijklmnopqrstuvwxyz0123456789"; + let mut rng = rand::rng(); + let suffix: String = (0..6) + .map(|_| ALPHABET[rng.random_range(0..ALPHABET.len())] as char) + .collect(); + format!("buzz-agents-{suffix}") +} + +/// The `config_schema` returned by `info`. Drives the UI form: +/// `properties[*].default` prefill, scalar coercion, `required` gating +/// (`:407-411`). +pub fn config_schema() -> serde_json::Value { + let defaults = Resources::default(); + serde_json::json!({ + "type": "object", + "properties": { + "context": { + "type": "string", + "title": "Kubeconfig context", + "description": "Context from your kubeconfig. Leave empty to use the current context." + }, + "namespace": { + "type": "string", + "title": "Namespace", + "description": "Created if it does not exist.", + "default": generated_namespace() + }, + "image": { + "type": "string", + "title": "Agent image", + "description": "Digest-pinned image containing the buzz-acp runtime ABI, e.g. ghcr.io/block/buzz-sprig@sha256:. Tags are not accepted: this pod holds the agent's private key." + }, + "cpu_request": { + "type": "string", "title": "CPU request", "default": defaults.cpu_request + }, + "memory_request": { + "type": "string", "title": "Memory request", "default": defaults.memory_request + }, + "cpu_limit": { + "type": "string", "title": "CPU limit", "default": defaults.cpu_limit + }, + "memory_limit": { + "type": "string", "title": "Memory limit", "default": defaults.memory_limit + }, + "inactivity_seconds": { + "type": "number", + "title": "Stop after inactivity (seconds)", + "description": "The agent exits after this long with no work, and can be started again at any time.", + "default": DEFAULT_INACTIVITY_SECONDS + }, + "service_account": { + "type": "string", + "title": "Service account", + "description": "Scheduling/RBAC identity only. No API token is mounted." + } + }, + "required": ["namespace", "image"] + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn digest_ref() -> String { + format!("ghcr.io/block/buzz-sprig@sha256:{}", "a".repeat(64)) + } + + fn minimal() -> serde_json::Value { + serde_json::json!({"namespace": "buzz-agents-abc123", "image": digest_ref()}) + } + + #[test] + fn applies_spec_defaults() { + let c = parse(&minimal()).unwrap(); + assert_eq!(c.resources, Resources::default()); + assert_eq!(c.resources.cpu_request, "1"); + assert_eq!(c.resources.memory_request, "2Gi"); + assert_eq!(c.resources.cpu_limit, "2"); + assert_eq!(c.resources.memory_limit, "4Gi"); + assert_eq!(c.inactivity_seconds, Some(DEFAULT_INACTIVITY_SECONDS)); + assert_eq!(c.context, None); + assert_eq!(c.service_account, None); + } + + #[test] + fn all_four_resources_are_configurable() { + let mut cfg = minimal(); + cfg["cpu_request"] = "500m".into(); + cfg["memory_request"] = "1Gi".into(); + cfg["cpu_limit"] = "4".into(); + cfg["memory_limit"] = "8Gi".into(); + let c = parse(&cfg).unwrap(); + assert_eq!( + c.resources, + Resources { + cpu_request: "500m".into(), + memory_request: "1Gi".into(), + cpu_limit: "4".into(), + memory_limit: "8Gi".into(), + } + ); + } + + /// The desktop's form omits blank numeric fields; a hand-crafted payload + /// may send a numeric string. Both must mean the same thing. + #[test] + fn inactivity_accepts_number_string_and_omission() { + let mut cfg = minimal(); + cfg["inactivity_seconds"] = serde_json::json!(300); + assert_eq!(parse(&cfg).unwrap().inactivity_seconds, Some(300)); + + cfg["inactivity_seconds"] = serde_json::json!("300"); + assert_eq!(parse(&cfg).unwrap().inactivity_seconds, Some(300)); + + cfg["inactivity_seconds"] = serde_json::json!(""); + assert_eq!( + parse(&cfg).unwrap().inactivity_seconds, + Some(DEFAULT_INACTIVITY_SECONDS) + ); + } + + /// Indefinite lifetime selects `OnFailure`, which is gated. Refuse rather + /// than silently downgrade — a downgraded agent dies on its first crash + /// while the user believes they asked for indefinite. + #[test] + fn refuses_indefinite_lifetime() { + let mut cfg = minimal(); + cfg["inactivity_seconds"] = serde_json::json!(0); + let err = parse(&cfg).unwrap_err(); + assert!(err.contains("inactivity_seconds"), "got: {err}"); + assert!( + err.contains("OnFailure"), + "error should name the gate: {err}" + ); + } + + #[test] + fn rejects_negative_and_non_numeric_inactivity() { + for bad in [ + serde_json::json!(-1), + serde_json::json!(1.5), + serde_json::json!("soon"), + serde_json::json!(true), + ] { + let mut cfg = minimal(); + cfg["inactivity_seconds"] = bad.clone(); + assert!(parse(&cfg).is_err(), "accepted {bad}"); + } + } + + #[test] + fn image_is_required_and_must_be_digest_pinned() { + let mut cfg = minimal(); + cfg.as_object_mut().unwrap().remove("image"); + assert!(parse(&cfg).unwrap_err().contains("provider_config.image")); + + cfg["image"] = "ghcr.io/block/buzz-sprig:latest".into(); + assert!(parse(&cfg).unwrap_err().contains("digest-pinned")); + } + + #[test] + fn rejects_invalid_namespace_names() { + for bad in [ + "", + "Buzz-Agents", + "-leading", + "trailing-", + "has_underscore", + &"n".repeat(64), + ] { + let mut cfg = minimal(); + cfg["namespace"] = bad.into(); + assert!(parse(&cfg).is_err(), "accepted namespace {bad:?}"); + } + } + + /// I2 corollary: there is no config path for cluster credentials, so a + /// caller that tries to supply one gets no effect from it. Asserting the + /// parsed struct has no such field is the closest a test can get to + /// "the type makes it impossible". + #[test] + fn credential_fields_have_no_effect() { + let mut cfg = minimal(); + cfg["token"] = "hunter2".into(); + cfg["client_key"] = "hunter2".into(); + let c = parse(&cfg).unwrap(); + let rendered = format!("{c:?}"); + assert!( + !rendered.contains("hunter2"), + "config absorbed a credential: {rendered}" + ); + } + + #[test] + fn mistyped_string_fields_are_named() { + let mut cfg = minimal(); + cfg["namespace"] = serde_json::json!(42); + assert!(parse(&cfg) + .unwrap_err() + .contains("provider_config.namespace")); + } + + #[test] + fn generated_namespaces_are_fresh_and_valid() { + let a = generated_namespace(); + let b = generated_namespace(); + assert_ne!(a, b, "namespace default is not random"); + assert!(valid_namespace(&a), "{a} is not a valid namespace"); + assert!(a.starts_with("buzz-agents-")); + assert_eq!(a.len(), "buzz-agents-".len() + 6); + } + + /// The schema's own namespace default must be a value the parser accepts — + /// otherwise the UI prefills a form that fails on submit. + #[test] + fn schema_default_namespace_round_trips_through_parse() { + let schema = config_schema(); + let default = schema["properties"]["namespace"]["default"] + .as_str() + .unwrap(); + let cfg = serde_json::json!({"namespace": default, "image": digest_ref()}); + assert_eq!(parse(&cfg).unwrap().namespace, default); + } + + /// Nine fields exactly (§`provider_config` v1 fields). The cap is 20; the + /// count is pinned so a field added without a spec change is caught here. + #[test] + fn schema_declares_exactly_the_nine_v1_fields() { + let schema = config_schema(); + let props = schema["properties"].as_object().unwrap(); + let mut keys: Vec<&str> = props.keys().map(String::as_str).collect(); + keys.sort(); + assert_eq!( + keys, + [ + "context", + "cpu_limit", + "cpu_request", + "image", + "inactivity_seconds", + "memory_limit", + "memory_request", + "namespace", + "service_account" + ] + ); + assert_eq!( + schema["required"], + serde_json::json!(["namespace", "image"]) + ); + } + + /// I2's key lint rejects any field whose word-split contains + /// secret|password|token|key|credential. A schema field tripping it would + /// make every deploy fail validation desktop-side (`:185-198`). + #[test] + fn no_schema_field_trips_the_i2_key_lint() { + const BANNED: [&str; 5] = ["secret", "password", "token", "key", "credential"]; + let schema = config_schema(); + for field in schema["properties"].as_object().unwrap().keys() { + for word in field.split(['_', '-']) { + assert!( + !BANNED.contains(&word), + "field {field:?} contains I2-banned word {word:?}" + ); + } + } + } +} diff --git a/crates/buzz-backend-kubernetes/src/env.rs b/crates/buzz-backend-kubernetes/src/env.rs new file mode 100644 index 000000000..badff621e --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/env.rs @@ -0,0 +1,732 @@ +//! Building the pod environment (spec §Launch data, §Entrypoint mapping table). +//! +//! The three tiers are resolved *here*, before serialization, because a +//! Kubernetes Secret's `data` is a flat map with no precedence of its own: if +//! two tiers supplied the same key, whichever entry landed in the map would +//! win silently. Resolving in-provider makes later-wins explicit and testable. + +use crate::wire::{AgentPayload, LaunchBlock}; +use std::collections::BTreeMap; + +/// Keys the authoritative tier owns. +/// +/// Load-bearing, not documentation: tier 3 *clears* every key on this list +/// before writing its own values, so a key the authoritative tier has no value +/// for is **removed** rather than left holding a lower-tier value. Plain +/// overwrite is not enough — most of these are written conditionally +/// (`BUZZ_ACP_AGENT_ARGS` only when `launch.args` is non-empty, +/// `BUZZ_ACP_RESPOND_TO` only when set), and without the clear, a lower tier +/// could supply the value for exactly the cases the authoritative tier stays +/// silent on. Clearing is also what the local spawn does: the desktop strips +/// reserved keys from user env before the authoritative layer is written +/// (`env_vars.rs:54-57`), so absent-means-absent in both paths. +const AUTHORITATIVE_KEYS: &[&str] = &[ + "BUZZ_RELAY_URL", + "BUZZ_PRIVATE_KEY", + "NOSTR_PRIVATE_KEY", + "BUZZ_AUTH_TAG", + "BUZZ_ACP_AGENT_OWNER", + "BUZZ_ACP_AGENT_COMMAND", + "BUZZ_ACP_AGENT_ARGS", + "BUZZ_ACP_RESPOND_TO", + "BUZZ_ACP_RESPOND_TO_ALLOWLIST", + "BUZZ_ACP_MCP_COMMAND", + "BUZZ_ACP_EXIT_AFTER_INACTIVITY", + START_NONCE_KEY, +]; + +/// The attempt's generation, as the harness sees it. Also the Secret's name +/// suffix — one generation, one identity — so the reconciler restamps this on +/// every create attempt rather than letting the caller's value persist across +/// a retry. +pub const START_NONCE_KEY: &str = "BUZZ_MANAGED_AGENT_START_NONCE"; + +/// Presence is the only remote liveness signal (I3), so a launch that +/// suppresses it is non-conforming (L1 item 2) — and unlike a reserved-key +/// collision, there is no "authoritative value" to overwrite it with. Refuse. +const FORBIDDEN_KEY: &str = "BUZZ_ACP_NO_PRESENCE"; + +/// Kubernetes' own cap on the summed value bytes of a Secret +/// (`MaxSecretSize`, `pkg/apis/core/types.go`). Enforced here so an oversized +/// env surfaces as a named provider error rather than an apiserver rejection +/// partway through a deploy. +const MAX_SECRET_BYTES: usize = 1024 * 1024; + +/// A POSIX-shaped env var name: `[A-Za-z_][A-Za-z0-9_]*`. +/// +/// Kubernetes validates Secret *keys* as `IsConfigMapKey` +/// (`[-._a-zA-Z0-9]+`), which is looser — `foo.bar` is a legal Secret key. +/// What the kubelet then does with such a key **changed between versions**: +/// through 1.29 it filtered invalid env names out of `envFrom` and emitted an +/// `InvalidEnvironmentVariableNames` warning event +/// (`pkg/kubelet/kubelet_pods.go:646,654` at v1.29.0); from 1.30 that filter +/// is gone (KEP-4369) and the key is injected verbatim. The same manifest +/// would silently drop a variable on one cluster and set it on another, so we +/// fail closed on the provider side and get one deterministic behavior. +fn is_posix_env_key(key: &str) -> bool { + let mut chars = key.chars(); + match chars.next() { + Some(c) if c == '_' || c.is_ascii_alphabetic() => {} + _ => return false, + } + chars.all(|c| c == '_' || c.is_ascii_alphanumeric()) +} + +/// An identity component (L1 item 1) is present only if it is nonempty after +/// trimming — and the **trimmed form is what gets stored**. The validator and +/// the writer must never disagree about the value: a guard that accepts +/// `" wss://relay "` and then writes it with the padding intact has only +/// moved the failure from a loud refusal to a connect error in the harness. +fn identity_component(value: &str) -> Option<&str> { + let trimmed = value.trim(); + (!trimmed.is_empty()).then_some(trimmed) +} + +/// The harness's `allowlist` gate mode, spelled as the desktop serializes +/// `RespondTo` (kebab-case) and as `buzz-acp`'s CLI parses it. +const RESPOND_TO_ALLOWLIST: &str = "allowlist"; + +/// Every gate mode `buzz-acp` accepts, spelled as its `clap::ValueEnum` parses +/// them (`config.rs:95-101`, kebab-case via `RespondTo`'s `Display`). +/// +/// Deliberately the **harness's** four and not the desktop's three: the desktop +/// rejects `nobody` on purpose (`managed_agents/types.rs:871-880`), but the +/// harness starts fine with it. This guard exists to cover non-desktop callers, +/// so inheriting a desktop-only narrowing would refuse a launch that works. +const RESPOND_TO_MODES: [&str; 4] = ["owner-only", RESPOND_TO_ALLOWLIST, "anyone", "nobody"]; + +/// Refuse a respond-to gate the harness will reject at config parse. +/// +/// The local spawn path re-validates this before spawning — "doing it here +/// means we never spawn a doomed process" (`runtime.rs:378`) — but the deploy +/// path projects the record's fields straight through. Without this, a gate +/// the harness refuses becomes a pod that exits 1 at startup; `restartPolicy: +/// Never` turns that into `Terminated` → `Delete` → recreate, and each cycle +/// leaves a Secret the in-call path never reaps (only a later deploy's orphan +/// sweep does, at `ORPHAN_SECRET_MIN_AGE_SECS`). The user-visible ending is +/// "startup not confirmed", indistinguishable from a slow cluster. +/// +/// Mirrors `buzz-acp`'s own rules exactly (`config.rs:95-101,996-1004,629-641`), +/// deliberately including their asymmetry: the allowlist is validated **only** +/// in allowlist mode, and merely warned about otherwise. Validating it in +/// every mode would refuse a deploy whose identical local spawn succeeds — +/// and a stale list is already harmless here, since +/// `BUZZ_ACP_RESPOND_TO_ALLOWLIST` is an authoritative key that tier 3 clears. +fn validate_respond_to_gate(respond_to: &str, allowlist: Option<&[String]>) -> Result<(), String> { + // Exact, untrimmed: `clap` does not trim, so `" allowlist "` is `rc=2` at + // the harness — a parse failure even earlier than the config errors below. + if !RESPOND_TO_MODES.contains(&respond_to) { + return Err(format!( + "deploy refused: respond_to {respond_to:?} is not a mode the \ + harness accepts (expected one of {}) — the pod would fail to \ + parse its arguments, be replaced, and leave a Secret behind on \ + every attempt", + RESPOND_TO_MODES.join(", ") + )); + } + if respond_to != RESPOND_TO_ALLOWLIST { + return Ok(()); + } + let entries = allowlist.unwrap_or_default(); + if entries.is_empty() { + return Err(format!( + "deploy refused: respond_to is {RESPOND_TO_ALLOWLIST:?} but the \ + allowlist is empty — the harness refuses this at startup, so the \ + pod would fail, be replaced, and leave a Secret behind on every \ + attempt" + )); + } + for entry in entries { + let trimmed = entry.trim(); + if trimmed.len() != 64 || !trimmed.chars().all(|c| c.is_ascii_hexdigit()) { + return Err(format!( + "deploy refused: invalid pubkey in respond_to_allowlist: \ + {entry:?} (must be exactly 64 hex characters)" + )); + } + } + Ok(()) +} + +/// Inputs the provider itself supplies to the authoritative tier. +pub struct AuthoritativeInputs<'a> { + /// The attempt's generation token — also the Secret's name suffix, so the + /// lifecycle correlator and the Secret generation are one identity. + pub generation: &'a str, + /// Resolved from `provider_config.inactivity_seconds`; `None` when the + /// indefinite opt-in was chosen (which this version refuses elsewhere). + pub inactivity_seconds: Option, +} + +/// Resolve the full pod environment. +/// +/// Order is the spec's, and the function body is deliberately three writes in +/// that order — tier 1, tier 2, tier 3 — so "later wins" is visible rather +/// than argued. +pub fn build_env( + agent: &AgentPayload, + auth: AuthoritativeInputs<'_>, +) -> Result, String> { + let default_launch = LaunchBlock::default(); + let launch = agent.launch.as_ref().unwrap_or(&default_launch); + + let mut env: BTreeMap = BTreeMap::new(); + + // Tier 1 — overridable behavior defaults. + env.extend(launch.policy_env.clone()); + + // Tier 2 — user/layered env. The descriptor already merged + // global < persona < agent, so `agent.env_vars` is NOT re-merged on top + // (§Launch data tier 2) — doing so would resurrect a layer the desktop + // already resolved. When the desktop predates the `launch` block we fall + // back to the legacy field, which is the only case it is the truth. + if agent.launch.is_some() { + env.extend(launch.env.clone()); + } else { + env.extend(agent.env_vars.clone()); + } + + // Validate what the lower tiers contributed, before the authoritative + // tier overwrites any of it. A reserved-key collision is NOT fatal: the + // spec's precedence is later-wins, so tier 3 simply overwrites it, which + // is exactly what a local spawn does. Only a key that has no + // authoritative counterpart to overwrite it — presence suppression — is + // a refusal. + for key in env.keys() { + if !is_posix_env_key(key) { + return Err(format!( + "env key {key:?} is not a POSIX environment variable name \ + ([A-Za-z_][A-Za-z0-9_]*); Kubernetes would treat it \ + inconsistently across cluster versions" + )); + } + if key.eq_ignore_ascii_case(FORBIDDEN_KEY) { + return Err(format!( + "{FORBIDDEN_KEY} must not be set on a remote agent: presence \ + is the only signal that a remote agent is alive" + )); + } + } + + // Tier 3 — authoritative. Every key it owns is cleared first, then the + // values it has are written, so it wins at a key whether or not it has a + // value there (see [`AUTHORITATIVE_KEYS`]). + for key in AUTHORITATIVE_KEYS { + env.remove(*key); + } + // Identity comes from top-level payload fields, never from `env_vars` + // (§Reserved-key rule). All three components must be nonempty: an agent + // that cannot reach a relay is the identityless launch L1 item 1 exists to + // prevent, and a blank field would otherwise sail through into the Secret + // and produce a pod that starts, fails to connect, and looks like a + // network problem. + let Some(relay_url) = identity_component(&agent.relay_url) else { + return Err("deploy refused: relay_url is empty — the agent would have \ + no relay to connect to" + .to_string()); + }; + env.insert("BUZZ_RELAY_URL".into(), relay_url.to_string()); + env.insert("BUZZ_PRIVATE_KEY".into(), agent.private_key_nsec.clone()); + // The git credential/signing helpers read NOSTR_PRIVATE_KEY. + env.insert("NOSTR_PRIVATE_KEY".into(), agent.private_key_nsec.clone()); + + // Owner: at least one of these must resolve, or the harness cannot match + // `!shutdown` and §Stop describes a mechanism that does not work. + let auth_tag = agent.auth_tag.as_deref().and_then(identity_component); + let owner = launch.owner_pubkey.as_deref().and_then(identity_component); + match (auth_tag, owner) { + (None, None) => { + return Err("deploy refused: neither auth_tag nor launch.owner_pubkey \ + resolved — without an owner the agent cannot honor \ + !shutdown" + .to_string()) + } + (tag, own) => { + if let Some(t) = tag { + env.insert("BUZZ_AUTH_TAG".into(), t.to_string()); + } + if let Some(o) = own { + env.insert("BUZZ_ACP_AGENT_OWNER".into(), o.to_string()); + } + } + } + + // The harness and MCP binaries are resolved against the *image's* PATH. + // A host path forwarded from the desktop is guaranteed absent in the + // container (§Launch data, host-resolved values). + if let Some(command) = launch.command.as_deref().filter(|c| !c.is_empty()) { + env.insert("BUZZ_ACP_AGENT_COMMAND".into(), command.to_string()); + } + if !launch.args.is_empty() { + // Comma-joined because that is what the harness's CLI parser decodes, + // and what the desktop's local spawn does. An argument containing a + // comma is unrepresentable in both paths; inventing an escaping + // scheme here would produce args the harness cannot decode. + env.insert("BUZZ_ACP_AGENT_ARGS".into(), launch.args.join(",")); + } + env.insert("BUZZ_ACP_MCP_COMMAND".into(), "buzz-dev-mcp".into()); + + if let Some(respond_to) = agent.respond_to.as_deref().filter(|s| !s.is_empty()) { + validate_respond_to_gate(respond_to, agent.respond_to_allowlist.as_deref())?; + env.insert("BUZZ_ACP_RESPOND_TO".into(), respond_to.to_string()); + } + if let Some(list) = agent + .respond_to_allowlist + .as_ref() + .filter(|l| !l.is_empty()) + { + env.insert("BUZZ_ACP_RESPOND_TO_ALLOWLIST".into(), list.join(",")); + } + + if let Some(secs) = auth.inactivity_seconds { + env.insert("BUZZ_ACP_EXIT_AFTER_INACTIVITY".into(), secs.to_string()); + } + // The generation token doubles as the lifecycle-frame correlator, so pod + // logs and observer frames share one identity (§K8s Secrets). + env.insert(START_NONCE_KEY.into(), auth.generation.to_string()); + + let total: usize = env.values().map(String::len).sum(); + if total > MAX_SECRET_BYTES { + return Err(format!( + "agent environment is {total} bytes; Kubernetes caps Secret data \ + at {MAX_SECRET_BYTES}" + )); + } + + Ok(env) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn payload_json(extra_agent: serde_json::Value) -> AgentPayload { + let mut agent = serde_json::json!({ + "name": "a", + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1example", + "auth_tag": "tag-1", + }); + let (serde_json::Value::Object(base), serde_json::Value::Object(extra)) = + (&mut agent, extra_agent) + else { + panic!("expected objects") + }; + base.extend(extra); + serde_json::from_value(agent).unwrap() + } + + fn build(agent: &AgentPayload) -> Result, String> { + build_env( + agent, + AuthoritativeInputs { + generation: "gen0001", + inactivity_seconds: Some(7200), + }, + ) + } + + #[test] + fn identity_comes_from_top_level_fields() { + let env = build(&payload_json(serde_json::json!({}))).unwrap(); + assert_eq!(env["BUZZ_RELAY_URL"], "wss://relay.example"); + assert_eq!(env["BUZZ_PRIVATE_KEY"], "nsec1example"); + assert_eq!(env["NOSTR_PRIVATE_KEY"], "nsec1example"); + assert_eq!(env["BUZZ_AUTH_TAG"], "tag-1"); + } + + /// Wren's amendment, and the spec's later-wins rule: a lower tier that + /// spoofs an authoritative key is *overwritten*, not refused. Refusing + /// would diverge from the local spawn, where the same env is written + /// before the authoritative layer and simply loses. + #[test] + fn lower_tiers_cannot_spoof_authoritative_values() { + let agent = payload_json(serde_json::json!({ + "launch": { + "command": "goose", + "policy_env": { + "BUZZ_PRIVATE_KEY": "nsec1attacker", + "BUZZ_MANAGED_AGENT_START_NONCE": "forged", + }, + "env": { + "BUZZ_RELAY_URL": "wss://attacker.example", + "NOSTR_PRIVATE_KEY": "nsec1attacker", + "BUZZ_AUTH_TAG": "forged-tag", + "BUZZ_ACP_AGENT_OWNER": "cafe", + "BUZZ_ACP_AGENT_COMMAND": "/bin/sh", + "BUZZ_ACP_MCP_COMMAND": "/bin/sh", + "BUZZ_ACP_EXIT_AFTER_INACTIVITY": "0", + }, + "owner_pubkey": "beef" + } + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_PRIVATE_KEY"], "nsec1example"); + assert_eq!(env["NOSTR_PRIVATE_KEY"], "nsec1example"); + assert_eq!(env["BUZZ_RELAY_URL"], "wss://relay.example"); + assert_eq!(env["BUZZ_AUTH_TAG"], "tag-1"); + assert_eq!(env["BUZZ_ACP_AGENT_OWNER"], "beef"); + assert_eq!(env["BUZZ_ACP_AGENT_COMMAND"], "goose"); + assert_eq!(env["BUZZ_ACP_MCP_COMMAND"], "buzz-dev-mcp"); + assert_eq!(env["BUZZ_ACP_EXIT_AFTER_INACTIVITY"], "7200"); + assert_eq!(env["BUZZ_MANAGED_AGENT_START_NONCE"], "gen0001"); + } + + /// Tier 1 is *overridable* — user env beats policy defaults, matching the + /// local spawn, where the user layer is written after them. Getting this + /// backwards would make remote agents ignore overrides local agents honor. + #[test] + fn user_env_overrides_policy_defaults() { + let agent = payload_json(serde_json::json!({ + "launch": { + "policy_env": {"GOOSE_MODE": "auto", "BUZZ_ACP_MODEL": "sonnet"}, + "env": {"GOOSE_MODE": "chat"}, + "owner_pubkey": "beef" + } + })); + let env = build(&agent).unwrap(); + assert_eq!(env["GOOSE_MODE"], "chat"); + assert_eq!(env["BUZZ_ACP_MODEL"], "sonnet"); + } + + /// `launch.env` already contains the merged user env, so re-merging the + /// legacy field would undo a layering the desktop already resolved. + #[test] + fn legacy_env_vars_are_not_remerged_when_launch_present() { + let agent = payload_json(serde_json::json!({ + "env_vars": {"STALE": "yes", "SHARED": "legacy"}, + "launch": {"env": {"SHARED": "resolved"}, "owner_pubkey": "beef"} + })); + let env = build(&agent).unwrap(); + assert_eq!(env["SHARED"], "resolved"); + assert!(!env.contains_key("STALE"), "legacy env_vars re-merged"); + } + + /// ...but a desktop predating the `launch` block has nothing else to + /// offer, so the legacy field is the truth in exactly that case. + #[test] + fn legacy_env_vars_used_when_launch_absent() { + let agent = payload_json(serde_json::json!({"env_vars": {"API": "v"}})); + let env = build(&agent).unwrap(); + assert_eq!(env["API"], "v"); + } + + #[test] + fn refuses_when_no_owner_resolves() { + let agent = payload_json(serde_json::json!({"auth_tag": null})); + let err = build(&agent).unwrap_err(); + assert!(err.contains("!shutdown"), "unhelpful error: {err}"); + } + + /// An empty string is not an owner. Without this the refusal is + /// bypassable by a blank field and the pod launches unable to be stopped. + #[test] + fn empty_owner_fields_count_as_absent() { + for blank in ["", " "] { + let agent = payload_json(serde_json::json!({ + "auth_tag": blank, + "launch": {"owner_pubkey": blank} + })); + assert!( + build(&agent).is_err(), + "whitespace resolved as an owner: {blank:?}" + ); + } + } + + /// The other half of every identity guard: what is *stored*. A validator + /// that trims and a writer that doesn't disagree about the value, and the + /// padding reaches the harness inside the Secret. Assert on the stored + /// string — asserting only that the deploy was accepted passes either way. + #[test] + fn identity_components_are_stored_trimmed() { + let agent = payload_json(serde_json::json!({ + "relay_url": " wss://relay.example ", + "auth_tag": " tag-1 ", + "launch": {"owner_pubkey": " beefcafe "} + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_RELAY_URL"], "wss://relay.example"); + assert_eq!(env["BUZZ_AUTH_TAG"], "tag-1"); + assert_eq!(env["BUZZ_ACP_AGENT_OWNER"], "beefcafe"); + } + + /// L1 item 1's third identity component. The nsec arm is enforced in + /// `naming.rs` and the owner arm above; without this one an agent + /// deploys with nothing to connect to — a pod that starts, fails at the + /// relay, and reads as a network fault rather than a refused launch. + #[test] + fn refuses_empty_relay_url() { + for blank in ["", " "] { + let agent = payload_json(serde_json::json!({"relay_url": blank})); + let err = build(&agent).unwrap_err(); + assert!(err.contains("relay_url"), "unhelpful error: {err}"); + } + } + + #[test] + fn owner_pubkey_alone_is_sufficient() { + let agent = payload_json(serde_json::json!({ + "auth_tag": null, + "launch": {"owner_pubkey": "beefcafe"} + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_ACP_AGENT_OWNER"], "beefcafe"); + assert!(!env.contains_key("BUZZ_AUTH_TAG")); + } + + #[test] + fn refuses_presence_suppression() { + let agent = payload_json(serde_json::json!({ + "launch": {"env": {"BUZZ_ACP_NO_PRESENCE": "1"}, "owner_pubkey": "beef"} + })); + let err = build(&agent).unwrap_err(); + assert!(err.contains("BUZZ_ACP_NO_PRESENCE"), "got: {err}"); + } + + /// `foo.bar` is a legal Secret key but not a legal env name: pre-1.30 + /// kubelets drop it, 1.30+ inject it. Refuse rather than behave + /// differently depending on the cluster. + #[test] + fn refuses_non_posix_env_keys() { + for bad in ["foo.bar", "foo-bar", "1LEADING", "", "has space"] { + let agent = payload_json(serde_json::json!({ + "launch": {"env": {bad: "v"}, "owner_pubkey": "beef"} + })); + assert!(build(&agent).is_err(), "accepted non-POSIX key {bad:?}"); + } + } + + #[test] + fn args_are_comma_joined_and_omitted_when_empty() { + let agent = payload_json(serde_json::json!({ + "launch": {"command": "goose", "args": ["run", "--no-session"], "owner_pubkey": "b"} + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_ACP_AGENT_ARGS"], "run,--no-session"); + + let agent = payload_json(serde_json::json!({ + "launch": {"command": "goose", "args": [], "owner_pubkey": "b"} + })); + assert!(!build(&agent).unwrap().contains_key("BUZZ_ACP_AGENT_ARGS")); + } + + /// The top-level `model`/`provider` fields are display inputs; their + /// environment consequence is per-runtime and arrives already resolved + /// inside `launch`. A provider-side mapping is wrong for three of the + /// four built-in runtimes. + #[test] + fn provider_never_maps_model_or_provider_itself() { + let agent = payload_json(serde_json::json!({ + "model": "claude-opus", "provider": "anthropic", + "launch": {"owner_pubkey": "beef"} + })); + let env = build(&agent).unwrap(); + for key in [ + "BUZZ_AGENT_PROVIDER", + "BUZZ_AGENT_MODEL", + "GOOSE_PROVIDER", + "GOOSE_MODEL", + ] { + assert!(!env.contains_key(key), "provider mapped {key} itself"); + } + } + + /// `turn_timeout_seconds` is deprecated and ignored upstream; the local + /// spawn does not emit it either. + #[test] + fn turn_timeout_is_not_mapped() { + let agent = payload_json(serde_json::json!({ + "turn_timeout_seconds": 30, "launch": {"owner_pubkey": "b"} + })); + let env = build(&agent).unwrap(); + assert!(!env.keys().any(|k| k.contains("TURN_TIMEOUT"))); + } + + #[test] + fn inactivity_omitted_when_unset() { + let agent = payload_json(serde_json::json!({"launch": {"owner_pubkey": "b"}})); + let env = build_env( + &agent, + AuthoritativeInputs { + generation: "g", + inactivity_seconds: None, + }, + ) + .unwrap(); + assert!(!env.contains_key("BUZZ_ACP_EXIT_AFTER_INACTIVITY")); + } + + /// Structural guard over the whole authoritative list at once: whatever + /// the lower tiers contain, no authoritative key holds a lower-tier value + /// — including the conditionally-written ones the authoritative tier has + /// nothing to say about, which must be **absent** rather than spoofed. + /// This test caught exactly that: `BUZZ_ACP_AGENT_ARGS` is only written + /// when `launch.args` is non-empty, so plain later-wins overwrite left the + /// spoofed value in place. + #[test] + fn no_authoritative_key_retains_a_lower_tier_value() { + let spoofed: serde_json::Map = AUTHORITATIVE_KEYS + .iter() + .map(|k| ((*k).to_string(), serde_json::json!("SPOOFED"))) + .collect(); + // Split across both lower tiers: policy_env and env are separate + // insertion points, and a fix that only cleared one would pass a + // single-tier test. + let agent = payload_json(serde_json::json!({ + "launch": { + "command": "goose", + "policy_env": spoofed.clone(), + "env": spoofed, + "owner_pubkey": "beef" + } + })); + let env = build(&agent).unwrap(); + for key in AUTHORITATIVE_KEYS { + assert_ne!( + env.get(*key).map(String::as_str), + Some("SPOOFED"), + "{key} kept its lower-tier value" + ); + } + // The keys the authoritative tier had no value for are gone, not + // merely different. + for absent in [ + "BUZZ_ACP_AGENT_ARGS", + "BUZZ_ACP_RESPOND_TO", + "BUZZ_ACP_RESPOND_TO_ALLOWLIST", + ] { + assert!(!env.contains_key(absent), "{absent} survived the clear"); + } + } + + /// A 64-hex pubkey, the only allowlist entry shape the harness accepts. + fn pubkey(fill: char) -> String { + std::iter::repeat_n(fill, 64).collect() + } + + /// The gate the harness refuses first (`config.rs:996-1004`). Refusing it + /// here is the difference between one error message and an unbounded + /// fail-replace loop that leaves a Secret per attempt. + #[test] + fn allowlist_mode_with_an_empty_list_is_refused() { + for empty in [serde_json::json!([]), serde_json::Value::Null] { + let agent = payload_json(serde_json::json!({ + "respond_to": "allowlist", + "respond_to_allowlist": empty, + })); + let err = build(&agent).unwrap_err(); + assert!( + err.contains("the allowlist is empty"), + "unexpected error: {err}" + ); + } + } + + /// `config.rs:629-641` — each entry must be exactly 64 hex characters. + /// The rejects are the distinct ways to miss that: too short, right length + /// but not hex, empty, and one character short of valid. + #[test] + fn an_allowlist_entry_that_is_not_64_hex_is_refused() { + for bad in ["abc1234", &"z".repeat(64), "", &pubkey('a')[..63]] { + let agent = payload_json(serde_json::json!({ + "respond_to": "allowlist", + "respond_to_allowlist": [pubkey('a'), bad], + })); + let err = build(&agent).unwrap_err(); + assert!( + err.contains("must be exactly 64 hex characters"), + "{bad:?} was accepted; error was: {err}" + ); + } + } + + /// The positive control: the guard refuses bad gates, not every gate. + /// Without this, a validator that refused unconditionally would pass both + /// tests above. + #[test] + fn a_valid_allowlist_gate_is_accepted_and_comma_joined() { + let agent = payload_json(serde_json::json!({ + "respond_to": "allowlist", + "respond_to_allowlist": [pubkey('a'), pubkey('b')], + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_ACP_RESPOND_TO"], "allowlist"); + assert_eq!( + env["BUZZ_ACP_RESPOND_TO_ALLOWLIST"], + format!("{},{}", pubkey('a'), pubkey('b')) + ); + } + + /// The harness validates the allowlist **only** in allowlist mode and + /// merely warns otherwise (`config.rs:1005-1010`). A stricter provider + /// would refuse a deploy whose identical local spawn succeeds, so this + /// pins the asymmetry rather than leaving it to look like an oversight. + #[test] + fn a_junk_allowlist_is_tolerated_outside_allowlist_mode() { + for mode in ["owner-only", "anyone"] { + let agent = payload_json(serde_json::json!({ + "respond_to": mode, + "respond_to_allowlist": ["not-a-pubkey"], + })); + let env = build(&agent) + .unwrap_or_else(|e| panic!("{mode} with a stale list must deploy: {e}")); + assert_eq!(env["BUZZ_ACP_RESPOND_TO"], mode); + } + } + + /// `respond_to` is an opaque `String` on the wire but a `clap::ValueEnum` + /// at the harness, so an unrecognized mode dies at `rc=2` — before config + /// parsing runs at all, earlier than either refusal above. Measured + /// against the built binary: `invalid value 'npub1abc' for '--respond-to'`. + /// This is the shape our own fixture carried until it was corrected. + #[test] + fn a_mode_the_harness_cannot_parse_is_refused() { + for bad in ["npub1abc", "OWNER-ONLY", "owner_only", "allowlistt", "x"] { + let agent = payload_json(serde_json::json!({ "respond_to": bad })); + let err = build(&agent).unwrap_err(); + assert!( + err.contains("is not a mode the harness accepts"), + "{bad:?} was accepted; error was: {err}" + ); + } + } + + /// `clap` does not trim its value-enum input, so a padded mode is `rc=2` + /// even though the same string trimmed is valid. Measured: `invalid value + /// ' allowlist ' for '--respond-to'`. Trimming here would accept a deploy + /// the harness refuses — the exact direction this guard exists to prevent. + #[test] + fn a_padded_mode_is_refused_because_clap_does_not_trim() { + for padded in [" allowlist ", "allowlist ", " owner-only", "\tnobody"] { + let agent = payload_json(serde_json::json!({ + "respond_to": padded, + "respond_to_allowlist": [pubkey('a')], + })); + let err = build(&agent).unwrap_err(); + assert!( + err.contains("is not a mode the harness accepts"), + "{padded:?} was accepted; error was: {err}" + ); + } + } + + /// Positive control for the mode check, and the reason it validates the + /// harness's four rather than the desktop's three: `nobody` is rejected by + /// `parse_wire` on purpose (`managed_agents/types.rs:871-880`) but starts + /// fine at the harness. A guard mirroring the desktop enum would refuse a + /// working launch from a non-desktop caller — the callers this guard is + /// for. Without this test, refusing `nobody` would pass everything above. + #[test] + fn every_mode_the_harness_accepts_is_deployable() { + for mode in ["owner-only", "allowlist", "anyone", "nobody"] { + let agent = payload_json(serde_json::json!({ + "respond_to": mode, + "respond_to_allowlist": [pubkey('a')], + })); + let env = build(&agent) + .unwrap_or_else(|e| panic!("{mode} is valid at the harness but was refused: {e}")); + assert_eq!(env["BUZZ_ACP_RESPOND_TO"], mode); + } + } +} diff --git a/crates/buzz-backend-kubernetes/src/gc.rs b/crates/buzz-backend-kubernetes/src/gc.rs new file mode 100644 index 000000000..2e18825aa --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/gc.rs @@ -0,0 +1,368 @@ +//! Preflight garbage collection (spec §K8s GC, `docs/remote-agents.md:1282-1335`). +//! +//! GC runs on every deploy, after identity derivation and before the state +//! transition. It deletes terminated pods (and their referenced Secrets) and +//! age-eligible orphan Secrets — every one of which must pass the full-pubkey +//! annotation check *and* carry the management marker. An unmarked object is +//! never GC'd regardless of its labels. +//! +//! The decision layer here is pure. The effectful caller supplies the observed +//! objects and the apiserver's clock; this module decides what may be deleted. + +use crate::naming::AgentIdentity; +use crate::observe::{referenced_secret, secret_is_ours}; +use chrono::{DateTime, Utc}; +use k8s_openapi::api::core::v1::{Pod, Secret}; + +/// The deploy operation deadline (spec §Deploy: `timeout: 600s`). +pub const OPERATION_DEADLINE_SECS: i64 = 600; + +/// An unreferenced Secret is GC-eligible only once it is older than **twice** +/// the deploy deadline. Rationale: Secret-create → pod-create is not atomic +/// against an independent GC pass, so without the gate a concurrent attempt's +/// preflight GC can delete a Secret whose pod has not been created yet and +/// strand that deploy. The age bound makes "unreferenced" mean "provably +/// abandoned" — any attempt that could still reference it has exceeded its own +/// deadline (`:1301-1319`). +pub const ORPHAN_SECRET_MIN_AGE_SECS: i64 = 2 * OPERATION_DEADLINE_SECS; + +/// What a GC pass decided to delete. Names only: the caller re-reads each +/// object's own fence at delete time. +#[derive(Debug, Default, PartialEq, Eq)] +pub struct GcPlan { + /// Terminated, verified, marker-bearing pods. + pub pods: Vec, + /// Age-eligible, verified, marker-bearing orphan Secrets. + pub secrets: Vec, +} + +/// Plan a GC pass. +/// +/// `now` is the apiserver's clock — the HTTP `Date` header from the very list +/// call that produced `secrets`. `None` means the header was absent or +/// unparseable, in which case **orphan-Secret GC is skipped entirely** rather +/// than falling back to local time: this provider runs on a user's desktop, +/// and a local clock fast by more than the margin does not race — it +/// deterministically computes every in-flight Secret as expired, on every +/// pass, reopening exactly the interleaving the gate exists to close +/// (`:1321-1335`). A deferred cleanup is free; a wrong deletion is not. +/// +/// Terminated-pod GC does not use the clock and is unaffected. +pub fn plan( + identity: &AgentIdentity, + pods: &[Pod], + secrets: &[Secret], + terminated: impl Fn(&Pod) -> bool, + now: Option>, +) -> GcPlan { + // Only pods that pass the full fence participate — in either direction. + // An unverified pod is neither deleted nor allowed to protect a Secret: + // it cannot be ours, so its `envFrom` cannot reference our generation. + let ours: Vec<&Pod> = pods + .iter() + .filter(|p| { + crate::observe::verify(p, identity, crate::classify::Startup::Started).is_some() + }) + .collect(); + + let doomed_pods: Vec<&&Pod> = ours.iter().filter(|p| terminated(p)).collect(); + + // A Secret referenced by ANY existing pod is protected — deliberately + // including not-yet-started pods, whose `envFrom` is exactly as + // load-bearing as a running pod's (`:1262-1264`). Pods being GC'd in this + // same pass are excluded, so their Secrets go with them. + let doomed_names: Vec<&str> = doomed_pods + .iter() + .filter_map(|p| p.metadata.name.as_deref()) + .collect(); + let protected: Vec = ours + .iter() + .filter(|p| !doomed_names.contains(&p.metadata.name.as_deref().unwrap_or_default())) + .filter_map(|p| referenced_secret(p)) + .collect(); + + let mut plan = GcPlan { + pods: doomed_names.iter().map(|n| n.to_string()).collect(), + secrets: doomed_pods + .iter() + .filter_map(|p| referenced_secret(p)) + .collect(), + }; + + // Orphan sweep: only with a server clock. + if let Some(now) = now { + for secret in secrets { + if !secret_is_ours(secret, identity) { + continue; + } + let Some(name) = secret.metadata.name.as_deref() else { + continue; + }; + if protected.contains(&name.to_string()) || plan.secrets.iter().any(|s| s == name) { + continue; + } + let Some(created) = secret.metadata.creation_timestamp.as_ref() else { + // No server-assigned timestamp means no age proof. Skip. + continue; + }; + if (now - created.0).num_seconds() >= ORPHAN_SECRET_MIN_AGE_SECS { + plan.secrets.push(name.to_string()); + } + } + } + + plan +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::naming::{ANNOTATION_PUBKEY_FULL, LABEL_MANAGED_BY}; + use k8s_openapi::api::core::v1::{Container, EnvFromSource, PodSpec, SecretEnvSource}; + use k8s_openapi::apimachinery::pkg::apis::meta::v1::{ObjectMeta, Time}; + use std::collections::BTreeMap; + + fn identity() -> AgentIdentity { + use nostr::nips::nip19::ToBech32; + let keys = nostr::Keys::generate(); + AgentIdentity::from_nsec(&keys.secret_key().to_bech32().unwrap()).unwrap() + } + + fn pod_named(id: &AgentIdentity, name: &str, secret: Option<&str>) -> Pod { + Pod { + metadata: ObjectMeta { + name: Some(name.into()), + uid: Some(format!("uid-{name}")), + resource_version: Some("1".into()), + labels: Some(id.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + id.pubkey_hex().to_string(), + )] + .into_iter() + .collect::>(), + ), + ..Default::default() + }, + spec: secret.map(|s| PodSpec { + containers: vec![Container { + name: "agent".into(), + env_from: Some(vec![EnvFromSource { + secret_ref: Some(SecretEnvSource { + name: s.into(), + optional: Some(false), + }), + ..Default::default() + }]), + ..Default::default() + }], + ..Default::default() + }), + ..Default::default() + } + } + + fn secret_named(id: &AgentIdentity, name: &str, age_secs: i64, now: DateTime) -> Secret { + Secret { + metadata: ObjectMeta { + name: Some(name.into()), + labels: Some(id.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + id.pubkey_hex().to_string(), + )] + .into_iter() + .collect::>(), + ), + creation_timestamp: Some(Time(now - chrono::Duration::seconds(age_secs))), + ..Default::default() + }, + ..Default::default() + } + } + + fn never(_: &Pod) -> bool { + false + } + fn always(_: &Pod) -> bool { + true + } + + #[test] + fn terminated_pods_and_their_secrets_are_collected_together() { + let id = identity(); + let now = Utc::now(); + let pod = pod_named(&id, "buzz-agent-dead", Some("buzz-agent-dead-gen1")); + let plan = plan(&id, &[pod], &[], always, Some(now)); + assert_eq!(plan.pods, ["buzz-agent-dead"]); + assert_eq!(plan.secrets, ["buzz-agent-dead-gen1"]); + } + + #[test] + fn live_pods_are_never_collected() { + let id = identity(); + let pod = pod_named(&id, "buzz-agent-live", Some("buzz-agent-live-gen1")); + let plan = plan(&id, &[pod], &[], never, Some(Utc::now())); + assert_eq!(plan, GcPlan::default()); + } + + /// The auto-repair fence applies to GC identically: an object that lacks + /// the marker, or carries a different pubkey, is never touched — however + /// well its labels match. + #[test] + fn unmarked_and_mismatched_objects_are_never_collected() { + let id = identity(); + let other = identity(); + let now = Utc::now(); + + let mut unmarked = pod_named(&id, "look-alike", Some("look-alike-gen1")); + let mut labels = id.labels(); + labels.remove(LABEL_MANAGED_BY); + unmarked.metadata.labels = Some(labels); + + let mut foreign = pod_named(&id, "someone-elses", Some("someone-elses-gen1")); + foreign.metadata.annotations = Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + other.pubkey_hex().to_string(), + )] + .into_iter() + .collect(), + ); + + let mut unmarked_secret = secret_named(&id, "orphan-unmarked", 100_000, now); + unmarked_secret.metadata.labels = Some(BTreeMap::new()); + let mut foreign_secret = secret_named(&id, "orphan-foreign", 100_000, now); + foreign_secret.metadata.annotations = Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + other.pubkey_hex().to_string(), + )] + .into_iter() + .collect(), + ); + + let plan = plan( + &id, + &[unmarked, foreign], + &[unmarked_secret, foreign_secret], + always, + Some(now), + ); + assert_eq!( + plan, + GcPlan::default(), + "GC touched an object it does not own" + ); + } + + /// The interleaving the age gate exists to close: attempt A creates its + /// Secret; concurrent attempt B's preflight GC runs before A creates its + /// pod. Without the gate B deletes A's Secret and strands A. + #[test] + fn young_unreferenced_secrets_are_protected() { + let id = identity(); + let now = Utc::now(); + let fresh = secret_named(&id, "buzz-agent-x-gen-inflight", 5, now); + assert_eq!( + plan(&id, &[], &[fresh], never, Some(now)), + GcPlan::default() + ); + } + + /// Past twice the deadline, any attempt that could still reference the + /// Secret has exceeded its own deadline — so it is provably abandoned. + #[test] + fn secrets_older_than_twice_the_deadline_are_collected() { + let id = identity(); + let now = Utc::now(); + let old = secret_named( + &id, + "buzz-agent-x-gen-abandoned", + ORPHAN_SECRET_MIN_AGE_SECS + 1, + now, + ); + let plan = plan(&id, &[], &[old], never, Some(now)); + assert_eq!(plan.secrets, ["buzz-agent-x-gen-abandoned"]); + } + + /// The boundary itself, both sides. `>= 1200s` is eligible. + #[test] + fn age_gate_boundary_is_exact() { + let id = identity(); + let now = Utc::now(); + let just_under = secret_named(&id, "under", ORPHAN_SECRET_MIN_AGE_SECS - 1, now); + let exactly = secret_named(&id, "exact", ORPHAN_SECRET_MIN_AGE_SECS, now); + assert!(plan(&id, &[], &[just_under], never, Some(now)) + .secrets + .is_empty()); + assert_eq!( + plan(&id, &[], &[exactly], never, Some(now)).secrets, + ["exact"] + ); + } + + /// The same-clock rule. No apiserver `Date` header → skip the orphan + /// sweep entirely. A local clock fast by more than the margin would + /// silently delete every in-flight Secret on every pass. + #[test] + fn without_a_server_clock_the_orphan_sweep_is_skipped() { + let id = identity(); + let now = Utc::now(); + let ancient = secret_named(&id, "buzz-agent-x-gen-ancient", 10_000_000, now); + let plan = plan(&id, &[], &[ancient], never, None); + assert!( + plan.secrets.is_empty(), + "orphan swept without a server clock — a fast local clock would delete live Secrets" + ); + } + + /// ...but terminated-pod GC does not consult the clock, so it still runs. + #[test] + fn terminated_pod_gc_runs_without_a_server_clock() { + let id = identity(); + let pod = pod_named(&id, "buzz-agent-dead", Some("buzz-agent-dead-gen1")); + let plan = plan(&id, &[pod], &[], always, None); + assert_eq!(plan.pods, ["buzz-agent-dead"]); + assert_eq!(plan.secrets, ["buzz-agent-dead-gen1"]); + } + + /// "Existing" includes not-yet-started pods: a Secret referenced by a pod + /// still pulling its image must not be swept, however old it is. + #[test] + fn secrets_referenced_by_a_pending_pod_are_protected() { + let id = identity(); + let now = Utc::now(); + let pending = pod_named(&id, "buzz-agent-pending", Some("buzz-agent-pending-gen1")); + let old = secret_named(&id, "buzz-agent-pending-gen1", 10_000_000, now); + let plan = plan(&id, &[pending], &[old], never, Some(now)); + assert!(plan.secrets.is_empty(), "swept a referenced Secret"); + } + + /// A Secret with no server-assigned creationTimestamp has no age proof, + /// so it is skipped rather than assumed old. + #[test] + fn secrets_without_a_creation_timestamp_are_skipped() { + let id = identity(); + let now = Utc::now(); + let mut no_timestamp = secret_named(&id, "buzz-agent-x-gen-unknown", 10_000_000, now); + no_timestamp.metadata.creation_timestamp = None; + assert!(plan(&id, &[], &[no_timestamp], never, Some(now)) + .secrets + .is_empty()); + } + + /// A Secret belonging to a pod being collected in this same pass goes with + /// it, and must not be listed twice. + #[test] + fn a_collected_pods_secret_is_listed_once() { + let id = identity(); + let now = Utc::now(); + let dead = pod_named(&id, "buzz-agent-dead", Some("buzz-agent-dead-gen1")); + let its_secret = secret_named(&id, "buzz-agent-dead-gen1", 10_000_000, now); + let plan = plan(&id, &[dead], &[its_secret], always, Some(now)); + assert_eq!(plan.secrets, ["buzz-agent-dead-gen1"]); + } +} diff --git a/crates/buzz-backend-kubernetes/src/image.rs b/crates/buzz-backend-kubernetes/src/image.rs new file mode 100644 index 000000000..b35e6bab1 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/image.rs @@ -0,0 +1,181 @@ +//! Image reference validation (spec §Image). +//! +//! The object holding this reference runs with an nsec, so the reference must +//! be **immutable**. Registry tags are mutable pointers — Kubernetes itself +//! distinguishes them from digests for exactly this reason — so a tag-only +//! reference is rejected, not just `:latest`. +//! +//! v1 ships no baked default (there is no published `ghcr.io/block/buzz-sprig` +//! image yet, so a compile-time digest would be a placeholder). `image` is +//! therefore required, and its absence fails closed with a named field. + +/// A validated, digest-qualified image reference. +/// +/// The inner string is always in canonical tagless form `name@sha256:`: +/// `name:tag@sha256:…` normalizes by dropping the tag, because the tag is +/// decorative once a digest pins the content, and leaving it in would make +/// two references to identical bytes produce different create-intent +/// fingerprints. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ImageRef(String); + +impl ImageRef { + pub fn as_str(&self) -> &str { + &self.0 + } +} + +impl std::fmt::Display for ImageRef { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +/// Parse and normalize a user-supplied image reference. +pub fn parse(raw: &str) -> Result { + let reference = raw.trim(); + if reference.is_empty() { + return Err("provider_config.image is required: v1 ships no default \ + image, so the digest-pinned image to run must be given \ + explicitly" + .to_string()); + } + + let mut parts = reference.split('@'); + let name_and_tag = parts.next().unwrap_or_default(); + let digest = match (parts.next(), parts.next()) { + (Some(d), None) => d, + (None, _) => { + return Err(format!( + "provider_config.image {reference:?} is not digest-pinned: a \ + tag is a mutable pointer, and this object runs with the \ + agent's private key. Use name@sha256:<64 hex chars>" + )) + } + (Some(_), Some(_)) => { + return Err(format!( + "provider_config.image {reference:?} contains more than one '@'" + )) + } + }; + + let hex = digest.strip_prefix("sha256:").ok_or_else(|| { + format!("provider_config.image digest {digest:?} must start with 'sha256:'") + })?; + // Lowercase only: OCI canonicalizes digest hex, and accepting uppercase + // would let two spellings of one digest produce two fingerprints. + if hex.len() != 64 + || !hex + .chars() + .all(|c| c.is_ascii_digit() || ('a'..='f').contains(&c)) + { + return Err(format!( + "provider_config.image digest {digest:?} must be exactly 64 \ + lowercase hex characters" + )); + } + + // Drop any tag: `name:tag@sha256:…` and `name@sha256:…` name the same + // bytes and must fingerprint identically. Only a *final* colon segment + // that isn't a port counts as a tag — `host:5000/name` has no tag. + let name = match name_and_tag.rfind(':') { + Some(colon) if !name_and_tag[colon + 1..].contains('/') => &name_and_tag[..colon], + _ => name_and_tag, + }; + if name.is_empty() { + return Err(format!( + "provider_config.image {reference:?} has no repository name" + )); + } + + Ok(ImageRef(format!("{name}@sha256:{hex}"))) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn accepts_digest_pinned_reference() { + let d = "a".repeat(64); + let r = parse(&format!("ghcr.io/block/buzz-sprig@sha256:{d}")).unwrap(); + assert_eq!(r.as_str(), format!("ghcr.io/block/buzz-sprig@sha256:{d}")); + } + + /// The normalization that keeps the fingerprint stable: two spellings of + /// the same bytes must produce one reference. + #[test] + fn strips_tag_from_tag_plus_digest_form() { + let d = "b".repeat(64); + let tagged = parse(&format!("ghcr.io/block/buzz-sprig:v1.2@sha256:{d}")).unwrap(); + let plain = parse(&format!("ghcr.io/block/buzz-sprig@sha256:{d}")).unwrap(); + assert_eq!(tagged, plain); + } + + /// A registry port is not a tag. `host:5000/name` must keep its port. + #[test] + fn registry_port_is_not_mistaken_for_a_tag() { + let d = "c".repeat(64); + let r = parse(&format!("localhost:5000/buzz-sprig@sha256:{d}")).unwrap(); + assert_eq!(r.as_str(), format!("localhost:5000/buzz-sprig@sha256:{d}")); + } + + #[test] + fn port_and_tag_together_drops_only_the_tag() { + let d = "d".repeat(64); + let r = parse(&format!("localhost:5000/buzz-sprig:dev@sha256:{d}")).unwrap(); + assert_eq!(r.as_str(), format!("localhost:5000/buzz-sprig@sha256:{d}")); + } + + /// Wren's amendment: *every* tag-only reference is rejected, not just + /// `:latest`. A `sha-` tag is traceable but still movable. + #[test] + fn rejects_every_tag_only_reference() { + for bad in [ + "ghcr.io/block/buzz-sprig:latest", + "ghcr.io/block/buzz-sprig:v1.2.3", + "ghcr.io/block/buzz-sprig:sha-abc1234", + "ghcr.io/block/buzz-sprig", + "localhost:5000/buzz-sprig", + ] { + let err = parse(bad).unwrap_err(); + assert!(err.contains("digest-pinned"), "for {bad:?} got: {err}"); + } + } + + /// Uppercase hex is a second spelling of one digest; accepting it would + /// let the same image fingerprint two ways. + #[test] + fn rejects_uppercase_digest_hex() { + let d = "A".repeat(64); + assert!(parse(&format!("img@sha256:{d}")).is_err()); + } + + #[test] + fn rejects_malformed_digests() { + let short = "a".repeat(63); + let long = "a".repeat(65); + let ok = "a".repeat(64); + for bad in [ + format!("img@sha256:{short}"), + format!("img@sha256:{long}"), + format!("img@sha512:{ok}"), + format!("img@{ok}"), + format!("img@sha256:{}", "g".repeat(64)), + format!("img@sha256:{ok}@sha256:{ok}"), + format!("@sha256:{ok}"), + ] { + assert!(parse(&bad).is_err(), "accepted {bad:?}"); + } + } + + /// v1 has no baked default, so an absent image is an error that names the + /// field rather than a silent fallback. + #[test] + fn empty_reference_names_the_field() { + for empty in ["", " "] { + let err = parse(empty).unwrap_err(); + assert!(err.contains("provider_config.image"), "got: {err}"); + } + } +} diff --git a/crates/buzz-backend-kubernetes/src/intent.rs b/crates/buzz-backend-kubernetes/src/intent.rs new file mode 100644 index 000000000..73218f83b --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/intent.rs @@ -0,0 +1,309 @@ +//! The create-intent fingerprint (spec §Deploy State Machine, create-intent +//! fingerprint, `docs/remote-agents.md:796-828`). +//! +//! The fingerprint is an unkeyed SHA-256 over a canonical serialization of the +//! provider's non-secret create-intent template. A plain hash is safe *only* +//! because of the scope rule: the input covers exactly the provider-controlled +//! fields that can affect scheduling or container creation, and **never Secret +//! data or attempt identity**. Hashing low-entropy secrets into a +//! world-readable annotation would be a dictionary oracle. +//! +//! That rule is enforced structurally rather than remembered. [`IntentTemplate`] +//! is a *pre-binding* type: it has no field that can hold Secret material or a +//! generation token, so there is no expression that hashes one. The +//! per-attempt Secret name never appears — the pod's `envFrom` is represented +//! by the fixed [`SECRET_PLACEHOLDER`], because otherwise every attempt would +//! diverge from every other by construction. +//! +//! Server- and admission-produced output (UID, `resourceVersion`, timestamps, +//! defaulted fields, the annotation itself) is excluded the same way: the +//! serializer is only ever handed this template, never a live `Pod`, so the +//! exclusion is checkable by inspection. + +use crate::image::ImageRef; +use serde::Serialize; +use sha2::{Digest, Sha256}; + +/// Stands in for the per-attempt Secret name in the `envFrom` position. +/// A real generation token here would make every attempt diverge. +const SECRET_PLACEHOLDER: &str = ""; + +/// The recorded/computed create intent: a hex SHA-256 digest. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Fingerprint(String); + +impl Fingerprint { + pub fn as_str(&self) -> &str { + &self.0 + } + + /// Read a fingerprint off a pod annotation. Any recorded string is + /// accepted verbatim: comparison is equality against a freshly computed + /// value, so a malformed annotation simply reads as divergence — which is + /// the correct outcome for a pod this provider version did not write. + pub fn from_annotation(value: &str) -> Self { + Self(value.to_string()) + } + + #[cfg(test)] + pub fn for_test(seed: &str) -> Self { + Self(format!("test-{seed}")) + } +} + +impl std::fmt::Display for Fingerprint { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +/// The non-secret, pre-binding description of the pod this deploy would +/// create. Every field is provider-controlled and scheduling-relevant; there +/// is deliberately no field for env values, Secret data, or the generation. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct IntentTemplate { + /// Schema version of the template itself. Bumping it re-fingerprints every + /// pod, which is the intended way to roll out a pod-shape change. + pub template_version: u32, + pub namespace: String, + /// Normalized, digest-qualified image reference. + pub image: String, + pub cpu_request: String, + pub memory_request: String, + pub cpu_limit: String, + pub memory_limit: String, + pub service_account: Option, + pub restart_policy: &'static str, + pub termination_grace_period_seconds: i64, + /// Env *keys* only, sorted. Keys are pod-shape (a renamed key changes the + /// container's contract); values are Secret material and must not be here. + pub env_keys: Vec, + /// Fixed placeholder for the per-attempt Secret in `envFrom`. + pub env_from_secret: &'static str, + pub workspace_mount_path: String, + pub run_as_user: i64, + pub run_as_group: i64, +} + +/// Current template schema version. +pub const TEMPLATE_VERSION: u32 = 1; + +impl IntentTemplate { + /// Compute the fingerprint. `serde_json` on a struct with declared field + /// order plus pre-sorted `env_keys` is a canonical serialization: the same + /// template always produces the same bytes. + pub fn fingerprint(&self) -> Fingerprint { + let canonical = serde_json::to_vec(self).expect("intent template is plain data"); + Fingerprint(hex::encode(Sha256::digest(&canonical))) + } + + /// Build from resolved pod-shape inputs. `env_keys` is sorted here rather + /// than at the call site so key ordering can never leak into the digest. + /// + /// The fixed pod-shape constants are read from [`crate::config`] rather + /// than passed in: `pod::build_pod` stamps the pod from those same + /// constants, so the fingerprint cannot describe a pod shape different + /// from the one actually created. Threading them through as arguments + /// would make that agreement a thing to test instead of a thing that holds. + pub fn new( + namespace: &str, + image: &ImageRef, + resources: &crate::config::Resources, + service_account: Option<&str>, + env_keys: impl IntoIterator, + ) -> Self { + let mut env_keys: Vec = env_keys.into_iter().collect(); + env_keys.sort(); + Self { + template_version: TEMPLATE_VERSION, + namespace: namespace.to_string(), + image: image.as_str().to_string(), + cpu_request: resources.cpu_request.clone(), + memory_request: resources.memory_request.clone(), + cpu_limit: resources.cpu_limit.clone(), + memory_limit: resources.memory_limit.clone(), + service_account: service_account.map(str::to_string), + restart_policy: crate::config::RESTART_POLICY, + termination_grace_period_seconds: crate::config::TERMINATION_GRACE_SECONDS, + env_keys, + env_from_secret: SECRET_PLACEHOLDER, + workspace_mount_path: crate::config::WORKSPACE_PATH.to_string(), + run_as_user: crate::config::RUN_AS_UID, + run_as_group: crate::config::RUN_AS_GID, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::config::Resources; + + fn image(byte: char) -> ImageRef { + crate::image::parse(&format!( + "ghcr.io/block/buzz-sprig@sha256:{}", + byte.to_string().repeat(64) + )) + .unwrap() + } + + fn template() -> IntentTemplate { + IntentTemplate::new( + "buzz-agents", + &image('a'), + &Resources::default(), + None, + ["BUZZ_RELAY_URL".to_string(), "GOOSE_MODE".to_string()], + ) + } + + #[test] + fn fingerprint_is_deterministic() { + assert_eq!(template().fingerprint(), template().fingerprint()); + } + + #[test] + fn fingerprint_is_hex_sha256() { + let fp = template().fingerprint(); + assert_eq!(fp.as_str().len(), 64); + assert!(fp.as_str().chars().all(|c| c.is_ascii_hexdigit())); + } + + /// Key *order* must not reach the digest, or two identical environments + /// built in different orders would look like a config change. + #[test] + fn env_key_order_does_not_affect_the_digest() { + let a = IntentTemplate::new( + "ns", + &image('a'), + &Resources::default(), + None, + ["A".to_string(), "B".to_string(), "C".to_string()], + ); + let b = IntentTemplate::new( + "ns", + &image('a'), + &Resources::default(), + None, + ["C".to_string(), "A".to_string(), "B".to_string()], + ); + assert_eq!(a.fingerprint(), b.fingerprint()); + } + + /// A mutation applied to a fresh template clone, named for its assertion + /// message. + type Mutation = (&'static str, Box); + + /// Every scheduling-relevant knob must move the digest — this is the + /// wedge escape (§Deploy State Machine never-started recoverable row). + /// Exhaustive by construction: each mutation is applied to a fresh clone. + #[test] + fn every_scheduling_field_changes_the_digest() { + let base = template(); + let baseline = base.fingerprint(); + + let mutations: Vec = vec![ + ( + "template_version", + Box::new(|t: &mut IntentTemplate| t.template_version += 1), + ), + ( + "namespace", + Box::new(|t: &mut IntentTemplate| t.namespace = "other".into()), + ), + ( + "image", + Box::new(|t: &mut IntentTemplate| t.image = image('b').as_str().into()), + ), + ( + "cpu_request", + Box::new(|t: &mut IntentTemplate| t.cpu_request = "4".into()), + ), + ( + "memory_request", + Box::new(|t: &mut IntentTemplate| t.memory_request = "8Gi".into()), + ), + ( + "cpu_limit", + Box::new(|t: &mut IntentTemplate| t.cpu_limit = "8".into()), + ), + ( + "memory_limit", + Box::new(|t: &mut IntentTemplate| t.memory_limit = "16Gi".into()), + ), + ( + "service_account", + Box::new(|t: &mut IntentTemplate| t.service_account = Some("sa".into())), + ), + ( + "restart_policy", + Box::new(|t: &mut IntentTemplate| t.restart_policy = "OnFailure"), + ), + ( + "grace_period", + Box::new(|t: &mut IntentTemplate| t.termination_grace_period_seconds = 30), + ), + ( + "env_keys", + Box::new(|t: &mut IntentTemplate| t.env_keys.push("NEW_KEY".into())), + ), + ( + "workspace_mount_path", + Box::new(|t: &mut IntentTemplate| t.workspace_mount_path = "/w".into()), + ), + ( + "run_as_user", + Box::new(|t: &mut IntentTemplate| t.run_as_user = 2000), + ), + ( + "run_as_group", + Box::new(|t: &mut IntentTemplate| t.run_as_group = 2000), + ), + ]; + + for (name, mutate) in mutations { + let mut t = base.clone(); + mutate(&mut t); + assert_ne!( + t.fingerprint(), + baseline, + "{name} did not affect the digest" + ); + } + } + + /// The scope rule, asserted on the bytes: no Secret value and no + /// generation token can appear in the serialization, because the type has + /// nowhere to put them. The placeholder is what `envFrom` contributes. + #[test] + fn serialization_contains_no_secret_material_or_attempt_identity() { + let json = serde_json::to_string(&template()).unwrap(); + for forbidden in ["nsec1", "SPOOFED", "wss://", "gen0001"] { + assert!( + !json.contains(forbidden), + "template leaked {forbidden}: {json}" + ); + } + assert!(json.contains(SECRET_PLACEHOLDER)); + } + + /// Two attempts for the same agent differ only in generation, which is + /// absent from the template — so their fingerprints must be equal, or the + /// divergence discriminator would fire on every single deploy. + #[test] + fn attempts_differing_only_by_generation_do_not_diverge() { + // There is no generation input to pass; that *is* the property. The + // test states it explicitly so a future field addition breaks here. + assert_eq!(template().fingerprint(), template().fingerprint()); + let json = serde_json::to_string(&template()).unwrap(); + assert_eq!(json.matches(SECRET_PLACEHOLDER).count(), 1); + } + + /// A recorded annotation this provider version did not write reads as + /// divergence rather than an error. + #[test] + fn unrecognized_annotation_reads_as_divergence() { + let recorded = Fingerprint::from_annotation("not-a-digest"); + assert_ne!(recorded, template().fingerprint()); + } +} diff --git a/crates/buzz-backend-kubernetes/src/main.rs b/crates/buzz-backend-kubernetes/src/main.rs new file mode 100644 index 000000000..5d521a9d3 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/main.rs @@ -0,0 +1,199 @@ +//! Kubernetes backend provider for Buzz remote agents +//! (spec `docs/remote-agents.md`). +//! +//! One process per operation: read exactly one JSON request from stdin, write +//! exactly one JSON response to stdout, exit. The exit code carries exactly +//! one bit — 0 for a response that was produced, 1 for a failure to produce +//! one. Everything a caller needs to distinguish is *inside* the response's +//! `ok` field, because a provider that encoded outcomes in exit codes would +//! have a second, redundant error channel to keep in sync (§Provider Protocol). + +mod classify; +mod client; +mod cluster; +mod config; +mod env; +mod gc; +mod image; +mod intent; +mod naming; +mod observe; +mod pod; +mod reconcile; +mod wire; + +use std::io::Read; +use wire::{Request, Response}; + +/// The provider a shared-compute agent resolves to. Refused here as the +/// spec's backstop: a mesh agent runs on the relay's compute, so deploying it +/// as a pod would create a second, contending consumer of the same agent +/// identity (`:214-219`). +const RELAY_MESH_PROVIDER: &str = "relay-mesh"; + +fn main() { + // rustls needs a process-level provider before the first TLS connection. + // The release build compiles every sidecar in one cargo invocation, which + // unifies the `ring` and `aws-lc-rs` features and leaves rustls unable to + // auto-select — so this is an explicit install, not a default. + let _ = rustls::crypto::ring::default_provider().install_default(); + + let mut input = String::new(); + if let Err(e) = std::io::stdin().read_to_string(&mut input) { + // No request means no request_id and no response contract to honor. + // This is the one path that exits nonzero. + eprintln!("could not read the request from stdin: {e}"); + std::process::exit(1); + } + + let response = respond(&input); + println!( + "{}", + serde_json::to_string(&response).unwrap_or_else(|e| { + // The response types are plain data; this cannot fail in practice, + // and a hand-built object is still a conforming response. + format!(r#"{{"ok":false,"error":"could not serialize a response: {e}"}}"#) + }) + ); +} + +/// Produce the single response for one request. Separated from `main` so the +/// whole dispatch is testable without a process. +fn respond(input: &str) -> Response { + // Parsed as raw JSON first: the relay-mesh refusal below MUST see the wire + // value, and `AgentPayload` deliberately does not carry `provider`. + let raw: serde_json::Value = match serde_json::from_str(input) { + Ok(value) => value, + Err(e) => return Response::error(format!("request is not valid JSON: {e}")), + }; + + if let Some(refusal) = refuse_relay_mesh(&raw) { + return Response::error(refusal); + } + + let request: Request = match serde_json::from_value(raw) { + Ok(request) => request, + Err(e) => return Response::error(format!("could not understand the request: {e}")), + }; + + match request { + Request::Info => Response::info(), + Request::Deploy(deploy) => { + let runtime = match tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + { + Ok(runtime) => runtime, + Err(e) => return Response::error(format!("could not start the runtime: {e}")), + }; + match runtime.block_on(deploy_agent(&deploy)) { + Ok(agent_id) => Response::deployed(agent_id), + Err(e) => Response::error(e), + } + } + } +} + +/// Refuse a shared-compute agent, reading the **raw wire value**. +/// +/// Trimmed before comparing: the desktop's own layers disagree about padding +/// (`relay_mesh.rs:17` and `effective_config/mod.rs:46` trim; the deploy guard +/// at `agents_deploy.rs:116` did not), and `non_blank` preserves surrounding +/// whitespace on a non-blank value. A backstop that shares its bypass with the +/// layer it backs is not a backstop. +fn refuse_relay_mesh(raw: &serde_json::Value) -> Option { + let provider = raw.get("agent")?.get("provider")?.as_str()?; + (provider.trim() == RELAY_MESH_PROVIDER).then(|| { + "deploy refused: this agent is configured for shared compute \ + (relay-mesh), which runs on the relay rather than in a pod. \ + Switch the agent to a local runtime before deploying it to \ + Kubernetes." + .to_string() + }) +} + +/// Run one deploy to a terminal outcome. +async fn deploy_agent(request: &wire::DeployRequest) -> Result { + let cfg = config::parse(&request.provider_config)?; + // Identity before any cluster contact: a malformed nsec is a refusal, not + // a failed connection (§Deploy State Machine step 0). + let identity = naming::AgentIdentity::from_nsec(&request.agent.private_key_nsec)?; + + // One generation for this operation's first attempt; the reconciler mints + // its own per attempt and restamps the correlator to match. + let env = env::build_env( + &request.agent, + env::AuthoritativeInputs { + generation: &naming::new_generation(), + inactivity_seconds: cfg.inactivity_seconds, + }, + )?; + + let client = client::connect(cfg.context.as_deref()).await?; + let substrate = cluster::Cluster::new(client, &cfg.namespace); + reconcile::deploy(&substrate, &identity, &cfg, env).await +} + +#[cfg(test)] +mod tests { + use super::*; + + fn error_of(response: &Response) -> String { + let json = serde_json::to_value(response).unwrap(); + assert_eq!(json["ok"], false, "expected a refusal: {json}"); + json["error"].as_str().unwrap().to_string() + } + + /// The spec's backstop for the relay-mesh MUST. The desktop refuses first + /// (`agents_deploy.rs:116`); this is the layer that owes the obligation. + #[test] + fn refuses_a_relay_mesh_agent() { + let request = r#"{"op":"deploy","agent":{ + "relay_url":"wss://r","private_key_nsec":"nsec1x","provider":"relay-mesh"}, + "provider_config":{"namespace":"ns"}}"#; + assert!(error_of(&respond(request)).contains("relay-mesh")); + } + + /// Padding must not bypass the backstop. Reachable by construction: + /// `GlobalConfig.provider` is a bare `Option` with no trim on + /// write, and `non_blank` rejects whitespace-only while preserving + /// surrounding whitespace on everything else. + #[test] + fn refuses_a_padded_relay_mesh_agent() { + let request = r#"{"op":"deploy","agent":{ + "relay_url":"wss://r","private_key_nsec":"nsec1x","provider":" relay-mesh "}, + "provider_config":{"namespace":"ns"}}"#; + assert!(error_of(&respond(request)).contains("relay-mesh")); + } + + /// The refusal must not fire on a normal agent — a guard that refuses + /// everything passes its own test and ships a provider that deploys + /// nothing. + #[test] + fn does_not_refuse_a_normal_provider() { + let raw: serde_json::Value = + serde_json::from_str(r#"{"agent":{"provider":"openai"}}"#).unwrap(); + assert!(refuse_relay_mesh(&raw).is_none()); + // …nor when the field is absent entirely, which is the common case: + // `AgentPayload` does not carry `provider`. + let bare: serde_json::Value = serde_json::from_str(r#"{"agent":{}}"#).unwrap(); + assert!(refuse_relay_mesh(&bare).is_none()); + } + + /// Malformed input still produces exactly one conforming response. + #[test] + fn malformed_input_is_an_in_band_error() { + assert!(error_of(&respond("not json")).contains("valid JSON")); + assert!(error_of(&respond(r#"{"op":"undeploy"}"#)).contains("understand")); + } + + /// `info` answers without touching a cluster — it is what the desktop + /// calls to render the config form, before any kubeconfig exists. + #[test] + fn info_answers_with_the_protocol_version_and_schema() { + let json = serde_json::to_value(respond(r#"{"op":"info"}"#)).unwrap(); + assert_eq!(json["ok"], true); + assert_eq!(json["protocol_version"], wire::PROTOCOL_VERSION); + assert!(json["config_schema"]["properties"]["namespace"].is_object()); + } +} diff --git a/crates/buzz-backend-kubernetes/src/naming.rs b/crates/buzz-backend-kubernetes/src/naming.rs new file mode 100644 index 000000000..4b9d7ea03 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/naming.rs @@ -0,0 +1,223 @@ +//! Identity derivation and the object-naming contract (spec §Pod shape). +//! +//! Every name, label, and annotation below is derived from the pubkey the +//! provider decoded itself from `private_key_nsec` — never from a +//! caller-supplied pubkey (§Deploy State Machine step 0). + +use nostr::nips::nip19::FromBech32; + +/// `app.kubernetes.io/managed-by` value: the management marker's identity half. +pub const MANAGED_BY: &str = "buzz-backend-kubernetes"; + +/// Label key carrying [`MANAGED_BY`]. +pub const LABEL_MANAGED_BY: &str = "app.kubernetes.io/managed-by"; + +/// Label key carrying [`BINDING_VERSION`] — the marker's schema half. +pub const LABEL_BINDING_VERSION: &str = "buzz.block.xyz/binding-version"; + +/// Schema version of the object layout this provider writes. Bumped when the +/// pod/Secret shape changes in a way a older provider would mis-handle. +pub const BINDING_VERSION: &str = "1"; + +/// Label key: truncated pubkey, the reconciliation and GC selector. +pub const LABEL_AGENT_PUBKEY: &str = "buzz.block.xyz/agent-pubkey"; + +/// Annotation key: full pubkey. Load-bearing — the truncated label is +/// collision-*resistant*, this is what makes it safe (§Deploy State Machine +/// step 1). +pub const ANNOTATION_PUBKEY_FULL: &str = "buzz.block.xyz/agent-pubkey-full"; + +/// Annotation key: the recorded create-intent fingerprint. +pub const ANNOTATION_CREATE_INTENT: &str = "buzz.block.xyz/create-intent"; + +/// Annotation key: the image reference this generation actually resolved to, +/// for post-hoc attribution (§Image). +pub const ANNOTATION_IMAGE: &str = "buzz.block.xyz/image"; + +/// An agent identity the provider derived itself, plus every name it implies. +/// +/// Constructing this type is the *only* way to obtain the names — so a +/// caller-supplied pubkey cannot reach a selector by any path. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct AgentIdentity { + pubkey_hex: String, +} + +impl AgentIdentity { + /// Derive from the payload's `private_key_nsec`. + /// + /// Accepts bech32 `nsec1…`; a malformed or undecodable key is an + /// immediate error, before any substrate read or mutation + /// (§Deploy State Machine step 0). + pub fn from_nsec(nsec: &str) -> Result { + let secret = nostr::SecretKey::from_bech32(nsec.trim()) + .map_err(|_| "private_key_nsec is not a decodable nsec1 key".to_string())?; + let keys = nostr::Keys::new(secret); + Ok(Self { + pubkey_hex: keys.public_key().to_hex(), + }) + } + + /// Full 64-hex public key — the annotation value and the comparison + /// operand for candidate authentication. + pub fn pubkey_hex(&self) -> &str { + &self.pubkey_hex + } + + /// Selector label value: first 32 hex chars (128 bits). A full hex pubkey + /// is 64 chars and label values cap at 63, which is why this is truncated + /// and why the annotation check is normative rather than decorative. + pub fn label_pubkey(&self) -> &str { + &self.pubkey_hex[..32] + } + + /// Deterministic pod name, also the returned `agent_id`. + pub fn pod_name(&self) -> String { + format!("buzz-agent-{}", &self.pubkey_hex[..12]) + } + + /// Per-attempt Secret name. `generation` is a fresh random token per + /// create attempt — never reused — which is what makes payload and Secret + /// atomic at the pod-spec boundary (§K8s Secrets). + pub fn secret_name(&self, generation: &str) -> String { + format!("buzz-agent-{}-{}", &self.pubkey_hex[..12], generation) + } + + /// Label selector matching this identity's objects *and* our management + /// marker. Selecting on the marker as well as the identity means an + /// unmarked look-alike never even enters the candidate list. + pub fn selector(&self) -> String { + format!( + "{LABEL_AGENT_PUBKEY}={},{LABEL_MANAGED_BY}={MANAGED_BY}", + self.label_pubkey() + ) + } + + /// The label set stamped on every object this provider creates. + pub fn labels(&self) -> std::collections::BTreeMap { + [ + ( + LABEL_AGENT_PUBKEY.to_string(), + self.label_pubkey().to_string(), + ), + (LABEL_MANAGED_BY.to_string(), MANAGED_BY.to_string()), + ( + LABEL_BINDING_VERSION.to_string(), + BINDING_VERSION.to_string(), + ), + ] + .into_iter() + .collect() + } +} + +/// A fresh generation token: 8 lowercase hex chars from the OS RNG. +/// +/// Appears in the Secret name and as `BUZZ_MANAGED_AGENT_START_NONCE`, so the +/// Secret generation and the harness's lifecycle-frame correlator are one +/// identity (§Launch data tier 3). +pub fn new_generation() -> String { + use rand::RngExt; + let n: u32 = rand::rng().random(); + format!("{n:08x}") +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A fixed test key. Deriving the pubkey (rather than hardcoding both + /// halves) is the point: the test exercises the same derivation the + /// reconciler depends on. + fn identity() -> AgentIdentity { + let keys = nostr::Keys::generate(); + let nsec = { + use nostr::nips::nip19::ToBech32; + keys.secret_key().to_bech32().unwrap() + }; + let id = AgentIdentity::from_nsec(&nsec).unwrap(); + assert_eq!(id.pubkey_hex(), keys.public_key().to_hex()); + id + } + + #[test] + fn rejects_malformed_nsec() { + for bad in ["", "nsec1", "not-a-key", "npub1abc"] { + assert!( + AgentIdentity::from_nsec(bad).is_err(), + "accepted malformed key {bad:?}" + ); + } + } + + #[test] + fn tolerates_surrounding_whitespace() { + let keys = nostr::Keys::generate(); + use nostr::nips::nip19::ToBech32; + let nsec = keys.secret_key().to_bech32().unwrap(); + let padded = format!(" {nsec}\n"); + assert_eq!( + AgentIdentity::from_nsec(&padded).unwrap().pubkey_hex(), + keys.public_key().to_hex() + ); + } + + /// Kubernetes label *values* cap at 63 chars; a full hex pubkey is 64, + /// one over. That one-char overflow is the whole reason the selector is + /// truncated, so it gets an explicit test. + #[test] + fn label_value_fits_kubernetes_limit() { + let id = identity(); + assert_eq!(id.pubkey_hex().len(), 64); + assert_eq!(id.label_pubkey().len(), 32); + assert!(id.label_pubkey().len() <= 63); + } + + #[test] + fn pod_name_is_deterministic_and_dns_safe() { + let id = identity(); + assert_eq!(id.pod_name(), id.pod_name()); + assert_eq!( + id.pod_name(), + format!("buzz-agent-{}", &id.pubkey_hex()[..12]) + ); + assert!(id.pod_name().len() <= 253); + assert!(id + .pod_name() + .chars() + .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-')); + } + + /// Two attempts must never share a Secret name — that uniqueness is what + /// stops a losing contender from overwriting the winner's identity. + #[test] + fn secret_names_are_per_attempt() { + let id = identity(); + let a = id.secret_name(&new_generation()); + let b = id.secret_name(&new_generation()); + assert_ne!(a, b); + assert!(a.starts_with(&id.pod_name())); + assert!(a.len() <= 253); + } + + #[test] + fn selector_requires_the_management_marker() { + let id = identity(); + let sel = id.selector(); + assert!(sel.contains(&format!("{LABEL_AGENT_PUBKEY}={}", id.label_pubkey()))); + assert!(sel.contains(&format!("{LABEL_MANAGED_BY}={MANAGED_BY}"))); + } + + #[test] + fn every_created_object_carries_the_marker() { + let labels = identity().labels(); + assert_eq!( + labels.get(LABEL_MANAGED_BY).map(String::as_str), + Some(MANAGED_BY) + ); + assert_eq!( + labels.get(LABEL_BINDING_VERSION).map(String::as_str), + Some(BINDING_VERSION) + ); + } +} diff --git a/crates/buzz-backend-kubernetes/src/observe.rs b/crates/buzz-backend-kubernetes/src/observe.rs new file mode 100644 index 000000000..1c6c88356 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/observe.rs @@ -0,0 +1,592 @@ +//! Decoding API objects into verified observations (spec §Deploy State +//! Machine step 1). +//! +//! Pure: `Pod` in, [`VerifiedPod`] out. Keeping the decode here means the +//! conformance tests drive the *shipped* decoder with real API types rather +//! than a test-only stand-in, and it keeps `classify.rs` free of API types. +//! +//! Verification is the gate, not a filter: [`verify`] returns `None` for any +//! object whose full-pubkey annotation does not equal the derived pubkey or +//! that lacks the management marker, so an unverified object cannot reach +//! classification, deletion, or the returned `agent_id`. + +use crate::classify::{Fence, PullFailure, Startup, VerifiedPod}; +use crate::intent::Fingerprint; +use crate::naming::{ + AgentIdentity, ANNOTATION_CREATE_INTENT, ANNOTATION_PUBKEY_FULL, BINDING_VERSION, + LABEL_BINDING_VERSION, LABEL_MANAGED_BY, MANAGED_BY, +}; +use k8s_openapi::api::core::v1::{Pod, Secret}; + +/// Container name the provider creates; status is read from this container. +pub const CONTAINER_NAME: &str = "agent"; + +/// The startup state, or a state that cannot be settled without one more read. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum StartupObservation { + Resolved(Startup), + /// `CreateContainerConfigError` — recoverable *unless* the referenced + /// Secret is confirmed absent by a most-recent read. The kubelet's reason + /// string is a hint; the provider verifies before treating it as fatal + /// (§Deploy State Machine: "provably" means a verified absence, never a + /// reason string). + ConfigErrorPendingSecretCheck { + secret_name: String, + }, +} + +/// Does this object carry the management marker (§Pod shape)? +/// +/// Identity labels prove identity; the marker asserts protocol ownership. +/// Without it an object that merely matches our schema fails closed. +fn has_marker(labels: Option<&std::collections::BTreeMap>) -> bool { + let Some(labels) = labels else { return false }; + labels.get(LABEL_MANAGED_BY).map(String::as_str) == Some(MANAGED_BY) + && labels.get(LABEL_BINDING_VERSION).map(String::as_str) == Some(BINDING_VERSION) +} + +/// Does the full-pubkey annotation equal the derived pubkey? +/// +/// The 32-hex label is collision-*resistant*, not collision-free, which is +/// why this check is normative rather than decorative (`:1152-1166`). +fn annotation_matches( + annotations: Option<&std::collections::BTreeMap>, + identity: &AgentIdentity, +) -> bool { + annotations + .and_then(|a| a.get(ANNOTATION_PUBKEY_FULL)) + .map(|v| v == identity.pubkey_hex()) + .unwrap_or(false) +} + +/// Is this Secret ours and this identity's? The same fence GC applies before +/// deleting anything. +pub fn secret_is_ours(secret: &Secret, identity: &AgentIdentity) -> bool { + has_marker(secret.metadata.labels.as_ref()) + && annotation_matches(secret.metadata.annotations.as_ref(), identity) +} + +/// Decode a pod's startup state from its status. +/// +/// "Started" means `state.running` on our container — not pod phase. A pod can +/// sit in phase `Running` with a container that never started, and a pod being +/// gracefully deleted stays in phase `Running` for its whole grace period. +pub fn decode_startup(pod: &Pod) -> StartupObservation { + use StartupObservation::Resolved; + + let status = pod.status.as_ref(); + let phase = status.and_then(|s| s.phase.as_deref()); + + let container = status + .and_then(|s| s.container_statuses.as_ref()) + .and_then(|cs| cs.iter().find(|c| c.name == CONTAINER_NAME)); + + if let Some(state) = container.and_then(|c| c.state.as_ref()) { + if state.running.is_some() { + return Resolved(Startup::Started); + } + if state.terminated.is_some() { + return Resolved(Startup::Terminated); + } + if let Some(waiting) = state.waiting.as_ref() { + let reason = waiting.reason.as_deref().unwrap_or_default(); + let message = waiting.message.as_deref().unwrap_or_default(); + return match reason { + // Structurally invalid reference: no retry can fix it. + "InvalidImageName" => Resolved(Startup::NeverStartedProvablyBroken), + "ErrImagePull" | "ImagePullBackOff" => match classify_pull_failure(message) { + Some(failure) => Resolved(Startup::NeverStartedPullFailing(failure)), + None => Resolved(Startup::NeverStartedRecoverable), + }, + "CreateContainerConfigError" => match referenced_secret(pod) { + Some(secret_name) => { + StartupObservation::ConfigErrorPendingSecretCheck { secret_name } + } + None => Resolved(Startup::NeverStartedRecoverable), + }, + _ => Resolved(Startup::NeverStartedRecoverable), + }; + } + } + + // No container status yet (unscheduled, image pulling before the kubelet + // reports, quota-blocked). A terminal phase without container status still + // means the pod is done. + match phase { + Some("Succeeded") | Some("Failed") => Resolved(Startup::Terminated), + _ => Resolved(Startup::NeverStartedRecoverable), + } +} + +/// The Secret name this pod's `envFrom` references, if any. +pub fn referenced_secret(pod: &Pod) -> Option { + pod.spec + .as_ref()? + .containers + .iter() + .flat_map(|c| c.env_from.iter().flatten()) + .find_map(|source| source.secret_ref.as_ref().map(|r| r.name.clone())) +} + +/// Classify a pull failure from the kubelet's message. +/// +/// Reporting only — [`PullFailure`] is structurally excluded from +/// `Action::Delete`, so a wrong guess here can delay a report but can never +/// destroy anything. `None` means "no permanent cause recognized", which +/// leaves the pod on the ordinary observational path. +fn classify_pull_failure(message: &str) -> Option { + let m = message.to_ascii_lowercase(); + if m.contains("401") + || m.contains("unauthorized") + || m.contains("403") + || m.contains("denied") + || m.contains("authentication required") + { + return Some(PullFailure::Unauthorized); + } + if m.contains("manifest unknown") + || m.contains("not found") + || m.contains("manifest_unknown") + || m.contains("repository does not exist") + { + return Some(PullFailure::ManifestUnknown); + } + if m.contains("no match for platform") || m.contains("no matching manifest") { + return Some(PullFailure::ArchMismatch); + } + None +} + +/// The redacted, actionable condition text for a pull failure. +/// +/// Names the registry and the immutable reference — never credentials, and +/// never the kubelet's raw message, which can echo a registry token. +pub fn pull_failure_message(failure: PullFailure, image: &str) -> String { + let registry = image.split('/').next().unwrap_or(image); + match failure { + PullFailure::Unauthorized => format!( + "the cluster is not authorized to pull {image} from {registry}. \ + This pull retries indefinitely and will not succeed on its own: \ + grant the cluster's nodes access to that registry." + ), + PullFailure::ManifestUnknown => { + format!("{registry} has no image at {image}. Check the digest and repository.") + } + PullFailure::ArchMismatch => format!( + "{image} has no variant for the architecture of the nodes it was \ + scheduled on." + ), + } +} + +/// The latest actionable condition for a pod that has not started, redacted. +/// +/// Two sources, deliberately treated differently: +/// +/// * The container's waiting **reason** is included; its **message** is not. +/// Waiting messages are kubelet-composed and echo the thing that failed — +/// for a pull that is the registry request, which can carry credential +/// material. The reason token alone (`ImagePullBackOff`, +/// `CreateContainerConfigError`) is the diagnostic; the message adds +/// exposure, not information the user can act on. +/// * Pod-condition messages **are** included. They are scheduler- and +/// kubelet-composed from the pod's own spec and cluster capacity +/// ("0/3 nodes are available: Insufficient memory"), which is precisely the +/// actionable half and contains nothing derived from Secret data. +pub fn condition(pod: &Pod) -> Option { + let status = pod.status.as_ref()?; + + if let Some(state) = status + .container_statuses + .as_ref() + .and_then(|cs| cs.iter().find(|c| c.name == CONTAINER_NAME)) + .and_then(|c| c.state.as_ref()) + { + if let Some(waiting) = state.waiting.as_ref() { + if let Some(reason) = waiting.reason.as_deref() { + return Some(format!("the container is waiting, reason {reason}")); + } + } + // Exit code and reason only — the terminated `message` is + // process-composed output and falls under the same redaction rule as + // waiting messages. + if let Some(terminated) = state.terminated.as_ref() { + return Some(match terminated.reason.as_deref() { + Some(reason) => format!( + "the container exited with code {} ({reason})", + terminated.exit_code + ), + None => format!("the container exited with code {}", terminated.exit_code), + }); + } + } + + if let Some((type_, reason, message)) = status.conditions.as_ref().and_then(|cs| { + cs.iter().find(|c| c.status == "False").map(|c| { + ( + c.type_.clone(), + c.reason.clone().unwrap_or_default(), + c.message.clone().unwrap_or_default(), + ) + }) + }) { + let detail = [reason, message] + .into_iter() + .filter(|s| !s.is_empty()) + .collect::>() + .join(": "); + return Some(if detail.is_empty() { + format!("pod condition {type_} is false") + } else { + format!("pod condition {type_} is false: {detail}") + }); + } + + status.phase.as_deref().map(|p| format!("the pod is {p}")) +} + +/// Verify a label-selected pod and decode it, or reject it. +/// +/// `startup` is supplied by the caller because settling +/// `CreateContainerConfigError` needs a most-recent Secret read the pure layer +/// must not perform. +pub fn verify(pod: &Pod, identity: &AgentIdentity, startup: Startup) -> Option { + if !has_marker(pod.metadata.labels.as_ref()) { + return None; + } + if !annotation_matches(pod.metadata.annotations.as_ref(), identity) { + return None; + } + Some(VerifiedPod { + name: pod.metadata.name.clone()?, + fence: Fence { + uid: pod.metadata.uid.clone()?, + resource_version: pod.metadata.resource_version.clone()?, + }, + deletion_marked: pod.metadata.deletion_timestamp.is_some(), + startup, + recorded_intent: pod + .metadata + .annotations + .as_ref() + .and_then(|a| a.get(ANNOTATION_CREATE_INTENT)) + .map(|v| Fingerprint::from_annotation(v)), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use k8s_openapi::api::core::v1::{ + ContainerState, ContainerStateRunning, ContainerStateTerminated, ContainerStateWaiting, + ContainerStatus, PodStatus, + }; + use k8s_openapi::apimachinery::pkg::apis::meta::v1::{ObjectMeta, Time}; + use std::collections::BTreeMap; + + fn identity() -> AgentIdentity { + use nostr::nips::nip19::ToBech32; + let keys = nostr::Keys::generate(); + AgentIdentity::from_nsec(&keys.secret_key().to_bech32().unwrap()).unwrap() + } + + fn base_pod(id: &AgentIdentity) -> Pod { + Pod { + metadata: ObjectMeta { + name: Some(id.pod_name()), + uid: Some("uid-1".into()), + resource_version: Some("100".into()), + labels: Some(id.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + id.pubkey_hex().to_string(), + )] + .into_iter() + .collect::>(), + ), + ..Default::default() + }, + ..Default::default() + } + } + + fn with_container_state(mut pod: Pod, state: ContainerState) -> Pod { + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: CONTAINER_NAME.into(), + state: Some(state), + ..Default::default() + }]), + ..Default::default() + }); + pod + } + + fn waiting(reason: &str, message: &str) -> ContainerState { + ContainerState { + waiting: Some(ContainerStateWaiting { + reason: Some(reason.into()), + message: Some(message.into()), + }), + ..Default::default() + } + } + + #[test] + fn running_container_is_started() { + let id = identity(); + let pod = with_container_state( + base_pod(&id), + ContainerState { + running: Some(ContainerStateRunning::default()), + ..Default::default() + }, + ); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::Started) + ); + } + + /// Pod phase is not the criterion. A pod in phase `Running` whose + /// container never started must NOT read as started, or the reconciler + /// no-ops on a pod that will never serve. + #[test] + fn phase_running_with_waiting_container_is_not_started() { + let id = identity(); + let pod = with_container_state(base_pod(&id), waiting("ContainerCreating", "")); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::NeverStartedRecoverable) + ); + } + + #[test] + fn terminated_container_is_terminated() { + let id = identity(); + let pod = with_container_state( + base_pod(&id), + ContainerState { + terminated: Some(ContainerStateTerminated { + exit_code: 0, + ..Default::default() + }), + ..Default::default() + }, + ); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::Terminated) + ); + } + + /// A terminal phase with no container status (evicted before the kubelet + /// reported) is still terminated — otherwise the residue is never GC'd. + #[test] + fn terminal_phase_without_container_status_is_terminated() { + let id = identity(); + for phase in ["Succeeded", "Failed"] { + let mut pod = base_pod(&id); + pod.status = Some(PodStatus { + phase: Some(phase.into()), + ..Default::default() + }); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::Terminated), + "phase {phase}" + ); + } + } + + #[test] + fn invalid_image_name_is_provably_broken() { + let id = identity(); + let pod = with_container_state(base_pod(&id), waiting("InvalidImageName", "bad ref")); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::NeverStartedProvablyBroken) + ); + } + + /// Permanent pull failures are recognized from the message; anything + /// unrecognized stays on the ordinary observational path rather than + /// being guessed at. + #[test] + fn permanent_pull_failures_are_classified() { + let id = identity(); + let cases = [ + ("401 Unauthorized", PullFailure::Unauthorized), + ("pull access denied", PullFailure::Unauthorized), + ("manifest unknown", PullFailure::ManifestUnknown), + ( + "no match for platform in manifest", + PullFailure::ArchMismatch, + ), + ]; + for (message, expected) in cases { + let pod = with_container_state(base_pod(&id), waiting("ErrImagePull", message)); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::NeverStartedPullFailing(expected)), + "message {message:?}" + ); + } + + let pod = with_container_state( + base_pod(&id), + waiting("ImagePullBackOff", "dial tcp: i/o timeout"), + ); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::NeverStartedRecoverable), + "a transient network failure must not be reported as permanent" + ); + } + + /// The kubelet's reason string is a hint, not proof: a config error defers + /// to a most-recent Secret read before anything is called broken. + #[test] + fn config_error_defers_to_a_secret_read() { + let id = identity(); + let mut pod = + with_container_state(base_pod(&id), waiting("CreateContainerConfigError", "")); + pod.spec = Some(k8s_openapi::api::core::v1::PodSpec { + containers: vec![k8s_openapi::api::core::v1::Container { + name: CONTAINER_NAME.into(), + env_from: Some(vec![k8s_openapi::api::core::v1::EnvFromSource { + secret_ref: Some(k8s_openapi::api::core::v1::SecretEnvSource { + name: "buzz-agent-abc-gen1".into(), + optional: Some(false), + }), + ..Default::default() + }]), + ..Default::default() + }], + ..Default::default() + }); + assert_eq!( + decode_startup(&pod), + StartupObservation::ConfigErrorPendingSecretCheck { + secret_name: "buzz-agent-abc-gen1".into() + } + ); + } + + /// The auto-repair fence: an object that matches our schema but lacks the + /// marker, or carries someone else's pubkey, is never verified — so it can + /// never be no-op'd against, deleted, or returned as an `agent_id`. + #[test] + fn unmarked_or_mismatched_objects_fail_verification() { + let id = identity(); + let other = identity(); + + let mut unmarked = base_pod(&id); + unmarked.metadata.labels = Some(BTreeMap::new()); + assert!( + verify(&unmarked, &id, Startup::Started).is_none(), + "unmarked pod verified" + ); + + let mut wrong_version = base_pod(&id); + let mut labels = id.labels(); + labels.insert(LABEL_BINDING_VERSION.to_string(), "999".to_string()); + wrong_version.metadata.labels = Some(labels); + assert!(verify(&wrong_version, &id, Startup::Started).is_none()); + + let mut mismatched = base_pod(&id); + mismatched.metadata.annotations = Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + other.pubkey_hex().to_string(), + )] + .into_iter() + .collect(), + ); + assert!( + verify(&mismatched, &id, Startup::Started).is_none(), + "collision verified" + ); + + let mut missing = base_pod(&id); + missing.metadata.annotations = Some(BTreeMap::new()); + assert!(verify(&missing, &id, Startup::Started).is_none()); + + assert!( + verify(&base_pod(&id), &id, Startup::Started).is_some(), + "own pod rejected" + ); + } + + /// The fence must come from the observed object, and the deletion mark + /// must be read even though the phase says `Running`. + #[test] + fn verified_pod_carries_the_fence_and_deletion_mark() { + let id = identity(); + let mut pod = base_pod(&id); + pod.metadata.deletion_timestamp = Some(Time(chrono::Utc::now())); + let verified = verify(&pod, &id, Startup::Started).unwrap(); + assert_eq!(verified.fence.uid, "uid-1"); + assert_eq!(verified.fence.resource_version, "100"); + assert!(verified.deletion_marked); + } + + /// A pod with no recorded intent reads as `None`, which the classifier + /// groups with divergence. + #[test] + fn missing_intent_annotation_decodes_as_none() { + let id = identity(); + assert!(verify(&base_pod(&id), &id, Startup::Started) + .unwrap() + .recorded_intent + .is_none()); + } + + /// A pull-failure report must name the registry and the immutable + /// reference and nothing else — never the kubelet's raw message, which + /// can echo a registry token. + #[test] + fn pull_failure_messages_are_actionable_and_redacted() { + let image = format!("ghcr.io/block/buzz-sprig@sha256:{}", "a".repeat(64)); + for failure in [ + PullFailure::Unauthorized, + PullFailure::ManifestUnknown, + PullFailure::ArchMismatch, + ] { + let msg = pull_failure_message(failure, &image); + assert!(msg.contains("ghcr.io"), "{msg}"); + assert!(msg.contains(&image), "{msg}"); + for secret in ["Bearer", "password", "nsec1", "token"] { + assert!(!msg.contains(secret), "leaked {secret}: {msg}"); + } + } + } + + #[test] + fn secret_ownership_requires_marker_and_annotation() { + let id = identity(); + let other = identity(); + let ours = Secret { + metadata: ObjectMeta { + labels: Some(id.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + id.pubkey_hex().to_string(), + )] + .into(), + ), + ..Default::default() + }, + ..Default::default() + }; + assert!(secret_is_ours(&ours, &id)); + assert!(!secret_is_ours(&ours, &other)); + + let mut unmarked = ours.clone(); + unmarked.metadata.labels = Some(BTreeMap::new()); + assert!(!secret_is_ours(&unmarked, &id)); + } +} diff --git a/crates/buzz-backend-kubernetes/src/pod.rs b/crates/buzz-backend-kubernetes/src/pod.rs new file mode 100644 index 000000000..725f98fc7 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/pod.rs @@ -0,0 +1,446 @@ +//! Pod and Secret construction (spec §Pod shape, §K8s Secrets). +//! +//! The builder is pure: it turns resolved inputs into API objects and performs +//! no I/O, so every normative field is a unit assertion. + +use crate::config::{ + ProviderConfig, RESTART_POLICY, RUN_AS_GID, RUN_AS_UID, TERMINATION_GRACE_SECONDS, + WORKSPACE_PATH, +}; +use crate::intent::{Fingerprint, IntentTemplate}; +use crate::naming::{ + AgentIdentity, ANNOTATION_CREATE_INTENT, ANNOTATION_IMAGE, ANNOTATION_PUBKEY_FULL, +}; +use k8s_openapi::api::core::v1::{ + Capabilities, Container, EmptyDirVolumeSource, EnvFromSource, Pod, PodSecurityContext, PodSpec, + ResourceRequirements, SeccompProfile, Secret, SecretEnvSource, SecurityContext, Volume, + VolumeMount, +}; +use k8s_openapi::apimachinery::pkg::api::resource::Quantity; +use k8s_openapi::apimachinery::pkg::apis::meta::v1::ObjectMeta; +use std::collections::BTreeMap; + +/// Volume name for the agent's writable workspace. +const WORKSPACE_VOLUME: &str = "workspace"; + +/// The container name. Fixed: log and exec tooling addresses it by name. +const CONTAINER_NAME: &str = "agent"; + +/// Build the per-attempt Secret holding the resolved environment. +/// +/// `immutable: true` — the Secret is written once per attempt and never +/// updated, which is what lets the pod's `envFrom` reference be treated as an +/// atomic binding to this exact payload (§K8s Secrets). +pub fn build_secret( + identity: &AgentIdentity, + namespace: &str, + generation: &str, + env: BTreeMap, +) -> Secret { + Secret { + metadata: ObjectMeta { + name: Some(identity.secret_name(generation)), + namespace: Some(namespace.to_string()), + labels: Some(identity.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + identity.pubkey_hex().to_string(), + )] + .into_iter() + .collect(), + ), + ..Default::default() + }, + string_data: Some(env), + immutable: Some(true), + ..Default::default() + } +} + +/// Build the pod for one create attempt. +/// +/// The `fingerprint` is computed from [`intent_template`] over a type that +/// cannot contain the generation or any Secret value, so it is stable across +/// attempts of the same configuration. +pub fn build_pod( + identity: &AgentIdentity, + cfg: &ProviderConfig, + generation: &str, + fingerprint: &Fingerprint, +) -> Pod { + let annotations: BTreeMap = [ + ( + ANNOTATION_PUBKEY_FULL.to_string(), + identity.pubkey_hex().to_string(), + ), + ( + ANNOTATION_CREATE_INTENT.to_string(), + fingerprint.as_str().to_string(), + ), + (ANNOTATION_IMAGE.to_string(), cfg.image.as_str().to_string()), + ] + .into_iter() + .collect(); + + let requests: BTreeMap = [ + ( + "cpu".to_string(), + Quantity(cfg.resources.cpu_request.clone()), + ), + ( + "memory".to_string(), + Quantity(cfg.resources.memory_request.clone()), + ), + ] + .into_iter() + .collect(); + let limits: BTreeMap = [ + ("cpu".to_string(), Quantity(cfg.resources.cpu_limit.clone())), + ( + "memory".to_string(), + Quantity(cfg.resources.memory_limit.clone()), + ), + ] + .into_iter() + .collect(); + + let container = Container { + name: CONTAINER_NAME.to_string(), + image: Some(cfg.image.as_str().to_string()), + // No `command`/`args`: the image's entrypoint execs the harness as + // PID 1 (§Entrypoint). Overriding it here would be how a provider + // accidentally puts a shell in front of the signal receiver. + env_from: Some(vec![EnvFromSource { + secret_ref: Some(SecretEnvSource { + name: identity.secret_name(generation), + optional: Some(false), + }), + ..Default::default() + }]), + resources: Some(ResourceRequirements { + requests: Some(requests), + limits: Some(limits), + ..Default::default() + }), + volume_mounts: Some(vec![VolumeMount { + name: WORKSPACE_VOLUME.to_string(), + mount_path: WORKSPACE_PATH.to_string(), + ..Default::default() + }]), + security_context: Some(SecurityContext { + allow_privilege_escalation: Some(false), + capabilities: Some(Capabilities { + drop: Some(vec!["ALL".to_string()]), + ..Default::default() + }), + // `readOnlyRootFilesystem` is deliberately unset: the sprig + // toolchain writes outside the workspace mount (§Pod shape). + ..Default::default() + }), + ..Default::default() + }; + + Pod { + metadata: ObjectMeta { + name: Some(identity.pod_name()), + namespace: Some(cfg.namespace.clone()), + labels: Some(identity.labels()), + annotations: Some(annotations), + ..Default::default() + }, + spec: Some(PodSpec { + containers: vec![container], + restart_policy: Some(RESTART_POLICY.to_string()), + termination_grace_period_seconds: Some(TERMINATION_GRACE_SECONDS), + // The agent runs prompted, untrusted code while holding an nsec; + // an ambient ServiceAccount token would be an API-stealable + // credential it never needs (§Pod shape hardening). Naming a + // service account selects a scheduling/RBAC identity and MUST NOT + // re-enable token mounting. + automount_service_account_token: Some(false), + service_account_name: cfg.service_account.clone(), + security_context: Some(PodSecurityContext { + run_as_non_root: Some(true), + run_as_user: Some(RUN_AS_UID), + run_as_group: Some(RUN_AS_GID), + fs_group: Some(RUN_AS_GID), + seccomp_profile: Some(SeccompProfile { + type_: "RuntimeDefault".to_string(), + ..Default::default() + }), + ..Default::default() + }), + volumes: Some(vec![Volume { + name: WORKSPACE_VOLUME.to_string(), + empty_dir: Some(EmptyDirVolumeSource::default()), + ..Default::default() + }]), + ..Default::default() + }), + ..Default::default() + } +} + +/// The create-intent template for this configuration (§Deploy State Machine). +pub fn intent_template( + cfg: &ProviderConfig, + env_keys: impl IntoIterator, +) -> IntentTemplate { + IntentTemplate::new( + &cfg.namespace, + &cfg.image, + &cfg.resources, + cfg.service_account.as_deref(), + env_keys, + ) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::config; + + fn identity() -> AgentIdentity { + use nostr::nips::nip19::ToBech32; + let keys = nostr::Keys::generate(); + AgentIdentity::from_nsec(&keys.secret_key().to_bech32().unwrap()).unwrap() + } + + fn provider_config() -> ProviderConfig { + config::parse(&serde_json::json!({ + "namespace": "buzz-agents-test", + "image": format!("ghcr.io/block/buzz-sprig@sha256:{}", "a".repeat(64)), + })) + .unwrap() + } + + fn pod() -> Pod { + let cfg = provider_config(); + build_pod( + &identity(), + &cfg, + "gen00001", + &intent_template(&cfg, ["BUZZ_RELAY_URL".to_string()]).fingerprint(), + ) + } + + fn spec(pod: &Pod) -> &PodSpec { + pod.spec.as_ref().unwrap() + } + + /// Every hardening default from §Pod shape, asserted individually so a + /// dropped one names itself. + #[test] + fn hardening_defaults_are_all_present() { + let pod = pod(); + let spec = spec(&pod); + assert_eq!(spec.automount_service_account_token, Some(false)); + + let sc = spec + .security_context + .as_ref() + .expect("pod security context"); + assert_eq!(sc.run_as_non_root, Some(true)); + assert_eq!(sc.run_as_user, Some(RUN_AS_UID)); + assert_ne!(sc.run_as_user, Some(0), "root UID"); + assert_eq!(sc.run_as_group, Some(RUN_AS_GID)); + assert_eq!( + sc.seccomp_profile.as_ref().map(|p| p.type_.as_str()), + Some("RuntimeDefault") + ); + + let csc = spec.containers[0] + .security_context + .as_ref() + .expect("container sc"); + assert_eq!(csc.allow_privilege_escalation, Some(false)); + assert_eq!( + csc.capabilities.as_ref().and_then(|c| c.drop.clone()), + Some(vec!["ALL".to_string()]) + ); + assert_ne!(csc.privileged, Some(true)); + } + + /// The forbidden host-namespace and hostPath escapes, asserted as absence. + #[test] + fn never_uses_host_namespaces_or_host_paths() { + let pod = pod(); + let spec = spec(&pod); + assert!(spec.host_pid.is_none() || spec.host_pid == Some(false)); + assert!(spec.host_network.is_none() || spec.host_network == Some(false)); + assert!(spec.host_ipc.is_none() || spec.host_ipc == Some(false)); + for volume in spec.volumes.as_ref().unwrap() { + assert!( + volume.host_path.is_none(), + "hostPath volume {}", + volume.name + ); + } + } + + /// `Never` only. `OnFailure` is gated on the harness exit-code contract + /// *and* a crash-loop classification row (`:1121-1139`); the config layer + /// refuses `inactivity_seconds: 0` so this arm is unreachable, and the + /// assertion keeps it that way. + #[test] + fn restart_policy_is_never() { + assert_eq!(spec(&pod()).restart_policy.as_deref(), Some("Never")); + } + + /// 60s, not Kubernetes' default 30s — which would SIGKILL the harness + /// mid-drain and leave presence stale-online (§Pod shape). + #[test] + fn declares_the_sixty_second_grace_budget() { + assert_eq!(spec(&pod()).termination_grace_period_seconds, Some(60)); + } + + /// The pod must not override the image's entrypoint: the image execs the + /// harness as PID 1, and a `command` here is how a shell ends up in front + /// of the signal receiver (§Entrypoint). + #[test] + fn does_not_override_the_image_entrypoint() { + let pod = pod(); + let container = &spec(&pod).containers[0]; + assert!(container.command.is_none(), "overrode the entrypoint"); + assert!(container.args.is_none()); + } + + #[test] + fn workspace_is_an_emptydir_mounted_at_home() { + let pod = pod(); + let spec = spec(&pod); + let volume = &spec.volumes.as_ref().unwrap()[0]; + assert!(volume.empty_dir.is_some()); + assert!(volume.persistent_volume_claim.is_none()); + let mount = &spec.containers[0].volume_mounts.as_ref().unwrap()[0]; + assert_eq!(mount.name, volume.name); + assert_eq!(mount.mount_path, WORKSPACE_PATH); + } + + #[test] + fn resources_carry_the_configured_requests_and_limits() { + let mut cfg = provider_config(); + cfg.resources.cpu_limit = "4".into(); + let pod = build_pod(&identity(), &cfg, "g", &Fingerprint::from_annotation("f")); + let r = spec(&pod).containers[0].resources.as_ref().unwrap(); + assert_eq!(r.requests.as_ref().unwrap()["cpu"], Quantity("1".into())); + assert_eq!( + r.requests.as_ref().unwrap()["memory"], + Quantity("2Gi".into()) + ); + assert_eq!(r.limits.as_ref().unwrap()["cpu"], Quantity("4".into())); + assert_eq!(r.limits.as_ref().unwrap()["memory"], Quantity("4Gi".into())); + } + + /// `envFrom` must point at this attempt's Secret and must NOT be optional: + /// an optional reference starts the container with no identity at all, + /// turning a missing-Secret bug into an agent that silently cannot + /// authenticate. + #[test] + fn env_from_references_this_attempts_secret_and_is_required() { + let id = identity(); + let cfg = provider_config(); + let pod = build_pod(&id, &cfg, "gen00042", &Fingerprint::from_annotation("f")); + let source = &spec(&pod).containers[0].env_from.as_ref().unwrap()[0]; + let secret_ref = source.secret_ref.as_ref().unwrap(); + assert_eq!(secret_ref.name, id.secret_name("gen00042")); + assert_eq!(secret_ref.optional, Some(false)); + assert!(source.config_map_ref.is_none()); + } + + /// Identity, ownership marker, and the recorded intent all travel on the + /// pod — the GC and reconciliation fences read exactly these. + #[test] + fn pod_carries_identity_marker_and_recorded_intent() { + let id = identity(); + let cfg = provider_config(); + let fp = intent_template(&cfg, ["A".to_string()]).fingerprint(); + let pod = build_pod(&id, &cfg, "g", &fp); + let meta = &pod.metadata; + assert_eq!(meta.name.as_deref(), Some(id.pod_name().as_str())); + assert_eq!(meta.namespace.as_deref(), Some("buzz-agents-test")); + assert_eq!(meta.labels.as_ref().unwrap(), &id.labels()); + let ann = meta.annotations.as_ref().unwrap(); + assert_eq!(ann[ANNOTATION_PUBKEY_FULL], id.pubkey_hex()); + assert_eq!(ann[ANNOTATION_CREATE_INTENT], fp.as_str()); + assert_eq!(ann[ANNOTATION_IMAGE], cfg.image.as_str()); + } + + /// The Secret is immutable and marker-bearing: immutability is what makes + /// the pod's `envFrom` an atomic binding, and the marker is what GC + /// requires before it will delete anything. + #[test] + fn secret_is_immutable_marked_and_holds_the_env() { + let id = identity(); + let env: BTreeMap = + [("BUZZ_RELAY_URL".to_string(), "wss://r".to_string())].into(); + let secret = build_secret(&id, "ns", "gen1", env.clone()); + assert_eq!(secret.immutable, Some(true)); + assert_eq!(secret.string_data.as_ref().unwrap(), &env); + assert_eq!( + secret.metadata.name.as_deref(), + Some(id.secret_name("gen1").as_str()) + ); + assert_eq!(secret.metadata.labels.as_ref().unwrap(), &id.labels()); + assert_eq!( + secret.metadata.annotations.as_ref().unwrap()[ANNOTATION_PUBKEY_FULL], + id.pubkey_hex() + ); + // `data` must stay unset — setting both is an apiserver rejection. + assert!(secret.data.is_none()); + } + + /// Naming a service account selects a scheduling identity; it must not + /// re-enable token mounting (§Pod shape hardening, `:1221-1225`). + #[test] + fn service_account_does_not_re_enable_token_mounting() { + let mut cfg = provider_config(); + cfg.service_account = Some("agent-sa".into()); + let pod = build_pod(&identity(), &cfg, "g", &Fingerprint::from_annotation("f")); + assert_eq!(spec(&pod).service_account_name.as_deref(), Some("agent-sa")); + assert_eq!(spec(&pod).automount_service_account_token, Some(false)); + } + + /// The fingerprint recorded on the pod is the one the classifier will + /// recompute — pinned end-to-end so a builder change that forgets to feed + /// the template a field cannot pass silently. + #[test] + fn recorded_fingerprint_matches_a_fresh_computation() { + let cfg = provider_config(); + let keys = ["BUZZ_RELAY_URL".to_string(), "GOOSE_MODE".to_string()]; + let fp = intent_template(&cfg, keys.clone()).fingerprint(); + let pod = build_pod(&identity(), &cfg, "gen-a", &fp); + let recorded = Fingerprint::from_annotation( + &pod.metadata.annotations.as_ref().unwrap()[ANNOTATION_CREATE_INTENT], + ); + assert_eq!(recorded, intent_template(&cfg, keys).fingerprint()); + } + + /// Two attempts differing only in generation must record the *same* + /// fingerprint, or the divergence discriminator fires on every deploy and + /// the never-started row deletes healthy pending pods. + #[test] + fn generation_does_not_change_the_recorded_fingerprint() { + let cfg = provider_config(); + let keys = ["BUZZ_RELAY_URL".to_string()]; + let a = intent_template(&cfg, keys.clone()).fingerprint(); + let b = intent_template(&cfg, keys).fingerprint(); + let id = identity(); + let pod_a = build_pod(&id, &cfg, "gen-1", &a); + let pod_b = build_pod(&id, &cfg, "gen-2", &b); + let read = + |p: &Pod| p.metadata.annotations.as_ref().unwrap()[ANNOTATION_CREATE_INTENT].clone(); + assert_eq!(read(&pod_a), read(&pod_b)); + // ...while the Secret they reference differs. + let secret_of = |p: &Pod| { + spec(p).containers[0].env_from.as_ref().unwrap()[0] + .secret_ref + .as_ref() + .unwrap() + .name + .clone() + }; + assert_ne!(secret_of(&pod_a), secret_of(&pod_b)); + } +} diff --git a/crates/buzz-backend-kubernetes/src/reconcile.rs b/crates/buzz-backend-kubernetes/src/reconcile.rs new file mode 100644 index 000000000..df2f99789 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/reconcile.rs @@ -0,0 +1,1585 @@ +//! The deploy loop: executes [`crate::classify`]'s actions against a substrate +//! and re-enters (spec §Deploy State Machine). +//! +//! The substrate is a trait so the conformance tests drive this exact +//! reconciler — the shipped code path, not a test-only reimplementation — with +//! a fake cluster and a fake clock. +//! +//! Two shapes are worth naming up front, because they are what keep the loop +//! terminating: +//! +//! * **Success means the harness container started** (`:696-699`). There is no +//! "deployed but not confirmed" success: `deploy` returns an `agent_id` or an +//! in-band error carrying the latest condition. The wire has no third form. +//! * **A create-conflict loser never repairs.** It verifies the winner, drops +//! its own Secret, and *observes* until the winner starts. Applying the +//! divergence row to the pod that just beat it is exactly the ping-pong the +//! spec forbids (`:845-850`), and the escape it names is the *next* deploy, +//! not this one. + +use crate::classify::{self, Action, Fence, Startup, VerifiedPod}; +use crate::config::ProviderConfig; +use crate::gc; +use crate::naming::AgentIdentity; +use crate::observe::{self, StartupObservation}; +use chrono::{DateTime, Utc}; +use k8s_openapi::api::core::v1::{Pod, Secret}; +use std::collections::BTreeMap; +use std::time::Duration; + +/// Outcome of a create against the deterministic pod name. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum CreateOutcome { + Created, + /// 409 with `Status.reason: AlreadyExists` — a concurrent attempt won. + /// Discriminated on the typed reason, never on the HTTP code alone: 409 is + /// also `Conflict`, which means a failed precondition (`:780-794`). + AlreadyExists, +} + +/// Outcome of a fenced delete. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum DeleteOutcome { + Accepted, + /// Already gone. Delete-not-found is success (`:842`). + NotFound, + /// 409 with `Status.reason: Conflict` — the object changed since the + /// observation that authorized this delete. Neither an error nor + /// permission to retry: re-enter and classify what exists now + /// (`:775-777`). + PreconditionFailed, +} + +/// The cluster operations the reconciler needs. Everything here is I/O; +/// everything that decides is pure and lives in `classify`/`gc`. +#[allow(async_fn_in_trait)] +pub trait Substrate { + /// Create the namespace if absent. On RBAC denial the error MUST name the + /// literal `kubectl create namespace ` command and MUST NOT fall + /// back to `default` (`:1002-1005`). + async fn ensure_namespace(&self, namespace: &str) -> Result<(), String>; + + /// Most-recent read of the pods matching this identity's selector, plus + /// the apiserver's clock from the same call's HTTP `Date` header. `None` + /// clock means the header was absent or unparseable, which makes the + /// orphan-Secret sweep skip (`:1321-1335`). + async fn list_pods(&self, selector: &str) -> Result<(Vec, Option>), String>; + + async fn list_secrets(&self, selector: &str) -> Result, String>; + + /// Most-recent existence check (`resourceVersion` explicitly unset, not + /// `"0"`): the classifier treats a confirmed absence as proof, so a + /// possibly-stale cache read would be proof of nothing (`:761-769`). + async fn secret_exists(&self, name: &str) -> Result; + + async fn create_secret(&self, secret: &Secret) -> Result<(), String>; + + async fn create_pod(&self, pod: &Pod) -> Result; + + /// Compare-and-delete against the fence from the authorizing observation. + /// Uses the object's own grace period — never `grace_period_seconds: 0`, + /// which is a force-kill that discards the declared 60s shutdown budget + /// (`:1185-1189`). + async fn delete_pod(&self, name: &str, fence: &Fence) -> Result; + + /// Best-effort: the Secret may already be gone, which is success. + async fn delete_secret(&self, name: &str) -> Result<(), String>; + + /// Read one pod by name, most-recent. `None` is a confirmed absence. + async fn get_pod(&self, name: &str) -> Result, String>; + + async fn sleep(&self, duration: Duration); + + /// Monotonic elapsed time since the operation began. Fake-clock driven in + /// tests; the deadline must not depend on wall-clock adjustments. + fn elapsed(&self) -> Duration; +} + +/// Interval between reconciler polls. Short enough that a fast start is +/// reported promptly, long enough not to hammer the apiserver for 600s. +const POLL_INTERVAL: Duration = Duration::from_secs(2); + +/// The deploy operation deadline (spec §Deploy: `timeout: 600s`). +const DEADLINE: Duration = Duration::from_secs(gc::OPERATION_DEADLINE_SECS as u64); + +/// Settle a pod's startup state, performing the one most-recent Secret read +/// that `CreateContainerConfigError` requires. +async fn settle(substrate: &impl Substrate, pod: &Pod) -> Result { + Ok(match observe::decode_startup(pod) { + StartupObservation::Resolved(startup) => startup, + StartupObservation::ConfigErrorPendingSecretCheck { secret_name } => { + // A confirmed absence is proof; anything else stays recoverable, + // because the kubelet's reason string alone is not evidence. + if substrate.secret_exists(&secret_name).await? { + Startup::NeverStartedRecoverable + } else { + Startup::NeverStartedProvablyBroken + } + } + }) +} + +/// Observe the single pod owned by this identity, verified. +/// +/// `Ok(None)` conflates "absent" with "present but not ours" on purpose *here* +/// — both mean the classifier has nothing it may act on. The create path +/// separates them, because only there is the difference actionable. +async fn observe_pod( + substrate: &impl Substrate, + identity: &AgentIdentity, +) -> Result, String> { + let Some(pod) = substrate.get_pod(&identity.pod_name()).await? else { + return Ok(None); + }; + let startup = settle(substrate, &pod).await?; + Ok(observe::verify(&pod, identity, startup)) +} + +/// The latest condition to report, read fresh at the moment of reporting. +/// +/// Reading it here rather than threading it through every loop iteration is +/// what "the *latest* redacted condition" (`:689`) asks for, and it costs a +/// read only on the paths that are already failing. +async fn latest_condition(substrate: &impl Substrate, identity: &AgentIdentity) -> String { + match substrate.get_pod(&identity.pod_name()).await { + Ok(Some(pod)) => observe::condition(&pod) + .unwrap_or_else(|| "no condition reported by the cluster".to_string()), + Ok(None) => "the pod no longer exists".to_string(), + Err(e) => format!("the pod's condition could not be read: {e}"), + } +} + +/// Preflight GC (§K8s GC). Failures are logged and swallowed: GC is hygiene, +/// and a deploy must not fail because a stale object could not be listed or +/// removed. A denial that actually blocks this deploy resurfaces at create, +/// where the message names the operation the user was denied. +async fn preflight_gc(substrate: &impl Substrate, identity: &AgentIdentity) { + if let Err(e) = try_preflight_gc(substrate, identity).await { + eprintln!("gc: preflight pass skipped: {e}"); + } +} + +async fn try_preflight_gc( + substrate: &impl Substrate, + identity: &AgentIdentity, +) -> Result<(), String> { + let selector = identity.selector(); + let (pods, server_now) = substrate.list_pods(&selector).await?; + let secrets = substrate.list_secrets(&selector).await?; + + let mut terminated: Vec = Vec::new(); + for pod in &pods { + if matches!(settle(substrate, pod).await?, Startup::Terminated) { + if let Some(name) = pod.metadata.name.clone() { + terminated.push(name); + } + } + } + + let plan = gc::plan( + identity, + &pods, + &secrets, + |pod| { + pod.metadata + .name + .as_deref() + .map(|n| terminated.iter().any(|t| t == n)) + .unwrap_or(false) + }, + server_now, + ); + + for name in &plan.pods { + // Re-read to fence the delete against the object we just observed; a + // pod that changed since the list is simply skipped this pass. + let Some(pod) = substrate.get_pod(name).await? else { + continue; + }; + let (Some(uid), Some(rv)) = ( + pod.metadata.uid.clone(), + pod.metadata.resource_version.clone(), + ) else { + continue; + }; + if !matches!(settle(substrate, &pod).await?, Startup::Terminated) { + continue; + } + let fence = Fence { + uid, + resource_version: rv, + }; + if let Err(e) = substrate.delete_pod(name, &fence).await { + eprintln!("gc: could not delete terminated pod {name}: {e}"); + } + } + for name in &plan.secrets { + if let Err(e) = substrate.delete_secret(name).await { + eprintln!("gc: could not delete secret {name}: {e}"); + } + } + Ok(()) +} + +/// Wait for a pod to actually disappear. +/// +/// Mandatory before recreating: `DELETE` returns success while the object +/// still exists, and the deterministic name stays taken for the whole grace +/// period (`:1177-1195`). +async fn await_disappearance(substrate: &impl Substrate, name: &str) -> Result<(), String> { + while substrate.elapsed() < DEADLINE { + if substrate.get_pod(name).await?.is_none() { + return Ok(()); + } + substrate.sleep(POLL_INTERVAL).await; + } + Err(format!( + "timed out after {}s waiting for {name} to finish terminating", + DEADLINE.as_secs() + )) +} + +/// Is `secret` referenced by any pod that currently exists under this +/// identity's selector? +/// +/// Protection deliberately spans *all* our pods, not just the winner: an +/// `envFrom` reference from a pod still pulling its image is exactly as +/// load-bearing as one from a running pod (`:1261-1264`). +async fn secret_is_referenced( + substrate: &impl Substrate, + identity: &AgentIdentity, + secret: &str, +) -> Result { + let (pods, _) = substrate.list_pods(&identity.selector()).await?; + Ok(pods + .iter() + .filter_map(observe::referenced_secret) + .any(|name| name == secret)) +} + +/// Drop this attempt's own Secret once nothing references it. +/// +/// Only ever called with a name this process generated, and gated on the +/// reference check: "never the winner's, never any Secret referenced by an +/// existing pod" (`:1259-1264`). Failure is logged, not fatal — a leaked +/// Secret is collected by the age-gated sweep. +async fn drop_own_secret(substrate: &impl Substrate, identity: &AgentIdentity, secret: &str) { + match secret_is_referenced(substrate, identity, secret).await { + Ok(false) => { + if let Err(e) = substrate.delete_secret(secret).await { + eprintln!("could not clean up own unreferenced secret {secret}: {e}"); + } + } + Ok(true) => {} + Err(e) => eprintln!("could not check whether {secret} is still referenced: {e}"), + } +} + +/// Lost the create race: adopt the elected winner. +/// +/// Observe-only by construction — this function has no delete edge for the +/// pod. A winner that is terminated or provably broken is reported, not +/// repaired; the spec's escape is "a *subsequent* deploy that walks in and +/// observes that never-started divergent winner replaces it normally" +/// (`:849-850`). +async fn adopt_winner( + substrate: &impl Substrate, + identity: &AgentIdentity, + own_secret: &str, +) -> Result { + let name = identity.pod_name(); + + // Verify before adopting. A pod under our deterministic name that fails + // the marker/annotation check is not ours to adopt, wait for, or touch — + // and it will never become ours, so this is terminal rather than a retry. + match substrate.get_pod(&name).await? { + None => {} + Some(pod) => { + let startup = settle(substrate, &pod).await?; + if observe::verify(&pod, identity, startup).is_none() { + drop_own_secret(substrate, identity, own_secret).await; + return Err(format!( + "a pod named {name} already exists in this namespace but is not \ + managed by this provider for this agent (it lacks the management \ + marker or carries a different agent identity). Remove it, or \ + deploy this agent to a different namespace." + )); + } + } + } + + drop_own_secret(substrate, identity, own_secret).await; + + // Then wait for the winner exactly as we would wait for our own pod: + // success still means the harness container started. + loop { + if substrate.elapsed() >= DEADLINE { + return Err(format!( + "startup not confirmed within {}s for {name} (another deploy of this \ + agent created it): {}", + DEADLINE.as_secs(), + latest_condition(substrate, identity).await + )); + } + match observe_pod(substrate, identity).await? { + Some(pod) if matches!(pod.startup, Startup::Started) => return Ok(pod.name), + Some(pod) if pod.deletion_marked => { + return Err(format!( + "{name} was created by another deploy of this agent and is already \ + being deleted; try again" + )) + } + Some(pod) + if matches!( + pod.startup, + Startup::Terminated | Startup::NeverStartedProvablyBroken + ) => + { + return Err(format!( + "{name} was created by another deploy of this agent and did not \ + start: {}", + latest_condition(substrate, identity).await + )) + } + // Gone again, or still coming up: keep observing under this + // operation's deadline. + _ => substrate.sleep(POLL_INTERVAL).await, + } + } +} + +/// Run the deploy state machine to a terminal outcome: the started pod's name, +/// or an in-band error carrying the latest condition. +pub async fn deploy( + substrate: &impl Substrate, + identity: &AgentIdentity, + cfg: &ProviderConfig, + env: BTreeMap, +) -> Result { + substrate.ensure_namespace(&cfg.namespace).await?; + preflight_gc(substrate, identity).await; + + let desired = crate::pod::intent_template(cfg, env.keys().cloned()).fingerprint(); + + // Has THIS call created a pod? Set once its create lands. The replacement + // rows below are for residue from a previous life; once this call has made + // its own attempt, a replace-classification means that attempt failed — + // and startup verification is part of create, so the failure is reported + // in-band rather than retried. Without this bound a deterministic startup + // failure (the harness starts, rejects its configuration, exits) is + // delete-recreated every poll for the whole deadline, minting an immutable + // Secret per cycle — measured live at 107 Secrets in one 600s call, every + // one younger than the orphan sweep's age gate. + let mut created_this_call = false; + + loop { + if substrate.elapsed() >= DEADLINE { + return Err(format!( + "startup not confirmed within {}s for {}: {}", + DEADLINE.as_secs(), + identity.pod_name(), + latest_condition(substrate, identity).await + )); + } + + let observed = observe_pod(substrate, identity).await?; + match classify::classify(observed.as_ref(), &desired) { + // The only success edge: the harness container is running. + Action::NoOp { agent_id } => return Ok(agent_id), + + // Self-healing states. Never delete, on this call or any later one + // — what replaces a never-started pod is a config change, never a + // deadline (`:717-729`). + Action::Observe { .. } => substrate.sleep(POLL_INTERVAL).await, + + // A pull that will not self-heal: report now rather than spend the + // remaining deadline on it. Still no delete authority. + Action::Report { name, failure } => { + return Err(format!( + "{name} did not start: {}", + observe::pull_failure_message(failure, cfg.image.as_str()) + )) + } + + Action::AwaitDisappearance { name } => await_disappearance(substrate, &name).await?, + + Action::Delete { name, fence } => { + // This call already made its own attempt, and that attempt is + // what the classification wants replaced: it terminated (the + // deterministic startup failure — the harness starts, rejects + // its configuration, exits) or was proven broken. Replacing it + // here retries the identical configuration against the same + // cluster: a hot delete/mint/create cycle every poll for the + // whole deadline, an immutable Secret per cycle — measured + // live at 107 Secrets in one 600s call, all younger than the + // orphan sweep's age gate. Report in-band instead. The residue + // is deliberate: the next Start's preflight GC collects the + // terminated pod and its referenced Secret together, so retry + // is gated on fresh owner intent and litter stays bounded at + // one pod + one Secret per press. + if created_this_call { + return Err(format!( + "{name} was created by this deploy and did not stay \ + running: {}. Not retrying in this call — an immediate \ + exit recurs until its cause is fixed. Check the \ + agent's configuration and press Start to try again.", + latest_condition(substrate, identity).await + )); + } + match substrate.delete_pod(&name, &fence).await? { + // Accepted or already gone: both need the disappearance + // poll before the name is free again. + DeleteOutcome::Accepted | DeleteOutcome::NotFound => { + await_disappearance(substrate, &name).await? + } + // The object changed since the observation that authorized + // this delete. Discard the action and re-classify — never + // retry with a fresher fence, which would delete something + // we never examined. Sleep before re-entering: the losing + // race is against another writer, and re-reading at full + // speed is a busy-retry with no better odds than a paced + // one. + DeleteOutcome::PreconditionFailed => substrate.sleep(POLL_INTERVAL).await, + } + } + + Action::Create => { + let generation = crate::naming::new_generation(); + let secret_name = identity.secret_name(&generation); + + // The generation is minted *per attempt*, and it is two things + // at once: the Secret's name suffix and the lifecycle + // correlator the harness reports. `build_env` stamped the + // caller's generation, so on any attempt after the first the + // two would name different generations — pod logs correlating + // to a Secret that is not the one mounted. Restamp so there is + // exactly one generation per attempt (§K8s Secrets). + let mut env = env.clone(); + env.insert(crate::env::START_NONCE_KEY.to_string(), generation.clone()); + + // Secret first: the pod's spec references this exact name, so + // payload and Secret are atomic at the pod-spec boundary. + let secret = crate::pod::build_secret(identity, &cfg.namespace, &generation, env); + substrate.create_secret(&secret).await?; + + let pod = crate::pod::build_pod(identity, cfg, &generation, &desired); + match substrate.create_pod(&pod).await? { + // Re-enter rather than wait inline: the next iteration + // observes what we just created and runs the same rows + // every other state runs through. One loop, one table. + // + // Sleep first. A just-created pod cannot already be + // started, so the immediate observation has no outcome but + // "still coming up" — and if it ever came back + // unverifiable, re-entering without advancing the clock + // would hot-spin creates against the apiserver for the + // whole deadline. + CreateOutcome::Created => { + created_this_call = true; + substrate.sleep(POLL_INTERVAL).await + } + CreateOutcome::AlreadyExists => { + return adopt_winner(substrate, identity, &secret_name).await + } + } + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::config::Resources; + use crate::naming::{ANNOTATION_CREATE_INTENT, ANNOTATION_PUBKEY_FULL, LABEL_MANAGED_BY}; + use k8s_openapi::api::core::v1::{ + ContainerState, ContainerStateRunning, ContainerStateTerminated, ContainerStateWaiting, + ContainerStatus, PodStatus, + }; + use k8s_openapi::apimachinery::pkg::apis::meta::v1::Time; + use std::cell::RefCell; + use std::future::Future; + use std::pin::pin; + use std::task::{Context, Poll, RawWaker, RawWakerVTable, Waker}; + + /// Mutates the pod map after a poll, so a test can script a pod that + /// starts (or vanishes) partway through an observation loop. + type PollHook = Box)>; + + /// A scripted cluster. Single-threaded on purpose: the reconciler is one + /// process per operation, and `RefCell` keeps the assertions readable. + /// + /// This drives the *shipped* `deploy` — the point of the `Substrate` seam. + /// Every fake here answers with real `k8s_openapi` objects, so the decode + /// and verification layers under test are the ones that run in a cluster. + #[derive(Default)] + struct Fake { + pods: RefCell>, + secrets: RefCell>, + /// Server clock for the GC age gate; `None` models a missing `Date`. + server_now: Option>, + /// Elapsed time, advanced only by `sleep` — a fake clock, so a 600s + /// deadline test runs instantly and cannot flake on a slow machine. + elapsed: RefCell, + /// Queued create outcomes; the default is `Created`. + create_outcomes: RefCell>, + /// Installed when a create loses the race. A winner must be *absent* + /// at the observation that decides to create and *present* by the time + /// the create lands — pre-seeding it instead makes the loop no-op + /// before it ever reaches the create edge. + winner: RefCell>, + /// Every Secret ever created, retained across deletion. + created_secrets: RefCell>, + /// Queued delete outcomes; the default is `Accepted`. + delete_outcomes: RefCell>, + /// Every mutating call, in order — the anti-mutation assertions read + /// this rather than guessing from final state. + calls: RefCell>, + /// Applied to the pod map after each poll, so a test can script a pod + /// that starts (or vanishes) partway through an observation loop. + on_poll: RefCell>, + /// `ensure_namespace` fails with this, if set. + namespace_error: Option, + } + + impl Fake { + fn with_pod(self, pod: Pod) -> Self { + self.pods + .borrow_mut() + .insert(pod.metadata.name.clone().unwrap(), pod); + self + } + fn log(&self, entry: impl Into) { + self.calls.borrow_mut().push(entry.into()); + } + fn mutations(&self) -> Vec { + self.calls.borrow().clone() + } + } + + impl Substrate for Fake { + async fn ensure_namespace(&self, namespace: &str) -> Result<(), String> { + match &self.namespace_error { + Some(e) => Err(e.clone()), + None => { + self.log(format!("ensure_namespace {namespace}")); + Ok(()) + } + } + } + + async fn list_pods( + &self, + _selector: &str, + ) -> Result<(Vec, Option>), String> { + Ok(( + self.pods.borrow().values().cloned().collect(), + self.server_now, + )) + } + + async fn list_secrets(&self, _selector: &str) -> Result, String> { + Ok(self.secrets.borrow().clone()) + } + + async fn secret_exists(&self, name: &str) -> Result { + Ok(self + .secrets + .borrow() + .iter() + .any(|s| s.metadata.name.as_deref() == Some(name))) + } + + async fn create_secret(&self, secret: &Secret) -> Result<(), String> { + let name = secret.metadata.name.clone().unwrap(); + self.log(format!("create_secret {name}")); + self.secrets.borrow_mut().push(secret.clone()); + // Kept even after the Secret is deleted: assertions about what an + // attempt *wrote* must not be silently vacuous once cleanup runs. + self.created_secrets.borrow_mut().push(secret.clone()); + Ok(()) + } + + async fn create_pod(&self, pod: &Pod) -> Result { + let name = pod.metadata.name.clone().unwrap(); + self.log(format!("create_pod {name}")); + let outcome = if self.create_outcomes.borrow().is_empty() { + CreateOutcome::Created + } else { + self.create_outcomes.borrow_mut().remove(0) + }; + if outcome == CreateOutcome::Created { + // The apiserver stamps these on admission; a builder never + // carries them. Without them the pod fails `verify`'s fence + // extraction and the loop can never see what it just created. + let mut pod = pod.clone(); + pod.metadata.uid = Some(format!("uid-created-{}", self.calls.borrow().len())); + pod.metadata.resource_version = Some("1".into()); + self.pods.borrow_mut().insert(name, pod); + } else if let Some(winner) = self.winner.borrow_mut().take() { + // The concurrent attempt's pod becomes visible exactly when our + // create is rejected — the ordering a real race produces. + self.pods.borrow_mut().insert(name, winner); + } + Ok(outcome) + } + + async fn delete_pod(&self, name: &str, fence: &Fence) -> Result { + self.log(format!( + "delete_pod {name} uid={} rv={}", + fence.uid, fence.resource_version + )); + let outcome = if self.delete_outcomes.borrow().is_empty() { + DeleteOutcome::Accepted + } else { + self.delete_outcomes.borrow_mut().remove(0) + }; + if outcome == DeleteOutcome::Accepted { + self.pods.borrow_mut().remove(name); + } + Ok(outcome) + } + + async fn delete_secret(&self, name: &str) -> Result<(), String> { + self.log(format!("delete_secret {name}")); + self.secrets + .borrow_mut() + .retain(|s| s.metadata.name.as_deref() != Some(name)); + Ok(()) + } + + async fn get_pod(&self, name: &str) -> Result, String> { + Ok(self.pods.borrow().get(name).cloned()) + } + + async fn sleep(&self, duration: Duration) { + *self.elapsed.borrow_mut() += duration; + let hooks = self.on_poll.borrow(); + let mut pods = self.pods.borrow_mut(); + for hook in hooks.iter() { + hook(&mut pods); + } + } + + fn elapsed(&self) -> Duration { + *self.elapsed.borrow() + } + } + + fn identity() -> AgentIdentity { + use nostr::nips::nip19::ToBech32; + let keys = nostr::Keys::generate(); + AgentIdentity::from_nsec(&keys.secret_key().to_bech32().unwrap()).unwrap() + } + + fn config() -> ProviderConfig { + ProviderConfig { + context: None, + namespace: "buzz-agents-test".into(), + image: crate::image::parse(&format!( + "ghcr.io/block/buzz-sprig@sha256:{}", + "a".repeat(64) + )) + .unwrap(), + resources: Resources::default(), + inactivity_seconds: Some(7200), + service_account: None, + } + } + + fn env() -> BTreeMap { + [("BUZZ_RELAY_URL".to_string(), "wss://r".to_string())] + .into_iter() + .collect() + } + + /// A pod exactly as this provider would have created it — same builder the + /// reconciler uses, so verification is exercised rather than bypassed. + fn our_pod(id: &AgentIdentity, cfg: &ProviderConfig, state: Option) -> Pod { + let fp = crate::pod::intent_template(cfg, env().keys().cloned()).fingerprint(); + let mut pod = crate::pod::build_pod(id, cfg, "gen-existing", &fp); + pod.metadata.uid = Some("uid-1".into()); + pod.metadata.resource_version = Some("100".into()); + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: state.map(|s| { + vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(s), + ..Default::default() + }] + }), + ..Default::default() + }); + pod + } + + fn running() -> ContainerState { + ContainerState { + running: Some(ContainerStateRunning::default()), + ..Default::default() + } + } + + fn terminated() -> ContainerState { + ContainerState { + terminated: Some(ContainerStateTerminated::default()), + ..Default::default() + } + } + + fn waiting(reason: &str) -> ContainerState { + ContainerState { + waiting: Some(ContainerStateWaiting { + reason: Some(reason.into()), + message: None, + }), + ..Default::default() + } + } + + fn run(fake: &Fake, id: &AgentIdentity, cfg: &ProviderConfig) -> Result { + block_on(deploy(fake, id, cfg, env())) + } + + /// Minimal executor: the fake never yields to a reactor (no timers, no + /// I/O — `sleep` just advances a counter), so polling to completion is + /// sufficient and avoids pulling a runtime into the unit job. + fn block_on(fut: impl Future) -> T { + fn noop_waker() -> Waker { + fn nop(_: *const ()) {} + fn clone(_: *const ()) -> RawWaker { + RawWaker::new(std::ptr::null(), &VTABLE) + } + static VTABLE: RawWakerVTable = RawWakerVTable::new(clone, nop, nop, nop); + unsafe { Waker::from_raw(RawWaker::new(std::ptr::null(), &VTABLE)) } + } + + let waker = noop_waker(); + let mut cx = Context::from_waker(&waker); + let mut fut = pin!(fut); + match fut.as_mut().poll(&mut cx) { + Poll::Ready(v) => v, + Poll::Pending => panic!("fake substrate future parked — it has no reactor"), + } + } + + // ---- the state machine's rows, end to end ------------------------------- + + /// First deploy: Secret before pod, and the returned id is the pod name. + #[test] + fn creates_secret_then_pod_and_returns_the_pod_name() { + let id = identity(); + let cfg = config(); + let fake = Fake::default(); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + if let Some(pod) = pods.get_mut(&name) { + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]), + ..Default::default() + }); + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + + let calls = fake.mutations(); + let secret_at = calls + .iter() + .position(|c| c.starts_with("create_secret")) + .unwrap(); + let pod_at = calls + .iter() + .position(|c| c.starts_with("create_pod")) + .unwrap(); + assert!( + secret_at < pod_at, + "pod created before its Secret: {calls:?}" + ); + } + + /// The strict no-op row: a started pod returns its id having mutated + /// nothing at all. Asserted on the *call log*, not on final state — a + /// delete-then-recreate would leave identical final state. + #[test] + fn started_pod_is_a_zero_mutation_no_op() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(running()))); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert_eq!( + fake.mutations(), + [format!("ensure_namespace {}", cfg.namespace)], + "the no-op row mutated something" + ); + } + + /// ...including when the desired intent has diverged. Edits reach a + /// started pod only via the next generation (`:861-865`). + #[test] + fn started_pod_no_ops_under_divergent_intent() { + let id = identity(); + let cfg = config(); + let mut pod = our_pod(&id, &cfg, Some(running())); + pod.metadata.annotations.as_mut().unwrap().insert( + ANNOTATION_CREATE_INTENT.to_string(), + "stale-fingerprint".into(), + ); + let fake = Fake::default().with_pod(pod); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "divergence deleted a started pod" + ); + } + + /// Terminated → fenced delete → disappearance → recreate. The normal + /// restart path, and the fence must carry the observed uid/rv. + #[test] + fn terminated_pod_is_replaced_with_a_fenced_delete() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(terminated()))); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + if let Some(pod) = pods.get_mut(&name) { + if pod + .status + .as_ref() + .and_then(|s| s.container_statuses.as_ref()) + .is_none() + { + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]), + ..Default::default() + }); + } + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + let calls = fake.mutations(); + assert!( + calls + .iter() + .any(|c| c == &format!("delete_pod {} uid=uid-1 rv=100", id.pod_name())), + "delete was not fenced to the observed uid+resourceVersion: {calls:?}" + ); + assert!(calls.iter().any(|c| c.starts_with("create_pod"))); + } + + /// A failed precondition is neither an error nor permission to retry the + /// delete: re-enter and classify what exists now (`:775-777`). Here the + /// object has become a *started* pod, so the correct outcome is the no-op + /// row — never a second delete with a fresher fence. + #[test] + fn precondition_failure_reclassifies_instead_of_retrying() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(terminated()))); + fake.delete_outcomes + .borrow_mut() + .push(DeleteOutcome::PreconditionFailed); + // The writer we lost the race to: between our observation and our + // delete, the pod under this name became a *started* one with a new + // resourceVersion. Installed on the poll that follows the failed + // precondition, so the sequence is observe → delete → lose → re-observe. + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + let started = { + let mut p = our_pod(&id, &cfg, Some(running())); + p.metadata.resource_version = Some("200".into()); + p + }; + Box::new(move |pods: &mut BTreeMap| { + pods.insert(name.clone(), started.clone()); + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + let deletes: Vec<_> = fake + .mutations() + .into_iter() + .filter(|c| c.starts_with("delete_pod")) + .collect(); + // Two call sites legitimately target a terminated pod: preflight GC's + // sweep (which is the one that loses the precondition here) and the + // state machine's terminated row. Both are fenced on their own + // observation. What must never happen is a *third* — a retry of the + // failed delete — so the invariant is the fence, not the count: every + // delete carries rv=100, the version we observed. A retry would carry + // the racing writer's rv=200. + assert_eq!(deletes.len(), 2, "unexpected delete traffic: {deletes:?}"); + assert!( + deletes.iter().all(|d| d.contains("rv=100")), + "retried with a fresher fence: {deletes:?}" + ); + } + + /// The anti-livelock rule, at the loop level: a recoverable pod whose + /// intent matches is observed until the deadline and **never** deleted — + /// the case that would otherwise reset the pod age Cluster Autoscaler + /// keys on (`:689`). + #[test] + fn recoverable_pod_with_matching_intent_is_never_deleted() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(waiting("Unschedulable")))); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("startup not confirmed"), "got: {err}"); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "deleted a recoverable pod: {:?}", + fake.mutations() + ); + assert!(fake.elapsed() >= DEADLINE, "gave up before the deadline"); + } + + /// The same pod, provisioned late: the observation loop must *succeed* + /// when the autoscaler eventually lands the node, not merely avoid + /// deleting. + #[test] + fn recoverable_pod_that_starts_late_succeeds() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(waiting("Unschedulable")))); + let polls = RefCell::new(0); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + *polls.borrow_mut() += 1; + if *polls.borrow() >= 5 { + if let Some(pod) = pods.get_mut(&name) { + pod.status.as_mut().unwrap().container_statuses = + Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]); + } + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!(fake.elapsed() < DEADLINE); + } + + /// A permanent pull failure reports immediately rather than burning the + /// deadline — and still never deletes. + #[test] + fn permanent_pull_failure_reports_without_deleting() { + let id = identity(); + let cfg = config(); + let mut pod = our_pod(&id, &cfg, None); + pod.status = Some(PodStatus { + phase: Some("Pending".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(ContainerState { + waiting: Some(ContainerStateWaiting { + reason: Some("ImagePullBackOff".into()), + message: Some("manifest unknown".into()), + }), + ..Default::default() + }), + ..Default::default() + }]), + ..Default::default() + }); + let fake = Fake::default().with_pod(pod); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("no image at"), "got: {err}"); + assert!( + fake.elapsed() < DEADLINE, + "burned the deadline on a permanent failure" + ); + assert!(!fake.mutations().iter().any(|c| c.starts_with("delete_pod"))); + } + + /// The kubelet's pull *message* can echo a registry request; only the + /// classified outcome and the image reference reach the user. + #[test] + fn pull_failure_message_is_not_echoed_verbatim() { + let id = identity(); + let cfg = config(); + let mut pod = our_pod(&id, &cfg, None); + pod.status = Some(PodStatus { + phase: Some("Pending".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(ContainerState { + waiting: Some(ContainerStateWaiting { + reason: Some("ErrImagePull".into()), + message: Some( + "unauthorized: authentication required, token=SUPERSECRET".into(), + ), + }), + ..Default::default() + }), + ..Default::default() + }]), + ..Default::default() + }); + let fake = Fake::default().with_pod(pod); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!( + !err.contains("SUPERSECRET"), + "kubelet message echoed: {err}" + ); + } + + /// A pod being gracefully deleted stays in phase `Running` for its whole + /// grace period. The reconciler must wait it out and recreate, not return + /// an id that evaporates. + #[test] + fn deletion_marked_pod_is_awaited_then_recreated() { + let id = identity(); + let cfg = config(); + let mut dying = our_pod(&id, &cfg, Some(running())); + dying.metadata.deletion_timestamp = Some(Time(Utc::now())); + let fake = Fake::default().with_pod(dying); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + match pods + .get(&name) + .map(|p| p.metadata.deletion_timestamp.is_some()) + { + // The grace period elapses: the object disappears. + Some(true) => { + pods.remove(&name); + } + // The replacement we create then starts. + Some(false) => { + let pod = pods.get_mut(&name).unwrap(); + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]), + ..Default::default() + }); + } + None => {} + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "deleted a pod that was already terminating" + ); + } + + /// `CreateContainerConfigError` is settled by a most-recent Secret read, + /// never by the reason string: Secret present → recoverable (observe). + #[test] + fn config_error_with_a_present_secret_is_recoverable() { + let id = identity(); + let cfg = config(); + let pod = our_pod(&id, &cfg, Some(waiting("CreateContainerConfigError"))); + let referenced = crate::observe::referenced_secret(&pod).unwrap(); + let fake = Fake::default().with_pod(pod); + fake.secrets.borrow_mut().push(crate::pod::build_secret( + &id, + &cfg.namespace, + "gen-existing", + env(), + )); + assert_eq!(referenced, id.secret_name("gen-existing")); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("startup not confirmed"), "got: {err}"); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "deleted a pod whose Secret exists" + ); + } + + /// ...and Secret confirmed absent → provably broken → fenced replace. + #[test] + fn config_error_with_a_confirmed_absent_secret_is_replaced() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod( + &id, + &cfg, + Some(waiting("CreateContainerConfigError")), + )); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + if let Some(pod) = pods.get_mut(&name) { + if pod + .status + .as_ref() + .and_then(|s| s.container_statuses.as_ref()) + .is_none() + { + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]), + ..Default::default() + }); + } + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!(fake.mutations().iter().any(|c| c.starts_with("delete_pod"))); + } + + /// The generation is the Secret's name suffix *and* the lifecycle + /// correlator inside it. They must name the same generation, or pod logs + /// point at a Secret that was never mounted. The caller stamps one + /// generation into the env once; each create attempt mints its own, so + /// only a restamp keeps the pair together across a retry. + #[test] + fn each_attempt_stamps_its_own_generation_into_its_own_secret() { + let id = identity(); + let cfg = config(); + // First attempt loses the create race and is cleaned up; the operation + // then adopts. Two creates, two generations. + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(our_pod(&id, &cfg, Some(running()))); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + run(&fake, &id, &cfg).unwrap(); + + let created = fake.created_secrets.borrow(); + // Guard the guard: this assertion is over a list that cleanup empties, + // so an empty list would make every check below vacuously true. + assert_eq!(created.len(), 1, "expected one create attempt"); + for secret in created.iter() { + let name = secret.metadata.name.as_deref().unwrap(); + let nonce = secret + .string_data + .as_ref() + .and_then(|d| d.get(crate::env::START_NONCE_KEY)) + .expect("no lifecycle correlator in the Secret"); + assert!( + name.ends_with(nonce.as_str()), + "secret {name} carries a correlator for a different generation ({nonce})" + ); + } + } + + // ---- bounded replacement ------------------------------------------------ + + /// Installs a hook that makes every pod under `name` crash-exit before the + /// next poll — a deterministic startup failure (the harness starts, rejects + /// its configuration, and exits) as observed live: the container reaches + /// `state.terminated` with a nonzero exit code. + fn crash_exits_immediately(fake: &Fake, name: String) { + fake.on_poll + .borrow_mut() + .push(Box::new(move |pods: &mut BTreeMap| { + if let Some(pod) = pods.get_mut(&name) { + if pod + .status + .as_ref() + .and_then(|s| s.container_statuses.as_ref()) + .is_none() + { + pod.status = Some(PodStatus { + phase: Some("Failed".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(ContainerState { + terminated: Some(ContainerStateTerminated { + exit_code: 1, + reason: Some("Error".into()), + ..Default::default() + }), + ..Default::default() + }), + ..Default::default() + }]), + ..Default::default() + }); + } + } + })); + } + + /// A pod that this call created and that crash-exits deterministically is + /// reported in-band after exactly ONE attempt — never hot-replaced until + /// the deadline. The live failure this pins: one delete/create cycle every + /// ~4s minted 107 immutable Secrets in a single 600s deploy call, all + /// younger than the orphan sweep's age gate. + #[test] + fn a_deterministic_crash_exit_is_reported_not_hot_replaced() { + let id = identity(); + let cfg = config(); + let fake = Fake::default(); + crash_exits_immediately(&fake, id.pod_name()); + + let err = run(&fake, &id, &cfg).unwrap_err(); + + let creates = fake + .mutations() + .iter() + .filter(|c| c.starts_with("create_secret")) + .count(); + assert_eq!( + creates, 1, + "hot replacement loop: {creates} Secrets minted in one deploy call" + ); + assert!( + err.contains("exited with code 1"), + "error does not carry the exit: {err}" + ); + // The failed attempt is left in place as evidence — no cleanup on the + // error path. Its Secret stays referenced by the terminated pod, so + // the next deploy's preflight GC collects both together. + assert!( + !fake + .mutations() + .iter() + .any(|c| c.starts_with("delete_pod") || c.starts_with("delete_secret")), + "the failed attempt was cleaned up on the error path: {:?}", + fake.mutations() + ); + assert!( + fake.elapsed() < DEADLINE, + "burned the whole deadline on a deterministic failure" + ); + } + + /// The revive path is untouched: terminated residue from a *previous* + /// life is still replaced — the bound is on pods this call created, not + /// on the row. + #[test] + fn pre_existing_terminated_residue_is_still_replaced_once() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(terminated()))); + crash_exits_immediately(&fake, id.pod_name()); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("exited with code 1"), "got: {err}"); + + let calls = fake.mutations(); + let deletes = calls.iter().filter(|c| c.starts_with("delete_pod")).count(); + let creates = calls + .iter() + .filter(|c| c.starts_with("create_secret")) + .count(); + // Exactly one residue delete (the normal restart path) and one fresh + // attempt — then report, not another cycle. + assert_eq!(deletes, 1, "unexpected delete traffic: {calls:?}"); + assert_eq!(creates, 1, "unexpected create traffic: {calls:?}"); + } + + /// Retry is gated on fresh owner intent: the *next* deploy call clears + /// the crashed attempt (preflight GC collects the terminated pod and its + /// referenced Secret together) and makes exactly one new attempt — total + /// litter stays one pod + one Secret however many times Start is pressed. + #[test] + fn the_next_deploy_collects_the_crashed_attempt_before_its_own_attempt() { + let id = identity(); + let cfg = config(); + let fake = Fake::default(); + crash_exits_immediately(&fake, id.pod_name()); + + run(&fake, &id, &cfg).unwrap_err(); + // Second Start: fresh call, fresh clock. + *fake.elapsed.borrow_mut() = Duration::ZERO; + run(&fake, &id, &cfg).unwrap_err(); + + assert_eq!( + fake.secrets.borrow().len(), + 1, + "crashed attempts accumulated Secrets across calls" + ); + assert_eq!(fake.pods.borrow().len(), 1, "crashed pods accumulated"); + } + + // ---- the auto-repair fence ---------------------------------------------- + + /// An object under our deterministic name that lacks the management + /// marker is not ours. The reconciler must not adopt it, delete it, or + /// return its name — it fails closed to the operator (`:1156-1162`). + #[test] + fn an_unmarked_look_alike_is_never_touched_or_adopted() { + let id = identity(); + let cfg = config(); + let mut look_alike = our_pod(&id, &cfg, Some(running())); + look_alike + .metadata + .labels + .as_mut() + .unwrap() + .remove(LABEL_MANAGED_BY); + let fake = Fake::default(); + // Our create loses to the object already sitting on the name. + *fake.winner.borrow_mut() = Some(look_alike); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("not managed by this provider"), "got: {err}"); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "deleted an object we do not own" + ); + } + + /// Same fence, other direction: the marker is present but the annotation + /// carries a different agent's pubkey. The 32-hex label is + /// collision-resistant, not collision-free (`:1152-1155`). + #[test] + fn a_pubkey_mismatch_is_never_adopted() { + let id = identity(); + let other = identity(); + let cfg = config(); + let mut foreign = our_pod(&id, &cfg, Some(running())); + foreign.metadata.annotations.as_mut().unwrap().insert( + ANNOTATION_PUBKEY_FULL.to_string(), + other.pubkey_hex().to_string(), + ); + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(foreign); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("different agent identity"), "got: {err}"); + assert!(!fake.mutations().iter().any(|c| c.starts_with("delete_pod"))); + } + + // ---- create-conflict convergence --------------------------------------- + + /// The loser adopts the winner, returns the winner's id, and deletes + /// **only its own** Secret — never the winner's (`:1259-1264`). + #[test] + fn create_loser_adopts_the_winner_and_drops_only_its_own_secret() { + let id = identity(); + let cfg = config(); + let winner = our_pod(&id, &cfg, Some(running())); + let winners_secret = crate::observe::referenced_secret(&winner).unwrap(); + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(winner); + fake.secrets.borrow_mut().push(crate::pod::build_secret( + &id, + &cfg.namespace, + "gen-existing", + env(), + )); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + + let deleted: Vec<_> = fake + .mutations() + .into_iter() + .filter(|c| c.starts_with("delete_secret")) + .collect(); + assert_eq!( + deleted.len(), + 1, + "expected exactly our own Secret dropped: {deleted:?}" + ); + assert!( + !deleted[0].contains(&winners_secret), + "deleted the winner's Secret: {deleted:?}" + ); + assert!( + fake.secrets + .borrow() + .iter() + .any(|s| s.metadata.name.as_deref() == Some(winners_secret.as_str())), + "the winner's Secret is gone" + ); + } + + /// The loser must not apply the divergence row to the pod that just beat + /// it — that is the ping-pong the spec forbids (`:845-850`). A divergent + /// *started* winner is adopted as-is. + #[test] + fn create_loser_does_not_replace_a_divergent_winner() { + let id = identity(); + let cfg = config(); + let mut winner = our_pod(&id, &cfg, Some(running())); + winner.metadata.annotations.as_mut().unwrap().insert( + ANNOTATION_CREATE_INTENT.to_string(), + "a-different-intent".into(), + ); + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(winner); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "the create loser deleted the winner" + ); + } + + /// The loser waits for the winner to *start* — adopting a not-yet-started + /// winner as success would report an agent that is not running. + #[test] + fn create_loser_waits_for_the_winner_to_start() { + let id = identity(); + let cfg = config(); + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(our_pod(&id, &cfg, Some(waiting("ContainerCreating")))); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + let polls = RefCell::new(0); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + *polls.borrow_mut() += 1; + if *polls.borrow() >= 3 { + if let Some(pod) = pods.get_mut(&name) { + pod.status.as_mut().unwrap().container_statuses = + Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]); + } + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!(fake.elapsed() > Duration::ZERO, "did not wait at all"); + } + + // ---- deadline and namespace -------------------------------------------- + + /// Deadline expiry reports the *latest* condition, not a generic timeout + /// (`:699-701`), and triggers no cleanup (`:720-724`). + #[test] + fn deadline_expiry_reports_the_condition_and_cleans_up_nothing() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(waiting("ContainerCreating")))); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("ContainerCreating"), "generic timeout: {err}"); + let calls = fake.mutations(); + assert!( + !calls.iter().any(|c| c.starts_with("delete_")), + "deadline expiry triggered cleanup: {calls:?}" + ); + } + + /// An RBAC denial on namespace create fails the deploy with the literal + /// command to run, before any Secret is written (`:1002-1005`). + #[test] + fn namespace_denial_fails_before_writing_any_secret() { + let id = identity(); + let cfg = config(); + let fake = Fake { + namespace_error: Some(format!( + "not authorized to create namespaces: run `kubectl create namespace {}`", + cfg.namespace + )), + ..Default::default() + }; + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("kubectl create namespace"), "got: {err}"); + assert!( + fake.mutations().is_empty(), + "wrote something after a namespace denial: {:?}", + fake.mutations() + ); + } + + /// GC failures are hygiene, not deploy failures: a list the user cannot + /// perform must not block a deploy they can. + #[test] + fn a_gc_failure_does_not_fail_the_deploy() { + struct GcDenied(Fake); + impl Substrate for GcDenied { + async fn ensure_namespace(&self, ns: &str) -> Result<(), String> { + self.0.ensure_namespace(ns).await + } + async fn list_pods( + &self, + _s: &str, + ) -> Result<(Vec, Option>), String> { + Err("forbidden: cannot list pods".into()) + } + async fn list_secrets(&self, _s: &str) -> Result, String> { + Err("forbidden: cannot list secrets".into()) + } + async fn secret_exists(&self, n: &str) -> Result { + self.0.secret_exists(n).await + } + async fn create_secret(&self, s: &Secret) -> Result<(), String> { + self.0.create_secret(s).await + } + async fn create_pod(&self, p: &Pod) -> Result { + self.0.create_pod(p).await + } + async fn delete_pod(&self, n: &str, f: &Fence) -> Result { + self.0.delete_pod(n, f).await + } + async fn delete_secret(&self, n: &str) -> Result<(), String> { + self.0.delete_secret(n).await + } + async fn get_pod(&self, n: &str) -> Result, String> { + self.0.get_pod(n).await + } + async fn sleep(&self, d: Duration) { + self.0.sleep(d).await + } + fn elapsed(&self) -> Duration { + self.0.elapsed() + } + } + + let id = identity(); + let cfg = config(); + let inner = Fake::default().with_pod(our_pod(&id, &cfg, Some(running()))); + let fake = GcDenied(inner); + + assert_eq!( + block_on(deploy(&fake, &id, &cfg, env())).unwrap(), + id.pod_name() + ); + } +} diff --git a/crates/buzz-backend-kubernetes/src/wire.rs b/crates/buzz-backend-kubernetes/src/wire.rs new file mode 100644 index 000000000..89b78b364 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/wire.rs @@ -0,0 +1,250 @@ +//! The stdin/stdout JSON protocol (spec §Provider Protocol). +//! +//! One process per operation: one JSON object in, one JSON object out. +//! These types are this provider's view of the contract; the golden fixtures +//! in `tests/fixtures/provider-wire/` are the arbiter shared with the desktop. + +use serde::{Deserialize, Serialize}; +use std::collections::BTreeMap; + +/// The wire-contract version this provider speaks (spec §Info). +pub const PROTOCOL_VERSION: u32 = 1; + +/// Request envelope. `op` discriminates; unknown ops are an in-band error. +/// +/// `request_id` is deliberately absent from every variant. The desktop sends +/// it, but the exchange is one request and one response per process, so there +/// is nothing to correlate and nothing in the response schema to echo it into +/// (`:387`, `:416` — neither response shape carries it). Serde ignores it on +/// the way in, so a caller that sends it is accepted; typing it would only +/// create a field nothing reads. +#[derive(Debug, Deserialize)] +#[serde(tag = "op", rename_all = "lowercase")] +pub enum Request { + Info, + Deploy(Box), +} + +#[derive(Debug, Deserialize)] +pub struct DeployRequest { + pub agent: AgentPayload, + #[serde(default)] + pub provider_config: serde_json::Value, +} + +/// The agent payload (spec §Deploy). +/// +/// Only the fields this binding actually consumes are typed. `name` (display +/// name), `model`, `provider`, and `turn_timeout_seconds` are deliberately +/// absent: object names derive from the pubkey, not the display name +/// (`:1150-1152`), the model/provider pair arrives already resolved inside +/// `launch`, and the timeout is ignored upstream — typing any of them would +/// invite a provider-side remap the spec forbids. +#[derive(Debug, Deserialize)] +pub struct AgentPayload { + pub relay_url: String, + pub private_key_nsec: String, + #[serde(default)] + pub auth_tag: Option, + #[serde(default)] + pub respond_to: Option, + #[serde(default)] + pub respond_to_allowlist: Option>, + /// User env, already merged global < persona < agent by the desktop and + /// already stripped of reserved keys. Superseded by `launch.env` when + /// `launch` is present — a provider MUST NOT re-merge it on top + /// (§Launch data, precedence tier 2). + #[serde(default)] + pub env_vars: BTreeMap, + /// The desktop-resolved launch contract. Absent only from a desktop + /// predating Known Defect 3's fix. + #[serde(default)] + pub launch: Option, +} + +/// Desktop-resolved launch data (spec §Launch data). +#[derive(Debug, Default, Deserialize)] +pub struct LaunchBlock { + /// Command *name*, resolved against the image's PATH — never a host path. + #[serde(default)] + pub command: Option, + #[serde(default)] + pub args: Vec, + /// Layered env: baked → runtime metadata → definition → global → persona + /// → agent. Precedence tier 2. + #[serde(default)] + pub env: BTreeMap, + /// Overridable behavior defaults. Precedence tier 1 — user env beats + /// these, matching the local spawn. + #[serde(default)] + pub policy_env: BTreeMap, + /// Resolved workspace owner (hex). The respond-to gate's one + /// irreducible input; without it or `auth_tag` the harness cannot match + /// `!shutdown`. + #[serde(default)] + pub owner_pubkey: Option, +} + +/// Response envelope. Serialized flat — `{"ok": true, …}` — because the +/// desktop reads `ok`, `error`, and `agent_id` off the top level. +#[derive(Debug, Serialize)] +#[serde(untagged)] +pub enum Response { + Info(InfoResponse), + Deploy(DeployResponse), + Error(ErrorResponse), +} + +#[derive(Debug, Serialize)] +pub struct InfoResponse { + pub ok: bool, + pub name: &'static str, + pub version: &'static str, + pub protocol_version: u32, + pub description: &'static str, + pub config_schema: serde_json::Value, +} + +#[derive(Debug, Serialize)] +pub struct DeployResponse { + pub ok: bool, + pub agent_id: String, +} + +#[derive(Debug, Serialize)] +pub struct ErrorResponse { + pub ok: bool, + pub error: String, +} + +impl Response { + pub fn error(message: impl Into) -> Self { + Response::Error(ErrorResponse { + ok: false, + error: message.into(), + }) + } + + /// The provider's self-description (spec §Info). Pure — no cluster + /// contact — because the desktop calls it to render the config form + /// before a kubeconfig is known to exist. + pub fn info() -> Self { + Response::Info(InfoResponse { + ok: true, + name: "kubernetes", + version: env!("CARGO_PKG_VERSION"), + protocol_version: PROTOCOL_VERSION, + description: "Runs agents as pods in a Kubernetes cluster", + config_schema: crate::config::config_schema(), + }) + } + + pub fn deployed(agent_id: impl Into) -> Self { + Response::Deploy(DeployResponse { + ok: true, + agent_id: agent_id.into(), + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parses_info_request() { + let r: Request = serde_json::from_str(r#"{"op":"info","request_id":"abc"}"#).unwrap(); + assert!(matches!(r, Request::Info)); + } + + /// The desktop sends `request_id` on every call, but the exchange is 1:1 + /// per process — a provider that hard-required it would fail a + /// conforming-but-minimal caller for no safety gain. + #[test] + fn request_id_is_optional() { + let r: Request = serde_json::from_str(r#"{"op":"info"}"#).unwrap(); + assert!(matches!(r, Request::Info)); + } + + #[test] + fn rejects_unknown_op() { + assert!(serde_json::from_str::(r#"{"op":"undeploy"}"#).is_err()); + } + + /// Payload fields this binding does not consume must not break parsing: + /// the desktop sends `model`, `provider`, `system_prompt` and more, and a + /// provider that rejected them would break on every real deploy. + #[test] + fn ignores_unconsumed_payload_fields() { + let json = r#"{ + "op":"deploy","request_id":"r1", + "agent":{ + "name":"a","relay_url":"wss://r","private_key_nsec":"nsec1x", + "model":"gpt-5","provider":"openai","system_prompt":"hi", + "turn_timeout_seconds":30,"parallelism":10, + "agent_command":"goose","agent_args":[] + }, + "provider_config":{"namespace":"ns"} + }"#; + let r: Request = serde_json::from_str(json).unwrap(); + let Request::Deploy(d) = r else { + panic!("wrong op") + }; + assert_eq!(d.agent.relay_url, "wss://r"); + assert!(d.agent.launch.is_none()); + } + + #[test] + fn parses_launch_block() { + let json = r#"{ + "op":"deploy", + "agent":{ + "relay_url":"wss://r","private_key_nsec":"nsec1x", + "launch":{ + "command":"goose","args":["run","--x"], + "env":{"GOOSE_MODEL":"m"}, + "policy_env":{"GOOSE_MODE":"auto"}, + "owner_pubkey":"deadbeef" + } + } + }"#; + let Request::Deploy(d) = serde_json::from_str::(json).unwrap() else { + panic!("wrong op") + }; + let l = d.agent.launch.unwrap(); + assert_eq!(l.command.as_deref(), Some("goose")); + assert_eq!(l.args, ["run", "--x"]); + assert_eq!(l.env["GOOSE_MODEL"], "m"); + assert_eq!(l.policy_env["GOOSE_MODE"], "auto"); + assert_eq!(l.owner_pubkey.as_deref(), Some("deadbeef")); + } + + /// A null `owner_pubkey`/`auth_tag` must parse (the refusal is a policy + /// decision made later, with a specific message), not fail as a type error. + #[test] + fn null_owner_fields_parse() { + let json = r#"{"op":"deploy","agent":{ + "relay_url":"wss://r","private_key_nsec":"nsec1x","auth_tag":null, + "launch":{"owner_pubkey":null} + }}"#; + let Request::Deploy(d) = serde_json::from_str::(json).unwrap() else { + panic!("wrong op") + }; + assert!(d.agent.auth_tag.is_none()); + assert!(d.agent.launch.unwrap().owner_pubkey.is_none()); + } + + /// The desktop reads `ok`/`error`/`agent_id` off the top level, so the + /// enum must serialize flat with no variant tag. + #[test] + fn responses_serialize_flat() { + let v = serde_json::to_value(Response::deployed("buzz-agent-abc")).unwrap(); + assert_eq!(v["ok"], true); + assert_eq!(v["agent_id"], "buzz-agent-abc"); + assert!(v.get("Deploy").is_none()); + + let v = serde_json::to_value(Response::error("boom")).unwrap(); + assert_eq!(v["ok"], false); + assert_eq!(v["error"], "boom"); + } +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/README.md b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/README.md new file mode 100644 index 000000000..56a972f89 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/README.md @@ -0,0 +1,39 @@ +# Provider wire fixtures + +The shared arbiter for the stdin/stdout contract between the desktop +(`agents_deploy.rs`) and this provider (spec §Provider Protocol). + +Each `*.request.json` is a request the desktop can emit; each matching +`*.response.json` is the exact response this provider produces for it. The +provider side is asserted by `tests/wire_fixtures.rs`; the desktop side should +assert that its emitted payloads parse as the corresponding request. + +Three rules keep these useful rather than decorative: + +* **Requests are recorded, not invented.** A fixture that no caller emits + tests a contract nobody has. "Recorded" means *executed and transcribed* — + `deploy-full-launch.request.json` is the output of the desktop's real + `build_launch_block` → `deploy_payload_json` path, not a shape derived by + reading those functions. Deriving it is how this fixture acquired four + impossible values at once: a `respond_to` that was a pubkey where the + desktop serializes a kebab-case `RespondTo` enum, allowlist and owner + values failing `validate_respond_to_allowlist`'s 64-hex rule + (`types.rs:897`), an invented `BUZZ_ACP_PARALLELISM` where the emitter + writes `BUZZ_ACP_AGENTS` (`runtime.rs:729`), and a `launch.env` key from + no layer of `resolve_effective_harness_descriptor`. +* **The provider cannot police this file, so the desktop must.** Every field + above is one this provider is deliberately indifferent to — `respond_to` is + an opaque `Option`, the allowlist an opaque `Vec`, + `policy_env` an arbitrary map — so `the_full_desktop_payload_is_accepted` + passes on invented data exactly as happily as on recorded data. The + enforcement is the desktop's whole-object equality test, which *builds* the + payload and compares it to this file. A completeness guard (the case-list + directory scan in `wire_fixtures.rs`) stops a case from going missing; it + cannot tell you a case is false. +* **Responses are byte-compared after key-sorted re-serialization**, so a + field rename or a type change fails here rather than in a desktop that + silently reads `undefined`. + +`deploy-*` fixtures cover only responses reachable without a cluster — +refusals and malformed input. A successful deploy needs an apiserver and is +covered by the conformance suite, not by a static fixture. diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-full-launch.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-full-launch.request.json new file mode 100644 index 000000000..28fe6ce90 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-full-launch.request.json @@ -0,0 +1,52 @@ +{ + "op": "deploy", + "request_id": "req-6", + "agent": { + "agent_args": [], + "agent_command": "goose", + "auth_tag": "tag-1", + "env_vars": { + "USER_KEY": "user-value" + }, + "idle_timeout_seconds": null, + "launch": { + "args": [ + "acp" + ], + "command": "goose", + "env": { + "GOOSE_MODEL": "gpt-5", + "GOOSE_PROVIDER": "openai", + "USER_KEY": "user-value" + }, + "owner_pubkey": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "policy_env": { + "BUZZ_ACP_AGENTS": "10", + "BUZZ_ACP_LAZY_POOL": "true", + "BUZZ_ACP_MODEL": "gpt-5", + "BUZZ_ACP_RELAY_OBSERVER": "true", + "BUZZ_ACP_SESSION_TITLE": "worker", + "GOOSE_MODE": "auto" + } + }, + "max_turn_duration_seconds": null, + "model": "gpt-5", + "name": "worker", + "parallelism": 10, + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "provider": "openai", + "relay_url": "wss://relay.example", + "respond_to": "allowlist", + "respond_to_allowlist": [ + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + ], + "system_prompt": null, + "turn_timeout_seconds": 300 + }, + "provider_config": { + "namespace": "buzz-agents-test", + "image": "ghcr.io/block/buzz-sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "inactivity_seconds": 3600 + } +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.request.json new file mode 100644 index 000000000..4f56082bb --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.request.json @@ -0,0 +1,9 @@ +{ + "op": "deploy", + "request_id": "req-5", + "agent": { + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5" + }, + "provider_config": {"namespace": "buzz-agents-test", "image": "ghcr.io/block/buzz-sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"} +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.response.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.response.json new file mode 100644 index 000000000..c6011b899 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.response.json @@ -0,0 +1 @@ +{"ok":false,"error":"deploy refused: neither auth_tag nor launch.owner_pubkey resolved — without an owner the agent cannot honor !shutdown"} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.request.json new file mode 100644 index 000000000..2160909b8 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.request.json @@ -0,0 +1,11 @@ +{ + "op": "deploy", + "request_id": "req-3", + "agent": { + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "auth_tag": "tag-1", + "provider": " relay-mesh " + }, + "provider_config": {"namespace": "buzz-agents-test", "image": "ghcr.io/block/buzz-sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"} +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.response.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.response.json new file mode 100644 index 000000000..0f0c995fd --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.response.json @@ -0,0 +1 @@ +{"ok":false,"error":"deploy refused: this agent is configured for shared compute (relay-mesh), which runs on the relay rather than in a pod. Switch the agent to a local runtime before deploying it to Kubernetes."} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.request.json new file mode 100644 index 000000000..71ce9f067 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.request.json @@ -0,0 +1,12 @@ +{ + "op": "deploy", + "request_id": "req-2", + "agent": { + "name": "mesh-agent", + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "auth_tag": "tag-1", + "provider": "relay-mesh" + }, + "provider_config": {"namespace": "buzz-agents-test", "image": "ghcr.io/block/buzz-sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"} +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.response.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.response.json new file mode 100644 index 000000000..0f0c995fd --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.response.json @@ -0,0 +1 @@ +{"ok":false,"error":"deploy refused: this agent is configured for shared compute (relay-mesh), which runs on the relay rather than in a pod. Switch the agent to a local runtime before deploying it to Kubernetes."} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.request.json new file mode 100644 index 000000000..232d9dcf4 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.request.json @@ -0,0 +1,10 @@ +{ + "op": "deploy", + "request_id": "req-4", + "agent": { + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "auth_tag": "tag-1" + }, + "provider_config": {"namespace": "buzz-agents-test", "image": "ghcr.io/block/buzz-sprig:latest"} +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.response.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.response.json new file mode 100644 index 000000000..e67fa6a47 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.response.json @@ -0,0 +1 @@ +{"ok":false,"error":"provider_config.image \"ghcr.io/block/buzz-sprig:latest\" is not digest-pinned: a tag is a mutable pointer, and this object runs with the agent's private key. Use name@sha256:<64 hex chars>"} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/info.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/info.request.json new file mode 100644 index 000000000..db99c86c0 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/info.request.json @@ -0,0 +1 @@ +{"op":"info","request_id":"req-1"} diff --git a/crates/buzz-backend-kubernetes/tests/wire_fixtures.rs b/crates/buzz-backend-kubernetes/tests/wire_fixtures.rs new file mode 100644 index 000000000..2c3af82b1 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/wire_fixtures.rs @@ -0,0 +1,214 @@ +//! Golden wire fixtures (spec §Provider Protocol). +//! +//! These drive the **built binary** over a real pipe rather than calling an +//! in-process function: the contract the desktop depends on is +//! `stdin → one JSON object on stdout → exit code`, and an in-process test +//! would assert the shape of a value while skipping the three things that +//! actually break — the process writing nothing, writing two objects, or +//! signalling the outcome through the exit code. + +use std::io::Write; +use std::path::{Path, PathBuf}; +use std::process::{Command, Stdio}; + +fn fixtures() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/provider-wire") +} + +/// Feed one request to the binary; return `(stdout, exit code)`. +fn run(request: &str) -> (String, i32) { + let mut child = Command::new(env!("CARGO_BIN_EXE_buzz-backend-kubernetes")) + // A kubeconfig that does not exist, so a fixture that accidentally + // reaches the cluster fails loudly here instead of depending on + // whatever cluster the developer is pointed at. + .env("KUBECONFIG", "/nonexistent/kubeconfig-for-fixture-tests") + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .spawn() + .expect("could not run the provider binary"); + child + .stdin + .take() + .expect("no stdin") + .write_all(request.as_bytes()) + .expect("could not write the request"); + let out = child.wait_with_output().expect("provider did not exit"); + ( + String::from_utf8(out.stdout).expect("stdout was not UTF-8"), + out.status.code().unwrap_or(-1), + ) +} + +fn read(name: &str) -> String { + std::fs::read_to_string(fixtures().join(name)) + .unwrap_or_else(|e| panic!("could not read fixture {name}: {e}")) +} + +/// Every response fixture, byte-compared after key-sorted re-serialization so +/// a field rename fails here rather than in a desktop reading `undefined`. +#[test] +fn responses_match_their_fixtures() { + let cases = [ + "deploy-relay-mesh", + "deploy-relay-mesh-padded", + "deploy-tag-image", + "deploy-no-owner", + ]; + // The list must cover every response fixture on disk. A literal array is + // never empty, so `!is_empty()` would assert nothing; what can actually go + // wrong is a fixture added to the directory and never added here, which + // reads as a passing suite that exercises one case fewer than it appears to. + let mut on_disk: Vec = std::fs::read_dir(fixtures()) + .expect("could not read the fixture directory") + .filter_map(|entry| entry.ok()?.file_name().into_string().ok()) + .filter_map(|name| Some(name.strip_suffix(".response.json")?.to_string())) + .collect(); + on_disk.sort(); + let mut listed: Vec = cases.iter().map(|c| c.to_string()).collect(); + listed.sort(); + assert_eq!(on_disk, listed, "response fixtures and cases disagree"); + + for case in cases { + let (stdout, code) = run(&read(&format!("{case}.request.json"))); + assert_eq!(code, 0, "{case}: a produced response must exit 0"); + + // Exactly one object, terminated by exactly one newline. Two responses + // would leave the desktop's reader holding a second one forever. + assert_eq!( + stdout.matches('\n').count(), + 1, + "{case}: expected exactly one line, got {stdout:?}" + ); + + let actual: serde_json::Value = + serde_json::from_str(&stdout).unwrap_or_else(|e| panic!("{case}: {e}: {stdout:?}")); + let expected: serde_json::Value = + serde_json::from_str(&read(&format!("{case}.response.json"))).unwrap(); + assert_eq!( + actual, expected, + "{case}: response drifted from its fixture" + ); + } +} + +/// `info` is checked on the fields the desktop reads rather than byte-for-byte: +/// the namespace default is randomly generated per call (§K8s Namespace), so a +/// golden copy of it would be a test that fails every run. +#[test] +fn info_response_carries_the_contract_fields() { + let (stdout, code) = run(&read("info.request.json")); + assert_eq!(code, 0); + let info: serde_json::Value = serde_json::from_str(&stdout).unwrap(); + assert_eq!(info["ok"], true); + assert_eq!(info["protocol_version"], 1); + assert_eq!(info["name"], "kubernetes"); + let schema = &info["config_schema"]; + assert_eq!( + schema["required"], + serde_json::json!(["namespace", "image"]) + ); + let default = schema["properties"]["namespace"]["default"] + .as_str() + .expect("no generated namespace default"); + assert!( + default.starts_with("buzz-agents-"), + "unexpected namespace default: {default}" + ); +} + +/// The desktop's richest payload must parse. No response fixture: this one +/// reaches the cluster, so its outcome depends on a kubeconfig. What it +/// guards is that every field the desktop sends is *accepted* — a payload the +/// provider rejects at parse time is a deploy that never starts. +#[test] +fn the_full_desktop_payload_is_accepted() { + let (stdout, code) = run(&read("deploy-full-launch.request.json")); + assert_eq!(code, 0); + let response: serde_json::Value = serde_json::from_str(&stdout).unwrap(); + let error = response["error"].as_str().unwrap_or_default(); + // It fails — there is no cluster — but it must fail at the *connection*, + // having accepted every field above it. + assert!( + error.contains("kubeconfig"), + "the full payload was rejected before reaching the cluster: {error}" + ); +} + +/// Sami's pre-registered respond-to matrix, driven through the built binary. +/// +/// `build_env` runs at `main.rs:124`, `client::connect` at `:132`, so under a +/// kubeconfig that cannot exist the error string *is* the ordering assertion: +/// "kubeconfig" means the gate passed and we reached the cluster, anything +/// else means we refused before writing a Secret. A test asserting only +/// `ok: false` would pass on the connection error and prove nothing. +/// +/// The cases are applied to the real full-launch request so each one differs +/// from a known-good deploy in exactly the field under test. +#[test] +fn the_respond_to_gate_matches_the_harness_acceptance_surface() { + let key_a = "a".repeat(64); + let padded_upper = format!(" {} ", "A".repeat(64)); + // (name, respond_to, allowlist, must reach the cluster) + let cases: Vec<(&str, &str, Option>, bool)> = vec![ + ("allowlist + []", "allowlist", Some(vec![]), false), + ("allowlist + absent", "allowlist", None, false), + ( + "allowlist + junk", + "allowlist", + Some(vec!["beefcafe".into()]), + false, + ), + ("unparseable mode", "npub1abc", None, false), + ("padded mode", " allowlist ", None, false), + ( + "allowlist + two valid", + "allowlist", + Some(vec![key_a.clone(), "b".repeat(64)]), + true, + ), + ( + "owner-only + junk list", + "owner-only", + Some(vec!["beefcafe".into()]), + true, + ), + ( + "allowlist + padded upper", + "allowlist", + Some(vec![padded_upper]), + true, + ), + ("nobody", "nobody", None, true), + ("anyone", "anyone", None, true), + ]; + + let base: serde_json::Value = + serde_json::from_str(&read("deploy-full-launch.request.json")).unwrap(); + + for (name, mode, allowlist, reaches_cluster) in cases { + let mut request = base.clone(); + let agent = &mut request["agent"]; + agent["respond_to"] = serde_json::json!(mode); + agent["respond_to_allowlist"] = match &allowlist { + Some(list) => serde_json::json!(list), + None => serde_json::Value::Null, + }; + + let (stdout, code) = run(&request.to_string()); + assert_eq!(code, 0, "{name}: provider did not exit cleanly"); + let response: serde_json::Value = serde_json::from_str(&stdout).unwrap(); + let error = response["error"].as_str().unwrap_or_default(); + let reached = error.contains("kubeconfig"); + + assert_eq!( + reached, reaches_cluster, + "{name}: expected reaches_cluster={reaches_cluster}, got error: {error}" + ); + if !reaches_cluster { + assert!( + error.contains("deploy refused"), + "{name}: refused, but not by the gate: {error}" + ); + } + } +} diff --git a/desktop/scripts/build-release-config.mjs b/desktop/scripts/build-release-config.mjs index 389d18aec..d1cd8181e 100644 --- a/desktop/scripts/build-release-config.mjs +++ b/desktop/scripts/build-release-config.mjs @@ -52,6 +52,15 @@ const releaseConfig = { }, }; +// Tauri applies --config after platform-specific config using RFC 7396. +// Any externalBin value here would therefore replace the platform sidecar list, +// while null would silently delete it. This delta must never own that key. +if (Object.hasOwn(releaseConfig.bundle, "externalBin")) { + throw new Error( + "Release config must not define bundle.externalBin; sidecars are platform-specific", + ); +} + console.log(`Updater enabled -> ${updaterEndpoint}`); writeFileSync(outputConfigPath, `${JSON.stringify(releaseConfig, null, 2)}\n`); diff --git a/desktop/src-tauri/Cargo.lock b/desktop/src-tauri/Cargo.lock index 00d3fba3b..cf024b228 100644 --- a/desktop/src-tauri/Cargo.lock +++ b/desktop/src-tauri/Cargo.lock @@ -1106,6 +1106,7 @@ dependencies = [ "tauri-plugin-single-instance", "tauri-plugin-updater", "tauri-plugin-window-state", + "tauri-utils", "tempfile", "tokio", "tokio-tungstenite 0.29.0", diff --git a/desktop/src-tauri/Cargo.toml b/desktop/src-tauri/Cargo.toml index b80684f95..0b5563477 100644 --- a/desktop/src-tauri/Cargo.toml +++ b/desktop/src-tauri/Cargo.toml @@ -138,6 +138,7 @@ strip-ansi-escapes = "0.2" tracing = "0.1" [dev-dependencies] +tauri-utils = "2" # `test-util` enables tokio's paused-clock (`start_paused`) so the relay # admission gate tests can assert exact wait durations without real sleeps. tokio = { version = "1", features = ["test-util"] } diff --git a/desktop/src-tauri/src/commands/agents.rs b/desktop/src-tauri/src/commands/agents.rs index 0758fc3aa..3b114b047 100644 --- a/desktop/src-tauri/src/commands/agents.rs +++ b/desktop/src-tauri/src/commands/agents.rs @@ -1362,7 +1362,7 @@ use deploy::build_deploy_payload; #[cfg(test)] use deploy::deploy_payload_json; #[cfg(test)] -pub(crate) use deploy::resolve_deploy_model_provider; +use deploy::{ensure_remote_provider_supported, resolve_deploy_model_provider}; #[path = "agents_profile.rs"] mod profile; diff --git a/desktop/src-tauri/src/commands/agents_deploy.rs b/desktop/src-tauri/src/commands/agents_deploy.rs index af785711d..9bb0f6230 100644 --- a/desktop/src-tauri/src/commands/agents_deploy.rs +++ b/desktop/src-tauri/src/commands/agents_deploy.rs @@ -1,6 +1,8 @@ //! Provider deploy payload construction, split from `agents.rs` (file-size -//! guard). `build_deploy_payload` gathers live state; `deploy_payload_json` -//! is the pure serialization half so payload completeness stays testable. +//! guard). The launch block is derived from the same effective descriptor and +//! policy helpers as local spawn so remote execution does not reimplement them. + +use std::collections::BTreeMap; use tauri::AppHandle; @@ -13,17 +15,6 @@ use crate::{ }; /// Resolve the deploy-specific structured model/provider for a managed agent. -/// -/// Delegates to the single effective-config resolver which enforces -/// definition-authoritative semantics for linked instances: -/// - **Linked:** definition → global. Stale record bytes are never consulted. -/// - **Definition-less:** instance → global. -/// - **Orphaned:** returns `(None, None)` — spawn is blocked elsewhere. -/// -/// Both local spawn and deploy now use the same resolver, so they can never -/// disagree on what model/provider an agent runs with. -/// -/// Exported `pub(crate)` for unit testing. #[cfg(test)] pub(crate) fn resolve_deploy_model_provider( record: &ManagedAgentRecord, @@ -36,58 +27,116 @@ pub(crate) fn resolve_deploy_model_provider( .unwrap_or((None, None)) } +/// Serialize the portable launch contract shared with provider-backed agents. +/// +/// `descriptor.env` is the authoritative six-layer environment. Policy values +/// are deliberately separate because providers apply them below that layered +/// environment, preserving the local spawn's power-user override semantics. +pub(super) fn build_launch_block( + record: &ManagedAgentRecord, + descriptor: &crate::managed_agents::readiness::EffectiveHarnessDescriptor, + teams: &[crate::managed_agents::TeamRecord], + effective_prompt: Option<&str>, + effective_model: Option<&str>, + owner_pubkey: &str, +) -> serde_json::Value { + use crate::managed_agents::{known_acp_runtime, resolve_session_title, SESSION_TITLE_ENV_VAR}; + + let runtime = known_acp_runtime(&descriptor.command); + let mut policy_env = BTreeMap::new(); + + if let Some(runtime) = runtime { + policy_env.extend( + runtime + .default_env + .iter() + .map(|(key, value)| ((*key).to_string(), (*value).to_string())), + ); + if runtime.mcp_hooks { + policy_env.insert("MCP_HOOK_SERVERS".into(), "*".into()); + } + } + policy_env.insert("BUZZ_ACP_RELAY_OBSERVER".into(), "true".into()); + policy_env.insert("BUZZ_ACP_LAZY_POOL".into(), "true".into()); + policy_env.insert("BUZZ_ACP_AGENTS".into(), record.parallelism.to_string()); + + if let Some(value) = effective_prompt { + policy_env.insert("BUZZ_ACP_SYSTEM_PROMPT".into(), value.to_string()); + } + if let Some(value) = effective_model { + policy_env.insert("BUZZ_ACP_MODEL".into(), value.to_string()); + } + if let Some(value) = record.idle_timeout_seconds { + policy_env.insert("BUZZ_ACP_IDLE_TIMEOUT".into(), value.to_string()); + } + if let Some(value) = record.max_turn_duration_seconds { + policy_env.insert("BUZZ_ACP_MAX_TURN_DURATION".into(), value.to_string()); + } + if let Some(value) = resolve_session_title(record.display_name.as_deref(), &record.name) { + policy_env.insert(SESSION_TITLE_ENV_VAR.into(), value); + } + if let Some(value) = + crate::managed_agents::spawn_hash::effective_team_instructions(record, teams) + { + policy_env.insert("BUZZ_ACP_TEAM_INSTRUCTIONS".into(), value); + } + + serde_json::json!({ + "command": descriptor.command, + "args": descriptor.args, + "env": descriptor.env, + "policy_env": policy_env, + "owner_pubkey": owner_pubkey, + }) +} + +pub(super) fn ensure_remote_provider_supported(provider: Option<&str>) -> Result<(), String> { + if provider.map(str::trim) == Some(crate::managed_agents::RELAY_MESH_PROVIDER_ID) { + return Err( + "shared-compute agents cannot be deployed remotely because the mesh endpoint is local to the desktop" + .to_string(), + ); + } + Ok(()) +} + /// Build the standard agent JSON payload for provider deploy calls. -/// -/// Like local spawn, provider deploy re-reads live persona env vars and -/// structured model/provider so remote agents receive current credentials -/// and the same authoritative values that local spawn derives from -/// `runtime_metadata_env_vars`. The only field still pinned is -/// `agent_command`/`agent_args` — those were captured at create time. -/// The only read-time resolution is `relay_url`: a blank pin resolves to -/// the active workspace relay here, matching the create-path contract. -/// -/// Fails closed when the private key is unavailable (keyring outage leaves -/// it empty after hydration): without this guard a provider deploy would -/// serialize `"private_key_nsec": ""` and launch the agent with no -/// identity — the same hazard the local spawn path refuses via -/// `spawn_key_refusal`. pub(super) fn build_deploy_payload( app: &AppHandle, state: &AppState, record: &ManagedAgentRecord, ) -> Result { - // Fails closed when the private key is unavailable — same guard as local - // spawn. Without this, a keyring outage would serialize `"private_key_nsec": ""` - // and launch the agent with no identity. if let Some(err) = crate::managed_agents::spawn_key_refusal(record) { return Err(err); } - // Merge global + persona + agent env_vars for provider deploy — the same - // live-persona-under-overrides semantics as local spawn. Global env vars - // are the lowest user-settable layer: global < persona < agent (last-wins - // on key collision). Without this, provider-backed agents wouldn't receive - // credentials saved on the persona or the agent itself. - let global_config = crate::managed_agents::load_global_agent_config(app).unwrap_or_default(); - let global_env = global_config.env_vars.clone(); - let persona_env = - crate::managed_agents::resolve_persona_env(app, record.persona_id.as_deref())?; - // Merge: global < persona (persona wins over global). - let global_persona_merged = crate::managed_agents::merged_user_env(&global_env, &persona_env); - // Merge: global+persona < agent (agent wins over everything). - let merged_env = - crate::managed_agents::merged_user_env(&global_persona_merged, &record.env_vars); - + let global = crate::managed_agents::load_global_agent_config(app).unwrap_or_default(); let personas = load_personas(app).unwrap_or_default(); - let cfg = crate::managed_agents::effective_config::resolve_effective_config( - record, - &personas, - &global_config, + let teams = crate::managed_agents::load_teams(app).unwrap_or_default(); + let persona_env = + crate::managed_agents::live_persona_env(&personas, record.persona_id.as_deref()); + let global_persona_env = crate::managed_agents::merged_user_env(&global.env_vars, &persona_env); + let merged_user_env = + crate::managed_agents::merged_user_env(&global_persona_env, &record.env_vars); + let effective = crate::managed_agents::effective_config::resolve_effective_config( + record, &personas, &global, ) .require_resolved()?; - let effective_model = cfg.model.value; - let effective_provider = cfg.provider.value; - let effective_prompt = cfg.system_prompt.value; + + ensure_remote_provider_supported(effective.provider.value.as_deref())?; + + let descriptor = + crate::managed_agents::resolve_effective_harness_descriptor(record, &personas, &global) + .map_err(|error| crate::managed_agents::user_facing_harness_error(&error))?; + let owner_pubkey = super::workspace_owner_hex(state)?; + let launch = build_launch_block( + record, + &descriptor, + &teams, + effective.system_prompt.value.as_deref(), + effective.model.value.as_deref(), + &owner_pubkey, + ); Ok(deploy_payload_json( record, @@ -95,23 +144,24 @@ pub(super) fn build_deploy_payload( &record.relay_url, &relay_ws_url_with_override(state), ), - effective_model, - effective_provider, - effective_prompt, - merged_env, + effective.model.value, + effective.provider.value, + effective.system_prompt.value, + merged_user_env, + launch, )) } -/// Pure serialization half of [`build_deploy_payload`] — every field the -/// provider harness receives is deliberately listed here, so payload -/// completeness is testable without an `AppHandle`. +/// Pure serialization half of [`build_deploy_payload`]. Legacy top-level fields +/// remain for display/bookkeeping; providers execute the resolved `launch` block. pub(super) fn deploy_payload_json( record: &ManagedAgentRecord, relay_url: String, effective_model: Option, effective_provider: Option, effective_prompt: Option, - merged_env: std::collections::BTreeMap, + merged_env: BTreeMap, + launch: serde_json::Value, ) -> serde_json::Value { serde_json::json!({ "name": &record.name, @@ -130,5 +180,81 @@ pub(super) fn deploy_payload_json( "respond_to": record.respond_to, "respond_to_allowlist": &record.respond_to_allowlist, "env_vars": merged_env, + "launch": launch, }) } + +#[cfg(test)] +mod tests { + use super::*; + use crate::managed_agents::{readiness::EffectiveHarnessDescriptor, RespondTo, TeamRecord}; + + fn record() -> ManagedAgentRecord { + serde_json::from_value(serde_json::json!({ + "pubkey": "abcd1234", + "name": "agent-handle", + "display_name": "Agent\u{0000} Name", + "private_key_nsec": "nsec1fake", + "relay_url": "wss://relay.example", + "acp_command": "buzz-acp", + "agent_command": "goose", + "agent_args": [], + "mcp_command": "", + "turn_timeout_seconds": 320, + "idle_timeout_seconds": 17, + "max_turn_duration_seconds": 23, + "parallelism": 4, + "respond_to": RespondTo::OwnerOnly, + "respond_to_allowlist": [], + "team_id": "team-1", + "created_at": "2026-01-01T00:00:00Z", + "updated_at": "2026-01-01T00:00:00Z" + })) + .unwrap() + } + + #[test] + fn launch_block_preserves_descriptor_and_spawn_policy() { + let record = record(); + let descriptor = EffectiveHarnessDescriptor { + command: "goose".into(), + args: vec!["acp".into()], + env: BTreeMap::from([ + ("GOOSE_MODE".into(), "custom".into()), + ("SECRET_FROM_PERSONA".into(), "secret".into()), + ]), + }; + let teams: Vec = serde_json::from_value(serde_json::json!([{ + "id": "team-1", "name": "Team", "instructions": "Coordinate", "persona_ids": [], "created_at": "2026-01-01T00:00:00Z", "updated_at": "2026-01-01T00:00:00Z" + }])).unwrap(); + + let launch = build_launch_block( + &record, + &descriptor, + &teams, + Some("prompt"), + Some("model"), + "owner-hex", + ); + + assert_eq!(launch["command"], "goose"); + assert_eq!(launch["args"], serde_json::json!(["acp"])); + assert_eq!(launch["env"]["GOOSE_MODE"], "custom"); + // policy_env is applied first, so this default remains separate from + // the descriptor value that wins in launch.env. + assert_eq!(launch["policy_env"]["GOOSE_MODE"], "auto"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_LAZY_POOL"], "true"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_RELAY_OBSERVER"], "true"); + assert_eq!( + launch["policy_env"]["BUZZ_ACP_TEAM_INSTRUCTIONS"], + "Coordinate" + ); + assert_eq!(launch["policy_env"]["BUZZ_ACP_SESSION_TITLE"], "Agent Name"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_SYSTEM_PROMPT"], "prompt"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_MODEL"], "model"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_IDLE_TIMEOUT"], "17"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_MAX_TURN_DURATION"], "23"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_AGENTS"], "4"); + assert_eq!(launch["owner_pubkey"], "owner-hex"); + } +} diff --git a/desktop/src-tauri/src/commands/agents_tests.rs b/desktop/src-tauri/src/commands/agents_tests.rs index 03389d1d1..df135298c 100644 --- a/desktop/src-tauri/src/commands/agents_tests.rs +++ b/desktop/src-tauri/src/commands/agents_tests.rs @@ -263,6 +263,19 @@ fn normalize_relay_mesh_trims_and_preserves_valid_config() { ); } +#[test] +fn deploy_refuses_resolved_relay_mesh_provider_with_padding() { + let record = bare_agent_record(Some("p1"), None, None); + let personas = vec![persona_record("p1", None, Some(" relay-mesh "))]; + let global = crate::managed_agents::GlobalAgentConfig::default(); + + let (_, provider) = resolve_deploy_model_provider(&record, &personas, &global); + let error = ensure_remote_provider_supported(provider.as_deref()) + .expect_err("resolved shared-compute provider must not deploy remotely"); + + assert!(error.contains("cannot be deployed remotely"), "{error}"); +} + #[test] fn created_avatar_prefers_explicit_input() { let resolved = resolve_created_avatar_url( @@ -398,50 +411,93 @@ fn legacy_avatar_empty_when_nothing_resolves() { // ── Provider deploy payload completeness ───────────────────────────────────── -/// Regression (PR #1667 review, Thufir): the provider deploy payload must -/// carry every behavioral field the local spawn path applies — a field -/// missing here silently strips it from provider-backed agents. +/// The shared provider fixture is the contract arbiter: it must be the exact +/// richest deploy request produced by the real desktop serializers. #[test] -fn deploy_payload_carries_the_full_behavioral_quad() { - let allow = "a".repeat(64); - let record: ManagedAgentRecord = serde_json::from_str(&format!( - r#"{{ - "pubkey": "abcd1234", - "name": "test-agent", - "private_key_nsec": "nsec1fake", - "relay_url": "wss://localhost:3000", - "acp_command": "buzz-acp", - "agent_command": "goose", - "agent_args": [], - "mcp_command": "", - "turn_timeout_seconds": 320, - "system_prompt": null, - "parallelism": 4, - "respond_to": "allowlist", - "respond_to_allowlist": ["{allow}"], - "created_at": "2026-01-01T00:00:00Z", - "updated_at": "2026-01-01T00:00:00Z", - "last_started_at": null, - "last_stopped_at": null, - "last_exit_code": null, - "last_error": null - }}"# - )) - .expect("sample record"); - - let payload = deploy_payload_json( +fn deploy_payload_matches_the_shared_full_launch_fixture() { + let fixture_path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join( + "../../crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-full-launch.request.json", + ); + let fixture: serde_json::Value = serde_json::from_str( + &std::fs::read_to_string(&fixture_path) + .unwrap_or_else(|error| panic!("read {}: {error}", fixture_path.display())), + ) + .expect("parse shared provider fixture"); + let record: ManagedAgentRecord = serde_json::from_value(serde_json::json!({ + "pubkey": "abcd1234", + "name": "worker", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "relay_url": "wss://localhost:3000", + "auth_tag": "tag-1", + "acp_command": "buzz-acp", + "agent_command": "goose", + "runtime": "goose", + "model": "gpt-5", + "provider": "openai", + "env_vars": {"USER_KEY": "user-value"}, + "agent_args": [], + "mcp_command": "", + "turn_timeout_seconds": 300, + "system_prompt": null, + "idle_timeout_seconds": null, + "max_turn_duration_seconds": null, + "parallelism": 10, + "respond_to": "allowlist", + "respond_to_allowlist": ["aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "created_at": "2026-01-01T00:00:00Z", + "updated_at": "2026-01-01T00:00:00Z" + })) + .expect("fixture source record"); + let descriptor = crate::managed_agents::resolve_effective_harness_descriptor( &record, - "wss://relay.example".to_string(), - Some("gpt-x".to_string()), - Some("openai".to_string()), + &[], + &crate::managed_agents::GlobalAgentConfig::default(), + ) + .expect("resolve fixture source record descriptor"); + let launch = super::deploy::build_launch_block( + &record, + &descriptor, + &[], None, - std::collections::BTreeMap::new(), + Some("gpt-5"), + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + ); + let agent = deploy_payload_json( + &record, + "wss://relay.example".into(), + Some("gpt-5".into()), + Some("openai".into()), + None, + std::collections::BTreeMap::from([("USER_KEY".into(), "user-value".into())]), + launch, ); - assert_eq!(payload["parallelism"], 4); - assert_eq!(payload["respond_to"], "allowlist"); - assert_eq!(payload["respond_to_allowlist"][0], "a".repeat(64)); - assert_eq!(payload["model"], "gpt-x"); - assert_eq!(payload["provider"], "openai"); - assert_eq!(payload["relay_url"], "wss://relay.example"); + assert_eq!( + agent, fixture["agent"], + "desktop payload drifted from the shared provider fixture" + ); +} + +#[test] +fn tauri_platform_configs_bundle_kubernetes_only_on_supported_hosts() { + use tauri_utils::{config::parse::read_from, platform::Target}; + + let config_root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")); + for (target, expected) in [ + (Target::MacOS, true), + (Target::Linux, true), + (Target::Windows, false), + ] { + let (config, paths) = read_from(target, config_root).expect("read Tauri config"); + let external_bins = config["bundle"]["externalBin"] + .as_array() + .expect("bundle.externalBin array"); + let has_kubernetes = external_bins + .iter() + .any(|value| value == "binaries/buzz-backend-kubernetes"); + assert_eq!( + has_kubernetes, expected, + "unexpected Kubernetes externalBin for {target}; merged {paths:?}" + ); + } } diff --git a/desktop/src-tauri/src/managed_agents/backend.rs b/desktop/src-tauri/src/managed_agents/backend.rs index 5debae41c..84dd7e99d 100644 --- a/desktop/src-tauri/src/managed_agents/backend.rs +++ b/desktop/src-tauri/src/managed_agents/backend.rs @@ -1,3 +1,4 @@ +use sha2::{Digest, Sha256}; use std::io::{BufReader, Read, Write}; use std::path::{Path, PathBuf}; use std::sync::mpsc; @@ -7,6 +8,61 @@ const STDERR_CAP: usize = 65536; /// Provider responses should be small JSON objects. Cap stdout to prevent a /// buggy or malicious provider from OOM-ing the desktop process. const STDOUT_CAP: usize = 1_048_576; // 1 MB +const PROVIDER_PROTOCOL_VERSION: u64 = 1; + +fn validate_provider_info(info: &serde_json::Value) -> Result<(), String> { + let object = info + .as_object() + .ok_or_else(|| "provider info response must be a JSON object".to_string())?; + let actual_version = object + .get("protocol_version") + .and_then(serde_json::Value::as_u64); + if actual_version != Some(PROVIDER_PROTOCOL_VERSION) { + return Err(match actual_version { + Some(version) => format!( + "unsupported provider protocol version {version}; desktop requires {PROVIDER_PROTOCOL_VERSION}" + ), + None => "provider info response missing integer protocol_version".to_string(), + }); + } + if object.get("ok") != Some(&serde_json::Value::Bool(true)) { + return Err("provider info response must contain ok: true".to_string()); + } + for field in ["name", "version", "description"] { + if object + .get(field) + .is_none_or(|value| value.as_str().is_none_or(str::is_empty)) + { + return Err(format!( + "provider info response missing non-empty string {field}" + )); + } + } + if !object + .get("config_schema") + .is_some_and(serde_json::Value::is_object) + { + return Err("provider info response missing object config_schema".to_string()); + } + + const FIELDS: &[&str] = &[ + "ok", + "name", + "version", + "protocol_version", + "description", + "config_schema", + ]; + if let Some(field) = object + .keys() + .find(|field| !FIELDS.contains(&field.as_str())) + { + return Err(format!( + "provider info response contains unknown field {field}" + )); + } + Ok(()) +} /// Invoke a provider binary: write JSON to stdin, read JSON from stdout. /// @@ -333,23 +389,29 @@ pub(crate) fn redact_secrets_with(s: &str, extras: &[&str]) -> String { result } -/// Collect string values from `request["agent"]["env_vars"]` (if present) -/// to feed into [`redact_secrets_with`]. Returns an empty Vec if the -/// request shape doesn't match, which is fine — falls back to the default -/// prefix-based scrubbing. +/// Collect string values from every environment map a deploy request can +/// carry. Providers may echo any of these values in diagnostics, including +/// definition/baked values that exist only in the resolved launch block. fn env_secrets_from_request(request: &serde_json::Value) -> Vec { - request - .get("agent") - .and_then(|a| a.get("env_vars")) - .and_then(|e| e.as_object()) - .map(|obj| { - obj.values() - .filter_map(|v| v.as_str()) - .filter(|s| !s.is_empty()) - .map(String::from) - .collect() - }) - .unwrap_or_default() + let agent = request.get("agent"); + let maps = [ + agent.and_then(|value| value.get("env_vars")), + agent + .and_then(|value| value.get("launch")) + .and_then(|value| value.get("env")), + agent + .and_then(|value| value.get("launch")) + .and_then(|value| value.get("policy_env")), + ]; + + maps.into_iter() + .flatten() + .filter_map(serde_json::Value::as_object) + .flat_map(|map| map.values()) + .filter_map(serde_json::Value::as_str) + .filter(|value| !value.is_empty()) + .map(String::from) + .collect() } /// Public-in-crate helper: redact every non-empty value from `env` (plus @@ -368,22 +430,102 @@ pub(crate) fn redact_env_values_in( redact_secrets_with(s, &values) } -/// Deploy an agent via provider binary. Returns the provider-assigned agent_id. -/// -/// `request_id` is included for provider-side logging/correlation but is not -/// validated in the response — the stdin→stdout exchange is 1:1 per process. +/// Copy a resolved provider into a private staging directory while hashing +/// exactly the bytes copied. The staged file becomes non-writable before either +/// invocation, closing the path replacement and in-place rewrite races. +fn stage_provider( + binary: &Path, +) -> Result<(tempfile::TempDir, PathBuf, String, std::fs::File), String> { + let directory = tempfile::Builder::new() + .prefix("buzz-provider-") + .tempdir() + .map_err(|error| format!("failed to create provider staging directory: {error}"))?; + let suffix = if cfg!(windows) { ".exe" } else { "" }; + let staged_path = directory.path().join(format!("provider{suffix}")); + let mut source = std::fs::File::open(binary) + .map_err(|error| format!("failed to open provider for staging: {error}"))?; + let mut staged = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&staged_path) + .map_err(|error| format!("failed to create staged provider: {error}"))?; + let mut hasher = Sha256::new(); + let mut buffer = [0_u8; 64 * 1024]; + loop { + let count = source + .read(&mut buffer) + .map_err(|error| format!("failed to read provider for staging: {error}"))?; + if count == 0 { + break; + } + staged + .write_all(&buffer[..count]) + .map_err(|error| format!("failed to write staged provider: {error}"))?; + hasher.update(&buffer[..count]); + } + staged + .sync_all() + .map_err(|error| format!("failed to sync staged provider: {error}"))?; + + let mut permissions = staged + .metadata() + .map_err(|error| format!("failed to inspect staged provider: {error}"))? + .permissions(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + permissions.set_mode(0o500); + } + #[cfg(not(unix))] + permissions.set_readonly(true); + std::fs::set_permissions(&staged_path, permissions) + .map_err(|error| format!("failed to protect staged provider: {error}"))?; + drop(staged); + + #[cfg(windows)] + let execution_guard = { + use std::os::windows::fs::OpenOptionsExt; + std::fs::OpenOptions::new() + .read(true) + // Permit CreateProcess to read the image while denying replacement, + // writes, and deletion until both invocations finish. + .share_mode(windows_sys::Win32::Storage::FileSystem::FILE_SHARE_READ) + .open(&staged_path) + }; + #[cfg(not(windows))] + let execution_guard = std::fs::File::open(&staged_path); + let execution_guard = execution_guard + .map_err(|error| format!("failed to lock staged provider for execution: {error}"))?; + Ok(( + directory, + staged_path, + hex::encode(hasher.finalize()), + execution_guard, + )) +} + +/// Deploy through one immutable staged copy: negotiate protocol v1 before the +/// secret-bearing request, then invoke deploy on those exact same bytes. pub fn provider_deploy( binary: &Path, agent: &serde_json::Value, provider_config: &serde_json::Value, ) -> Result { + let (_directory, staged, _digest, _execution_guard) = stage_provider(binary)?; + let info_request = serde_json::json!({ + "op": "info", + "request_id": uuid::Uuid::new_v4().to_string(), + }); + let info = invoke_provider(&staged, &info_request, Duration::from_secs(10))?; + validate_provider_info(&info)?; + let request = serde_json::json!({ "op": "deploy", "request_id": uuid::Uuid::new_v4().to_string(), "agent": agent, "provider_config": provider_config, }); - let resp = invoke_provider(binary, &request, Duration::from_secs(600))?; + let resp = invoke_provider(&staged, &request, Duration::from_secs(600))?; resp["agent_id"] .as_str() .map(String::from) @@ -423,6 +565,24 @@ pub fn validate_provider_config(config: &serde_json::Value) -> Result<(), String Ok(()) } +/// Derive a provider id from the filename Tauri stages at runtime. Tauri +/// removes its target-triple suffix while copying an external binary, but on +/// Windows leaves the executable/script extension, which is not part of the +/// provider id. +fn provider_id_from_filename(name: &str) -> Option<&str> { + let raw = name.strip_prefix("buzz-backend-")?; + let id = [".exe", ".bat", ".cmd"] + .into_iter() + .find_map(|extension| { + raw.get(raw.len().saturating_sub(extension.len())..) + .filter(|suffix| suffix.eq_ignore_ascii_case(extension)) + .map(|_| &raw[..raw.len() - extension.len()]) + }) + .unwrap_or(raw); + + (!id.is_empty()).then_some(id) +} + /// Enumerate PATH for buzz-backend-* executables. Returns (id, path) pairs. /// Only includes files that are executable. Does NOT execute any binaries. /// @@ -464,10 +624,12 @@ pub fn discover_provider_candidates() -> Vec<(String, PathBuf)> { }; for entry in entries.flatten() { let name = entry.file_name().to_string_lossy().to_string(); - if let Some(id) = name.strip_prefix(prefix) { - if !id.is_empty() && !seen.contains(&name) && is_executable(&entry.path()) { - seen.insert(name.clone()); - results.push((id.to_string(), entry.path())); + if name.starts_with(prefix) { + if let Some(id) = provider_id_from_filename(&name) { + if !seen.contains(&name) && is_executable(&entry.path()) { + seen.insert(name.clone()); + results.push((id.to_string(), entry.path())); + } } } } @@ -538,203 +700,5 @@ pub struct BackendProviderInfo { } #[cfg(test)] -mod tests { - use super::*; - - #[test] - fn redact_secrets_replaces_nsec() { - let s = "key=nsec1abc123def456 other"; - let r = redact_secrets(s); - assert!(r.contains("[REDACTED]")); - assert!(!r.contains("nsec1abc123def456")); - } - - #[test] - fn redact_secrets_replaces_token() { - let s = r#"{"token":"sprt_tok_xyz789"}"#; - let r = redact_secrets(s); - assert!(r.contains("[REDACTED]")); - assert!(!r.contains("sprt_tok_xyz789")); - } - - #[test] - fn redact_secrets_with_extras_scrubs_user_env_values() { - // If a provider echoes back a user-supplied API key in its error - // output, the desktop must not surface that secret unredacted via - // `last_error`. We scrub the literal values that came from the - // request's `agent.env_vars`. - let secret = "sk-ant-api03-abc123def456"; - let stderr = format!("auth failed with key {secret} on host api.anthropic.com"); - let r = redact_secrets_with(&stderr, &[secret]); - assert!(r.contains("[REDACTED]")); - assert!(!r.contains(secret)); - } - - #[test] - fn redact_secrets_with_extras_skips_short_values() { - // Don't scrub values shorter than 4 chars — too noisy. - let r = redact_secrets_with("error code: 42", &["42"]); - assert!(r.contains("42")); - } - - /// GitHub tokens are recognised by shape, so one that never passed through - /// our environment — embedded in a remote URL an installer echoes — is - /// still scrubbed. The scan runs to the next whitespace or quote, so the - /// rest of the URL goes with it; over-redaction is the safe direction. - #[test] - fn redact_secrets_with_scrubs_github_token_prefixes() { - for token in [ - "ghp_abcdefghij0123456789", - "gho_abcdefghij0123456789", - "ghu_abcdefghij0123456789", - "ghs_abcdefghij0123456789", - "ghr_abcdefghij0123456789", - "github_pat_abcdefghij0123456789", - ] { - let r = - redact_secrets_with(&format!("cloning https://{token}@github.com/o/r now"), &[]); - assert!(!r.contains(token), "leaked {token}: {r}"); - assert!(r.contains("[REDACTED]"), "got: {r}"); - assert!(r.contains("cloning"), "scan must stop at whitespace: {r}"); - assert!(r.ends_with(" now"), "scan must stop at whitespace: {r}"); - } - } - - #[test] - fn redact_secrets_with_extras_terminates_when_value_substring_of_marker() { - // Regression: an earlier impl used `while let Some(pos) = find(value)` - // which never terminates if the user's env value is a substring of - // the replacement marker `[REDACTED]` — each replacement - // reintroduces the same text. Now uses `str::replace` (single-pass). - for value in ["REDACTED", "EDACTE", "REDA", "ACTED"] { - let r = redact_secrets_with(&format!("leak={value}"), &[value]); - assert!(r.contains("[REDACTED]")); - } - } - - #[test] - fn redact_secrets_with_extras_handles_overlapping_secrets() { - // Longer entries get scrubbed first so the substring "abc12" isn't - // matched before "abc123" is consumed. - let s = "key1=abc123 key2=abc12"; - let r = redact_secrets_with(s, &["abc12", "abc123"]); - assert!(!r.contains("abc123")); - assert!(!r.contains("abc12 ")); - } - - #[test] - fn env_secrets_from_request_extracts_string_values() { - let req = serde_json::json!({ - "op": "deploy", - "agent": { - "env_vars": { - "ANTHROPIC_API_KEY": "sk-ant-test", - "EMPTY": "", - "NUMERIC": 42, - }, - }, - }); - let secrets = env_secrets_from_request(&req); - assert!(secrets.iter().any(|v| v == "sk-ant-test")); - // Empty and non-string values are filtered out. - assert_eq!(secrets.len(), 1); - } - - #[test] - fn env_secrets_from_request_handles_missing_shape() { - assert!(env_secrets_from_request(&serde_json::json!({})).is_empty()); - assert!(env_secrets_from_request(&serde_json::json!({"agent": {}})).is_empty()); - assert!( - env_secrets_from_request(&serde_json::json!({"agent": {"env_vars": null}})).is_empty() - ); - } - - #[test] - fn redact_env_values_in_scrubs_map_values() { - let mut env = std::collections::BTreeMap::new(); - env.insert("ANTHROPIC_API_KEY".to_string(), "sk-ant-real".to_string()); - env.insert("EMPTY".to_string(), String::new()); - let stderr = "auth=sk-ant-real failed; other context"; - let r = redact_env_values_in(stderr, &env); - assert!(!r.contains("sk-ant-real")); - assert!(r.contains("[REDACTED]")); - } - - #[test] - fn validate_provider_config_rejects_secret_key() { - let cfg = serde_json::json!({"api_key": "val"}); - assert!(validate_provider_config(&cfg).is_err()); - } - - #[test] - fn validate_provider_config_rejects_nested() { - let cfg = serde_json::json!({"region": {"us": "east"}}); - assert!(validate_provider_config(&cfg).is_err()); - } - - #[test] - fn validate_provider_config_accepts_scalars() { - let cfg = serde_json::json!({"region": "us-east-1", "tier": "standard"}); - assert!(validate_provider_config(&cfg).is_ok()); - } - - #[test] - fn validate_provider_config_allows_key_as_substring() { - // "keyboard", "monkey" contain "key" as substring but not as a word segment. - let cfg = serde_json::json!({"keyboard_layout": "us", "monkey_wrench": "tight"}); - assert!(validate_provider_config(&cfg).is_ok()); - } - - #[test] - fn validate_provider_config_rejects_camel_case_secrets() { - assert!(validate_provider_config(&serde_json::json!({"apiKey": "val"})).is_err()); - assert!(validate_provider_config(&serde_json::json!({"accessToken": "val"})).is_err()); - assert!(validate_provider_config(&serde_json::json!({"clientSecret": "val"})).is_err()); - // ALL-CAPS variants - assert!(validate_provider_config(&serde_json::json!({"apiKEY": "val"})).is_err()); - assert!(validate_provider_config(&serde_json::json!({"accessTOKEN": "val"})).is_err()); - } - - #[test] - fn split_config_key_handles_all_styles() { - assert_eq!(split_config_key("apiKey"), vec!["api", "key"]); - assert_eq!(split_config_key("access_token"), vec!["access", "token"]); - assert_eq!(split_config_key("keyboard"), vec!["keyboard"]); - assert_eq!(split_config_key("client-secret"), vec!["client", "secret"]); - // Acronym runs stay together - assert_eq!(split_config_key("APIKey"), vec!["api", "key"]); - assert_eq!(split_config_key("apiKEY"), vec!["api", "key"]); - assert_eq!(split_config_key("accessTOKEN"), vec!["access", "token"]); - assert_eq!(split_config_key("MyAPIKey"), vec!["my", "api", "key"]); - } - - #[test] - fn resolve_provider_binary_rejects_invalid_ids() { - // Path traversal - assert!(resolve_provider_binary("../evil").is_err()); - // Empty - assert!(resolve_provider_binary("").is_err()); - // Uppercase - assert!(resolve_provider_binary("MyProvider").is_err()); - // Spaces - assert!(resolve_provider_binary("my provider").is_err()); - // Shell metacharacters - assert!(resolve_provider_binary("foo;rm -rf /").is_err()); - // Valid format but not on PATH — should fail with "not found" - assert!(resolve_provider_binary("nonexistent-test-id-12345").is_err()); - } - - #[test] - fn resolve_provider_binary_accepts_valid_id_format() { - // Valid ID format should pass validation. If the binary happens to - // exist on PATH, Ok is returned; otherwise Err contains "not found" - // (not "invalid provider ID"). Either outcome proves validation passed. - match resolve_provider_binary("zzz-nonexistent-test-provider") { - Ok(_) => {} // unlikely but fine — binary exists - Err(e) => assert!( - e.contains("not found"), - "expected 'not found' error, got: {e}" - ), - } - } -} +#[path = "backend_tests.rs"] +mod tests; diff --git a/desktop/src-tauri/src/managed_agents/backend_tests.rs b/desktop/src-tauri/src/managed_agents/backend_tests.rs new file mode 100644 index 000000000..ce1f81466 --- /dev/null +++ b/desktop/src-tauri/src/managed_agents/backend_tests.rs @@ -0,0 +1,452 @@ +use super::*; + +#[test] +fn redact_secrets_replaces_nsec() { + let s = "key=nsec1abc123def456 other"; + let r = redact_secrets(s); + assert!(r.contains("[REDACTED]")); + assert!(!r.contains("nsec1abc123def456")); +} + +#[test] +fn redact_secrets_replaces_token() { + let s = r#"{"token":"sprt_tok_xyz789"}"#; + let r = redact_secrets(s); + assert!(r.contains("[REDACTED]")); + assert!(!r.contains("sprt_tok_xyz789")); +} + +#[test] +fn redact_secrets_with_extras_scrubs_user_env_values() { + // If a provider echoes back a user-supplied API key in its error + // output, the desktop must not surface that secret unredacted via + // `last_error`. We scrub the literal values that came from the + // request's `agent.env_vars`. + let secret = "sk-ant-api03-abc123def456"; + let stderr = format!("auth failed with key {secret} on host api.anthropic.com"); + let r = redact_secrets_with(&stderr, &[secret]); + assert!(r.contains("[REDACTED]")); + assert!(!r.contains(secret)); +} + +#[test] +fn redact_secrets_with_extras_skips_short_values() { + // Don't scrub values shorter than 4 chars — too noisy. + let r = redact_secrets_with("error code: 42", &["42"]); + assert!(r.contains("42")); +} + +/// GitHub tokens are recognised by shape, so one that never passed through +/// our environment — embedded in a remote URL an installer echoes — is +/// still scrubbed. The scan runs to the next whitespace or quote, so the +/// rest of the URL goes with it; over-redaction is the safe direction. +#[test] +fn redact_secrets_with_scrubs_github_token_prefixes() { + for token in [ + "ghp_abcdefghij0123456789", + "gho_abcdefghij0123456789", + "ghu_abcdefghij0123456789", + "ghs_abcdefghij0123456789", + "ghr_abcdefghij0123456789", + "github_pat_abcdefghij0123456789", + ] { + let r = redact_secrets_with(&format!("cloning https://{token}@github.com/o/r now"), &[]); + assert!(!r.contains(token), "leaked {token}: {r}"); + assert!(r.contains("[REDACTED]"), "got: {r}"); + assert!(r.contains("cloning"), "scan must stop at whitespace: {r}"); + assert!(r.ends_with(" now"), "scan must stop at whitespace: {r}"); + } +} + +#[test] +fn redact_secrets_with_extras_terminates_when_value_substring_of_marker() { + // Regression: an earlier impl used `while let Some(pos) = find(value)` + // which never terminates if the user's env value is a substring of + // the replacement marker `[REDACTED]` — each replacement + // reintroduces the same text. Now uses `str::replace` (single-pass). + for value in ["REDACTED", "EDACTE", "REDA", "ACTED"] { + let r = redact_secrets_with(&format!("leak={value}"), &[value]); + assert!(r.contains("[REDACTED]")); + } +} + +#[test] +fn redact_secrets_with_extras_handles_overlapping_secrets() { + // Longer entries get scrubbed first so the substring "abc12" isn't + // matched before "abc123" is consumed. + let s = "key1=abc123 key2=abc12"; + let r = redact_secrets_with(s, &["abc12", "abc123"]); + assert!(!r.contains("abc123")); + assert!(!r.contains("abc12 ")); +} + +#[test] +fn env_secrets_from_request_extracts_string_values() { + let req = serde_json::json!({ + "op": "deploy", + "agent": { + "env_vars": { + "ANTHROPIC_API_KEY": "sk-ant-test", + "EMPTY": "", + "NUMERIC": 42, + }, + }, + }); + let secrets = env_secrets_from_request(&req); + assert!(secrets.iter().any(|v| v == "sk-ant-test")); + // Empty and non-string values are filtered out. + assert_eq!(secrets.len(), 1); +} + +#[test] +fn env_secrets_from_request_handles_missing_shape() { + assert!(env_secrets_from_request(&serde_json::json!({})).is_empty()); + assert!(env_secrets_from_request(&serde_json::json!({"agent": {}})).is_empty()); + assert!(env_secrets_from_request(&serde_json::json!({"agent": {"env_vars": null}})).is_empty()); +} + +#[test] +fn redact_env_values_in_scrubs_map_values() { + let mut env = std::collections::BTreeMap::new(); + env.insert("ANTHROPIC_API_KEY".to_string(), "sk-ant-real".to_string()); + env.insert("EMPTY".to_string(), String::new()); + let stderr = "auth=sk-ant-real failed; other context"; + let r = redact_env_values_in(stderr, &env); + assert!(!r.contains("sk-ant-real")); + assert!(r.contains("[REDACTED]")); +} + +#[test] +fn env_secrets_from_request_includes_resolved_launch_maps() { + let req = serde_json::json!({ + "agent": { + "env_vars": {"LEGACY": "legacy-secret"}, + "launch": { + "env": {"PERSONA": "persona-secret"}, + "policy_env": {"POLICY": "policy-secret"} + } + } + }); + let secrets = env_secrets_from_request(&req); + assert_eq!(secrets.len(), 3); + for secret in ["legacy-secret", "persona-secret", "policy-secret"] { + assert!(secrets.iter().any(|candidate| candidate == secret)); + } +} + +#[cfg(unix)] +fn write_test_provider(path: &Path, body: &str) { + use std::os::unix::fs::PermissionsExt; + std::fs::write(path, format!("#!/bin/sh\nset -eu\n{body}\n")).unwrap(); + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o700)).unwrap(); +} + +#[cfg(unix)] +#[test] +fn provider_deploy_negotiates_and_deploys_the_same_staged_bytes() { + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + let log = directory.path().join("invocations"); + let body = format!( + r#"read request +printf '%s\n' "$0" >> '{}' +case "$request" in + *\"op\":\"info\"*) printf '%s\n' '{{"ok":true,"name":"test","version":"1.0.0","protocol_version":1,"description":"test provider","config_schema":{{}}}}' ;; + *\"op\":\"deploy\"*) printf '%s\n' '{{"ok":true,"agent_id":"remote-1"}}' ;; +esac"#, + log.display() + ); + write_test_provider(&provider, &body); + + let id = provider_deploy(&provider, &serde_json::json!({}), &serde_json::json!({})) + .expect("staged deploy"); + assert_eq!(id, "remote-1"); + let paths: Vec<_> = std::fs::read_to_string(log) + .unwrap() + .lines() + .map(str::to_owned) + .collect(); + assert_eq!(paths.len(), 2); + assert_eq!(paths[0], paths[1]); + assert_ne!(Path::new(&paths[0]), provider); + assert!( + !Path::new(&paths[0]).exists(), + "staging directory must be deleted" + ); +} + +#[cfg(unix)] +fn replacement_provider() -> &'static str { + r#"#!/bin/sh +set -eu +read request +case "$request" in + *\"op\":\"info\"*) printf '%s\n' '{"ok":true,"name":"replacement","version":"9.9.9","protocol_version":1,"description":"replacement provider","config_schema":{}}' ;; + *\"op\":\"deploy\"*) printf '%s\n' '{"ok":true,"agent_id":"replacement-bytes-ran"}' ;; +esac +"# +} + +#[cfg(unix)] +#[test] +fn provider_deploy_uses_staged_bytes_after_same_inode_source_rewrite() { + use std::os::unix::fs::MetadataExt; + + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + let replacement = directory.path().join("replacement"); + std::fs::write(&replacement, replacement_provider()).unwrap(); + let body = format!( + r#"read request +case "$request" in + *\"op\":\"info\"*) + cat '{}' > '{}' + chmod 700 '{}' + printf '%s\n' '{{"ok":true,"name":"original","version":"1.0.0","protocol_version":1,"description":"original provider","config_schema":{{}}}}' + ;; + *\"op\":\"deploy\"*) printf '%s\n' '{{"ok":true,"agent_id":"original-staged-bytes"}}' ;; +esac"#, + replacement.display(), + provider.display(), + provider.display(), + ); + write_test_provider(&provider, &body); + let inode_before = std::fs::metadata(&provider).unwrap().ino(); + + let id = provider_deploy(&provider, &serde_json::json!({}), &serde_json::json!({})) + .expect("deploy from immutable staged copy"); + + assert_eq!(id, "original-staged-bytes"); + assert_eq!( + std::fs::metadata(&provider).unwrap().ino(), + inode_before, + "test must rewrite the source binary in place" + ); + assert_eq!( + std::fs::read_to_string(&provider).unwrap(), + replacement_provider(), + "source pathname must contain replacement bytes before deploy" + ); +} + +#[cfg(unix)] +#[test] +fn provider_deploy_uses_staged_bytes_after_source_pathname_replacement() { + use std::os::unix::fs::{MetadataExt, PermissionsExt}; + + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + let replacement = directory.path().join("replacement"); + std::fs::write(&replacement, replacement_provider()).unwrap(); + std::fs::set_permissions(&replacement, std::fs::Permissions::from_mode(0o700)).unwrap(); + let body = format!( + r#"read request +case "$request" in + *\"op\":\"info\"*) + mv '{}' '{}' + printf '%s\n' '{{"ok":true,"name":"original","version":"1.0.0","protocol_version":1,"description":"original provider","config_schema":{{}}}}' + ;; + *\"op\":\"deploy\"*) printf '%s\n' '{{"ok":true,"agent_id":"original-staged-bytes"}}' ;; +esac"#, + replacement.display(), + provider.display(), + ); + write_test_provider(&provider, &body); + let inode_before = std::fs::metadata(&provider).unwrap().ino(); + + let id = provider_deploy(&provider, &serde_json::json!({}), &serde_json::json!({})) + .expect("deploy from immutable staged copy"); + + assert_eq!(id, "original-staged-bytes"); + assert_ne!( + std::fs::metadata(&provider).unwrap().ino(), + inode_before, + "test must replace the source pathname with a different inode" + ); + assert_eq!( + std::fs::read_to_string(&provider).unwrap(), + replacement_provider(), + "source pathname must contain replacement bytes before deploy" + ); +} + +#[cfg(unix)] +#[test] +fn provider_deploy_refuses_mismatch_before_sending_agent_secret() { + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + let marker = directory.path().join("deploy-received"); + let body = format!( + r#"read request +case "$request" in + *\"op\":\"info\"*) printf '%s\n' '{{"ok":true,"name":"test","version":"2.0.0","protocol_version":2,"description":"test provider","config_schema":{{}}}}' ;; + *\"op\":\"deploy\"*) touch '{}'; printf '%s\n' '{{"ok":true,"agent_id":"bad"}}' ;; +esac"#, + marker.display() + ); + write_test_provider(&provider, &body); + + let error = provider_deploy( + &provider, + &serde_json::json!({"private_key_nsec": "nsec1must-not-cross"}), + &serde_json::json!({}), + ) + .unwrap_err(); + assert!(error.contains("protocol version 2"), "{error}"); + assert!(!marker.exists()); + assert!(!error.contains("nsec1must-not-cross")); +} + +#[cfg(unix)] +#[test] +fn provider_deploy_requires_an_explicit_integer_protocol_version() { + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + write_test_provider( + &provider, + r#"read request +printf '%s\n' '{"ok":true,"version":"1.0.0"}'"#, + ); + + let error = + provider_deploy(&provider, &serde_json::json!({}), &serde_json::json!({})).unwrap_err(); + assert!( + error.contains("missing integer protocol_version"), + "{error}" + ); +} + +#[test] +fn provider_info_requires_the_complete_flat_wire_shape() { + let complete = serde_json::json!({ + "ok": true, + "name": "kubernetes", + "version": "1.0.0", + "protocol_version": 1, + "description": "Kubernetes provider", + "config_schema": {} + }); + assert!(validate_provider_info(&complete).is_ok()); + + let mut missing = complete.clone(); + missing.as_object_mut().unwrap().remove("config_schema"); + assert!(validate_provider_info(&missing) + .unwrap_err() + .contains("config_schema")); + + let mut nested = complete; + nested.as_object_mut().unwrap().insert( + "provider".into(), + serde_json::json!({"protocol_version": 1}), + ); + assert!(validate_provider_info(&nested) + .unwrap_err() + .contains("unknown field provider")); +} + +#[test] +fn validate_provider_config_rejects_secret_key() { + let cfg = serde_json::json!({"api_key": "val"}); + assert!(validate_provider_config(&cfg).is_err()); +} + +#[test] +fn validate_provider_config_rejects_nested() { + let cfg = serde_json::json!({"region": {"us": "east"}}); + assert!(validate_provider_config(&cfg).is_err()); +} + +#[test] +fn validate_provider_config_accepts_scalars() { + let cfg = serde_json::json!({"region": "us-east-1", "tier": "standard"}); + assert!(validate_provider_config(&cfg).is_ok()); +} + +#[test] +fn validate_provider_config_allows_key_as_substring() { + // "keyboard", "monkey" contain "key" as substring but not as a word segment. + let cfg = serde_json::json!({"keyboard_layout": "us", "monkey_wrench": "tight"}); + assert!(validate_provider_config(&cfg).is_ok()); +} + +#[test] +fn validate_provider_config_rejects_camel_case_secrets() { + assert!(validate_provider_config(&serde_json::json!({"apiKey": "val"})).is_err()); + assert!(validate_provider_config(&serde_json::json!({"accessToken": "val"})).is_err()); + assert!(validate_provider_config(&serde_json::json!({"clientSecret": "val"})).is_err()); + // ALL-CAPS variants + assert!(validate_provider_config(&serde_json::json!({"apiKEY": "val"})).is_err()); + assert!(validate_provider_config(&serde_json::json!({"accessTOKEN": "val"})).is_err()); +} + +#[test] +fn split_config_key_handles_all_styles() { + assert_eq!(split_config_key("apiKey"), vec!["api", "key"]); + assert_eq!(split_config_key("access_token"), vec!["access", "token"]); + assert_eq!(split_config_key("keyboard"), vec!["keyboard"]); + assert_eq!(split_config_key("client-secret"), vec!["client", "secret"]); + // Acronym runs stay together + assert_eq!(split_config_key("APIKey"), vec!["api", "key"]); + assert_eq!(split_config_key("apiKEY"), vec!["api", "key"]); + assert_eq!(split_config_key("accessTOKEN"), vec!["access", "token"]); + assert_eq!(split_config_key("MyAPIKey"), vec!["my", "api", "key"]); +} + +#[test] +fn provider_filename_strips_the_windows_extension() { + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes.exe"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes.EXE"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes.bat"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes.CMD"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-my-provider"), + Some("my-provider") + ); + assert_eq!(provider_id_from_filename("other"), None); +} + +#[test] +fn resolve_provider_binary_rejects_invalid_ids() { + // Path traversal + assert!(resolve_provider_binary("../evil").is_err()); + // Empty + assert!(resolve_provider_binary("").is_err()); + // Uppercase + assert!(resolve_provider_binary("MyProvider").is_err()); + // Spaces + assert!(resolve_provider_binary("my provider").is_err()); + // Shell metacharacters + assert!(resolve_provider_binary("foo;rm -rf /").is_err()); + // Valid format but not on PATH — should fail with "not found" + assert!(resolve_provider_binary("nonexistent-test-id-12345").is_err()); +} + +#[test] +fn resolve_provider_binary_accepts_valid_id_format() { + // Valid ID format should pass validation. If the binary happens to + // exist on PATH, Ok is returned; otherwise Err contains "not found" + // (not "invalid provider ID"). Either outcome proves validation passed. + match resolve_provider_binary("zzz-nonexistent-test-provider") { + Ok(_) => {} // unlikely but fine — binary exists + Err(e) => assert!( + e.contains("not found"), + "expected 'not found' error, got: {e}" + ), + } +} diff --git a/desktop/src-tauri/src/managed_agents/env_vars.rs b/desktop/src-tauri/src/managed_agents/env_vars.rs index 592a5cbbd..1653371e7 100644 --- a/desktop/src-tauri/src/managed_agents/env_vars.rs +++ b/desktop/src-tauri/src/managed_agents/env_vars.rs @@ -77,6 +77,10 @@ pub(crate) const RESERVED_ENV_KEYS: &[&str] = &[ "BUZZ_ACP_RESPOND_TO", "BUZZ_ACP_RESPOND_TO_ALLOWLIST", "BUZZ_ACP_AGENT_OWNER", + // Remote lifetime/presence policy: user env must not disable the + // desktop/provider-owned bounds while the saved record still promises them. + "BUZZ_ACP_EXIT_AFTER_INACTIVITY", + "BUZZ_ACP_NO_PRESENCE", // Readiness handoff: desktop is the ONLY readiness source. A saved or // ambient env var must not be able to forge setup mode (NotReady) on a // Ready agent or suppress it (empty/stale payload) on a NotReady one. @@ -307,28 +311,5 @@ pub(crate) fn live_persona_env( .unwrap_or_default() } -/// Resolve live env_vars for a linked persona, loading personas from disk. -/// -/// Returns the persona's `env_vars` map if a persona_id is provided and found; -/// returns an empty map if no persona is linked. Errors if the linked persona -/// is missing. Used by the provider deploy path, which has no pre-loaded -/// persona slice. -pub(crate) fn resolve_persona_env( - app: &tauri::AppHandle, - persona_id: Option<&str>, -) -> Result, String> { - let Some(pid) = persona_id else { - return Ok(std::collections::BTreeMap::new()); - }; - let personas = super::load_personas(app).map_err(|e| { - format!("failed to load personas while resolving env for persona `{pid}`: {e}") - })?; - let persona = personas - .into_iter() - .find(|p| p.id == pid) - .ok_or_else(|| format!("persona `{pid}` not found while resolving env"))?; - Ok(persona.env_vars) -} - #[cfg(test)] mod tests; diff --git a/desktop/src-tauri/src/managed_agents/env_vars/tests.rs b/desktop/src-tauri/src/managed_agents/env_vars/tests.rs index cf57b1254..534c2e083 100644 --- a/desktop/src-tauri/src/managed_agents/env_vars/tests.rs +++ b/desktop/src-tauri/src/managed_agents/env_vars/tests.rs @@ -158,6 +158,15 @@ fn reserved_keys_include_respond_to_gate() { } } +#[test] +fn reserved_keys_include_remote_lifetime_policy() { + for key in ["BUZZ_ACP_EXIT_AFTER_INACTIVITY", "BUZZ_ACP_NO_PRESENCE"] { + assert!(is_reserved_env_key(key), "{key} should be reserved"); + let agent = map(&[(key, "0")]); + assert!(merged_user_env(&BTreeMap::new(), &agent).is_empty()); + } +} + #[test] fn reserved_keys_include_code_execution_surface() { // The agent/MCP command + args are what Buzz actually exec's. diff --git a/desktop/src-tauri/tauri.conf.json b/desktop/src-tauri/tauri.conf.json index 1ff8bd20e..93326b996 100644 --- a/desktop/src-tauri/tauri.conf.json +++ b/desktop/src-tauri/tauri.conf.json @@ -55,6 +55,7 @@ "externalBin": [ "binaries/buzz-acp", "binaries/buzz-agent", + "binaries/buzz-backend-kubernetes", "binaries/buzz-dev-mcp", "binaries/git-credential-nostr", "binaries/buzz" diff --git a/desktop/src-tauri/tauri.windows.conf.json b/desktop/src-tauri/tauri.windows.conf.json new file mode 100644 index 000000000..abf09b5bf --- /dev/null +++ b/desktop/src-tauri/tauri.windows.conf.json @@ -0,0 +1,11 @@ +{ + "bundle": { + "externalBin": [ + "binaries/buzz-acp", + "binaries/buzz-agent", + "binaries/buzz-dev-mcp", + "binaries/git-credential-nostr", + "binaries/buzz" + ] + } +} diff --git a/desktop/src/features/agents/ui/ProviderConfigFields.test.mjs b/desktop/src/features/agents/ui/ProviderConfigFields.test.mjs new file mode 100644 index 000000000..0dfd80d44 --- /dev/null +++ b/desktop/src/features/agents/ui/ProviderConfigFields.test.mjs @@ -0,0 +1,31 @@ +import assert from "node:assert/strict"; +import { describe, it } from "node:test"; + +import { coerceConfigValues } from "./ProviderConfigFields.tsx"; + +const schema = { + properties: { + inactivity_seconds: { type: "integer" }, + threshold: { type: "number" }, + label: { type: "string" }, + }, +}; + +describe("coerceConfigValues", () => { + it("omits cleared numeric fields without losing explicit zero", () => { + assert.deepEqual( + coerceConfigValues( + { inactivity_seconds: "", threshold: "0", label: "" }, + schema, + ), + { threshold: 0, label: "" }, + ); + }); + + it("preserves nonempty invalid numeric input for provider validation", () => { + assert.deepEqual( + coerceConfigValues({ inactivity_seconds: "not-a-number" }, schema), + { inactivity_seconds: "not-a-number" }, + ); + }); +}); diff --git a/desktop/src/features/agents/ui/ProviderConfigFields.tsx b/desktop/src/features/agents/ui/ProviderConfigFields.tsx index e922dd9c1..031b0b727 100644 --- a/desktop/src/features/agents/ui/ProviderConfigFields.tsx +++ b/desktop/src/features/agents/ui/ProviderConfigFields.tsx @@ -14,7 +14,8 @@ export function coerceConfigValues( for (const [key, value] of Object.entries(config)) { const prop = properties[key] as Record | undefined; const schemaType = prop?.type; - if ((schemaType === "integer" || schemaType === "number") && value !== "") { + if (schemaType === "integer" || schemaType === "number") { + if (value === "") continue; const num = Number(value); result[key] = Number.isNaN(num) ? value : num; } else if (schemaType === "boolean") { diff --git a/docs/remote-agents.md b/docs/remote-agents.md index 4664de21e..48ce85f5e 100644 --- a/docs/remote-agents.md +++ b/docs/remote-agents.md @@ -203,17 +203,20 @@ one. deployment axis (`deployed`/`not_deployed`, from the stored `backend_agent_id`) is bookkeeping, not liveness. Staleness bound: presence can be wrong for the window between an abnormal agent death (SIGKILL, node - loss) and the relay's presence expiry — **90 seconds** + loss) and the relay's presence expiry — **180 seconds** (`PRESENCE_TTL_SECS`, `buzz-pubsub/src/presence.rs:16`; the vision's - "ninety seconds of a wrong dot, never an indefinite one"), the accepted - cost of M1. + "a bounded wrong dot, never an indefinite one"), the accepted + cost of M1. The specific number is a relay-wide constant, not a + remote-agent choice: #3783 raised it from 90s to keep a three-heartbeat + expiry window after the desktop heartbeat moved to 60s. What I3 promises + is that the window is *bounded*, not its width. The Kubernetes binding minimizes the *avoidable* part of that window by sizing the termination grace period to the harness's full graceful-shutdown path (§K8s Grace). Two consequences the bound imposes: (a) the harness's presence-suppression knob, `BUZZ_ACP_NO_PRESENCE`, MUST join `RESERVED_ENV_KEYS` — locally the knob is cosmetic (the process and UI remain visible), but remotely M1 makes presence the *only* signal, so an - unreserved user env var would convert "wrong for ≤90s" into "wrong + unreserved user env var would convert "wrong for ≤180s" into "wrong indefinitely" and silently disarm the one bound in print; (b) presence is scoped to a **community**: the relay derives community from its host, so the deploy-time `relay_url` binds the body to one community for its whole @@ -273,7 +276,7 @@ one. directives. Any revive-on-abnormal-death policy carries a universal precondition: the supervisor can distinguish intent from accident only if the harness formally promises *clean exit = exit code 0* on every - intentional path and nonzero otherwise, pinned by test. At `c1bca1b56` + intentional path and nonzero otherwise, pinned by test. At `28ae6cd21` that property is emergent, not defended (Known Defect 6); restart-on-failure before the pinned contract is how a refactor silently converts every clean stop into a restart loop with no failing @@ -308,7 +311,7 @@ entry of `PATH`, and `~/.local/bin`, for executables named `buzz-backend-`. The suffix after the prefix is the provider id and MUST match `[a-z0-9][a-z0-9_-]*`. On Windows, an `.exe`/`.bat`/`.cmd` extension MUST be stripped before the id is derived (see §Known Defects — as of -`c1bca1b56` it is not, so Windows providers probe but cannot deploy). First +`28ae6cd21` it is not, so Windows providers probe but cannot deploy). First hit per filename wins. Discovery executes nothing. **Shadowing and invalid candidates are diagnosable, not silent.** First-hit @@ -319,7 +322,7 @@ deploy-time errors MUST be able to surface: the selected binary's full path, any shadowed candidates for the same id (later-PATH duplicates), and candidates rejected for malformed names. A deploy error that names which binary ran answers the first question a user with two copies of -`buzz-backend-kubernetes` will ask. (At `c1bca1b56` discovery records only +`buzz-backend-kubernetes` will ask. (At `28ae6cd21` discovery records only the winning path — a desktop change alongside Known Defect 3's.) **Resolution rule.** Every subsequent operation resolves the provider id @@ -330,7 +333,7 @@ to a binary discovery would not have found. **Pre-secret negotiation gate (normative).** Declaring `protocol_version` is worthless if nothing checks it before the nsec crosses the trust -boundary — and at `c1bca1b56` nothing does: `provider_deploy` invokes +boundary — and at `28ae6cd21` nothing does: `provider_deploy` invokes `deploy` directly, so a stale UI-time probe (or a binary replaced on PATH since that probe) can receive `private_key_nsec` unchecked (Known Defect 5). The deploy path MUST: resolve the provider id **once**; copy the @@ -420,7 +423,7 @@ timeout: 600s ``` The agent payload (field list per -`commands/agents_deploy.rs: deploy_payload_json` at `c1bca1b56`; the +`commands/agents_deploy.rs: deploy_payload_json` at `28ae6cd21`; the `launch` block is a normative addition not yet emitted — Known Defect 3): | field | meaning | @@ -429,7 +432,7 @@ The agent payload (field list per | `relay_url` | concrete WS URL (workspace fallback materialized — the remote side has no workspace notion) | | `private_key_nsec` | **the identity** (I1: never empty) | | `auth_tag` | NIP-OA owner attestation | -| `agent_command`, `agent_args` | the ACP agent under the harness (configurable-harness support). At `c1bca1b56` these are raw record bytes — see Known Defect 3: the normative source is the resolved descriptor in `launch` | +| `agent_command`, `agent_args` | the ACP agent under the harness (configurable-harness support). At `28ae6cd21` these are raw record bytes — see Known Defect 3: the normative source is the resolved descriptor in `launch` | | `system_prompt`, `model`, `provider` | effective values, live-persona-first resolution | | `turn_timeout_seconds`, `idle_timeout_seconds`, `max_turn_duration_seconds` | harness timeout knobs | | `parallelism` | concurrent-turn bound | @@ -466,7 +469,7 @@ deploy of the same key, or manually). Reproducing the local spawn's launch semantics requires state only the desktop can resolve: the runtime-metadata table (`model_env_var`, `provider_env_var`, `provider_locked`, `default_env` — -`discovery.rs:74-193`), the six-layer env resolution, harness-definition +`discovery.rs:75-207`), the six-layer env resolution, harness-definition command/args fallback, team instructions, session title, the respond-to gate's legacy owner fallback, and the mesh rewrite. A provider MUST NOT reimplement that derivation — it would be a second copy of desktop runtime @@ -529,12 +532,12 @@ mis-tiered (below): verbatim would bake a host accident into the pod. Launch data MUST be computed from record + config alone. - `BUZZ_ACP_LAZY_POOL=true` — a **deliberate pick, not a transcription**: - the two local paths disagree (manual Start is eager, `runtime.rs:1006`; + the two local paths disagree (manual Start is eager, `runtime.rs:1001`; launch restore is lazy, `restore.rs:333`, precisely to avoid "N idle brains on every launch"). Remote pods take the lazy arm: an idle LLM pool in a cluster is billable waste with no user watching it warm up. - `MCP_HOOK_SERVERS=*` when the resolved runtime has `mcp_hooks` - (`runtime.rs:594-598`; buzz-agent only at `c1bca1b56`) — gates the + (`runtime.rs:594-598`; buzz-agent only at `28ae6cd21`) — gates the `_Stop`/`_PostCompact` hook tools. - `BUZZ_ACP_SYSTEM_PROMPT`, `BUZZ_ACP_IDLE_TIMEOUT`, `BUZZ_ACP_MAX_TURN_DURATION`, `BUZZ_ACP_AGENTS` — resolved by the desktop @@ -728,6 +731,31 @@ keys on it. Whether ten minutes fits the intended cluster class is a product ruling, not a correctness one. +**One create attempt per call (normative).** The replacement rows above — +terminated, never-started provably broken, never-started divergent — exist +to clear residue from a *previous* life. Once a deploy call has created its +own pod, a classification that would replace that pod means the attempt this +call just made has already failed: the harness started, rejected its +configuration, and exited (the deterministic startup failure), or the pod +was proven broken. Re-running the identical create against the same cluster +inside the same call cannot produce a different outcome; what it produces is +a hot delete/mint/create cycle every poll interval for the whole operation +deadline, one immutable Secret per cycle (measured live: 107 Secrets in a +single 600s call), every one younger than §K8s GC's orphan age gate — a +bounded-call resource DoS and nsec-bearing-Secret amplifier. A binding MUST +NOT delete-recreate a pod created by the same deploy call: it MUST return +the in-band error carrying the latest condition (for a terminated container, +the exit code and reason — never the terminated `message`, which is +process-composed output under the same redaction rule as pull messages). +The failed attempt's pod and Secret are deliberately left in place: the pod +is terminated, so the *next* Start's preflight GC collects the pod and its +referenced Secret together before that call's own single attempt — retry is +thereby gated on fresh owner intent, and litter is bounded at one pod plus +one Secret per press, not per poll. This bounds attempts, not observation: +the recoverable rows still observe a slow startup for the full deadline, and +residue from previous lives is still replaced exactly once on the way to +this call's attempt. + **Destructive decisions come from views you control — reads and writes both (normative).** This is one rule with three instances, stated once so nobody optimizes an instance away. §K8s GC's same-clock rule is the time @@ -853,13 +881,13 @@ can yield two live instances in one scope. derive an upper bound for this path from its segment timeouts, because review proved that arithmetic wrong twice**: the visible constants (30s drain, 2s presence, 5s relay close) omit terms that are *variable*, not - constant — at this PR's base `b4f4ed1a6` the post-drain reap segment + constant — at `28ae6cd21` the post-drain reap segment (late-arriving reap `lib.rs:2664`, idle-slot reap loop `:2670`, respawn drain `:2684-2688`) runs *outside* the 30s drain timeout (opened at `:2636`, closed at `:2657`) and serially awaits a 5s post-SIGKILL wait per occupied pool slot (`acp.rs:436`). **That segment alone can reach `30 + 5×parallelism + 7` — ~87s at the desktop's default parallelism - of 10** (`DEFAULT_AGENT_PARALLELISM`, `types.rs:809`; lowered from 24 by + of 10** (`DEFAULT_AGENT_PARALLELISM`, `types.rs:814`; lowered from 24 by #3038), ~197s at the harness cap of 32 (`config.rs:293`) — already exceeding a 60s grace. And it is a *lower* bound on the tail, not the worst case: the same path runs earlier segments before the prompt drain @@ -925,7 +953,7 @@ I5's enforcement point. A new harness knob: could disable the reaper and reopen unbounded lifetime through the front door. `BUZZ_ACP_NO_PRESENCE` (`config.rs:378`) MUST join in the same change, for the same shape of reason at I3 instead of I5: unreserved, it - lets user env silently defeat the 90s presence bound (I3). One knob + lets user env silently defeat the 180s presence bound (I3). One knob guards "knows when to leave", the other "you can see that it left"; both are promises users must not be able to un-make by typo. - Distinctness note: this is a **fourth** timeout concept, deliberately named @@ -1173,7 +1201,7 @@ regardless of `HOME`. would SIGKILL the harness mid-drain, leaving presence stale-online — the avoidable half of I3's staleness window — so the binding declares 60s. But the shutdown tail is *variable*, not constant (§Stop: the post-drain - reap segment alone reaches ~87s at default parallelism at `b4f4ed1a6`, + reap segment alone reaches ~87s at default parallelism at `28ae6cd21`, and earlier untimed segments precede it — the total is not bounded by today's segment timeouts), so no fixed grace can be proven sufficient by adding segment timeouts. The two halves of the requirement: @@ -1541,14 +1569,11 @@ the wrong tool here: the failure modes found in review were wrong delete, phase-as-readiness, non-atomic Secret→pod against GC), which a hand-written model would have reproduced convincingly. -## Known Defects (at `c1bca1b56`) +## Known Defects (at `28ae6cd21`) -**Citation-pin caveat:** `c1bca1b56` is an unmerged feature-branch commit -that diverged from main on Jul 18 and predates #3038 (default parallelism -24 → 10). Line references marked `at b4f4ed1a6` were re-verified against -this PR's own base; unmarked `c1bca1b56` references may be offset on -current main. A follow-up re-pins the whole document to one merged -commit. +**Citation pin:** every `file:line` reference in this document was verified +against `28ae6cd21` — the commit at which this spec merged to `main`. +References are to that tree; a later commit may offset them. Desktop- and harness-side, discovered during this design: @@ -1562,7 +1587,7 @@ Desktop- and harness-side, discovered during this design: a desktop-side PATH augmentation would fix the class. 3. **Deploy payload bypasses the launch resolver** (the prerequisite this spec names for §Launch data — a desktop code change, not spec text). - At `c1bca1b56`, `deploy_payload_json` serializes raw record bytes and a + At `28ae6cd21`, `deploy_payload_json` serializes raw record bytes and a three-layer `merged_user_env` where the local spawn uses `resolve_effective_harness_descriptor`'s six-layer resolution. Concrete consequences, each verified in review: (a) no per-runtime model/provider @@ -1584,14 +1609,14 @@ Desktop- and harness-side, discovered during this design: semantics-preserving remote launch. **Security follow-through:** once secrets can arrive via `launch.env`, desktop redaction MUST collect candidate values from `launch.env` (and `launch.policy_env`) as well as - legacy `agent.env_vars` — at `c1bca1b56`, `env_secrets_from_request` + legacy `agent.env_vars` — at `28ae6cd21`, `env_secrets_from_request` reads only `agent.env_vars` (`backend.rs`), leaving a definition/persona-layer secret outside the literal-value scrub. Conformance: a provider that echoes a launch-only secret into an error must come back redacted. 4. **The I5 reaper does not exist, and its natural home is a trap** (harness code prerequisite). `BUZZ_ACP_EXIT_AFTER_INACTIVITY` appears - nowhere in the harness at `c1bca1b56`; §Auto-Stop is a design, not a + nowhere in the harness at `28ae6cd21`; §Auto-Stop is a design, not a description. Worse, the obvious attachment point — the existing 30s maintenance tick — is gated on `pool_ready` (`lib.rs:1743`), which under `lazy_pool` only becomes true when work arrives, so a never-mentioned @@ -1607,7 +1632,7 @@ Desktop- and harness-side, discovered during this design: resolve-once → stage-and-digest → `info` → explicit-version check → `deploy`, both invocations running the staged bytes. 6. **The clean-exit contract is emergent, not defended** (harness code - prerequisite; gates `OnFailure`). At `b4f4ed1a6`: the graceful path + prerequisite; gates `OnFailure`). At `28ae6cd21`: the graceful path returns `Ok(())` (`lib.rs:2723`), and owner `!shutdown` (`:2045`), Ctrl-C (`:1635`), and SIGTERM (`:1644`) all route into the same shutdown channel — so clean @@ -1620,12 +1645,12 @@ Desktop- and harness-side, discovered during this design: timeout would silently convert every clean stop into a restart loop — I5 defeated with no failing test (I5 ordering rule). 7. **The shutdown tail overruns the declared grace budget at default - config** (harness code prerequisite). At `b4f4ed1a6`: the post-drain + config** (harness code prerequisite). At `28ae6cd21`: the post-drain reap segment (`lib.rs:2664-2688`) runs *after* the 30s drain timeout closes (`:2636,:2657`) and serially awaits a 5s post-SIGKILL wait per occupied slot (`acp.rs:436`) — that segment alone reaches ~87s at the desktop's - default parallelism of 10 (`types.rs:809`; #3038 lowered it from 24), + default parallelism of 10 (`types.rs:814`; #3038 lowered it from 24), ~197s at the harness cap of 32 (`config.rs:293`), against the binding's 60s grace; and it is not the whole tail — the wake-task drain (`:2612`) and awakened-pool shutdown (`:2620-2624`, per-slot loop @@ -1640,7 +1665,7 @@ Desktop- and harness-side, discovered during this design: 8. **Cleared numeric config fields ship as strings** (desktop code prerequisite, raised by blessing `0`). `coerceConfigValues` (`desktop/src/features/agents/ui/ProviderConfigFields.tsx:6` at - `b4f4ed1a6`) skips numeric coercion when the value is `""`, so a + `28ae6cd21`) skips numeric coercion when the value is `""`, so a *cleared* numeric field reaches the provider as a JSON string instead of a number. Blessing `inactivity_seconds: 0` makes clearing that field a legitimate user action, so the empty-string arm now sits on a diff --git a/scripts/bundle-sidecars.sh b/scripts/bundle-sidecars.sh index 07de47740..8ea5fe2bb 100755 --- a/scripts/bundle-sidecars.sh +++ b/scripts/bundle-sidecars.sh @@ -4,6 +4,12 @@ set -euo pipefail SIDECARS=(buzz-acp buzz-agent buzz-dev-mcp git-credential-nostr buzz) HOST=$(rustc -vV | sed -n 's|host: ||p') TARGET=${1:-$HOST} +if [[ "$TARGET" != *windows* ]]; then + SIDECARS+=(buzz-backend-kubernetes) + BUILD_HINT="cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli" +else + BUILD_HINT="cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli" +fi BINARIES_DIR="desktop/src-tauri/binaries" # When --target is passed explicitly to cargo (even if it matches the host), @@ -29,7 +35,7 @@ for bin in "${SIDECARS[@]}"; do done if [[ ${#missing[@]} -gt 0 ]]; then echo "Error: missing release binaries in $SRC_DIR: ${missing[*]}" >&2 - echo "Run 'cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli' first." >&2 + echo "Run '$BUILD_HINT' first." >&2 exit 1 fi diff --git a/scripts/run-tests.sh b/scripts/run-tests.sh index 2e7e94448..0e5946ca1 100755 --- a/scripts/run-tests.sh +++ b/scripts/run-tests.sh @@ -106,6 +106,12 @@ run_unit_tests() { run_test_step "buzz-push-gateway tests" \ cargo test -p buzz-push-gateway -- --nocapture + + # Kubernetes backend provider: pure decision layers driven by a fake + # substrate, no cluster. Mirrors the nextest path in `just test-unit` — + # the two lists must stay in step or the fallback silently covers less. + run_test_step "buzz-backend-kubernetes tests" \ + cargo test -p buzz-backend-kubernetes -- --nocapture } # ---- DB / integration tests (infra required) -------------------------------- diff --git a/scripts/sprig-entrypoint.sh b/scripts/sprig-entrypoint.sh new file mode 100755 index 000000000..16abf7bc5 --- /dev/null +++ b/scripts/sprig-entrypoint.sh @@ -0,0 +1,16 @@ +#!/bin/bash +set -euo pipefail + +# Match desktop's URL-scoped git credential configuration without installing a +# helper globally (which would make it answer for unrelated remotes). +if [[ -n "${BUZZ_RELAY_URL:-}" ]]; then + relay_http_url="${BUZZ_RELAY_URL/#ws:/http:}" + relay_http_url="${relay_http_url/#wss:/https:}" + relay_http_url="${relay_http_url%/}" + git config --global "credential.${relay_http_url}/git.helper" \ + /usr/local/bin/git-credential-nostr + git config --global "credential.${relay_http_url}/git.useHttpPath" true +fi + +# The harness must receive Kubernetes' termination signal directly. +exec buzz-acp "$@" diff --git a/scripts/test-k8s-provider-release.sh b/scripts/test-k8s-provider-release.sh new file mode 100755 index 000000000..40c78e268 --- /dev/null +++ b/scripts/test-k8s-provider-release.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +# Reproduce the desktop release's multi-package build shape, then prove the +# Kubernetes provider reports an unreachable cluster in-band rather than +# panicking when rustls has multiple compiled crypto providers available. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$ROOT" + +TARGET_DIR="${CARGO_TARGET_DIR:-$ROOT/target}" +PROVIDER_BINARY="$TARGET_DIR/release/buzz-backend-kubernetes" + +CARGO="${CARGO:-cargo}" +"$CARGO" build --release \ + -p buzz-acp \ + -p buzz-agent \ + -p buzz-dev-mcp \ + -p git-credential-nostr \ + -p buzz-cli \ + -p buzz-backend-kubernetes + +TMP="$(mktemp -d)" +trap 'rm -rf "$TMP"' EXIT +cat >"$TMP/kubeconfig" <<'YAML' +apiVersion: v1 +kind: Config +clusters: + - name: unreachable + cluster: + server: http://127.0.0.1:9 +contexts: + - name: unreachable + context: + cluster: unreachable + user: none +current-context: unreachable +users: + - name: none + user: {} +YAML + +cat >"$TMP/request.json" <<'JSON' +{ + "op": "deploy", + "agent": { + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "auth_tag": "owner", + "launch": {"command": "goose", "args": [], "env": {}, "policy_env": {}} + }, + "provider_config": { + "context": "unreachable", + "namespace": "never-created", + "image": "example.invalid/sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "inactivity_seconds": 60 + } +} +JSON + +set +e +KUBECONFIG="$TMP/kubeconfig" \ + "$PROVIDER_BINARY" \ + <"$TMP/request.json" >"$TMP/stdout" 2>"$TMP/stderr" +status=$? +set -e + +[[ "$status" == 0 ]] || { + echo "provider exited $status instead of returning an in-band failure" >&2 + cat "$TMP/stderr" >&2 + exit 1 +} +[[ "$(wc -l <"$TMP/stdout" | tr -d ' ')" == 1 ]] +jq -e '.ok == false and (.error | type == "string" and length > 0)' "$TMP/stdout" >/dev/null +if rg -i 'panic|crypto provider|no process-level' "$TMP/stdout" "$TMP/stderr"; then + echo "provider emitted a panic/crypto-provider failure" >&2 + exit 1 +fi +printf 'PASS: release-shaped provider returned one in-band unreachable-cluster failure\n' diff --git a/scripts/test-k8s-sprig-image-live.sh b/scripts/test-k8s-sprig-image-live.sh new file mode 100755 index 000000000..ec9f25905 --- /dev/null +++ b/scripts/test-k8s-sprig-image-live.sh @@ -0,0 +1,97 @@ +#!/usr/bin/env bash +# Prove that a cluster can resolve and start the exact digest-qualified Sprig +# image that will be passed to the Kubernetes provider. This is intentionally a +# separate preflight: a Docker image existing in the host daemon does not imply +# that the kubelet's container runtime can resolve the same name and digest. +set -euo pipefail + +: "${BUZZ_K8S_TEST_CONTEXT:?set the explicit disposable/local kubectl context}" +: "${BUZZ_SPRIG_IMAGE:?set an immutable image reference (name@sha256:<64 hex>)}" + +if [[ ! "$BUZZ_SPRIG_IMAGE" =~ ^[^[:space:]@]+@sha256:[0-9a-fA-F]{64}$ ]]; then + echo "error: BUZZ_SPRIG_IMAGE must be name@sha256:<64 hex>" >&2 + exit 2 +fi + +CONTEXT="$BUZZ_K8S_TEST_CONTEXT" +IMAGE="$BUZZ_SPRIG_IMAGE" +PULL_POLICY="${BUZZ_K8S_TEST_PULL_POLICY:-IfNotPresent}" +case "$PULL_POLICY" in + Always|IfNotPresent|Never) ;; + *) echo "error: invalid BUZZ_K8S_TEST_PULL_POLICY: $PULL_POLICY" >&2; exit 2 ;; +esac + +MANAGED_BY="buzz-backend-kubernetes" +BINDING_VERSION="v1" +NAMESPACE="buzz-k8s-sprig-$(date +%s)-$RANDOM" +CREATED=0 + +cleanup() { + (( CREATED == 1 )) || return 0 + local managed binding foreign + managed="$(kubectl --context "$CONTEXT" get namespace "$NAMESPACE" \ + -o jsonpath='{.metadata.labels.app\.kubernetes\.io/managed-by}' 2>/dev/null || true)" + binding="$(kubectl --context "$CONTEXT" get namespace "$NAMESPACE" \ + -o jsonpath='{.metadata.labels.buzz\.block\.xyz/binding-version}' 2>/dev/null || true)" + if [[ "$managed" != "$MANAGED_BY" || "$binding" != "$BINDING_VERSION" ]]; then + echo "REFUSING cleanup: namespace ownership markers changed: $NAMESPACE" >&2 + return 1 + fi + foreign="$(kubectl --context "$CONTEXT" --namespace "$NAMESPACE" get pods -o json \ + | jq '[.items[] | select(.metadata.labels["app.kubernetes.io/managed-by"] != "buzz-backend-kubernetes" or .metadata.labels["buzz.block.xyz/binding-version"] != "v1")] | length')" + if [[ "$foreign" != 0 ]]; then + echo "REFUSING cleanup: namespace contains an unowned pod: $NAMESPACE" >&2 + return 1 + fi + kubectl --context "$CONTEXT" delete namespace "$NAMESPACE" --wait=true +} +trap cleanup EXIT + +# Fail before mutation if the named context is absent or inaccessible. Print the +# cluster identity so evidence cannot be mistaken for a different kubeconfig. +kubectl config get-contexts "$CONTEXT" >/dev/null +SERVER="$(kubectl config view --minify --context "$CONTEXT" -o jsonpath='{.clusters[0].cluster.server}')" +kubectl --context "$CONTEXT" get nodes -o name >/dev/null +printf 'context=%s\nserver=%s\nimage=%s\npull_policy=%s\nnamespace=%s\n' \ + "$CONTEXT" "$SERVER" "$IMAGE" "$PULL_POLICY" "$NAMESPACE" + +kubectl --context "$CONTEXT" create namespace "$NAMESPACE" +CREATED=1 +kubectl --context "$CONTEXT" label namespace "$NAMESPACE" \ + "app.kubernetes.io/managed-by=$MANAGED_BY" \ + "buzz.block.xyz/binding-version=$BINDING_VERSION" + +cat <&2 || true + exit 1 +fi + +kubectl --context "$CONTEXT" --namespace "$NAMESPACE" logs digest-resolution-probe +IMAGE_ID="$(kubectl --context "$CONTEXT" --namespace "$NAMESPACE" get pod digest-resolution-probe \ + -o jsonpath='{.status.containerStatuses[0].imageID}')" +RESOLVED_SPEC="$(kubectl --context "$CONTEXT" --namespace "$NAMESPACE" get pod digest-resolution-probe \ + -o jsonpath='{.spec.containers[0].image}')" +[[ "$RESOLVED_SPEC" == "$IMAGE" ]] +[[ -n "$IMAGE_ID" ]] +printf 'resolved_spec=%s\nimage_id=%s\nPASS: exact digest-qualified Sprig reference started\n' \ + "$RESOLVED_SPEC" "$IMAGE_ID" diff --git a/scripts/test-sprig-image.sh b/scripts/test-sprig-image.sh new file mode 100755 index 000000000..4ca42fdb4 --- /dev/null +++ b/scripts/test-sprig-image.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +set -euo pipefail + +IMAGE="${1:-buzz-sprig:contract-test}" +if [[ "${SKIP_BUILD:-0}" != 1 ]]; then + docker build --file Dockerfile.sprig --tag "$IMAGE" . +fi + +assert_run() { + docker run --rm --entrypoint /bin/bash "$IMAGE" -ceu "$1" +} + +assert_run ' + command -v bash git update-ca-certificates >/dev/null + test "$(readlink /usr/local/bin/buzz-acp)" = sprig + for name in buzz-agent buzz-dev-mcp rg tree buzz git-credential-nostr git-sign-nostr; do + test "$(readlink "/usr/local/bin/$name")" = sprig + done + test "$(git config --system gpg.x509.program)" = /usr/local/bin/git-sign-nostr + ! git config --system --get-all credential.helper + test "$HOME" = /home/agent + test "$(pwd)" = /home/agent +' + +assert_run ' + grep -Eq "^[[:space:]]*exec buzz-acp" /usr/local/bin/sprig-entrypoint + ! grep -Eq "^[[:space:]]*(buzz-acp|bash -c .*buzz-acp)" /usr/local/bin/sprig-entrypoint +' + +docker run --rm --entrypoint /bin/bash \ + -e BUZZ_RELAY_URL=wss://relay.example.test/ "$IMAGE" -ceu ' + /usr/local/bin/sprig-entrypoint --help >/dev/null 2>&1 & pid=$! + for _ in 1 2 3 4 5; do + git config --global --get credential.https://relay.example.test/git.helper >/dev/null 2>&1 && break + sleep 0.1 + done + test "$(git config --global --get credential.https://relay.example.test/git.helper)" = /usr/local/bin/git-credential-nostr + test "$(git config --global --get credential.https://relay.example.test/git.useHttpPath)" = true + ! git config --global --get-all credential.helper + wait "$pid" || true + ' + +echo "PASS: Sprig image runtime contract ($IMAGE)"