External/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-08-18 06:50:31 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix(agents): stop the Welcome intro loop with an ack that costs nothing

Browse Source 
Observed with Codex on a fresh Welcome channel: Fizz, Honey, and Bumble
reply to each other 21+ times, each message announcing it will now stop
replying, each announcement waking the others.

A mention is a wake (p-tag gated), so the only way an agent could say
"seen" was to summon someone. Combined with an unconditional MUST in the
shared base prompt, that has no exit:

  :44 you MUST @mention the delegator when you finish delegated work

"I won't reply again" is itself a summons. Note :40 already said "only
@mention when you need their attention" -- and they still looped, because
a soft "only when" loses to a hard MUST two lines below it. Codex reads
MUST literally; Claude Code treats it as defeasible, so the design only
ever worked on a model choosing to disobey.

Fix it in the base prompt rather than the personas (3 copies to drift) or
team instructions (Welcome-team only). Three bullets, all in Callback
Mentions:

- Bound the callback: it hands work back, is sent once, and a delegator
  can waive it. The unconditional MUST itself is kept -- stalled
  collaboration is the worse failure.
- Forbid the ack-mention and say why: a tag costs a whole turn, it is not
  punctuation, so "understood" restarts a conversation rather than ending
  it. A literal model needs the mechanism, not just the rule.
- Point at reactions as the outlet: "to acknowledge, use a reaction
  instead" (buzz reactions add). This is the load-bearing part. It
  redirects rather than suppresses, and it keeps the human-visible social
  signal so the fix does not read as stalling. The command already
  existed and is already in the command table; the prompt just never
  connected it to acking.

:61 ("every turn MUST publish a reply") is deliberately left alone. It is
not load-bearing: a loop needs a wake, wake is p-tag gated, and :61 forces
a reply rather than a tag -- an untagged reply ends the chain. It governs
noise, not the loop, and weakening "always reply" trades toward the silent
failures this doc already tracks. Corroborated by another Buzz agent
running this same base prompt including :61, plus a core memory saying
almost exactly the bullets above; it does not loop.

Also considered and rejected: a structural cap on consecutive
agent-authored turns. It cannot distinguish a loop from a long-running
task where agents legitimately trade many messages, so it would truncate
real work.

Fizz's opener is reduced to a plain welcome: an em dash becomes a sentence
break, and "Don't start any work yet" is dropped -- nothing has been asked
for yet, so it guarded against nothing concrete. An anti-loop instruction
there would work, but the opener is the first thing a new user reads and
should not advertise that the product needs prompting to behave; it would
also confound the test by suppressing the loop in this one channel while
proving nothing about the base prompt. A test pins the absence of
agent-steering instructions so re-adding one is a deliberate decision.

Also corrects the doc's rate-limit note. "rate-limited: quota exceeded" is
the relay rejecting a publish (buzz-relay/src/connection.rs:652), and
buzz-acp has no publish retry/backoff, so the tight-retry-loop theory was
unsupported. A rejected send produces no event, no p-tag, no wake -- it
breaks the loop rather than feeding it. Rate limiting is downstream; the
two can be fixed independently.

Not verified live. The prompt change is behavioral and needs a real Codex
session on a fresh Welcome channel.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
morgmart
2026-07-18 15:53:48 -07:00
co-authored by Claude
parent a73b7f7705
commit 0261e344d0
4 changed files with 328 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/buzz-acp/src/base_prompt.md
+3
View File
@@ -42,6 +42,9 @@ For explicit changes to an existing personal agent, use `buzz agents draft-updat
### Callback Mentions
- When you finish delegated work, you MUST `@mention` the delegator in your completion message. This is the #1 cause of stalled collaboration.
- The callback hands work **back**: a completion, a blocker, or a question. Send it once. If the delegator said no callback is needed, don't send one.
- **Never `@mention` anyone just to acknowledge, agree, confirm, or sign off.** A mention is a summons: it wakes that agent and obliges them to answer, so a tag costs a whole turn — it is not punctuation. "Understood", "standing by", and "I won't reply again" therefore do not end a conversation, they restart it, and two agents trading acknowledgments will loop until something external stops them.
- **To acknowledge, use a reaction instead** — `buzz reactions add`. It signals you saw something and agree, costs nobody a turn, and leaves the thread quiet. Reach for it whenever your reply would be a bare "thanks" or "got it".
### Threading
desktop/src/features/onboarding/welcomeKickoff.test.mjs
+21 -1
View File
@@ -51,7 +51,27 @@ test("opener uses current agent names and requests bounded simultaneous intros",
assert.match(opener, /@Honeybee and @Bumble/);
assert.doesNotMatch(opener, /@@/);
assert.match(opener, /sentence or two/);
assert.match(opener, /Don't start any work yet/);
});
test("opener carries no agent-steering instructions", () => {
// The opener is the first message a new user reads: it should read as a normal
// welcome, not as prompt engineering. The intro loop is fixed in the base
// prompt (`buzz-acp/src/base_prompt.md`), not here. Re-adding an anti-loop
// instruction is a product decision, not a quick fix — see
// docs/welcome-kickoff-silent-failures.md.
const opener = buildWelcomeKickoffOpener(fizz, [honey, bumble]);
assert.doesNotMatch(opener, /@mention/i);
assert.doesNotMatch(opener, /don't start any work/i);
});
test("solo opener has no mentions to loop on", () => {
// No teammates online: nothing delegated, so there is no callback to waive
// and the opener must not mention an agent at all.
const opener = buildWelcomeKickoffOpener(fizz, [], [honey, bumble], "morgan");
assert.match(opener, /Hi @morgan/);
assert.doesNotMatch(opener, /@Honey|@Bumble/);
});
test("teammates are not ready until every harness publishes online presence", () => {
desktop/src/features/onboarding/welcomeKickoff.ts
+6 -1
View File
@@ -159,7 +159,12 @@ export function buildWelcomeKickoffOpener(
return `${greeting} Welcome to Buzz. This is your private home base, and I'm here${teammatePhrase} to help you get oriented or work through something you're building.\n\n${WELCOME_KICKOFF_CTA}`;
}
return `${greeting} Welcome to Buzz. This is your private home base, and we're here to help you get oriented or work through something you're building.\n\n${introNames}, introduce ${introTeammates.length === 1 ? "yourself" : "yourselves"} in a sentence or two — share what you're good at and when to bring you in. Don't start any work yet.`;
// Deliberately carries no agent-steering instructions — no "don't @mention
// each other", no "don't start any work yet". This is the first message a new
// user reads, so it stays a normal welcome instead of advertising its own
// guardrails; the intro loop is fixed in the base prompt instead
// (`buzz-acp/src/base_prompt.md`). See docs/welcome-kickoff-silent-failures.md.
return `${greeting} Welcome to Buzz. This is your private home base, and we're here to help you get oriented or work through something you're building.\n\n${introNames}, introduce ${introTeammates.length === 1 ? "yourself" : "yourselves"} in a sentence or two. Share what you're good at and when to bring you in.`;
}
export function onlineWelcomeTeammates(
docs/welcome-kickoff-silent-failures.md
+298 -11
View File
@@ -1,12 +1,19 @@
# Welcome Kickoff — Silent Failure Paths
# Welcome Kickoff — Failure Paths
Status: **open** — the *perception* gap is handled (see below); the silent
paths themselves are not. Documented for follow-up work.
Status: **partly open.** The *perception* gap is handled (see below); the silent
paths themselves are not. The [intro loop](#the-intro-loop-a-loud-failure) is
mitigated as of 2026-07-18 by three bullets in the base prompt's Callback
Mentions section — **not yet confirmed live against Codex.** Prompt is the only
lever available: a structural cap was considered and rejected (see that section).
Context: the Welcome-channel kickoff choreography
(`desktop/src/features/onboarding/welcomeKickoff.ts`) where Fizz posts an
opener, teammates introduce themselves in-thread, and Fizz posts a closer.
## The problem
Two failure shapes live here: the **silent** ones (nobody speaks) and the
**loud** one (nobody stops). They have opposite symptoms and the same root
cause — the kickoff assumes agent behavior it never constrains.
## The silent-path problem
Every fallback message in the kickoff assumes Fizz — the lead agent and
sender — is alive and able to post. When Fizz itself fails, or an early step
@@ -34,7 +41,10 @@ real handling. Two things worth knowing before picking this up:
## Message inventory (what the user CAN receive today)
All hard-coded client-side; only teammate intro replies are LLM-generated.
All hard-coded client-side; only teammate intro replies are LLM-generated —
which is exactly where the [intro loop](#the-intro-loop-a-loud-failure) lives.
Everything the client authors is bounded; the one unbounded part is the part
that runs away.
| # | Message | Trigger | Sender |
|---|---------|---------|--------|
@@ -65,7 +75,258 @@ All hard-coded client-side; only teammate intro replies are LLM-generated.
Navigation away mid-kickoff also cancels silently, but that is intentional
(the kickoff resumes on next visit) — not a failure.
## Constraints for the fix
## The intro loop (a loud failure)
Observed 2026-07-18 with **Codex** as the provider: the intro thread never
terminates. Fizz, Honey, and Bumble keep replying to each other — 21+ replies,
each one an agent announcing it will now stop replying ("parked", "acknowledged",
"I won't reply again unless there's a task for me"), each announcement waking the
others. **Not reproduced with Claude Code.**
### Why it loops
The wake trigger is `p`-tag gated: an agent fires when a message mentions it
(`require_mention`, default on — `crates/buzz-acp/src/filter.rs:390`). A reply
adds `p` tags **only** for names explicitly `@`-mentioned in its body
(`buzz-sdk/src/builders.rs:218`; threading itself contributes `e` tags only). So
mentioning a teammate — for any reason, including to say goodbye — is
indistinguishable from summoning them.
Two unconditional rules in the shared base prompt then close the cycle:
| `crates/buzz-acp/src/base_prompt.md` | Rule |
|---|---|
| `:44` | "When you finish delegated work, you **MUST** `@mention` the delegator in your completion message." |
| `:61` | "**Every turn that processes a user message MUST publish a reply.** … A turn that ends without a published message is a silent failure." |
Composed, they have no exit:
```
Fizz's opener @-mentions Honey + Bumble (kickoff, by design)
→ Honey finishes her intro ("delegated work")
→ :44 — MUST @mention the delegator → Fizz wakes
→ Fizz processed a message
→ :61 — MUST publish a reply → Fizz replies
→ :44 — acknowledgment @-mentions the two → Honey + Bumble wake
→ … (forever)
```
**The sign-off is the trigger.** "I'll stay quiet until you have a task" is a
mention, and a mention is a wake. Politeness is the fuel.
Nothing structural brakes this:
- `ignore_self` (`lib.rs:1864`) filters an agent's **own** pubkey only — it does
not touch A↔B mutual mentions.
- The author gate (`RespondTo::OwnerOnly`, the default) admits **siblings** —
other agents launched by the same owner (`is_owner_or_sibling`,
`lib.rs:166–190`). Agent-to-agent traffic is allowed by design.
- `max_turns_per_session` defaults to `0` = unlimited (`config.rs:372`).
- There is no per-thread agent-to-agent turn cap and no human-in-the-loop check.
So the only brake is model discretion — which is precisely the variable that
differs between providers.
### Why Codex and not Claude Code
**No code path explains it.** Prompt content is identical for both (both are
protocol ≥ 2, both receive the same `base_prompt.md` + persona `[System]` via
`pool.rs:1072`). The only per-provider prompt divergence is Goose (custom
request) and legacy protocol `< 2` (inlined `[System]`) — neither is
Codex-vs-Claude. Codex's only special-casing is network-sandbox env injection
(`config.rs:640–668`), which has no prompt effect.
The difference is behavioral: Codex reads `MUST` as a hard constraint and obeys
it literally every turn; Claude Code treats it as defeasible and exercises
discretion about when silence is correct. **Which means the current design only
ever worked by luck** — it depends on a model choosing to violate an explicit
instruction. Any provider that follows instructions faithfully will loop. Codex
is not misbehaving here; it is doing what the prompt says.
Corollary for the fix: **prompting alone cannot be the whole answer.** Every
message in the observed thread is an agent *promising to stop*. They understood
the instruction and complied verbally while structurally continuing to loop. A
prompt that says "don't loop" is one more sentence for a literal-minded model to
acknowledge — in a reply, with a mention.
### Where the prompt fix belongs
`crates/buzz-acp/src/base_prompt.md` — **not** the personas, **not** the team
instructions. Reasoning:
| Layer | File | Verdict |
|---|---|---|
| Base prompt (all agents) | `crates/buzz-acp/src/base_prompt.md` | ✅ **Right home.** The two rules that *cause* the loop live here, unconditional. The cycle is a general property of any mutual-mention agent set — not a Welcome-team quirk. |
| Team instructions | `desktop/src-tauri/src/managed_agents/teams.rs:59` (`instructions: None` — slot exists, unused) | ⚠️ Scoped to the Welcome Team only. Same loop reappears for any other team or ad-hoc pair. |
| Per-persona | `desktop/src-tauri/src/managed_agents/personas.rs:22–26` (Fizz/Honey/Bumble) | ❌ Three copies to drift, and every new persona re-introduces the bug. |
Fixing it above the base prompt would leave `:44` and `:61` still saying "always
reply, always mention the delegator" and layer a contradiction on top. The rules
need a **termination clause at the source**: bound the "MUST reply" to turns that
advance work, and carve out acknowledgment-only replies from the "MUST mention
the delegator" rule (an ack that needs no action should either not be sent or not
mention). Both edits belong in the same two bullets that created the cycle.
### Constraints for the intro-loop fix
- **Don't break callback mentions.** `:44` exists because a missing callback is
"the #1 cause of stalled collaboration" — the reason the rule is unconditional.
Loosening it trades a loud failure for a silent one, which is the failure class
this doc already tracks. The carve-out must be narrow: acknowledgments that
close a loop, not completions that hand work back.
- **Don't break `:61` either.** It exists to prevent invisible turns. "Reply
unless you have nothing to add" invites a model to skip a real answer.
- Prompt change is **necessary but not sufficient** — pair it with a structural
brake so a literal provider cannot loop even if it ignores the prose.
### The fix (landed 2026-07-18)
**Give the acknowledgment somewhere else to go.** The loop is not caused by
agents wanting to talk too much; it is caused by the only available way to say
"seen" being a mention, and a mention being a wake. So bound the MUST that forces
the mention, and hand the impulse a zero-cost outlet.
Three bullets, all in **Callback Mentions** (`base_prompt.md:42`). Nothing else
changed — no persona, no team instructions, no kickoff copy, no harness code:
1. **Bound the callback.** It hands work *back* — a completion, a blocker, or a
question — is sent **once**, and a delegator can waive it. The unconditional
MUST itself is **kept**: stalled collaboration is the worse failure (see
Constraints).
2. **Forbid the ack-mention, and say why.** Never `@mention` anyone *just* to
acknowledge, agree, confirm, or sign off — *"a mention is a summons… a tag
costs a whole turn, it is not punctuation… 'I won't reply again' does not end
a conversation, it restarts it."* A literal model needs the mechanism, not
just the rule.
3. **Point at reactions as the alternative.** *"To acknowledge, use a reaction
instead — `buzz reactions add`."* This is the load-bearing addition: it
redirects rather than suppresses. Telling a model "don't" leaves it with an
unresolved impulse; giving it a `p`-tag-free way to signal "seen and agreed"
resolves it, **and keeps the human-visible social signal** so the fix doesn't
read as stalling. `buzz reactions add` already existed and is already in the
command table (`:13`) — the base prompt just never connected it to acking.
### Why `:61` was left alone
An earlier draft also rescoped `:61` ("Every turn that processes a user message
MUST publish a reply") to humans only, plus an explicit "end the turn silently
when the exchange is over". **Reverted.** Two reasons:
- **It isn't load-bearing.** A loop needs A's message to *wake* B, and wake is
`p`-tag gated. `:61` forces a *reply*, not a *tag* — an untagged reply ends the
chain after one message. `:61` governs **noise**, not the loop.
- **It was the riskiest line in the diff.** Any weakening of "always reply" trades
toward the silent failures this same doc tracks. Not worth it for noise.
Corroborating data point (2026-07-18): another Buzz agent (`Eva`) runs the same
base prompt **including** `:61`, plus a per-agent core memory that says almost
exactly bullets 2–3 above ("Don't @-tag another agent to merely acknowledge…
ack with a reaction instead"; "a tag = forced wake-up = a whole invocation").
It does not loop. So the mention rule alone is sufficient, and `:61` can stay
as-is.
That memory also **confirms the diagnosis from the other direction**: its
"complementary rule" (only mention when you need attention, never in narrative)
was *already in our base prompt at `:40`* the whole time — and the agents still
looped, because a soft "only… when" loses to a hard MUST two lines below it. The
per-agent memory works because it scopes the mandatory tag narrowly ("completion
callbacks are the one mandatory tag") — the same move as bullet 1. Memory fixed
one agent; the base prompt is where it holds for every agent, including a fresh
Fizz with no memory.
**Known residual tension:** `:61` still says a turn MUST publish a reply, which a
literal model may read as outranking "ack with a reaction". Worst case is an
extra untagged message — noise, not a loop, since it wakes nobody. Accepted
deliberately; revisit only if observed.
**Deliberately NOT done — telling the teammates not to mention each other in the
kickoff opener.** Tried and reverted, for two reasons:
- **Product.** The opener is the first thing a new user ever reads. A greeting
that visibly negotiates anti-loop rules with its own agents ("don't @mention me
or each other") advertises that the product needs intense prompting to behave
normally. The fix belongs somewhere the user never sees. The opener now carries
**no agent-steering instructions at all** — "Don't start any work yet" went too
(nothing has been asked for yet, so it guarded against nothing concrete, and
"in a sentence or two" already bounds the intros). Pinned by a test, so
re-adding one is a deliberate decision rather than a reflex.
- **Isolation.** Shipping it alongside the base-prompt edit confounds the test:
the opener waiver alone would suppress the loop in the Welcome channel, so a
clean run would prove nothing about whether the base prompt actually works —
and the base prompt is the part that has to hold for *every* agent in *every*
channel, not just this one thread. Keeping the opener ordinary makes the
Welcome kickoff an honest test of the general fix.
The `:44` delegator waiver stays in the base prompt as a general capability (any
delegator, human or agent, can say "no need to report back") — it is simply not
exercised by the kickoff.
**Deliberately NOT done — a structural turn cap.** Considered and rejected: a
cap on consecutive agent-authored turns cannot distinguish a loop from a
long-running task where agents legitimately trade many messages, so it would
truncate real work. There is no cheap structural signal that separates the two —
the difference between "parked, standing by" and a useful update is semantic.
That leaves prompting as the only lever, with the honest caveat below.
**The closer needs no change** — its happy path is the bare CTA and its degraded
paths use plain names, not `@`, so it emits no `p` tags and cannot wake anyone.
### Residual risk
Prompting is the only mitigation, and prompting is what failed here: every
message in the observed thread was an agent promising to stop. The bet is that
the old prompt left a compliant model **no way to say "seen" except a mention**,
so it complied verbally while looping structurally — and that bounding the MUST
plus offering reactions as the outlet removes the loop. That is a real bet, not a
guarantee. It cannot be verified by unit tests; it needs a live Codex run on the
Welcome flow (see Verification below).
The `Eva` data point (above) is the strongest evidence it will hold: an agent
carrying essentially these rules in memory demonstrably does not loop. The
untested part is whether the same words work at **base-prompt salience** rather
than per-agent-memory salience — memory sits closer to the model, and `:40`
already proved that a low-salience mention rule loses to a nearby MUST.
If it recurs, the next lever is not a blunt cap but a **semantic** one — e.g. the
harness declining to wake an agent on a sibling mention whose message adds no
new work — which needs a judgment call the harness can't currently make.
### Verification status
- **Not verified live.** The prompt change is behavioral; only a real Codex
session on a fresh Welcome channel can confirm it. Prompt edits have no test
that can fail on a literal-minded model, so the desktop/Rust suites say
nothing about whether this works.
- One run on 2026-07-18 went from 21+ replies to **2** (the two intros, no
loop) — but that run had the opener waiver in it too, so it is **confounded
and does not count as evidence for the base prompt**. It is the reason the
opener was reverted: the next run needs to isolate the variable.
- Unit-tested only: the solo opener mentions no agent at all (pins the
degraded path, which cannot loop by construction).
### Outstanding questions
- Which other providers loop? Only Codex is confirmed, and **Codex-only is not a
safe assumption** — the fix is deliberately provider-agnostic and written to
be correct for a *perfectly obedient* model. Goose and the legacy `< 2`
inlined-`[System]` path take a different prompt route (`pool.rs:181–191`) and
are untested.
- If the base prompt alone does **not** hold, the opener waiver is the known
fallback — it worked in the confounded run above. The cost is the product one:
the first message a new user reads would carry anti-loop instructions. Prefer
fixing the base prompt wording first.
- Do agents actually reach for `buzz reactions add`? The rule is only as good as
the model's willingness to use a tool instead of writing prose. If it gets
ignored, the ack-mention prohibition degrades from "redirect" to "suppress",
which is the weaker form.
- Does "never mention just to acknowledge" bleed into pickup acknowledgments for
real work (`base_prompt.md:66` expects a pickup ack before follow-up tools)? A
pickup ack *precedes* work and is a legitimate mention; a terminal ack follows
the end and is not. The wording targets "acknowledge, agree, confirm, or sign
off", which reads terminal — but the boundary is a model judgment, and reading
it too broadly converts this loud failure into the silent kind.
## Constraints for the silent-path fix
- **Fizz cannot be the messenger** for these paths: she is the thing that
failed. Any user-visible fallback must come from the client UI itself
@@ -81,7 +342,7 @@ Navigation away mid-kickoff also cancels silently, but that is intentional
in `desktop/src-tauri/src/managed_agents/readiness.rs` already classifies
the actionable ones.
## Sketch (to validate later)
## Sketch for the silent paths (to validate later)
1. Surface a `kickoffError` phase from `useWelcomeKickoff` when the catch
block fires or the lead's start rejects, with a coarse cause
@@ -100,7 +361,33 @@ Navigation away mid-kickoff also cancels silently, but that is intentional
## Related
- Rate-limiting incident: one Welcome agent produced a 42KB log of
"rate-limited: quota exceeded" retries within seconds
(2026-07-17, remote relay `onboarding.communities.buzz.xyz`). Worth a
separate look at buzz-acp publish backoff — a tight retry loop against a
quota makes every other send in the session fail too.
"rate-limited: quota exceeded" within seconds (2026-07-17, remote relay
`onboarding.communities.buzz.xyz`).
**Rate limiting is downstream of the intro loop, not a cause of it** — checked
2026-07-18, and this corrects the earlier note here that assumed "an agent
publishing in a tight retry loop":
- That string is emitted by the **relay rejecting a publish**
(`buzz-relay/src/connection.rs:652`), so the log is 42KB of *rejections*,
not of client retries.
- There is **no retry/backoff code in buzz-acp's publish path** — the only
`fetch_with_retry` usage is for context fetches (`pool.rs:2159`). Nothing
supports the tight-retry-loop reading.
- Rate limiting cannot *drive* a back-and-forth: a rejected send produces no
event, hence no `p` tag, hence no wake. A rate-limited send **breaks** the
loop rather than feeding it.
So the causal arrow is loop → publish volume → quota → rejections. **The intro
loop fix does not depend on any rate-limit work**, and the two can be worked
independently.
Still open (hypothesis, unobserved): a provider whose *send* fails may retry it
within its own agent loop, since `:61` frames an unpublished message as a silent
failure — a literal model may not accept a failed send either. If the 42KB came
from one agent's repeated `buzz messages send` attempts, that is the mechanism.
**This one is not addressed here** (`:61` was deliberately left as-is — see
"Why `:61` was left alone"), so it remains live: a quota rejection has no
client-side brake at all. Worth a look at publish backoff independently.
- The two failure classes interact: silent-path work adds messages, and every
new message is a potential wake. Any fallback that `@mentions` an agent to
recover from a failure can itself seed a loop.