3 people f53bbd1152 fix(bench): mention the orchestrator by pubkey when posting the task (#5136) ...

## Problem

The harness posts each trial's task via `buzz messages send`, relying on
`@<orchestrator-id>` name resolution. Task text is untrusted payload:
when it contains @-tokens of its own, the CLI's mention resolver tries
to resolve them as channel members, fails, and refuses to send — killing
the trial with `RuntimeLaunchError` before the agent ever saw the task.

Live occurrence: TB 2.1's `large-scale-text-editing` task embeds Vim
macros (`:%normal! @a`). In the tb21-solo-1 run the trial died at
launch:

```
RuntimeLaunchError: buzz messages send ... exited 1:
{"error":"user_error","message":"mention '@a' does not match a current channel member; retry with --mention <pubkey>"}
```

Any TB task whose statement contains @-syntax is silently zeroed this
way.

## Fix

Pass the orchestrator's pubkey as an explicit `--mention` when posting
the task. The CLI demotes unresolved @-tokens in the text to
presentation-only when any explicit identity is supplied, so delivery
still targets exactly the orchestrator and every @-token in the task
statement becomes inert. The harness already holds the orchestrator's
`AgentCredential` (it writes that pubkey into the worker roster tables),
so no persistence is needed — fresh key per trial, fresh `--mention` per
trial.

Verified both halves against a live relay: a fenced `@a` without
`--mention` still hard-fails (the resolver is not markdown-aware); the
same content with an explicit `--mention` sends clean with
`mention_pubkeys` containing only the target.

## Testing

- `benchmarks/harbor-buzz-orchestra`: full pytest suite — 35 passed (34
baseline + new `test_send_mentions_by_pubkey_so_task_text_stays_inert`),
ruff clean. Run against `origin/main` 769ac70b with exactly this patch
applied.
- `testbed`: full pytest suite — 23 passed, 1 skipped; ruff clean.

## Acceptance

A task statement containing arbitrary @-tokens (Vim registers, emails,
decorators) launches and delivers to the orchestrator instead of dying
in `_send`.

Originating Buzz thread:

`buzz://message?channel=c3252dd2-0142-4e01-88c7-a2183c3960a5&id=74a65a0990fd2197882b66b5ea2707169d4a3dbd2020d1610c45150fb99f140b`

Signed-off-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>

2026-08-06 22:27:29 -04:00

..

harbor-buzz-orchestra

fix(bench): mention the orchestrator by pubkey when posting the task (#5136)

2026-08-06 22:27:29 -04:00