External/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-08-18 06:50:31 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
397564b434dd757ff105072f2003a6653992efdc
buzz/TESTING.md
T

43 KiB
Raw Blame History

Sprout Testing Guide

This guide enables an AI agent (the operator) to run the full Sprout test suite: automated cargo test suites and a three-agent multi-agent E2E run that exercises all 41 MCP tools against a live relay.

Two Test Modes

Mode What It Does When to Use
Automated (cargo test) Unit tests + REST/WebSocket/MCP integration tests Fast CI check; verify no unit regressions
Multi-Agent E2E Three agents (Alice, Bob, Charlie) run via sprout-acp harness, exercising all 41 MCP tools via real Nostr identities Before merging relay/MCP/auth changes; full regression run; exploring new features

Run both modes for a complete regression check. Run automated-only for a fast sanity check.

Table of Contents

  1. Prerequisites
  2. Quick Start: Automated Tests Only
  3. Multi-Agent E2E Testing
  4. Advanced: ACP Harness Scenarios
  5. Workflow YAML Reference
  6. The 41 MCP Tools
  7. Cleanup
  8. Known Issues / Troubleshooting

1. Prerequisites

Verify each requirement before proceeding. All commands must succeed.

Docker

docker --version
# Required: any recent version

docker compose version
# Required: v2+ (uses "docker compose", not "docker-compose")

Rust 1.88+

# From the sprout repo root — use Hermit if system Rust is older than 1.88
. bin/activate-hermit

rustc --version
# Required: rustc 1.88.0 or newer

goose CLI

goose --version
# Must be on $PATH and configured with a valid provider/model

goose run --help | head -5
# Must not error

sqlx-cli

sqlx --version
# If missing:
cargo install sqlx-cli --no-default-features --features mysql

screen

screen --version
# Must print a version string (note: on macOS this exits with code 1 — that's fine)
# If missing: brew install screen

All clear

If all commands above print version info, proceed. If any binary is missing, install it first — the tests will not work without all prerequisites.

2. Quick Start: Automated Tests Only

Run this when you want a fast check without spinning up multi-agent infrastructure.

# Enter the repo and activate toolchain FIRST — all subsequent commands
# assume you are in the sprout repo root with hermit activated.
cd /path/to/sprout   # e.g. ~/Development/goosetown_oss/REPOS/sprout
. bin/activate-hermit

Check for existing infrastructure

If Docker services or a relay are already running from a previous session, you can leave the Docker services up and just reset the database and relay:

# Kill any existing relay
screen -S relay -X quit 2>/dev/null
lsof -ti :3000 | xargs kill -9 2>/dev/null

# Check Docker services — if already running, skip `docker compose up`
docker compose ps --format '{{.Name}} {{.Status}}' 2>/dev/null
# If mysql/redis/typesense show "Up", you can skip to "Setup and build" below.
# If not running:
docker compose up -d

Port conflicts: If docker compose up -d fails with "port already allocated", a container from another project may be using the port. Find it with docker ps --format '{{.Names}} {{.Ports}}' and stop it manually.

Keycloak: You may see sprout-keycloak as unhealthy or starting — this is fine. Keycloak is only needed for token-based auth and is not required for automated tests (which use dev-mode X-Pubkey header auth). You may also see extra containers like sprout-postgres from other projects — ignore them.

Setup and build

# Configure environment
[ -f .env ] || cp .env.example .env
# Load env vars — ALWAYS required, even if .env already existed
export $(cat .env | grep -v "^#" | grep -v "^$" | xargs) 2>/dev/null

# Reset database (fresh state for tests)
docker exec sprout-mysql mysql -u root -psprout_dev -e \
  "DROP DATABASE IF EXISTS sprout; CREATE DATABASE sprout;" 2>/dev/null
sqlx migrate run --database-url "$DATABASE_URL"

# Build the full workspace (relay, MCP server, ACP harness, test client, etc.)
cargo build --release --workspace

# Run unit tests
cargo test --workspace

Integration Tests (require running relay)

Start the relay (kill any stale instance first):

screen -S relay -X quit 2>/dev/null
lsof -ti :3000 | xargs kill -9 2>/dev/null; sleep 1
screen -dmS relay bash -c \
  'export $(cat .env | grep -v "^#" | grep -v "^$" | xargs) 2>/dev/null; \
   ./target/release/sprout-relay 2>&1 | tee /tmp/sprout-relay.log'
sleep 3 && curl -s http://localhost:3000/health
# Must print: ok

Then run the integration suites:

# REST API integration tests (40 tests)
RELAY_URL=ws://localhost:3000 \
  cargo test -p sprout-test-client --test e2e_rest_api -- --ignored

# WebSocket relay integration tests (14 tests)
RELAY_URL=ws://localhost:3000 \
  cargo test -p sprout-test-client --test e2e_relay -- --ignored

# MCP server integration tests (14 tests)
RELAY_URL=ws://localhost:3000 \
  cargo test -p sprout-test-client --test e2e_mcp -- --ignored

Expected Results

test result: ok. 40 passed; 0 failed; 0 ignored   ← REST API
test result: ok. 14 passed; 0 failed; 0 ignored   ← relay
test result: ok. 14 passed; 0 failed; 0 ignored   ← MCP

All 68 integration tests pass (across the three suites above). An additional 7 workflow integration tests exist in e2e_workflows.rs — run them separately if workflow changes are involved. If any fail, check that the relay is running and Docker services are healthy before proceeding to E2E.

3. Multi-Agent E2E Testing

3.1 Architecture

The E2E suite uses the sprout-acp harness — a process that bridges Sprout relay events to AI agents over the ACP protocol. The operator sends @mention events via the mention binary; each harness instance picks up mentions targeting its agent's pubkey and forwards them to a goose session with Sprout MCP tools pre-configured.

Operator (you)
    │
    │  mention <channel> <pubkey> "task instructions"
    ▼
Sprout Relay  ──WS (NIP-01)──►  sprout-acp (harness)  ──stdio (ACP)──►  goose
                                                                            │
                                                                       sprout-mcp-server
                                                                        (41 MCP tools)
                                                                            │
                                                                       Sprout Relay
                                                                    (send_message, etc.)

Three harness instances run simultaneously — one each for Alice, Bob, and Charlie. Each has its own Nostr keypair (identity) and responds only to @mentions targeting its pubkey.

Key properties of the harness:

  • Discovers and subscribes to all accessible channels on startup
  • Queues events per channel; one prompt in flight globally at a time
  • Batches multiple rapid @mentions into a single prompt
  • Auto-respawns the agent subprocess on crash
  • Reconnects to the relay with a since filter on disconnect (no missed events)
  • GOOSE_MODE=auto is mandatory — prevents goose from pausing for permission prompts

3.2 Infrastructure Setup

Run all commands from the sprout repo root.

cd /path/to/sprout
. bin/activate-hermit

# 1. Start Docker services (MySQL, Redis, Typesense, Keycloak)
docker compose down -v && docker compose up -d
docker compose ps   # All services should show "Up"

# 2. Configure environment
[ -f .env ] || cp .env.example .env
export $(cat .env | grep -v "^#" | grep -v "^$" | xargs) 2>/dev/null

# 3. Run database migrations
sqlx migrate run --database-url "$DATABASE_URL"

# 4. Build all binaries (sprout-acp, sprout-mcp-server, mention, sprout-admin)
cargo build --release --workspace

# 5. Add release binaries to PATH
export PATH="$PWD/target/release:$PATH"

# 6. Verify key binaries are present
ls -la target/release/sprout-acp target/release/sprout-mcp-server \
        target/release/mention target/release/sprout-admin

# 7. Start the relay
lsof -ti :3000 | xargs kill -9 2>/dev/null; sleep 1
screen -dmS relay bash -c \
  'export $(cat .env | grep -v "^#" | grep -v "^$" | xargs) 2>/dev/null; \
   ./target/release/sprout-relay 2>&1 | tee /tmp/sprout-relay.log'
sleep 3

# 8. Verify relay is up
curl -s http://localhost:3000/health
# Expected: {"status":"ok"} or similar

3.3 Mint Agent Keys

Each agent needs its own Nostr keypair. Use sprout-admin to mint them — it handles all database interaction internally.

# Mint keys for all three agents
for agent in alice bob charlie; do
  echo "=== $agent ==="
  cargo run -p sprout-admin -- mint-token \
    --name "$agent" \
    --scopes "messages:read,messages:write,channels:read"
  echo ""
done

Each invocation prints an nsec1... private key, the corresponding pubkey hex, and an API token. Save all three sets immediately — they are shown only once.

Set environment variables for the session:

# Replace with actual values from mint-token output
export ALICE_NSEC="nsec1..."
export ALICE_PUBKEY="<alice-pubkey-hex>"

export BOB_NSEC="nsec1..."
export BOB_PUBKEY="<bob-pubkey-hex>"

export CHARLIE_NSEC="nsec1..."
export CHARLIE_PUBKEY="<charlie-pubkey-hex>"

Tip: Pipe the mint output to a temp file during setup: cargo run -p sprout-admin -- mint-token --name alice ... | tee /tmp/alice-keys.txt

3.4 Launch Harness Instances

Start one sprout-acp instance per agent in a dedicated screen session. GOOSE_MODE=auto is required on all three.

# Alice's harness
SPROUT_PRIVATE_KEY="$ALICE_NSEC" \
SPROUT_RELAY_URL="ws://localhost:3000" \
GOOSE_MODE=auto \
  screen -dmS agent-alice bash -c \
    'sprout-acp 2>&1 | tee /tmp/agent-alice.log'

# Bob's harness
SPROUT_PRIVATE_KEY="$BOB_NSEC" \
SPROUT_RELAY_URL="ws://localhost:3000" \
GOOSE_MODE=auto \
  screen -dmS agent-bob bash -c \
    'sprout-acp 2>&1 | tee /tmp/agent-bob.log'

# Charlie's harness
SPROUT_PRIVATE_KEY="$CHARLIE_NSEC" \
SPROUT_RELAY_URL="ws://localhost:3000" \
GOOSE_MODE=auto \
  screen -dmS agent-charlie bash -c \
    'sprout-acp 2>&1 | tee /tmp/agent-charlie.log'

Wait ~5 seconds for all three to connect, then verify:

sleep 5

for agent in alice bob charlie; do
  echo "=== agent-$agent ==="
  grep -E "connected|discovered|subscribed|error" /tmp/agent-$agent.log 2>/dev/null \
    || echo "(no log yet)"
  echo ""
done

Expected startup output for each harness:

sprout-acp starting: relay=ws://localhost:3000 harness_pubkey=... agent_pubkey=<hex>
agent initialized: ...
connected to relay at ws://localhost:3000
discovered N channel(s)
subscribed to channel <uuid>

If you see discovered 0 channel(s), the agent is not yet a member of any channels. Alice will create channels in the first exercise — after that, all three will discover them on subsequent subscriptions (open channels are accessible to any authenticated pubkey).

Bootstrap channel timing: Harnesses discover channels only at startup. If you create the bootstrap channel (in exercise A-1) after launching harnesses, Alice's harness won't be subscribed to it. Two options:

  1. Create the bootstrap channel before launching harnesses (recommended): run the curl -X POST command from A-1 first, then start all three harnesses.
  2. Restart Alice's harness after creating the bootstrap channel — it will discover and subscribe to it on reconnect.

3.5 Test Exercises

All exercises are delivered via @mention events using the mention binary:

mention <channel_uuid> <target_pubkey_hex> "task instructions"

The mention binary generates ephemeral sender keys — it does not need its own nsec. It requires SPROUT_RELAY_URL (defaults to ws://localhost:3000).

Important: Channel UUIDs are dynamic. Alice creates the channels in Exercise A-1. After that step completes, query the REST API to get the UUIDs before proceeding with other exercises.

# Helper: get channel UUID by name (run after Alice creates channels)
get_channel_uuid() {
  local name="$1"
  curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
    "http://localhost:3000/api/channels" \
    | jq -r ".[] | select(.name == \"$name\") | .id"
}

Alice — Infrastructure Creator

Alice sets up the shared environment that Bob and Charlie will use.

A-1: Create channels and seed messages

Alice needs a bootstrap channel to receive her first @mention. Use the default test channel from the relay, or create one via the REST API first:

# Create a bootstrap channel for Alice's first mention
BOOTSTRAP_CHANNEL=$(curl -s -X POST \
  -H "Content-Type: application/json" \
  -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels" \
  -d '{"name":"bootstrap","channel_type":"stream","visibility":"open"}' \
  | jq -r '.id')
echo "Bootstrap channel: $BOOTSTRAP_CHANNEL"

Then send Alice her first task:

mention "$BOOTSTRAP_CHANNEL" "$ALICE_PUBKEY" \
  "Create 3 channels: 'general' (stream/open), 'alice-testing' (stream/open), and 'private-ops' (stream/private). Then send 3 messages to the 'general' channel introducing yourself and describing what you're testing."

Wait for Alice to complete (~30–60s), then capture channel UUIDs:

sleep 60
export GENERAL=$(get_channel_uuid "general")
export ALICE_TESTING=$(get_channel_uuid "alice-testing")
export PRIVATE_OPS=$(get_channel_uuid "private-ops")
echo "general=$GENERAL  alice-testing=$ALICE_TESTING  private-ops=$PRIVATE_OPS"

A-2: Channel metadata and canvas

mention "$GENERAL" "$ALICE_PUBKEY" \
  "Set the topic on the 'general' channel to 'Sprout E2E Testing'. Set the purpose to 'Multi-agent integration test run'. Then set the canvas on 'general' to a markdown document with a header '# Test Run Notes' and a bullet list of the 3 channels you created."

A-3: Thread and reactions

mention "$GENERAL" "$ALICE_PUBKEY" \
  "Get the history of the 'general' channel. Reply to your first message there with a thread reply saying 'This is a thread reply from Alice'. Then add a 👍 reaction and a 🚀 reaction to your own first message."

A-4: Workflow creation

mention "$GENERAL" "$ALICE_PUBKEY" \
  "Create a workflow named 'alice-notify' with a message_posted trigger on the 'general' channel. The workflow should have one step: send a message to the 'general' channel saying 'Workflow fired!'. Save the workflow ID and report it back."

A-5: Profile, NIP-05 identity, and presence

mention "$GENERAL" "$ALICE_PUBKEY" \
  "Set your display name to 'Alice (Test Agent)'. Set your about/bio to 'I am Alice, the infrastructure creator for the Sprout E2E test suite.' Set your NIP-05 handle to 'alice@localhost' using set_profile. Then use set_presence to set your status to 'online'. Finally, use get_presence to check your own presence status (pubkey: $ALICE_PUBKEY) and confirm it shows 'online'."

A-6: Feed, search, and membership

mention "$GENERAL" "$ALICE_PUBKEY" \
  "Get your feed and the channel feed for 'general'. Search for messages containing the word 'Alice'. List the members of the 'general' channel. Then invite Bob (pubkey: $BOB_PUBKEY) to the 'private-ops' channel."

Bob — Discoverer and Reactor

Bob explores the environment Alice created and interacts with her content.

B-1: Discovery and history

mention "$GENERAL" "$BOB_PUBKEY" \
  "List all channels you have access to. Get the message history from the 'general' channel (last 20 messages). Report what you find — how many channels exist, and what did Alice write in general?"

B-2: Reactions and DM

mention "$GENERAL" "$BOB_PUBKEY" \
  "React to the first message in the 'general' channel with a ❤️ reaction. Then send a direct message to Alice (pubkey: $ALICE_PUBKEY) saying 'Hi Alice, Bob here — I can see your channels and messages. The setup looks great!'"

B-3: DM history and canvas

mention "$GENERAL" "$BOB_PUBKEY" \
  "Get your DM conversation history with Alice. Read the canvas on the 'general' channel and report what it says. List all your DM conversations."

B-4: Thread participation

mention "$GENERAL" "$BOB_PUBKEY" \
  "Find Alice's thread in the 'general' channel (a message that has replies). Add a thread reply saying 'Bob joining the thread — everything looks good from my end.' Then get the thread replies and report how many there are."

B-5: Channel join and profile

mention "$GENERAL" "$BOB_PUBKEY" \
  "Join the 'alice-testing' channel. Get your own profile and set your display name to 'Bob (Test Agent)'. Search for any messages mentioning 'workflow' or 'canvas'."

B-6: Private channel access test

mention "$GENERAL" "$BOB_PUBKEY" \
  "Get the message history from the 'private-ops' channel (ID: $PRIVATE_OPS). Alice invited you in exercise A-6 — confirm you have access and report what you find."

B-7: Get presence

mention "$GENERAL" "$BOB_PUBKEY" \
  "Use set_presence to set your status to 'away'. Then get the presence status for Alice (pubkey: $ALICE_PUBKEY) and yourself. Report both statuses — Alice should be 'online' (from A-5) and you should be 'away'."

B-8: Profile resolution (public profiles)

mention "$GENERAL" "$BOB_PUBKEY" \
  "Use get_user_profile to look up Alice's profile (pubkey: $ALICE_PUBKEY). Report her display name and about text. Then use get_users_batch with all three pubkeys (yours: $BOB_PUBKEY, Alice: $ALICE_PUBKEY, Charlie: $CHARLIE_PUBKEY). Report which ones have display names set and which are in the missing list."

B-8: Profile resolution (public profiles)

mention "$GENERAL" "$BOB_PUBKEY" \
  "Use get_user_profile to look up Alice's profile (pubkey: $ALICE_PUBKEY). Report her display name and about text. Then use get_users_batch with all three pubkeys (yours: $BOB_PUBKEY, Alice: $ALICE_PUBKEY, Charlie: $CHARLIE_PUBKEY). Report which ones have display names set and which are in the missing list."

Charlie — Edge Case Specialist

Charlie tests error handling, idempotency, and lifecycle operations.

C-1: Non-existent channel error

mention "$GENERAL" "$CHARLIE_PUBKEY" \
  "Try to send a message to channel UUID '00000000-0000-0000-0000-000000000000'. Report the exact error you receive."

C-2: Unauthorized archive attempt

mention "$GENERAL" "$CHARLIE_PUBKEY" \
  "Try to archive the 'general' channel (ID: $GENERAL). You did not create it — report what error you get."

C-3: Canvas overwrite

mention "$GENERAL" "$CHARLIE_PUBKEY" \
  "Set the canvas on the 'general' channel to a new markdown document: '# Charlie Was Here\n\nCharlie overwrote the canvas on $(date -u +%Y-%m-%dT%H:%M:%SZ)'. Then immediately read the canvas back and confirm it shows your content."

C-4: Reaction idempotency

mention "$GENERAL" "$CHARLIE_PUBKEY" \
  "Find the first message in the 'general' channel. Add a 🎉 reaction to it. Then try to add the same 🎉 reaction again. Report what happens the second time — does it error or succeed silently?"

C-5: Channel lifecycle (create → archive → send → unarchive → send)

mention "$GENERAL" "$CHARLIE_PUBKEY" \
  "Create a new channel named 'charlie-lifecycle' (stream/open). Send a message to it saying 'Before archive'. Archive the channel. Try to send another message — report the error. Unarchive the channel. Send a message saying 'After unarchive'. Confirm the final message was accepted."

C-6: Join, send, leave, verify

mention "$GENERAL" "$CHARLIE_PUBKEY" \
  "Join the 'alice-testing' channel. Send a message there saying 'Charlie was here'. Then leave the channel. Try to send another message to 'alice-testing' — report whether it succeeds or fails after leaving."

C-7: Workflow trigger and cross-agent summary

mention "$GENERAL" "$CHARLIE_PUBKEY" \
  "Get the list of workflows. Find Alice's 'alice-notify' workflow and trigger it via webhook if it has a webhook trigger, or note that it uses message_posted. Then get the presence for both Alice (pubkey: $ALICE_PUBKEY) and Bob (pubkey: $BOB_PUBKEY). Finally, produce a summary report in the 'general' channel listing: (1) all channels created during this test run, (2) total messages sent, (3) any errors encountered."

C-8: NIP-05 identity verification

mention "$GENERAL" "$CHARLIE_PUBKEY" \
  "Verify the NIP-05 endpoint. Alice set her NIP-05 handle to 'alice@localhost' in exercise A-5. Use curl or an HTTP request to GET http://localhost:3000/.well-known/nostr.json?name=alice — it should return her pubkey in the 'names' map and a relay URL in the 'relays' map. Also try ?name=nonexistent and confirm it returns empty names/relays. Check that the response includes an Access-Control-Allow-Origin: * header. Report your findings."

C-9: Profile edge cases

mention "$GENERAL" "$CHARLIE_PUBKEY" \
  "Test profile edge cases. Use get_user_profile with a pubkey that doesn't exist — report the error. Use get_users_batch with a mix of valid pubkeys, an invalid-length string like 'tooshort', and a string that is 64 chars but not valid hex. Report what ends up in the profiles map vs the missing list."

3.6 Monitoring & Verification

Watch harness logs live

# Tail the log files (agent-safe — no TTY required)
tail -f /tmp/agent-alice.log &
tail -f /tmp/agent-bob.log &
tail -f /tmp/agent-charlie.log &

# Or view recent output from a specific agent
tail -50 /tmp/agent-alice.log

Note: Do NOT use screen -r to attach to harness sessions if you are an AI agent — it requires an interactive TTY and will hang indefinitely. Always use tail on the log files or grep for specific patterns instead.

Key log patterns to watch for:

# Good — turn completed
turn complete for channel <uuid>: end_turn

# Good — agent is working
prompting agent for channel <uuid> (session ..., N event(s))

# Investigate — agent had trouble
turn complete for channel <uuid>: max_tokens
turn timeout (300s) for channel <uuid> — cancelling

# Bad — needs attention
agent process exited — respawning
relay connection lost — reconnecting

Tail log files

# All three agents at once
tail -f /tmp/agent-alice.log /tmp/agent-bob.log /tmp/agent-charlie.log

# Filter for completions only
grep "turn complete\|turn timeout\|turn cancelled" \
  /tmp/agent-alice.log /tmp/agent-bob.log /tmp/agent-charlie.log

REST API verification

# List all channels (use any agent's pubkey)
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels" \
  | jq '.[] | {id: .id, name: .name, visibility: .visibility}'

# Recent messages in general
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels/$GENERAL/messages?limit=20" \
  | jq '.[] | {sender: .pubkey[:16], body: .content[:120]}'

# Messages from a specific agent
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels/$GENERAL/messages?limit=50" \
  | jq --arg pk "$CHARLIE_PUBKEY" \
    '[.[] | select(.pubkey == $pk)] | {count: length, messages: [.[] | .content[:100]]}'

# Channel members
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels/$GENERAL/members" \
  | jq '.[] | {pubkey: .pubkey[:16], role: .role}'

# Check private-ops membership (Bob should be there after A-6)
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels/$PRIVATE_OPS/members" \
  | jq '.[] | .pubkey[:16]'

Screen session management

# List all active sessions
screen -ls

# Check if a session is still running
screen -ls | grep agent-alice

# Capture current screen contents to file (non-destructive)
screen -S agent-alice -X hardcopy /tmp/alice-snapshot.txt
cat /tmp/alice-snapshot.txt

3.7 Expected Results

After all exercises complete, the following should be true:

Check Expected
Channels created At least 5: general, alice-testing, private-ops, charlie-lifecycle, bootstrap
Messages in general 10+ messages from Alice, Bob, and Charlie
Thread replies At least 2 replies on Alice's first message
Reactions 👍 🚀 (Alice), ❤️ (Bob), 🎉 (Charlie) on general messages
Canvas Charlie's content (last writer wins)
DM conversation Alice ↔ Bob DM exists
Bob in private-ops Yes (Alice invited him in A-6)
Workflow alice-notify created with message_posted trigger
Display names Alice and Bob have display names set
Profile resolution Bob can read Alice's profile via get_user_profile; get_users_batch returns Alice and Bob with display names, Charlie with null display name (all in profiles map)
NIP-05 verification Charlie queries /.well-known/nostr.json?name=alice and gets Alice's pubkey (Alice set alice@localhost in A-5)
Profile edge cases Charlie gets appropriate errors for invalid/unknown pubkeys
Presence Alice is 'online' (A-5), Bob is 'away' (B-7); Charlie can read both via get_presence (C-7)
Error handling Charlie's C-1, C-2, C-5 exercises report correct errors
charlie-lifecycle Unarchived and final message sent successfully

Verify the full picture:

# Channel count
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels" | jq 'length'

# Message count in general
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels/$GENERAL/messages?limit=200" | jq 'length'

# Workflow list
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/workflows" \
  | jq '.[] | {name: .name, trigger: .trigger.on}'

4. Advanced: ACP Harness Scenarios

These scenarios test the sprout-acp harness itself — crash recovery, relay reconnection, turn timeout, and concurrent multi-agent operation. Run them independently from the main E2E suite.

Prerequisites for Advanced Scenarios

# Use the single-agent test key from sprout-acp TESTING.md
export SPROUT_PRIVATE_KEY=nsec1ddyp0fufd6ejerfqkxcfqlmkktwzx7w45emalvgtcvyafefusj5q8fyllm
export AGENT_PUBKEY=ae670a075ac2446f445808ab5a1a796cec37c72c70b25e10ee39f7f0eab50feb
export TEST_CHANNEL=94a444a4-c0a3-5966-ab05-530c6ddc2301

# Start a single harness
SPROUT_PRIVATE_KEY="$SPROUT_PRIVATE_KEY" \
SPROUT_RELAY_URL="ws://localhost:3000" \
GOOSE_MODE=auto \
  screen -dmS harness bash -c 'sprout-acp 2>&1 | tee /tmp/harness.log'
sleep 5

Scenario A: Basic @mention → Agent Replies

mention "$TEST_CHANNEL" "$AGENT_PUBKEY" "What is 2 + 2? Reply with just the number."
sleep 30

# Verify reply via REST API
curl -s -H "X-Pubkey: $AGENT_PUBKEY" \
  "http://localhost:3000/api/channels/$TEST_CHANNEL/messages?limit=5" \
  | jq --arg pk "$AGENT_PUBKEY" \
    '.[] | select(.pubkey == $pk) | {body: .content[:200]}'

Expected: agent replies with "4" via send_message.

Scenario B: Multi-Event Batch

# Send 3 mentions in rapid succession
for i in 1 2 3; do
  mention "$TEST_CHANNEL" "$AGENT_PUBKEY" "Batch message $i" &
done
wait
sleep 30

# Check harness log for batch size
grep "prompting agent" /tmp/harness.log | tail -5
# Look for "(session ..., N event(s))" where N > 1

Scenario C: Agent Crash Recovery

# Kill the agent subprocess
kill -9 $(pgrep -f "goose acp") 2>/dev/null

# Send a new mention
sleep 2
mention "$TEST_CHANNEL" "$AGENT_PUBKEY" "Are you still alive after the crash?"
sleep 30

# Verify recovery in logs
grep -E "agent process exited|agent respawned|turn complete" /tmp/harness.log | tail -10

Expected log sequence: agent process exited — respawning → agent respawned successfully → agent initialized → turn complete.

Scenario D: Relay Disconnect Recovery

# Stop the relay
screen -S relay -X stuff $'\003'   # Ctrl-C
sleep 2

# Watch harness detect disconnect
grep "relay connection lost\|reconnecting" /tmp/harness.log

# Restart relay
screen -dmS relay bash -c \
  'export $(cat .env | grep -v "^#" | grep -v "^$" | xargs) 2>/dev/null; \
   ./target/release/sprout-relay 2>&1 | tee /tmp/sprout-relay.log'
sleep 5

# Verify reconnection
grep "relay reconnected\|subscribed" /tmp/harness.log | tail -5

# Confirm harness is functional again
mention "$TEST_CHANNEL" "$AGENT_PUBKEY" "Post-reconnect test — reply with OK"
sleep 30
grep "turn complete" /tmp/harness.log | tail -3

Scenario E: Turn Timeout

# Restart harness with 5-second timeout
screen -S harness -X quit
SPROUT_PRIVATE_KEY="$SPROUT_PRIVATE_KEY" \
SPROUT_RELAY_URL="ws://localhost:3000" \
SPROUT_ACP_TURN_TIMEOUT=5 \
GOOSE_MODE=auto \
  screen -dmS harness bash -c 'sprout-acp 2>&1 | tee /tmp/harness.log'
sleep 3

# Send a prompt that will take longer than 5 seconds
mention "$TEST_CHANNEL" "$AGENT_PUBKEY" \
  "Write a detailed 500-word essay on the history of computing, then list 50 prime numbers."
sleep 15

# Verify timeout was triggered
grep "turn timeout\|turn cancelled" /tmp/harness.log | tail -5

# Reset to normal timeout
screen -S harness -X quit
SPROUT_PRIVATE_KEY="$SPROUT_PRIVATE_KEY" \
SPROUT_RELAY_URL="ws://localhost:3000" \
GOOSE_MODE=auto \
  screen -dmS harness bash -c 'sprout-acp 2>&1 | tee /tmp/harness.log'

Expected: turn timeout (5s) for channel ... — cancelling then turn cancelled for channel .... Harness continues running.

Scenario F: Permission Handling (GOOSE_MODE=auto)

mention "$TEST_CHANNEL" "$AGENT_PUBKEY" \
  "Use your tools to get the last 5 messages from this channel and summarize them."
sleep 60

# Verify no permission prompts appeared
grep -i "permission\|approval\|waiting" /tmp/harness.log | head -5
# Should return nothing

grep "turn complete" /tmp/harness.log | tail -3
# Should show end_turn

Scenario G: Channel Discovery

# Restart harness fresh and watch discovery
screen -S harness -X quit
SPROUT_PRIVATE_KEY="$SPROUT_PRIVATE_KEY" \
SPROUT_RELAY_URL="ws://localhost:3000" \
GOOSE_MODE=auto \
  screen -dmS harness bash -c 'sprout-acp 2>&1 | tee /tmp/harness.log'
sleep 5

grep "discovered\|subscribed" /tmp/harness.log
# Expected: "discovered N channel(s)" then "subscribed to channel <uuid>" for each

Verify channel membership via REST API:

curl -s -H "X-Pubkey: $AGENT_PUBKEY" \
  "http://localhost:3000/api/channels" \
  | jq '.[] | {id: .id, name: .name}'

Scenario H: Concurrent Channels (FIFO Fairness)

# Get a second channel UUID (create one if needed)
CHANNEL_B=$(curl -s -X POST \
  -H "Content-Type: application/json" \
  -H "X-Pubkey: $AGENT_PUBKEY" \
  "http://localhost:3000/api/channels" \
  -d '{"name":"channel-b-test","channel_type":"stream","visibility":"open"}' \
  | jq -r '.id')

# Send to channel A first, then B immediately
mention "$TEST_CHANNEL" "$AGENT_PUBKEY" "Channel A message — process me first"
mention "$CHANNEL_B" "$AGENT_PUBKEY" "Channel B message — process me second"
sleep 60

# Verify FIFO ordering in logs
grep "prompting agent for channel" /tmp/harness.log | tail -5
# Channel A should appear before Channel B
# No two "prompting agent" lines without a "turn complete" between them

Scenario I: Multi-Agent (3 Agents, 1 Channel)

# Mint two additional keypairs
cargo run -p sprout-admin -- mint-token \
  --name "agent-b" --scopes "messages:read,messages:write,channels:read" \
  | tee /tmp/agent-b-keys.txt

cargo run -p sprout-admin -- mint-token \
  --name "agent-c" --scopes "messages:read,messages:write,channels:read" \
  | tee /tmp/agent-c-keys.txt

# Extract keys (adjust parsing as needed based on output format)
AGENT_B_NSEC=$(grep "nsec1" /tmp/agent-b-keys.txt | awk '{print $NF}')
AGENT_B_PUBKEY=$(grep "pubkey" /tmp/agent-b-keys.txt | awk '{print $NF}')
AGENT_C_NSEC=$(grep "nsec1" /tmp/agent-c-keys.txt | awk '{print $NF}')
AGENT_C_PUBKEY=$(grep "pubkey" /tmp/agent-c-keys.txt | awk '{print $NF}')

# Start three harnesses
SPROUT_PRIVATE_KEY="$SPROUT_PRIVATE_KEY" GOOSE_MODE=auto \
  screen -dmS harness-a bash -c 'sprout-acp 2>&1 | tee /tmp/harness-a.log'

SPROUT_PRIVATE_KEY="$AGENT_B_NSEC" GOOSE_MODE=auto \
  screen -dmS harness-b bash -c 'sprout-acp 2>&1 | tee /tmp/harness-b.log'

SPROUT_PRIVATE_KEY="$AGENT_C_NSEC" GOOSE_MODE=auto \
  screen -dmS harness-c bash -c 'sprout-acp 2>&1 | tee /tmp/harness-c.log'

sleep 5

# Send a targeted @mention to each agent
mention "$TEST_CHANNEL" "$AGENT_PUBKEY"  "Hello agent-a, reply with PONG-A"
mention "$TEST_CHANNEL" "$AGENT_B_PUBKEY" "Hello agent-b, reply with PONG-B"
mention "$TEST_CHANNEL" "$AGENT_C_PUBKEY" "Hello agent-c, reply with PONG-C"
sleep 60

# Verify three distinct replies
curl -s -H "X-Pubkey: $AGENT_PUBKEY" \
  "http://localhost:3000/api/channels/$TEST_CHANNEL/messages?limit=20" \
  | jq '.[] | {sender: (.pubkey[:16] + "..."), body: .content[:100]}'
# Look for three distinct sender prefixes, each with a PONG reply

# Cleanup
for s in harness-a harness-b harness-c; do screen -S $s -X quit; done

5. Workflow YAML Reference

Workflows are created via the create_workflow MCP tool. The YAML structure:

name: My Workflow
trigger:
  on: message_posted     # Valid: message_posted | reaction_added | webhook
  channel_id: "<uuid>"   # Optional: scope to a specific channel
steps:
  - id: notify           # Required: alphanumeric + underscores only
    action: send_message # Action type
    channel_id: "<uuid>" # Action-specific fields are DIRECT properties (not nested)
    text: "Workflow fired!"

Valid triggers:

  • message_posted — fires when any message is posted (optionally scoped to a channel)
  • reaction_added — fires when a reaction is added to a message
  • webhook — fires when the webhook URL is called via HTTP POST

Step field rules:

  • id is required on every step (alphanumeric and underscores)
  • Action fields (channel_id, text, etc.) are direct properties of the step object — do NOT nest them under a params key
  • create_workflow tool accepts the YAML as a string parameter

Example: message_posted workflow

name: welcome-new-messages
trigger:
  on: message_posted
  channel_id: "94a444a4-c0a3-5966-ab05-530c6ddc2301"
steps:
  - id: echo_reply
    action: send_message
    channel_id: "94a444a4-c0a3-5966-ab05-530c6ddc2301"
    text: "New message detected!"

Example: webhook workflow

name: external-trigger
trigger:
  on: webhook
steps:
  - id: notify_channel
    action: send_message
    channel_id: "94a444a4-c0a3-5966-ab05-530c6ddc2301"
    text: "Webhook triggered this workflow!"

6. The 41 MCP Tools

The sprout-mcp-server exposes 41 tools covering the full Sprout feature surface. All are available to agents running via the sprout-acp harness.

Channels (8)

Tool Description
list_channels List all channels accessible to the agent
get_channel Get metadata for a specific channel
create_channel Create a new channel (channel_type: stream|forum, visibility: open|private)
update_channel Update channel name or metadata
archive_channel Archive a channel (creator only)
unarchive_channel Restore an archived channel
join_channel Join an open channel
leave_channel Leave a channel

Messages (2)

Tool Description
send_message Post a message to a channel
get_channel_history Get recent messages from a channel

Threads (2)

Tool Description
send_reply Reply within a message thread
get_thread Get replies in a thread

Reactions (3)

Tool Description
add_reaction Add an emoji reaction to a message
remove_reaction Remove a reaction
get_reactions List all reactions on a message

Direct Messages (3)

Tool Description
open_dm Create or retrieve a DM channel with a user (optionally send an initial message)
add_dm_member Add a member to an existing DM conversation
list_dms List all DM conversations

Canvas (2)

Tool Description
get_canvas Read the canvas document for a channel
set_canvas Write/overwrite the canvas document (last writer wins)

Workflows (7)

Tool Description
list_workflows List all workflows
create_workflow Create a new workflow with trigger and steps
update_workflow Update an existing workflow
delete_workflow Delete a workflow
trigger_workflow Manually trigger a webhook workflow
get_workflow_runs Get execution history for a workflow
approve_workflow_step Approve a pending approval step in a workflow run

Feed (3)

Tool Description
get_feed Get the agent's personal activity feed
get_feed_mentions Get mentions from the agent's feed
get_feed_actions Get action items from the agent's feed

Search (1)

Tool Description
search Full-text search across messages and channels

Profile (3)

Tool Description
set_profile Set display name, about/bio, avatar URL, and NIP-05 handle
get_user_profile Get any user's profile by pubkey (omit pubkey for own profile)
get_users_batch Bulk resolve display names and NIP-05 handles for multiple pubkeys

Presence (1)

Tool Description
get_presence Bulk presence lookup by pubkey
set_presence Set presence status (online/away/offline) with TTL

Members (3)

Tool Description
add_channel_member Add a user (by pubkey) to a channel
remove_channel_member Remove a member from a channel
list_channel_members List members of a channel

Admin (2)

Tool Description
set_channel_topic Set the topic for a channel
set_channel_purpose Set the purpose for a channel

7. Cleanup

Stop harness instances

# E2E test harnesses
for s in agent-alice agent-bob agent-charlie; do
  screen -S $s -X quit 2>/dev/null && echo "stopped $s" || echo "$s not running"
done

# Advanced scenario harnesses
for s in harness harness-a harness-b harness-c; do
  screen -S $s -X quit 2>/dev/null
done

Stop relay

screen -S relay -X quit 2>/dev/null && echo "relay stopped"

Verify all sessions gone

screen -ls
# Should show "No Sockets found" or only unrelated sessions

Tear down Docker services

# Stop services and remove volumes (full reset)
docker compose down -v

# Stop services only (preserve data for next run)
docker compose down

Clean up temp files

rm -f /tmp/agent-alice.log /tmp/agent-bob.log /tmp/agent-charlie.log
rm -f /tmp/harness.log /tmp/harness-a.log /tmp/harness-b.log /tmp/harness-c.log
rm -f /tmp/sprout-relay.log
rm -f /tmp/alice-keys.txt /tmp/agent-b-keys.txt /tmp/agent-c-keys.txt

8. Known Issues / Troubleshooting

Current Status

All automated tests pass as of 2026-03-11:

  • ✅ 40/40 REST API integration tests
  • ✅ 14/14 WebSocket relay integration tests
  • ✅ 14/14 MCP server integration tests
  • ✅ Multi-agent E2E (Alice/Bob/Charlie) via sprout-acp harness

Harness exits immediately with "configuration error"

Cause: SPROUT_PRIVATE_KEY not set or invalid.

echo $SPROUT_PRIVATE_KEY
# Must be a valid nsec1... bech32 string

"relay connect error" on startup

Cause: Relay not running or wrong URL.

# Check relay session
screen -ls | grep relay

# If missing, start it
screen -dmS relay bash -c \
  'export $(cat .env | grep -v "^#" | grep -v "^$" | xargs) 2>/dev/null; \
   ./target/release/sprout-relay 2>&1 | tee /tmp/sprout-relay.log'

# Verify
curl -s http://localhost:3000/health

"discovered 0 channel(s)"

Cause: Agent pubkey is not a member of any open channels.

# Check what channels are accessible
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels" | jq 'length'

Open channels are accessible to any authenticated pubkey. If the relay has no open channels yet, Alice's bootstrap channel creation (Exercise A-1) will fix this. After Alice creates channels, restart Bob's and Charlie's harnesses so they rediscover.

"failed to spawn agent"

Cause: goose binary not found or not on $PATH.

which goose
goose --version

Ensure goose is installed and configured with a valid provider/model before starting the harness.

Agent hangs, turn never completes

Cause: GOOSE_MODE=auto not set — goose is waiting for permission approval.

# Kill and restart with GOOSE_MODE=auto
screen -S agent-alice -X quit
SPROUT_PRIVATE_KEY="$ALICE_NSEC" \
SPROUT_RELAY_URL="ws://localhost:3000" \
GOOSE_MODE=auto \
  screen -dmS agent-alice bash -c 'sprout-acp 2>&1 | tee /tmp/agent-alice.log'

GOOSE_MODE=auto is mandatory for all harness instances. Without it, the first MCP tool call will hang indefinitely.

No agent reply after @mention

Checklist:

  1. Is the harness running? pgrep -a sprout-acp
  2. Is the harness subscribed to the target channel? grep "subscribed" /tmp/agent-alice.log
  3. Did mention use the correct pubkey hex? Check $ALICE_PUBKEY is set correctly.
  4. Check harness logs for errors: tail -50 /tmp/agent-alice.log
  5. Verify via REST API that the @mention event arrived:
curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels/$GENERAL/messages?limit=10" \
  | jq '.[] | {kind: .kind, sender: .pubkey[:16], body: .content[:100]}'

MCP tool calls failing

Cause: sprout-mcp-server binary not found, or wrong relay URL passed to MCP.

which sprout-mcp-server
# If missing: cargo build --release -p sprout-mcp-server
# Then: export PATH="$PWD/target/release:$PATH"

Channel UUIDs not set after Alice's first exercise

If $GENERAL, $ALICE_TESTING, or $PRIVATE_OPS are empty, Alice may not have finished yet. Wait longer, then re-query:

sleep 30
export GENERAL=$(curl -s -H "X-Pubkey: $ALICE_PUBKEY" \
  "http://localhost:3000/api/channels" \
  | jq -r '.[] | select(.name == "general") | .id')
echo "GENERAL=$GENERAL"

If still empty, check Alice's harness logs for errors: tail -50 /tmp/agent-alice.log.

Stale events replayed on harness restart

Expected behavior. On startup, the harness replays all unprocessed @mentions since the last run. If you restart a harness mid-test, expect a burst of activity as it catches up on stale events. This is correct — the harness uses a since filter on reconnect to avoid missing events.

To start fresh with no stale events, use a new keypair (mint a new token) for the harness instance.

Docker services unhealthy

docker compose ps
# If any service is not "Up":
docker compose down -v && docker compose up -d
# Wait 30s then re-run migrations:
sqlx migrate run --database-url "$DATABASE_URL"
Reference in New Issue View Git Blame Copy Permalink