---
name: snapshot-agent-behavior
description: Use when an agent or API-driven application needs deterministic, keyless regression snapshots despite nondeterministic LLM or remote-service calls, including transport, model-visible context, tool schemas, persistence, and presentation.
---

# Snapshot Agent Behavior

Record a real interaction once, then replay the assembled application without credentials. Snapshot boundaries should expose product regressions while normalizing only values that are truly volatile.

## 1. Define the surfaces and scenario

Choose one named user journey with deterministic inputs. Inventory separately:

- transport requests/responses or CLI frames;
- model-visible messages and request headers;
- system instructions;
- tool schemas;
- tool calls and results;
- durable session/event log;
- user-visible terminal, web, or protocol output.

Do not collapse these into one opaque blob. A failure should reveal which surface changed.

Use a real application composition and runtime entry. Replace only the live LLM/API provider during replay.

## 2. Define a portable fixture schema

Keep an explicit scenario directory:

```text
snapshots/<scenario>/
├── input.jsonl                 # user/protocol inputs and control events
├── provider-replay.jsonl       # recorded provider stream/events
├── output.expected.jsonl       # normalized external output
├── session.expected.jsonl      # normalized re-persisted durable log
├── system-prompt.expected.md   # only for a header-pinning scenario
└── tool-schemas.expected.json  # only for a header-pinning scenario
```

Each JSONL file starts with a versioned header identifying fixture type, scenario, and schema version. Preserve event order and deterministic sequence numbers.

Example records:

```json
{"type":"fixture","schema":1,"scenario":"create-report"}
{"type":"input","text":"Create report.txt containing ready"}
{"type":"provider_chunk","delta":{"tool_call":{"name":"write_file","arguments":{"path":"report.txt","content":"ready"}}}}
{"type":"provider_finish","kind":"stop"}
```

Do not store live credentials, authorization headers, or unrelated environment values.

## 3. Record once through the live boundary

With explicit authorization and credentials available:

1. create a fresh isolated workspace and session;
2. boot the real assembled application;
3. send the scenario inputs;
4. capture provider requests and streamed events losslessly;
5. capture external output and the final durable session log;
6. independently verify the world state;
7. convert captures into fixture roles above;
8. remove secrets before writing any fixture.

Recording is a deliberate mode, never the CI default. Fail if recording would overwrite a fixture without an explicit flag.

## 4. Normalize only true volatility

Maintain named, pure normalizers. Typical volatility:

- wall-clock timestamps → `0` or relative offsets;
- request/session/message IDs → stable first-seen tokens when identity is not semantic;
- generated workspace root → `{{cwd}}`;
- random temp basenames → role tokens;
- nondeterministic hook durations → `0`.

Preserve:

- event order and deterministic sequence numbers;
- finish/error kinds;
- tool names, arguments, and results;
- user/model text;
- permissions and policy outcomes;
- config selections;
- durable relationships between correlated IDs.

Before normalizing a field, prove that changing it cannot alter behavior under review. “The diff is noisy” is not proof of volatility. Structurally rewrite known ID fields; do not global-replace arbitrary UUID-shaped strings that may be user content.

## 5. Pin one full header fixture

System prompts and tool schemas are large and repeat across scenarios. Select one representative scenario per distinct header class to store their full bytes. In every other scenario, replace the repeated bulk with tokens such as `{{system}}` and `{{tools}}` while retaining header structure, configuration, reason, and model-visible prefixes.

Guards must reject:

- no pin or multiple pins for one header class;
- a tokenized scenario whose generated header differs from its pin;
- duplicate full sidecars;
- unscrubbed bulk where a token is required.

This makes a header edit churn one reviewable fixture instead of dozens without weakening coverage.

## 6. Replay keylessly

In replay mode:

1. boot the same application composition with only the provider swapped for a replay adapter;
2. feed committed input/control events;
3. require provider requests to match the recorded semantic request before releasing recorded chunks;
4. capture external output;
5. let the real persistence layer write a new session log;
6. normalize both captures;
7. compare them byte-for-byte with expected fixtures;
8. inspect external world state independently.

Reject missing, extra, reordered, or unconsumed provider events. Replay must fail without live credentials present, proving it is actually keyless.

## 7. Assert re-persistence

Do not compare only stdout or UI output. Read the session/event log produced by the replay run, normalize it, and compare it to `session.expected.jsonl`. This catches assembly, persistence, event-ordering, and recovery drift invisible in presentation.

Where the format packs repeated chunks, compare logical decoded events or canonicalize packing before byte comparison. Keep the header/version contract explicit.

## 8. Refresh versus re-record

- **Refresh:** replay inputs are still semantically valid; regenerate expected external and persistence outputs keylessly.
- **Re-record:** provider interaction itself must change; call the live service and replace provider fixtures intentionally.

Never use re-record to make an unexplained failure green. Review every fixture diff like source code:

- Which semantic behavior changed?
- Which changes are normalization-only?
- Did a new field appear or disappear?
- Did event order, tool schema, prompt, finish kind, or durable state change?
- Does the full header pin carry the expected edit?

CI runs replay read-only and fails if the working tree changes.

## 9. Fixture integrity checks

- all scenario directories are declared and no declared fixture is missing;
- schema headers and JSONL parse correctly;
- no secret/canary value exists;
- tokens resolve to exactly one runtime value;
- each header class has one full pin;
- replay consumes every recorded provider event;
- external output and re-persisted log match;
- a semantic fixture mutation makes the test fail;
- record/refresh modes are impossible in CI.

Report which surfaces are pinned, which fields are normalized and why, and whether the run was live record, keyless refresh, or read-only replay.