open-design

1,703 Commits 2,747 Branches 46 Tags

2 Commits

Include renames

Author	SHA1	Message	Date
lefarcenandClaude Opus 4.7	a30b868a60	mocks: scrub synclo namespace; use OD_MOCKS_* env prefix throughout These mocks were copy-pasted from synclo-explore, where they originated, and inherited the SYNCLO_EXPLORE_MOCK_* env-var convention. That brand-bleed is not appropriate in OD: rename the public env surface to OD_MOCKS_* (matching OD-native prefixes like OD_MOCKS_CACHE_DIR, OD_TRACE_R2_UPLOAD, OD_EXPECT_TIMEOUT_SECONDS). Renames: SYNCLO_EXPLORE_MOCK_TRACE → OD_MOCKS_TRACE SYNCLO_EXPLORE_MOCK_BY_PROMPT_HASH → OD_MOCKS_BY_PROMPT_HASH SYNCLO_EXPLORE_MOCK_POOL → OD_MOCKS_POOL SYNCLO_EXPLORE_MOCK_SEED → OD_MOCKS_SEED SYNCLO_EXPLORE_MOCK_NO_DELAY → OD_MOCKS_NO_DELAY SYNCLO_EXPLORE_MOCK_RECORDINGS_DIR → OD_MOCKS_RECORDINGS_DIR SYNCLO_EXPLORE_MOCK_SMOKE_TRACE → OD_MOCKS_SMOKE_TRACE SYNCLO_OD_MOCKS_I_KNOW_WHAT_IM_DOING → OD_MOCKS_ALLOW_LOCAL_UPLOAD Also drop the inline harvester usage from README. The harvester is an external CLI in nexu-io/agent-pr-explore — its README is the right place for langfuse-import flags, anonymization options, etc. OD only documents its own staging→PR→Action workflow. Smoke test (21 checks) still green; OD_MOCKS_TRACE end-to-end verified to route correctly. Consumers of the OLD env names (notably the orchestrator in nexu-io/agent-pr-explore) need a matching rename. No back-compat shim here — the explore side has zero external users today and a one-line follow-up is cleaner than a permanent deprecation layer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-29 13:20:35 +08:00
lefarcenandClaude Opus 4.7	56656eae25	feat(mocks): replay-based mock CLIs for opencode/claude/codex/deepseek/qwen/grok Drops in a `mocks/` top-level dir that pretends to be the real agent CLIs by streaming pre-recorded sessions in each CLI's native stdout protocol. Zero LLM tokens. ## Use cases - E2E tests in `apps/daemon/tests/` — exercise the full chat-server pipeline against a known trace, assert UI events / artifacts. - Self-validation during dev — iterate on `claude-stream.ts` / `json-event-stream.ts` parser changes without burning provider budget. - Regression harness — replay the same trace before and after a charter / parser change; diff the daemon events the UI surfaces. - Demo / onboarding — show what a 17-tool claude editing session looks like end-to-end, offline. ## How - 6 bash wrappers (`mocks/bin/`) shadow the real CLIs when PATH-overlaid. - `mocks/mock-agent.mjs` reads `mocks/recordings/<trace>.jsonl`, picks one via env var (`SYNCLO_EXPLORE_MOCK_TRACE` / `_POOL` / `_BY_PROMPT_HASH`), streams the trace in the requested format. - Each format renderer matches the EXACT JSON shape the OD daemon parser expects, verified line-by-line against `apps/daemon/src/{json-event-stream,claude-stream}.ts`: \| CLI \| streamFormat \| parser source \| \| ------------------------- \| ------------------------- \| ------------------------------------------ \| \| `opencode` \| `json-event-stream` \| `handleOpenCodeEvent` \| \| `codex` \| `json-event-stream` \| `handleCodexEvent` \| \| `claude` \| `claude-stream-json` \| `createClaudeStreamHandler` \| \| `deepseek` `qwen` `grok` \| `plain` \| `server.ts` (raw stdout) \| ## Quick start ```bash export PATH="$PWD/mocks/bin:$PATH" export SYNCLO_EXPLORE_MOCK_TRACE=04097377 # 8-char prefix OK export SYNCLO_EXPLORE_MOCK_NO_DELAY=1 echo "any prompt" \| opencode run echo "any prompt" \| claude -p --output-format=stream-json echo "any prompt" \| codex exec ``` The mock binary announces the picked trace id on stderr: `[mock-opencode] picked 04097377… via fixed`. Recording selection (env, in priority order): - `SYNCLO_EXPLORE_MOCK_TRACE=<id>` — fixed (prefix OK) - `SYNCLO_EXPLORE_MOCK_BY_PROMPT_HASH=1` + stdin prompt — `sha256(prompt) % N` - `SYNCLO_EXPLORE_MOCK_POOL=<tag>` — random within `agent:claude` / `skill:agent-browser` / `outcome:failed` / etc. - (default) uniform random - `SYNCLO_EXPLORE_MOCK_SEED=<str>` — reproducible "random" - `SYNCLO_EXPLORE_MOCK_NO_DELAY=1` — skip inter-event waits ## Dataset 179 anonymized Langfuse traces from this project's own production telemetry: - 9 agents: claude 57 · opencode 41 · codex 38 · gemini 25 · cursor-agent 11 · qwen 2 · copilot 2 · deepseek 2 · antigravity 1 - outcomes: succeeded 144 · failed 35 - skills: default 71 · ad-creative 50 · algorithmic-art 30 · agent-browser 22 · video-hyperframes 2 · plus magazine-web-ppt / brainstorming / data-report / penpot-flutter-design-source 1 each - 124 multi-turn (sessions with ≥2 turns) - 18 produce `<artifact>` output - ~4.5 MB on disk total Anonymization: `/Users/<name>/` → `${HOME}/`, `C:\Users\<name>\` → `%USERPROFILE%\`, project UUIDs → stable `proj-001`, `proj-002`, …. Tool input/output payloads preserved verbatim (templated UI, no cell-level PII). ## Smoke test `bash mocks/scripts/smoke-test.sh` — 6 checks across all 6 agents. All pass on this branch (verified locally): ``` ✓ opencode first event = step_start ✓ codex first event = thread.started ✓ claude first event = system ✓ deepseek emitted plain text (144 chars on first line) ✓ qwen emitted plain text (144 chars on first line) ✓ grok emitted plain text (144 chars on first line) All mock CLIs working. ✅ ``` ## Adding more recordings The exporter that produced this set lives in [nexu-io/agent-pr-explore](https://github.com/nexu-io/agent-pr-explore) (see `cli/src/local/orchestrator/langfuse-import.ts` + the `local langfuse-import` CLI command). Operators with the Langfuse keys can pull more by tag / outcome / artifact / multi-turn filter, then run `local recordings anonymize --out-dir ~/Documents/open-design/mocks/recordings`. `mocks/README.md` has the full instructions. ## Out of scope (follow-ups) - ACP agents (`devin`, `hermes`, `kilo`, `kimi`, `kiro`, `vibe`) need a JSON-RPC server on stdio rather than a one-shot stream — separate `format-acp.mjs` module not yet written. - Per-agent json-event-stream variants (`cursor-agent`, `gemini`, `qoder`, `copilot`, `pi`) currently fall back to the `plain` renderer; their parsers are in `apps/daemon/src/json-event-stream.ts` and follow the same template as `format-codex.mjs`. ## AGENTS.md updates - Added `mocks/` to the top-level content directories listing - Added a Validation strategy bullet pointing here for agent-stream / parser changes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-29 01:16:29 +08:00

Author

SHA1

Message

Date

lefarcenandClaude Opus 4.7

a30b868a60

mocks: scrub synclo namespace; use OD_MOCKS_* env prefix throughout

These mocks were copy-pasted from synclo-explore, where they
originated, and inherited the SYNCLO_EXPLORE_MOCK_* env-var
convention. That brand-bleed is not appropriate in OD: rename the
public env surface to OD_MOCKS_* (matching OD-native prefixes like
OD_MOCKS_CACHE_DIR, OD_TRACE_R2_UPLOAD, OD_EXPECT_TIMEOUT_SECONDS).

Renames:
  SYNCLO_EXPLORE_MOCK_TRACE             → OD_MOCKS_TRACE
  SYNCLO_EXPLORE_MOCK_BY_PROMPT_HASH    → OD_MOCKS_BY_PROMPT_HASH
  SYNCLO_EXPLORE_MOCK_POOL              → OD_MOCKS_POOL
  SYNCLO_EXPLORE_MOCK_SEED              → OD_MOCKS_SEED
  SYNCLO_EXPLORE_MOCK_NO_DELAY          → OD_MOCKS_NO_DELAY
  SYNCLO_EXPLORE_MOCK_RECORDINGS_DIR    → OD_MOCKS_RECORDINGS_DIR
  SYNCLO_EXPLORE_MOCK_SMOKE_TRACE       → OD_MOCKS_SMOKE_TRACE
  SYNCLO_OD_MOCKS_I_KNOW_WHAT_IM_DOING  → OD_MOCKS_ALLOW_LOCAL_UPLOAD

Also drop the inline harvester usage from README. The harvester is an
external CLI in nexu-io/agent-pr-explore — its README is the right
place for langfuse-import flags, anonymization options, etc. OD only
documents its own staging→PR→Action workflow.

Smoke test (21 checks) still green; OD_MOCKS_TRACE end-to-end
verified to route correctly.

Consumers of the OLD env names (notably the orchestrator in
nexu-io/agent-pr-explore) need a matching rename. No back-compat
shim here — the explore side has zero external users today and a
one-line follow-up is cleaner than a permanent deprecation layer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-29 13:20:35 +08:00

lefarcenandClaude Opus 4.7

56656eae25

feat(mocks): replay-based mock CLIs for opencode/claude/codex/deepseek/qwen/grok

Drops in a `mocks/` top-level dir that pretends to be the real agent
CLIs by streaming pre-recorded sessions in each CLI's native stdout
protocol. Zero LLM tokens.

## Use cases

- **E2E tests** in `apps/daemon/tests/` — exercise the full chat-server
  pipeline against a known trace, assert UI events / artifacts.
- **Self-validation during dev** — iterate on `claude-stream.ts` /
  `json-event-stream.ts` parser changes without burning provider budget.
- **Regression harness** — replay the same trace before and after a
  charter / parser change; diff the daemon events the UI surfaces.
- **Demo / onboarding** — show what a 17-tool claude editing session
  looks like end-to-end, offline.

## How

- 6 bash wrappers (`mocks/bin/`) shadow the real CLIs when PATH-overlaid.
- `mocks/mock-agent.mjs` reads `mocks/recordings/<trace>.jsonl`, picks
  one via env var (`SYNCLO_EXPLORE_MOCK_TRACE` / `_POOL` /
  `_BY_PROMPT_HASH`), streams the trace in the requested format.
- Each format renderer matches the EXACT JSON shape the OD daemon
  parser expects, verified line-by-line against
  `apps/daemon/src/{json-event-stream,claude-stream}.ts`:

  | CLI                       | streamFormat              | parser source                              |
  | ------------------------- | ------------------------- | ------------------------------------------ |
  | `opencode`                | `json-event-stream`       | `handleOpenCodeEvent`                      |
  | `codex`                   | `json-event-stream`       | `handleCodexEvent`                         |
  | `claude`                  | `claude-stream-json`      | `createClaudeStreamHandler`                |
  | `deepseek` `qwen` `grok`  | `plain`                   | `server.ts` (raw stdout)                   |

## Quick start

```bash
export PATH="$PWD/mocks/bin:$PATH"
export SYNCLO_EXPLORE_MOCK_TRACE=04097377   # 8-char prefix OK
export SYNCLO_EXPLORE_MOCK_NO_DELAY=1

echo "any prompt" | opencode run
echo "any prompt" | claude -p --output-format=stream-json
echo "any prompt" | codex exec
```

The mock binary announces the picked trace id on stderr:
`[mock-opencode] picked 04097377… via fixed`.

Recording selection (env, in priority order):
- `SYNCLO_EXPLORE_MOCK_TRACE=<id>` — fixed (prefix OK)
- `SYNCLO_EXPLORE_MOCK_BY_PROMPT_HASH=1` + stdin prompt — `sha256(prompt) % N`
- `SYNCLO_EXPLORE_MOCK_POOL=<tag>` — random within `agent:claude` /
  `skill:agent-browser` / `outcome:failed` / etc.
- (default) uniform random
- `SYNCLO_EXPLORE_MOCK_SEED=<str>` — reproducible "random"
- `SYNCLO_EXPLORE_MOCK_NO_DELAY=1` — skip inter-event waits

## Dataset

179 anonymized Langfuse traces from this project's own production
telemetry:

- 9 agents: claude 57 · opencode 41 · codex 38 · gemini 25 ·
  cursor-agent 11 · qwen 2 · copilot 2 · deepseek 2 · antigravity 1
- outcomes: succeeded 144 · failed 35
- skills: default 71 · ad-creative 50 · algorithmic-art 30 ·
  agent-browser 22 · video-hyperframes 2 · plus magazine-web-ppt /
  brainstorming / data-report / penpot-flutter-design-source 1 each
- 124 multi-turn (sessions with ≥2 turns)
- 18 produce `<artifact>` output
- ~4.5 MB on disk total

Anonymization: `/Users/<name>/` → `${HOME}/`,
`C:\Users\<name>\` → `%USERPROFILE%\`, project UUIDs →
stable `proj-001`, `proj-002`, …. Tool input/output payloads
preserved verbatim (templated UI, no cell-level PII).

## Smoke test

`bash mocks/scripts/smoke-test.sh` — 6 checks across all 6 agents.
All pass on this branch (verified locally):

```
  ✓ opencode first event = step_start
  ✓ codex first event = thread.started
  ✓ claude first event = system
  ✓ deepseek emitted plain text (144 chars on first line)
  ✓ qwen emitted plain text (144 chars on first line)
  ✓ grok emitted plain text (144 chars on first line)
All mock CLIs working. ✅
```

## Adding more recordings

The exporter that produced this set lives in
[nexu-io/agent-pr-explore](https://github.com/nexu-io/agent-pr-explore)
(see `cli/src/local/orchestrator/langfuse-import.ts` + the `local
langfuse-import` CLI command). Operators with the Langfuse keys can pull
more by tag / outcome / artifact / multi-turn filter, then run
`local recordings anonymize --out-dir ~/Documents/open-design/mocks/recordings`.
`mocks/README.md` has the full instructions.

## Out of scope (follow-ups)

- **ACP agents** (`devin`, `hermes`, `kilo`, `kimi`, `kiro`, `vibe`) need
  a JSON-RPC server on stdio rather than a one-shot stream — separate
  `format-acp.mjs` module not yet written.
- **Per-agent json-event-stream variants** (`cursor-agent`, `gemini`,
  `qoder`, `copilot`, `pi`) currently fall back to the `plain` renderer;
  their parsers are in `apps/daemon/src/json-event-stream.ts` and follow
  the same template as `format-codex.mjs`.

## AGENTS.md updates

- Added `mocks/` to the top-level content directories listing
- Added a Validation strategy bullet pointing here for agent-stream /
  parser changes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-29 01:16:29 +08:00