mirror of
https://github.com/nexu-io/open-design.git
synced 2026-10-27 09:51:12 +01:00
* feat(daemon): export project transcript to disk for downstream consumption
Adds exportProjectTranscript(db, projectsRoot, projectId, options?) — a
pure function that walks SQLite-backed conversation history and writes a
structured, lossless JSONL transcript to <projectDir>/.transcript.jsonl.
This is the input primitive that #450's "Finalize design package"
synthesis step needs. Landed ahead of the synthesis endpoint as a small,
reviewable, well-tested unit — no HTTP route, no LLM call, no web UI in
this diff. PR 2 will wire POST /api/projects/:id/finalize on top of it.
Format: JSONL with header line + per-conversation marker lines +
per-message lines. Compact encoding saves ~20–30% on synth-call tokens
vs pretty-printed JSON. schemaVersion field reserved on the header for
incompatible changes later.
Coalescing: events_json carries streaming text_delta / thinking_delta
chunks plus tool_use / tool_result / thinking_start markers and
telemetry. The export collapses runs of same-type deltas into terminal
text / thinking blocks via arrival-order with type-change flush,
preserving any interleaving with tool blocks. Telemetry (status, usage,
raw) is dropped. thinking_start is treated as an explicit flush trigger
so multiple thinking blocks in one message survive intact.
Content fallback: user-typed messages persist as plain text in
messages.content with events_json = NULL because user input does not
flow through the streaming pipeline. When event-derived blocks come
back empty, fall back to a single text block from content so user
prompts are not silently dropped.
Atomic write: tmp filename includes pid + crypto-random suffix so
concurrent exports for the same project cannot collide. fsync before
rename so a crash between rename and power loss cannot lose bytes.
Hidden file: leading dot keeps .transcript.jsonl out of listFiles
(projects.ts:54), the archive zip, and the gallery.
Tests: 14 unit cases — empty project, text/streaming coalescing, tool
ordering, telemetry filtering, type-change flush, text↔thinking↔text
interleaving, thinking_start flush, multi-conversation chronological
order, atomic-write hygiene, content fallback (both directions),
malformed events_json, and unsafe project id rejection.
Refs: nexu-io/open-design#450
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(daemon): address PR #493 review feedback for transcript export
Addresses every blocker, P2, and P3 raised on
https://github.com/nexu-io/open-design/pull/493:
- Blocker (event-shape mismatch): coalescer now switches on the
PersistedAgentEvent kind discriminator (text/thinking/tool_use/
tool_result), reading the shared `text` field for content kinds,
matching what apps/web/src/providers/daemon.ts:347-394 actually
persists into messages.events_json. Empirical confirmation: live
SQLite contains zero `type` keys.
- Removed @ts-nocheck from the source file; added inline types for
Db (Database.Database), ConversationRow, MessageRow, AttachmentRef,
CommentAttachmentRef, and Block. Tests retain @ts-nocheck per
codebase convention. Note: db.ts still uses @ts-nocheck, so the
new types catch drift inside transcript-export.ts itself, not at
the SQLite-helper boundary.
- parseEvents now distinguishes null / malformed / not_array / ok
cases; non-null-but-unparseable rows emit a console.warn with
project+message id before falling back to content.
- Switched temp-write from writeSync (which can return short) to
writeFileSync({flag:'wx'}); explicit fsync via reopen before
rename, per reviewer concern about partial-write durability.
- Added per-project lockfile (.transcript.lock) acquired with
openSync(..., 'wx') and released in finally; concurrent exports
throw the new TranscriptExportLockedError. Stale-lock recovery
is documented as a known limitation in the file header.
- Header gains attachmentCount, commentAttachmentCount, and explicit
attachmentsInlined: false. Per-message lines gain attachments /
commentAttachments references (paths only, not bytes; synthesis
reads files from disk by path). schemaVersion bumped 1 -> 2 so
the change is explicit; v1 was never consumed.
- mkdirSync(dir, { recursive: true }) at entry covers projects
with DB rows but no on-disk directory yet (codex bot finding).
- Refactored node:fs imports from named to default
(import fs from 'node:fs') so vitest spies in tests #15-#17 can
redefine properties on the underlying CJS exports object. ESM
namespace imports of node:fs produce a frozen Module Namespace
Object that vi.spyOn cannot mutate; default-import returns the
CJS module.exports which is mutable.
- Inline PersistedAgentEvent union: the daemon tsconfig does not
resolve the `@open-design/contracts/api/chat` subpath export, so
the union is restated in the source. Schema-mismatch tests cover
the case where the contract would diverge.
- Test count 14 -> 24: failure injection for writeFileSync /
fsyncSync / renameSync, existing-file replacement, lockfile
contention (lockfile-pre-create design — synchronous API can't
race via Promise.allSettled), parse-warning cases (malformed +
not-array), attachments header + per-message coverage, missing-
project-dir case.
Refs nexu-io/open-design#450 (does not close).
* fix(daemon): preserve thinking-segment boundaries on status thinking-start
Codex flagged this as a P2 on PR #493 a39d430:
https://github.com/nexu-io/open-design/pull/493#discussion_r3188524878
The web translator emits `{ kind: 'status', label: 'thinking' }` at every
thinking_start (apps/web/src/providers/daemon.ts:367-369). The previous
default branch dropped all status events without flushing the active
accumulator, so two thinking segments separated only by that marker
merged into one block — losing the original boundary and making the
transcript non-lossless for downstream synthesis.
coalesceBlocks now matches `status` explicitly: when `label === 'thinking'`
the prior accumulator flushes; other status labels and usage / raw drop
without flushing as before. Behavior fix within schemaVersion 2; no
shape change.
Test #25 verifies the boundary preservation:
[thinking 'a', thinking 'b', status 'thinking', thinking 'c', thinking 'd']
→ blocks: [thinking 'ab', thinking 'cd'] (two blocks, not one)
Existing test #5 still passes because it uses status with label 'streaming'
which remains pure telemetry and does not flush.
Suite: 506 -> 507 daemon tests, all green.
Refs nexu-io/open-design#493
---------
Co-authored-by: DevForgeAI CI/CD Engineer <devforge-ai@development.ai>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
687 lines
26 KiB
TypeScript
687 lines
26 KiB
TypeScript
// @ts-nocheck
|
|
// Persisted event shape under test is `PersistedAgentEvent` from
|
|
// packages/contracts/src/api/chat.ts (the discriminator is `kind`, the
|
|
// thinking field is `text`). The daemon's claude-stream emits a different
|
|
// `type:`-shaped wire format — those events are translated to the persisted
|
|
// `kind:` shape by the web client before being PUT back for storage.
|
|
//
|
|
// All seeded events here mirror the canonical persisted shape, exactly as
|
|
// they appear in `messages.events_json` in production databases.
|
|
//
|
|
// Note on fs imports: both this file and `transcript-export.ts` use
|
|
// `import fs from 'node:fs'` (default import — the CJS module exports
|
|
// object) so that `vi.spyOn(fs, '<fn>')` in the failure-injection tests can
|
|
// actually redefine properties. ESM namespace imports of `node:fs` (`import
|
|
// * as fs from 'node:fs'`) produce a frozen Module Namespace Object that
|
|
// `vi.spyOn` cannot mutate; default-import sidesteps that restriction
|
|
// because it returns the underlying CJS `module.exports` object.
|
|
|
|
import { afterEach, describe, expect, it, vi } from 'vitest';
|
|
import fs from 'node:fs';
|
|
import os from 'node:os';
|
|
import path from 'node:path';
|
|
import {
|
|
closeDatabase,
|
|
insertConversation,
|
|
insertProject,
|
|
openDatabase,
|
|
upsertMessage,
|
|
} from '../src/db.js';
|
|
import {
|
|
exportProjectTranscript,
|
|
TranscriptExportLockedError,
|
|
} from '../src/transcript-export.js';
|
|
|
|
const PROJECT_ID = 'project-1';
|
|
const FIXED_NOW = () => new Date('2026-05-04T12:00:00.000Z');
|
|
|
|
let tempDir: string | null = null;
|
|
let projectsRoot: string | null = null;
|
|
|
|
afterEach(() => {
|
|
closeDatabase();
|
|
vi.restoreAllMocks();
|
|
if (tempDir) fs.rmSync(tempDir, { recursive: true, force: true });
|
|
tempDir = null;
|
|
projectsRoot = null;
|
|
});
|
|
|
|
function setup(opts: { skipMkdir?: boolean } = {}): { db: any; projectsRoot: string } {
|
|
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'od-tx-'));
|
|
const db = openDatabase(tempDir);
|
|
insertProject(db, {
|
|
id: PROJECT_ID,
|
|
name: 'Project',
|
|
createdAt: 1,
|
|
updatedAt: 1,
|
|
});
|
|
projectsRoot = path.join(tempDir, 'projects');
|
|
if (!opts.skipMkdir) {
|
|
fs.mkdirSync(path.join(projectsRoot, PROJECT_ID), { recursive: true });
|
|
}
|
|
return { db, projectsRoot };
|
|
}
|
|
|
|
function readLines(filePath: string): any[] {
|
|
const raw = fs.readFileSync(filePath, 'utf8');
|
|
expect(raw.endsWith('\n')).toBe(true);
|
|
return raw
|
|
.split('\n')
|
|
.filter((l) => l.length > 0)
|
|
.map((l) => JSON.parse(l));
|
|
}
|
|
|
|
function seedConversation(db: any, opts: { id: string; createdAt: number; updatedAt?: number; title?: string | null }) {
|
|
insertConversation(db, {
|
|
id: opts.id,
|
|
projectId: PROJECT_ID,
|
|
title: opts.title ?? null,
|
|
createdAt: opts.createdAt,
|
|
updatedAt: opts.updatedAt ?? opts.createdAt,
|
|
});
|
|
}
|
|
|
|
function seedMessage(
|
|
db: any,
|
|
conversationId: string,
|
|
m: {
|
|
id: string;
|
|
role: 'user' | 'assistant';
|
|
content?: string;
|
|
events?: any[];
|
|
attachments?: any[];
|
|
commentAttachments?: any[];
|
|
},
|
|
) {
|
|
upsertMessage(db, conversationId, {
|
|
id: m.id,
|
|
role: m.role,
|
|
content: m.content ?? '',
|
|
events: m.events,
|
|
attachments: m.attachments,
|
|
commentAttachments: m.commentAttachments,
|
|
});
|
|
}
|
|
|
|
describe('exportProjectTranscript', () => {
|
|
it('writes a header-only file when the project has no conversations', () => {
|
|
const { db, projectsRoot } = setup();
|
|
const result = exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
|
|
expect(result.conversationCount).toBe(0);
|
|
expect(result.messageCount).toBe(0);
|
|
expect(result.bytesWritten).toBeGreaterThan(0);
|
|
expect(result.path).toBe(path.join(projectsRoot, PROJECT_ID, '.transcript.jsonl'));
|
|
|
|
const lines = readLines(result.path);
|
|
expect(lines).toHaveLength(1);
|
|
expect(lines[0]).toEqual({
|
|
kind: 'header',
|
|
schemaVersion: 2,
|
|
projectId: PROJECT_ID,
|
|
exportedAt: '2026-05-04T12:00:00.000Z',
|
|
conversationCount: 0,
|
|
messageCount: 0,
|
|
attachmentCount: 0,
|
|
commentAttachmentCount: 0,
|
|
attachmentsInlined: false,
|
|
});
|
|
});
|
|
|
|
it('emits header, conversation marker, and one message line per message', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100, title: 'Greeting' });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm1',
|
|
role: 'user',
|
|
events: [{ kind: 'text', text: 'hello' }],
|
|
});
|
|
seedMessage(db, 'c1', {
|
|
id: 'm2',
|
|
role: 'assistant',
|
|
events: [{ kind: 'text', text: 'world' }],
|
|
});
|
|
|
|
const result = exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
const lines = readLines(result.path);
|
|
|
|
expect(lines).toHaveLength(4);
|
|
expect(lines[0].kind).toBe('header');
|
|
expect(lines[0].schemaVersion).toBe(2);
|
|
expect(lines[0].conversationCount).toBe(1);
|
|
expect(lines[0].messageCount).toBe(2);
|
|
expect(lines[1]).toEqual({
|
|
kind: 'conversation',
|
|
id: 'c1',
|
|
title: 'Greeting',
|
|
createdAt: 100,
|
|
updatedAt: expect.any(Number),
|
|
});
|
|
expect(lines[2].kind).toBe('message');
|
|
expect(lines[2].conversationId).toBe('c1');
|
|
expect(lines[2].id).toBe('m1');
|
|
expect(lines[2].role).toBe('user');
|
|
expect(lines[2].position).toBe(0);
|
|
expect(lines[2].blocks).toEqual([{ type: 'text', text: 'hello' }]);
|
|
expect(lines[3].id).toBe('m2');
|
|
expect(lines[3].position).toBe(1);
|
|
expect(lines[3].blocks).toEqual([{ type: 'text', text: 'world' }]);
|
|
});
|
|
|
|
it('coalesces adjacent text events into a single text block', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm1',
|
|
role: 'assistant',
|
|
events: [
|
|
{ kind: 'text', text: 'hel' },
|
|
{ kind: 'text', text: 'lo' },
|
|
{ kind: 'text', text: ' world' },
|
|
],
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
const msg = lines[2];
|
|
expect(msg.blocks).toEqual([{ type: 'text', text: 'hello world' }]);
|
|
});
|
|
|
|
it('preserves tool_use and tool_result ordering interleaved with text', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm1',
|
|
role: 'assistant',
|
|
events: [
|
|
{ kind: 'text', text: 'I will read.' },
|
|
{ kind: 'tool_use', id: 'tu_1', name: 'Read', input: { path: '/x' } },
|
|
{ kind: 'tool_result', toolUseId: 'tu_1', content: 'file contents', isError: false },
|
|
{ kind: 'text', text: ' Done.' },
|
|
],
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
expect(lines[2].blocks).toEqual([
|
|
{ type: 'text', text: 'I will read.' },
|
|
{ type: 'tool_use', id: 'tu_1', name: 'Read', input: { path: '/x' } },
|
|
{ type: 'tool_result', toolUseId: 'tu_1', content: 'file contents', isError: false },
|
|
{ type: 'text', text: ' Done.' },
|
|
]);
|
|
});
|
|
|
|
it('drops status / usage / raw telemetry events without breaking content', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm1',
|
|
role: 'assistant',
|
|
events: [
|
|
{ kind: 'status', label: 'streaming' },
|
|
{ kind: 'thinking', text: 'reasoning' },
|
|
{ kind: 'usage', inputTokens: 5 },
|
|
{ kind: 'text', text: 'answer' },
|
|
{ kind: 'raw', line: '??' },
|
|
],
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
expect(lines[2].blocks).toEqual([
|
|
{ type: 'thinking', thinking: 'reasoning' },
|
|
{ type: 'text', text: 'answer' },
|
|
]);
|
|
});
|
|
|
|
it('flushes accumulator on type change (thinking → text → tool)', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm1',
|
|
role: 'assistant',
|
|
events: [
|
|
{ kind: 'thinking', text: 'plan' },
|
|
{ kind: 'text', text: 'ok' },
|
|
{ kind: 'tool_use', id: 't', name: 'X', input: {} },
|
|
],
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
expect(lines[2].blocks).toEqual([
|
|
{ type: 'thinking', thinking: 'plan' },
|
|
{ type: 'text', text: 'ok' },
|
|
{ type: 'tool_use', id: 't', name: 'X', input: {} },
|
|
]);
|
|
});
|
|
|
|
it('emits text → thinking → text as three ordered blocks (arrival order, not heuristic)', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm1',
|
|
role: 'assistant',
|
|
events: [
|
|
{ kind: 'text', text: 'pre' },
|
|
{ kind: 'thinking', text: 'mid' },
|
|
{ kind: 'text', text: 'post' },
|
|
],
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
expect(lines[2].blocks).toEqual([
|
|
{ type: 'text', text: 'pre' },
|
|
{ type: 'thinking', thinking: 'mid' },
|
|
{ type: 'text', text: 'post' },
|
|
]);
|
|
});
|
|
|
|
it('coalesces consecutive thinking events into one thinking block', () => {
|
|
// A continuous thinking run with no intervening boundary marker
|
|
// produces one block. Boundary-preservation across thinking-start
|
|
// markers is exercised in test #25 below.
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm1',
|
|
role: 'assistant',
|
|
events: [
|
|
{ kind: 'thinking', text: 'first ' },
|
|
{ kind: 'thinking', text: 'second ' },
|
|
{ kind: 'thinking', text: 'third' },
|
|
{ kind: 'text', text: 'visible' },
|
|
],
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
expect(lines[2].blocks).toEqual([
|
|
{ type: 'thinking', thinking: 'first second third' },
|
|
{ type: 'text', text: 'visible' },
|
|
]);
|
|
});
|
|
|
|
it('orders multiple conversations chronologically by created_at (regardless of updated_at)', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'older', createdAt: 100, updatedAt: 999, title: 'Older' });
|
|
seedConversation(db, { id: 'newer', createdAt: 200, updatedAt: 200, title: 'Newer' });
|
|
seedMessage(db, 'older', { id: 'm-older', role: 'user', events: [{ kind: 'text', text: 'a' }] });
|
|
seedMessage(db, 'newer', { id: 'm-newer', role: 'user', events: [{ kind: 'text', text: 'b' }] });
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
const conversationLines = lines.filter((l) => l.kind === 'conversation');
|
|
expect(conversationLines.map((c) => c.id)).toEqual(['older', 'newer']);
|
|
});
|
|
|
|
it('atomic write: leaves no .tmp file at success and does not disturb unrelated tmp files', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', { id: 'm1', role: 'user', events: [{ kind: 'text', text: 'x' }] });
|
|
|
|
// Pre-existing orphan tmp file from a hypothetical prior failed run.
|
|
const orphan = path.join(projectsRoot, PROJECT_ID, '.transcript.jsonl.tmp.99999.deadbeef');
|
|
fs.writeFileSync(orphan, 'leftover');
|
|
|
|
exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
|
|
const dirEntries = fs.readdirSync(path.join(projectsRoot, PROJECT_ID));
|
|
const tmps = dirEntries.filter((n) => n.startsWith('.transcript.jsonl.tmp.'));
|
|
// Only the orphan should remain — our run's tmp must have been renamed away.
|
|
expect(tmps).toEqual(['.transcript.jsonl.tmp.99999.deadbeef']);
|
|
expect(fs.readFileSync(orphan, 'utf8')).toBe('leftover');
|
|
expect(dirEntries).toContain('.transcript.jsonl');
|
|
});
|
|
|
|
it('falls back to messages.content as a single text block when events_json is null', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
// User-typed messages persist as plain text in `content`; events_json is
|
|
// null because the user input does not flow through the streaming pipeline.
|
|
upsertMessage(db, 'c1', {
|
|
id: 'm-user',
|
|
role: 'user',
|
|
content: 'Make me a landing page.',
|
|
// events deliberately omitted
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
expect(lines[2].id).toBe('m-user');
|
|
expect(lines[2].blocks).toEqual([{ type: 'text', text: 'Make me a landing page.' }]);
|
|
});
|
|
|
|
it('prefers event-derived blocks over the content fallback when both are present', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
// Assistant rows in production carry a coalesced `content` AND the full
|
|
// `events` blocks. The event-derived blocks are richer (tool_use,
|
|
// thinking) so they must win.
|
|
upsertMessage(db, 'c1', {
|
|
id: 'm-asst',
|
|
role: 'assistant',
|
|
content: 'final coalesced text',
|
|
events: [
|
|
{ kind: 'text', text: 'final ' },
|
|
{ kind: 'text', text: 'coalesced text' },
|
|
{ kind: 'tool_use', id: 'tu_1', name: 'Read', input: { path: '/x' } },
|
|
],
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
expect(lines[2].blocks).toEqual([
|
|
{ type: 'text', text: 'final coalesced text' },
|
|
{ type: 'tool_use', id: 'tu_1', name: 'Read', input: { path: '/x' } },
|
|
]);
|
|
});
|
|
|
|
it('produces empty blocks (no throw) for messages with malformed events_json', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
// Bypass the helpers so we can inject a deliberately malformed value.
|
|
db.prepare(
|
|
`INSERT INTO messages (id, conversation_id, role, content, events_json, position, created_at)
|
|
VALUES ('mbad', 'c1', 'assistant', '', 'not json', 0, ${Date.now()})`,
|
|
).run();
|
|
|
|
// Suppress the now-emitted warning so test output stays clean.
|
|
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
|
|
|
const result = exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
const lines = readLines(result.path);
|
|
expect(lines).toHaveLength(3); // header + conversation + 1 message
|
|
expect(lines[2].id).toBe('mbad');
|
|
expect(lines[2].blocks).toEqual([]);
|
|
});
|
|
|
|
it('rejects unsafe project ids (path-traversal guard from projectDir)', () => {
|
|
const { db, projectsRoot } = setup();
|
|
expect(() =>
|
|
exportProjectTranscript(db, projectsRoot, '../etc', { now: FIXED_NOW }),
|
|
).toThrow(/invalid project id/);
|
|
});
|
|
|
|
// ---------- §1.8 atomic-write failure injection (tests #15-#17) ----------
|
|
|
|
it('cleans up tmp file when writeFileSync throws', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', { id: 'm1', role: 'user', events: [{ kind: 'text', text: 'x' }] });
|
|
|
|
const realWrite = fs.writeFileSync;
|
|
vi.spyOn(fs, 'writeFileSync').mockImplementation((p: any, ...rest: any[]) => {
|
|
// Fail only on the transcript tmp write. Other writes (e.g. test
|
|
// fixtures) must continue to work.
|
|
if (typeof p === 'string' && p.includes('.transcript.jsonl.tmp.')) {
|
|
throw new Error('disk full');
|
|
}
|
|
return (realWrite as any)(p, ...rest);
|
|
});
|
|
|
|
expect(() =>
|
|
exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }),
|
|
).toThrow(/disk full/);
|
|
|
|
const dirEntries = fs.readdirSync(path.join(projectsRoot, PROJECT_ID));
|
|
expect(dirEntries.filter((n) => n.startsWith('.transcript.jsonl.tmp.'))).toEqual([]);
|
|
expect(dirEntries).not.toContain('.transcript.jsonl');
|
|
// Lock should also have been released.
|
|
expect(dirEntries).not.toContain('.transcript.lock');
|
|
});
|
|
|
|
it('cleans up tmp file when fsyncSync throws', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', { id: 'm1', role: 'user', events: [{ kind: 'text', text: 'x' }] });
|
|
|
|
vi.spyOn(fs, 'fsyncSync').mockImplementation(() => {
|
|
throw new Error('fsync failed');
|
|
});
|
|
|
|
expect(() =>
|
|
exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }),
|
|
).toThrow(/fsync failed/);
|
|
|
|
const dirEntries = fs.readdirSync(path.join(projectsRoot, PROJECT_ID));
|
|
expect(dirEntries.filter((n) => n.startsWith('.transcript.jsonl.tmp.'))).toEqual([]);
|
|
expect(dirEntries).not.toContain('.transcript.jsonl');
|
|
expect(dirEntries).not.toContain('.transcript.lock');
|
|
});
|
|
|
|
it('cleans up tmp file when renameSync throws', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', { id: 'm1', role: 'user', events: [{ kind: 'text', text: 'x' }] });
|
|
|
|
vi.spyOn(fs, 'renameSync').mockImplementation(() => {
|
|
throw new Error('rename failed');
|
|
});
|
|
|
|
expect(() =>
|
|
exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }),
|
|
).toThrow(/rename failed/);
|
|
|
|
const dirEntries = fs.readdirSync(path.join(projectsRoot, PROJECT_ID));
|
|
expect(dirEntries.filter((n) => n.startsWith('.transcript.jsonl.tmp.'))).toEqual([]);
|
|
expect(dirEntries).not.toContain('.transcript.jsonl');
|
|
expect(dirEntries).not.toContain('.transcript.lock');
|
|
});
|
|
|
|
// ---------- §1.8 existing-file replacement (test #18) ----------
|
|
|
|
it('replaces existing transcript file on second export', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', { id: 'm1', role: 'user', events: [{ kind: 'text', text: 'x' }] });
|
|
|
|
// First export.
|
|
const result1 = exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
const finalPath = result1.path;
|
|
|
|
// Inject a sentinel — a downstream consumer / older transcript.
|
|
fs.writeFileSync(finalPath, '{"sentinel":true}\n');
|
|
expect(fs.readFileSync(finalPath, 'utf8')).toContain('sentinel');
|
|
|
|
// Second export should atomically replace the sentinel.
|
|
exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
|
|
const after = fs.readFileSync(finalPath, 'utf8');
|
|
expect(after).not.toContain('sentinel');
|
|
const lines = after.split('\n').filter((l) => l.length > 0).map((l) => JSON.parse(l));
|
|
expect(lines[0].kind).toBe('header');
|
|
expect(lines[2].id).toBe('m1');
|
|
});
|
|
|
|
// ---------- §1.5 lock contention (test #19, advisor-redesigned) ----------
|
|
|
|
it('throws TranscriptExportLockedError when lock held; succeeds after unlink', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', { id: 'm1', role: 'user', events: [{ kind: 'text', text: 'x' }] });
|
|
|
|
const lockPath = path.join(projectsRoot, PROJECT_ID, '.transcript.lock');
|
|
const finalPath = path.join(projectsRoot, PROJECT_ID, '.transcript.jsonl');
|
|
|
|
// Pre-create the lock to simulate a concurrent export in flight.
|
|
fs.writeFileSync(lockPath, '');
|
|
|
|
expect(() =>
|
|
exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }),
|
|
).toThrow(TranscriptExportLockedError);
|
|
// No transcript should have been written while the lock was held.
|
|
expect(fs.existsSync(finalPath)).toBe(false);
|
|
|
|
// Release the lock — a subsequent export must succeed.
|
|
fs.unlinkSync(lockPath);
|
|
|
|
const result = exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
expect(result.path).toBe(finalPath);
|
|
expect(fs.existsSync(finalPath)).toBe(true);
|
|
expect(fs.existsSync(lockPath)).toBe(false);
|
|
});
|
|
|
|
// ---------- §1.3 parse-warning surface (tests #20-#21) ----------
|
|
|
|
it('warns when events_json is malformed JSON and falls back to content', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
db.prepare(
|
|
`INSERT INTO messages (id, conversation_id, role, content, events_json, position, created_at)
|
|
VALUES ('mmal', 'c1', 'assistant', 'fallback content', '{not valid', 0, ${Date.now()})`,
|
|
).run();
|
|
|
|
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
|
const result = exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
expect(warn).toHaveBeenCalledTimes(1);
|
|
expect(warn.mock.calls[0][0]).toContain('mmal');
|
|
expect(warn.mock.calls[0][0]).toContain(PROJECT_ID);
|
|
expect(warn.mock.calls[0][0]).toContain('malformed');
|
|
|
|
const lines = readLines(result.path);
|
|
expect(lines[2].id).toBe('mmal');
|
|
expect(lines[2].blocks).toEqual([{ type: 'text', text: 'fallback content' }]);
|
|
});
|
|
|
|
it('warns when events_json is JSON but not an array', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
db.prepare(
|
|
`INSERT INTO messages (id, conversation_id, role, content, events_json, position, created_at)
|
|
VALUES ('mobj', 'c1', 'assistant', 'fallback content', '{"foo":1}', 0, ${Date.now()})`,
|
|
).run();
|
|
|
|
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
|
|
const result = exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
expect(warn).toHaveBeenCalledTimes(1);
|
|
expect(warn.mock.calls[0][0]).toContain('mobj');
|
|
expect(warn.mock.calls[0][0]).toContain('not_array');
|
|
|
|
const lines = readLines(result.path);
|
|
expect(lines[2].blocks).toEqual([{ type: 'text', text: 'fallback content' }]);
|
|
});
|
|
|
|
// ---------- §1.6 attachments (tests #22-#23) ----------
|
|
|
|
it('header carries attachmentCount + commentAttachmentCount totals', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm1',
|
|
role: 'user',
|
|
events: [{ kind: 'text', text: 'a' }],
|
|
attachments: [
|
|
{ path: 'a.png', name: 'a.png', kind: 'image', size: 100 },
|
|
{ path: 'b.png', name: 'b.png', kind: 'image', size: 200 },
|
|
],
|
|
commentAttachments: [
|
|
{
|
|
id: 'ca1',
|
|
order: 0,
|
|
filePath: 'p.html',
|
|
elementId: 'e1',
|
|
selector: '#x',
|
|
label: 'L',
|
|
comment: 'C',
|
|
currentText: '',
|
|
pagePosition: { x: 0, y: 0 },
|
|
htmlHint: '',
|
|
},
|
|
],
|
|
});
|
|
seedMessage(db, 'c1', {
|
|
id: 'm2',
|
|
role: 'user',
|
|
events: [{ kind: 'text', text: 'b' }],
|
|
attachments: [{ path: 'c.png', name: 'c.png', kind: 'image' }],
|
|
});
|
|
|
|
const result = exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
const lines = readLines(result.path);
|
|
expect(lines[0].attachmentCount).toBe(3);
|
|
expect(lines[0].commentAttachmentCount).toBe(1);
|
|
expect(lines[0].attachmentsInlined).toBe(false);
|
|
});
|
|
|
|
it('per-message line carries attachments / commentAttachments only when present', () => {
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm-with',
|
|
role: 'user',
|
|
events: [{ kind: 'text', text: 'q' }],
|
|
attachments: [{ path: 'a.png', name: 'a.png', kind: 'image', size: 99 }],
|
|
commentAttachments: [
|
|
{
|
|
id: 'ca1',
|
|
order: 0,
|
|
filePath: 'p.html',
|
|
elementId: 'e1',
|
|
selector: '#x',
|
|
label: 'Lab',
|
|
comment: 'Cmt',
|
|
currentText: '',
|
|
pagePosition: { x: 1, y: 2 },
|
|
htmlHint: '',
|
|
},
|
|
],
|
|
});
|
|
seedMessage(db, 'c1', {
|
|
id: 'm-bare',
|
|
role: 'user',
|
|
events: [{ kind: 'text', text: 'r' }],
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
const withAtt = lines.find((l) => l.id === 'm-with');
|
|
const bare = lines.find((l) => l.id === 'm-bare');
|
|
|
|
expect(withAtt.attachments).toEqual([
|
|
{ path: 'a.png', name: 'a.png', kind: 'image', size: 99 },
|
|
]);
|
|
expect(withAtt.commentAttachments).toEqual([
|
|
{ id: 'ca1', filePath: 'p.html', label: 'Lab', comment: 'Cmt' },
|
|
]);
|
|
expect(bare.attachments).toBeUndefined();
|
|
expect(bare.commentAttachments).toBeUndefined();
|
|
});
|
|
|
|
// ---------- §1.7 missing project directory (test #24) ----------
|
|
|
|
it('creates project directory if it does not exist on disk', () => {
|
|
const { db, projectsRoot } = setup({ skipMkdir: true });
|
|
expect(fs.existsSync(path.join(projectsRoot, PROJECT_ID))).toBe(false);
|
|
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', { id: 'm1', role: 'user', events: [{ kind: 'text', text: 'x' }] });
|
|
|
|
const result = exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW });
|
|
expect(fs.existsSync(result.path)).toBe(true);
|
|
const lines = readLines(result.path);
|
|
expect(lines[0].kind).toBe('header');
|
|
expect(lines[2].id).toBe('m1');
|
|
});
|
|
|
|
// ---------- Codex P2 (3188524878): thinking-start boundary preservation ----------
|
|
|
|
it('flushes thinking accumulator on status thinking-start marker so adjacent segments stay separate', () => {
|
|
// The web translator emits `{ kind: 'status', label: 'thinking' }` at
|
|
// every thinking_start (apps/web/src/providers/daemon.ts:367-369).
|
|
// Two thinking segments separated only by that marker must stay as two
|
|
// blocks; merging them would lose the original boundary and make the
|
|
// transcript non-lossless for synthesis.
|
|
const { db, projectsRoot } = setup();
|
|
seedConversation(db, { id: 'c1', createdAt: 100 });
|
|
seedMessage(db, 'c1', {
|
|
id: 'm1',
|
|
role: 'assistant',
|
|
events: [
|
|
{ kind: 'thinking', text: 'a' },
|
|
{ kind: 'thinking', text: 'b' },
|
|
{ kind: 'status', label: 'thinking' },
|
|
{ kind: 'thinking', text: 'c' },
|
|
{ kind: 'thinking', text: 'd' },
|
|
],
|
|
});
|
|
|
|
const lines = readLines(exportProjectTranscript(db, projectsRoot, PROJECT_ID, { now: FIXED_NOW }).path);
|
|
expect(lines[2].blocks).toEqual([
|
|
{ type: 'thinking', thinking: 'ab' },
|
|
{ type: 'thinking', thinking: 'cd' },
|
|
]);
|
|
});
|
|
});
|