diff --git a/docs/testing/amr-runtime-contract.md b/docs/testing/amr-runtime-contract.md new file mode 100644 index 0000000000..226e535509 --- /dev/null +++ b/docs/testing/amr-runtime-contract.md @@ -0,0 +1,177 @@ +# AMR prerelease runtime contract validation + +This is the regression entry point for OPEND-2886. It distinguishes package +identity, the Vela/ACP bridge contract, and actual OpenCode execution. Passing +one does not establish the others. + +## Evidence boundaries + +| Check | Real component | Synthetic component | What it establishes | +| --- | --- | --- | --- | +| `tools-pack verify-runtime` | Selected package's Vela and OpenCode executables | None | Exact versions, byte hashes, app/version/platform agreement with the supplied release manifest | +| E2E `tests/amr/vela-contract.test.ts` | Pinned Vela binary and current daemon ACP consumer | OpenCode HTTP/SSE server, Link model catalog | Complete/incomplete handling, unknown-tool evidence, cancellation, conservative failure | +| `scripts/vela-contract.ts` | Explicitly selected Vela binary | Same local HTTP/SSE fixtures | Cross-version error/capability/continuation and Write abort ordering | +| Full package acceptance | Installed app, Vela, OpenCode, daemon, provider | Only deliberate fault injection | Actual compaction, durable session state, side effects, UI/CLI, telemetry and release behavior | + +The bridge fixtures never call a model provider. The reported model context +budget is 128,000 because Vela 0.0.35 rejects smaller catalog budgets. Synthetic +SSE compaction events are **not** evidence of forcing the real compaction loop. +Reports set `realCompactionRuns: 0` and `packageAcceptance: not-performed`. +The POSIX fixture launcher runs on macOS/Linux; Windows bridge acceptance is +still pending. The identity command supports matching macOS, Windows and Linux +release manifests, but only macOS arm64 has been exercised against a real package. + +## Verify package identity + +Obtain the immutable platform manifest and payload/installer from the selected +release. Verify the archive against its published SHA-256 before extraction or +installation. Select the actual package's Resources directory, not a source +checkout or a developer CLI path. Daemon data directory handling follows the +[root contract](../../AGENTS.md#daemon-data-directory-contract). + +```sh +corepack pnpm tools-pack verify-runtime \ + --resources "$PACKAGE_RESOURCES" \ + --manifest "$RELEASE_PLATFORM_MANIFEST" \ + --expected-vela "$REVIEWED_VELA_VERSION" \ + --expected-opencode "$REVIEWED_OPENCODE_VERSION" \ + --json +``` + +The command reads `open-design-config.json`, then executes only the Vela and +OpenCode paths below `open-design/bin`. Missing companions, paths escaping the +package, version/platform mismatches and binaries changing during verification +fail the command. No PATH fallback is used. Expected binary versions must come +from the reviewed Vela package pin and its published runtime provenance and +expected self-reported OpenCode version, independently of the binary being checked. Current release manifests do not contain those +binary versions/hashes; the command records them alongside the manifest hash, +not as values purportedly supplied by that manifest. + +## Run bridge contracts + +The normal E2E suite resolves the exact optional dependency owned by +`tools/pack/package.json`, without using a developer PATH binary: + +```sh +corepack pnpm --filter @open-design/e2e test tests/amr/vela-contract.test.ts +``` + +Both `ci.yml`'s `e2e_vitest` lane and `release-prerelease-tests.yml` already run +the full E2E Vitest suite. The planner selects `e2e_vitest` for a Vela dependency +bump touching `tools/pack/package.json` and `pnpm-lock.yaml`; no routing omission +or new workflow is introduced. The legacy error assertion intentionally names +the current 0.0.35 contract: a changed protocol requires a reviewed update. + +Use the standalone entry point to compare package bytes with a candidate build: + +```sh +corepack pnpm -C e2e exec tsx scripts/vela-contract.ts \ + --binary "$EXACT_VELA_BINARY" \ + --expected-version "$EXACT_VELA_VERSION" \ + --out "$CONTRACT_REPORT_DIR" \ + --require-write-abort --require-continuation +``` + +The two target flags are independent while OPEND-2884 and OPEND-2885 are separate +branches. Omit a flag only when recording a baseline or testing the other +isolated fix; the report records which requirements were enabled. For the +combined candidate, enable **both**. Do not publish a dependency bump based on a +baseline-only pass. + +Each scenario records the raw ACP transcript and a local OpenCode request/response +ledger. `--require-write-abort` requires exactly one abort response to finish +before the ACP terminal response (monotonic timestamps), not cleanup after the +fact. Both the outstanding Write and completed-Write/missing-terminal scenarios +must remain unsuccessful and must submit only one prompt. + +`--require-continuation` requires the negotiated +`com.open-design.nativeSessionContinue` v1 capability and structured +`OPENCODE_COMPACTION_CONTINUATION_INCOMPLETE` error, with explicit committed-tool +evidence. It starts a fresh Vela process, loads the same durable session ID and +uses `_session/continue` with the original cursor. There must be no new session +or prompt replay, one continue request and one terminal response. A parallel +outstanding tool must report `toolResultsCommitted: false`. These are bridge +contract checks; the fixture does not prove OpenCode's persisted cursor validation +or real tool execution. OPEND-2884 owns those production tests and the host's +cancellation/attempt-limit policy. + +## Verified baseline, 2026-09-09 + +Host: macOS Darwin 24.6.0, arm64, Node 24.16.0. OpenDesign test source baseline: +`d54f5cf07d35261dce3c8c2e7fc187c6cc9efb86`. + +The immutable [0.22.1-prerelease.13 macOS manifest](https://releases.open-design.ai/prerelease/versions/0.22.1-prerelease.13/platforms/mac_arm64.json) +identifies app commit `ee76e93c75c5235bbb7a3464e832cff987d80c91`. Its downloaded +payload passed the published archive checksum. Selected executables were +extracted from that verified payload; the full app was not installed/launched. + +| Artifact | Version | SHA-256 | +| --- | --- | --- | +| macOS arm64 payload | 0.22.1-prerelease.13 | `6b6e0ac16e4f3ed4c54011ea771f73f81305c7a51d24b808dde6aa8732a9bdb7` | +| Package Vela | 0.0.35 | `5a2449f368b95cb15b29694906b4abd1abf7fe4afd71bde75f05db48708a323c` | +| Package OpenCode | 0.0.0--202609020336 | `30fc9f9f687460db9cb340df183b926973df554fb7f98aea6a43b5e6e765d704` | + +The [Vela 0.0.35 runtime manifest](https://github.com/powerformer/vela/releases/download/vela-cli/v0.0.35/vela-cli-runtime-manifest.json) +records `powerformer/opencode@powerformer-v1.18.1`, commit +`595fa8ec01bdec67546c3c0f79744dce598f85cf`. Its darwin-arm64 member hash +`c1dec4a2f722191c0e5c31c0414c348e3ef4d9a68e844194a549ca536d148f9e` +matches the npm companion exactly. The app package re-signs that companion. +After removing signatures from **copies only**, the npm and package binaries +have identical bytes except the Mach-O `__LINKEDIT.vmsize` field. Normalizing +that field makes them byte-identical. The repository's fallback OpenCode lock +(`anomalyco/opencode` 1.15.10) is therefore not the provenance of this release. +The fork tag and self-reported `--version` string are separate identifiers. + +- Package Vela: complete continuation, incomplete protection, cancellation and + unknown-error checks passed. Both Write failure/abort requirements failed, + the native continuation capability is absent, and the new structured + unknown-tool evidence requirement fails. This is an expected RED + against the target contract, not a passed package acceptance. +- [Vela PR #1951](https://github.com/powerformer/vela/pull/1951), commit + `43f35e29e50dad790d2c697f127c9c137ce05abd`: locally built `0.0.1-test`, SHA-256 + `6cf45efc55509138bd415cc6ad818532bc713ec1c5a58c3e1fc931aee33c0ca3`, passed + all seven baseline/Write scenarios, including both Write abort-order checks. + This is an unpublished development build, not an upgrade target. + The final PR head `28b5733f5d1b558d8966e76a094b65b74ef95e7e` adds only + ACP timeout/cancellation fixture tests; production source is unchanged from + the measured commit above. The seven-scenario result remains tied to that + measured binary, rather than claiming a new combined-package validation. + The host cancellation notification companion is + [OpenDesign #7959](https://github.com/nexu-io/open-design/pull/7959), commit + `65fa5dced1`; it has not been included in this PR's combined-package acceptance. +- [Vela PR #1952](https://github.com/powerformer/vela/pull/1952), commit + `1acdf79feaf88837763eb3afeb291421603ab76f`: locally built `0.0.1-test`, SHA-256 + `487a9cb1882aa0cc9badb170a51ecf7a86f14c40e0de19694ac57e665cf95cc1`, passed + all eight continuation-profile scenarios. Related production changes are + [OpenDesign #7958](https://github.com/nexu-io/open-design/pull/7958) and + [OpenCode #16](https://github.com/powerformer/opencode/pull/16). This is still + an unpublished, separate candidate; a combined version needs verification. + +## Remaining release acceptance + +The foundation does not finish OPEND-2886. Record each remaining item separately: + +1. Review and integrate the final OPEND-2884/2885 PRs, publish through the normal + authorized release process, and pin the actual published Vela/OpenCode versions. + Run both target flags against the same candidate and update the daemon consumer + tests for the negotiated protocol. Do not invent an unreleased npm version. +2. Build/install the real prerelease app and repeat identity verification. Run + actual compaction 100 consecutive times at a fixed, recorded provider/context + configuration. Retain machine, app/Vela/OpenCode versions, run IDs, final model + steps, durable-session identity, actual tool side-effect counts, cancellation, + unknown-state rejection and continuation attempt-limit evidence. +3. Separately test Write completion, stall, missing completion, cancellation and + daemon reconnect. Establish process quiescence and absence of duplicate writes; + bridge error return alone is insufficient. +4. Reconcile the real run cohort against existing PostHog project 420348 + `run_created`, `run_finished`, `run_retry_finished` and recovery fields. Filter + prerelease by app version/channel: `env=production` does not mean stable. + Keep the run denominator and mature unfinished-run policy unchanged. Synthetic + contract invocations are not product runs and have no PostHog run IDs. +5. Inspect current prerelease alert coverage and sample real runs after release. + Neither alert inspection nor post-release sampling has been performed by these + local contracts. The four original incident run IDs remain evidence, not new + acceptance runs: `6aa26b33-968a-4d5e-846a-2dbad009a3dc`, + `9ff005b6-f1d1-489f-b102-ac8af75fc078`, + `a5a5afc5-c37d-400d-a402-a68026a8718b`, and + `500540ac-6b4d-4c5f-83b8-23e41e27a17b`. diff --git a/e2e/lib/amr/vela-contract.ts b/e2e/lib/amr/vela-contract.ts new file mode 100644 index 0000000000..a2de5ab4f6 --- /dev/null +++ b/e2e/lib/amr/vela-contract.ts @@ -0,0 +1,119 @@ +import { spawn } from 'node:child_process'; +import { once } from 'node:events'; +import { createServer } from 'node:http'; +import { chmod, mkdir, readFile, writeFile } from 'node:fs/promises'; +import { join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import type { RuntimeScenario } from '../../resources/amr-continuation-frames.ts'; + +export interface RpcFrame { + id?: number; + method?: string; + params?: Record; + result?: Record; + error?: { code: number; message: string; data?: Record }; +} +const quote = (value: string) => `'${value.replaceAll("'", "'\\''")}'`; + +/** Real Vela, synthetic OpenCode SSE. No provider credentials or model calls. */ +export async function runVelaContract(options: { binary: string; directory: string; scenario: RuntimeScenario; cancel?: boolean; loadSessionId?: string; continuation?: Record }) { + if (process.platform === 'win32') throw new Error('the synthetic OpenCode launcher currently requires POSIX'); + const directory = resolve(options.directory); + await mkdir(directory, { recursive: true }); + const amrHome = join(directory, 'amr'); + await mkdir(amrHome, { recursive: true }); + const apiRequests: string[] = []; + const api = createServer((req, res) => { + apiRequests.push(`${req.method} ${req.url}`); + if (req.url?.startsWith('/v1/models')) { + res.setHeader('Content-Type', 'application/json'); + res.end(JSON.stringify({ data: [{ id: 'contract-model', context_budget: 128_000 }, { id: 'opencode-auxiliary', context_budget: 128_000 }] })); + } else { res.writeHead(404).end('unsupported synthetic API request'); } + }); + api.listen(0, '127.0.0.1'); + await once(api, 'listening'); + try { + const address = api.address(); + if (!address || typeof address === 'string') throw new Error('missing mock API address'); + const origin = `http://127.0.0.1:${address.port}`; + await writeFile(join(amrHome, 'config.json'), JSON.stringify({ profiles: { local: { runtimeKey: 'synthetic-runtime-key', apiUrl: origin, linkUrl: origin, user: { id: 'contract-user', email: 'contract@example.invalid' } } } })); + await chmod(join(amrHome, 'config.json'), 0o600); + const wrapper = join(directory, 'opencode-fixture'); + const fixture = fileURLToPath(new URL('../../resources/amr-opencode-server.ts', import.meta.url)); + await writeFile(wrapper, `#!/bin/sh\nexec ${quote(process.execPath)} --experimental-strip-types ${quote(fixture)} "$@"\n`, { mode: 0o755 }); + const ledgerPath = join(directory, 'requests.jsonl'); + await writeFile(ledgerPath, ''); + const received: RpcFrame[] = []; + const sent: RpcFrame[] = []; + const receivedAtNs = new Map(); + let stderr = ''; + let buffer = ''; + const child = spawn(resolve(options.binary), ['agent', 'run'], { + cwd: directory, + env: { PATH: process.env.PATH, HOME: process.env.HOME, TMPDIR: process.env.TMPDIR, AMR_HOME: amrHome, VELA_PROFILE: 'local', VELA_API_URL: origin, VELA_LINK_URL: origin, + VELA_OPENCODE_BIN: wrapper, OD_CONTRACT_SCENARIO: options.scenario, OD_CONTRACT_LEDGER: ledgerPath, + OTEL_SDK_DISABLED: 'true' }, + stdio: ['pipe', 'pipe', 'pipe'], + }); + const closed = new Promise((done) => child.once('close', () => done())); + let processError: Error | undefined; + child.on('error', error => { processError = error; }); + child.stderr.on('data', chunk => { stderr += chunk.toString(); }); + child.stdout.on('data', chunk => { + buffer += chunk.toString(); + let newline; + while ((newline = buffer.indexOf('\n')) >= 0) { + const line = buffer.slice(0, newline); buffer = buffer.slice(newline + 1); + if (line.trim()) { + try { + const frame = JSON.parse(line) as RpcFrame; + received.push(frame); receivedAtNs.set(frame, process.hrtime.bigint().toString()); + } + catch { processError = new Error('Vela emitted a non-JSON protocol frame'); } + } + } + }); + const send = (frame: RpcFrame) => { sent.push(frame); child.stdin.write(`${JSON.stringify({ jsonrpc: '2.0', ...frame })}\n`); }; + async function waitFor(predicate: (frame: RpcFrame) => boolean): Promise { + const deadline = Date.now() + 15_000; + while (Date.now() < deadline) { + const match = received.find(predicate); + if (match) return match; + if (processError || child.exitCode !== null || child.signalCode !== null) throw new Error(`Vela exited before contract response: ${processError?.message ?? stderr}`); + await new Promise(done => setTimeout(done, 10)); + } + throw new Error(`Vela contract response timed out: ${JSON.stringify(received)}\n${stderr}`); + } + try { + send({ id: 1, method: 'initialize', params: { protocolVersion: 1, clientCapabilities: {}, clientInfo: { name: 'open-design-contract', version: '1' } } }); + const initialize = await waitFor(frame => frame.id === 1); + if (initialize.error) throw new Error(initialize.error.message); + send({ id: 2, method: options.loadSessionId ? 'session/load' : 'session/new', params: { cwd: directory, mcpServers: [], ...(options.loadSessionId ? { sessionId: options.loadSessionId } : {}) } }); + const session = await waitFor(frame => frame.id === 2); + if (session.error) throw new Error(`session/new: ${session.error.message}\n${stderr}`); + const sessionId = session.result?.sessionId; + send({ id: 3, method: 'session/set_model', params: { sessionId, modelId: 'contract-model' } }); + const model = await waitFor(frame => frame.id === 3); + if (model.error) throw new Error(model.error.message); + send({ id: 4, method: options.continuation ? '_session/continue' : 'session/prompt', params: { sessionId, ...(options.continuation ? { continuation: options.continuation } : { prompt: [{ type: 'text', text: 'Run synthetic compaction contract.' }] }) } }); + if (options.cancel) { + await waitFor(frame => frame.method === 'session/update' && JSON.stringify(frame).includes('write-once')); + send({ method: 'session/cancel', params: { sessionId } }); + } + const terminal = await waitFor(frame => frame.id === 4); + const openCodeEvents = (await readFile(ledgerPath, 'utf8')).trim().split('\n').map(line => JSON.parse(line) as { atNs: string; phase: string; method: string; path: string; body: unknown }); + return { terminalReceivedAtNs: receivedAtNs.get(terminal)!, openCodeEvents, scope: 'real-vela-synthetic-opencode', initialize, session, terminal, sent, received, apiRequests, + openCodeRequests: openCodeEvents.filter(event => event.phase === 'request'), stderr }; + } finally { + child.stdin.end(); + const timer = setTimeout(() => child.kill('SIGTERM'), 3_000); + const force = setTimeout(() => child.kill('SIGKILL'), 6_000); + try { await closed; } finally { + clearTimeout(timer); clearTimeout(force); + } + } + } finally { + api.closeAllConnections(); + await new Promise(done => api.close(() => done())); + } +} diff --git a/e2e/resources/amr-continuation-frames.ts b/e2e/resources/amr-continuation-frames.ts new file mode 100644 index 0000000000..3efd965b59 --- /dev/null +++ b/e2e/resources/amr-continuation-frames.ts @@ -0,0 +1,36 @@ +/** Synthetic wire fixtures based on Vela 0.0.35's compaction integrity regression. + * These exercise the real bridge, not OpenCode's actual model/compaction loop. + */ +export type RuntimeScenario = 'complete' | 'incomplete' | 'incomplete-outstanding' | 'write-stall' | 'write-disconnect' | 'write-missing-terminal' | 'unknown-error'; +export function continuationFrames(userMessageId: string, scenario: RuntimeScenario) { + const sessionID = 'oc-contract-session'; + const assistantId = 'assistant-continuation'; + const message = (info: Record) => ({ type: 'message.updated', properties: { info: { sessionID, ...info } } }); + const part = (value: Record) => ({ type: 'message.part.updated', properties: { sessionID, part: { sessionID, messageID: assistantId, ...value } } }); + if (scenario === 'unknown-error') return [{ type: 'session.error', properties: { sessionID, error: { name: 'UnknownContractError', data: { message: 'unknown runtime failure' } } } }]; + if (scenario === 'write-stall' || scenario === 'write-disconnect' || scenario === 'write-missing-terminal') return [ + message({ id: userMessageId, role: 'user' }), + message({ id: assistantId, role: 'assistant', parentID: userMessageId }), + part({ id: 'write-part', type: 'tool', callID: 'write-once', tool: 'write', state: { status: scenario === 'write-missing-terminal' ? 'completed' : 'running', input: { filePath: 'result.txt', content: 'fixture' }, output: 'written', time: { start: 1, end: 2 } } }), + ]; + const frames = [ + message({ id: userMessageId, role: 'user' }), + message({ id: 'assistant-initial', role: 'assistant', parentID: userMessageId }), + part({ id: 'compact', messageID: userMessageId, type: 'compaction', auto: true }), + message({ id: 'assistant-summary', role: 'assistant', parentID: userMessageId, summary: true, finish: 'stop' }), + { type: 'session.compacted', properties: { sessionID } }, + message({ id: 'user-continuation', role: 'user' }), + message({ id: assistantId, role: 'assistant', parentID: 'user-continuation' }), + part({ id: 'tool-step', type: 'step-finish', reason: 'tool-calls' }), + part({ id: 'write-part', type: 'tool', callID: 'write-once', tool: 'write', state: { status: 'completed', input: { filePath: 'result.txt', content: 'fixture' }, output: 'written', time: { start: 1, end: 2 } } }), + message({ id: assistantId, role: 'assistant', finish: 'tool-calls', time: { completed: 2 } }), + ]; + if (scenario === 'incomplete-outstanding') frames.push(part({ id: 'unknown-write', type: 'tool', callID: 'write-unknown', tool: 'write', state: { status: 'running', input: { filePath: 'unknown.txt' }, time: { start: 3 } } })); + if (scenario === 'complete') frames.push( + part({ id: 'final-start', type: 'step-start' }), + part({ id: 'final-text', type: 'text', text: 'Contract fixture completed.' }), + part({ id: 'final-stop', type: 'step-finish', reason: 'stop' }), + ); + frames.push({ type: 'session.idle', properties: { sessionID } }); + return frames; +} diff --git a/e2e/resources/amr-opencode-server.ts b/e2e/resources/amr-opencode-server.ts new file mode 100644 index 0000000000..627615cf13 --- /dev/null +++ b/e2e/resources/amr-opencode-server.ts @@ -0,0 +1,66 @@ +import { appendFileSync } from 'node:fs'; +import { createServer, type ServerResponse } from 'node:http'; +import { continuationFrames, type RuntimeScenario } from './amr-continuation-frames.ts'; + +const port = Number(process.argv[process.argv.indexOf('--port') + 1]); +const scenario = process.env.OD_CONTRACT_SCENARIO as RuntimeScenario; +const ledger = process.env.OD_CONTRACT_LEDGER!; +const streams = new Set(); +const record = (entry: Record) => appendFileSync(ledger, `${JSON.stringify({ atNs: process.hrtime.bigint().toString(), ...entry })}\n`); +const server = createServer(async (req, res) => { + const url = new URL(req.url!, 'http://127.0.0.1'); + const chunks = []; + for await (const chunk of req) chunks.push(chunk); + const body = chunks.length ? JSON.parse(Buffer.concat(chunks).toString()) : null; + record({ phase: 'request', method: req.method, path: url.pathname, body }); + res.on('finish', () => record({ phase: 'response-complete', method: req.method, path: url.pathname })); + res.setHeader('Content-Type', 'application/json'); + if (url.pathname === '/global/health') return void res.end(JSON.stringify({ healthy: true, version: 'synthetic-contract-fixture' })); + if (url.pathname === '/event') { + res.setHeader('Content-Type', 'text/event-stream'); + res.flushHeaders(); + res.write('data: {"type":"server.connected","properties":{}}\n\n'); + streams.add(res); + res.on('close', () => streams.delete(res)); + return; + } + if (url.pathname === '/session' || url.pathname === '/session/oc-contract-session') return void res.end(JSON.stringify({ id: 'oc-contract-session' })); + if (url.pathname === '/session/oc-contract-session/prompt_async') { + res.writeHead(204).end(); + const frames = continuationFrames(body.messageID ?? 'user-original', scenario); + for (const frame of frames) { + for (const stream of streams) stream.write(`data: ${JSON.stringify(frame)}\n\n`); + } + if (scenario !== 'write-stall') for (const stream of streams) stream.end(); + return; + } + if (url.pathname === '/session/oc-contract-session/abort') { + for (const stream of streams) stream.end(); + return void res.end('true'); + } + if (url.pathname === '/session/oc-contract-session/message') return void res.end(JSON.stringify([ + { info: { id: 'user-original', role: 'user' } }, { info: { id: 'user-continuation', role: 'user' } }, + ])); + if (url.pathname === '/session/oc-contract-session/continue') { + if (body.userMessageID !== 'user-continuation' || body.assistantMessageID !== 'assistant-continuation' || Object.keys(body).length !== 2) { + res.writeHead(409).end('{"error":"cursor mismatch"}'); return; + } + res.end('{}'); + const sessionID = 'oc-contract-session'; + const info = { id: 'assistant-final', parentID: 'user-continuation', sessionID, role: 'assistant' }; + const frames = [ + { type: 'message.updated', properties: { info } }, + { type: 'message.part.updated', properties: { part: { id: 'final-text', sessionID, messageID: 'assistant-final', type: 'text', text: 'Continued without another Write.' } } }, + { type: 'message.part.updated', properties: { part: { id: 'final-step', sessionID, messageID: 'assistant-final', type: 'step-finish', reason: 'stop' } } }, + { type: 'session.idle', properties: { sessionID } }, + ]; + for (const frame of frames) for (const stream of streams) stream.write(`data: ${JSON.stringify(frame)}\n\n`); + for (const stream of streams) stream.end(); + return; + } + res.writeHead(404).end(JSON.stringify({ error: 'unexpected fixture request' })); +}); +server.listen(port, '127.0.0.1', () => process.stdout.write(`opencode server listening on http://127.0.0.1:${port}\n`)); +function stop() { for (const stream of streams) stream.end(); server.close(() => process.exit(0)); server.closeAllConnections(); } +process.on('SIGINT', stop); +process.on('SIGTERM', stop); diff --git a/e2e/scripts/vela-contract.ts b/e2e/scripts/vela-contract.ts new file mode 100644 index 0000000000..8710fb9792 --- /dev/null +++ b/e2e/scripts/vela-contract.ts @@ -0,0 +1,88 @@ +import { execFile } from 'node:child_process'; +import { createHash } from 'node:crypto'; +import { createReadStream } from 'node:fs'; +import { mkdir, writeFile } from 'node:fs/promises'; +import { hostname, platform, arch, release } from 'node:os'; +import { join, resolve } from 'node:path'; +import { parseArgs, promisify } from 'node:util'; +import { runVelaContract } from '../lib/amr/vela-contract.ts'; + +const { values } = parseArgs({ options: { + binary: { type: 'string' }, out: { type: 'string' }, 'expected-version': { type: 'string' }, + 'require-write-abort': { type: 'boolean', default: false }, 'require-continuation': { type: 'boolean', default: false }, help: { type: 'boolean' }, +} }); +if (values.help) { + console.log('tsx scripts/vela-contract.ts --binary --expected-version --out [--require-write-abort] [--require-continuation]'); + process.exit(0); +} +if (!values.binary || !values.out || !values['expected-version']) throw new Error('--binary, --expected-version and --out are required'); +const binary = resolve(values.binary); +const out = resolve(values.out); +await mkdir(out, { recursive: true }); +async function binaryHash() { + const hash = createHash('sha256'); + for await (const chunk of createReadStream(binary)) hash.update(chunk); + return hash.digest('hex'); +} +const sha256 = await binaryHash(); +const version = (await promisify(execFile)(binary, ['--version'], { encoding: 'utf8', timeout: 10_000 })).stdout.trim(); +if (version !== values['expected-version']) throw new Error(`Vela version ${version} does not match expected ${values['expected-version']}`); +const results: Array<{ scenario: string; passed: boolean; checks: Record; evidence: string; error?: string }> = []; +let incomplete: Awaited> | undefined; +for (const scenario of ['complete', 'incomplete', 'incomplete-outstanding', 'write-stall', 'write-disconnect', 'write-missing-terminal', 'unknown-error'] as const) { + try { + const run = await runVelaContract({ binary, directory: join(out, scenario), scenario, cancel: scenario === 'write-stall' }); + if (scenario === 'incomplete') incomplete = run; + const evidence = join(out, `${scenario}.json`); + await writeFile(evidence, `${JSON.stringify(run, null, 2)}\n`); + const promptCount = run.openCodeRequests.filter(request => request.path.endsWith('/prompt_async')).length; + const abortCount = run.openCodeRequests.filter(request => request.path.endsWith('/abort')).length; + const checks: Record = { promptSubmittedOnce: promptCount === 1, acpPromptSentOnce: run.sent.filter(frame => frame.method === 'session/prompt').length === 1 }; + if (scenario === 'complete') checks.terminalModelStepAccepted = run.terminal.result?.stopReason === 'end_turn'; + else if (scenario === 'write-stall') { checks.cancelled = run.terminal.result?.stopReason === 'cancelled'; checks.cancelAbortedOnce = abortCount === 1; } + else checks.noFalseSuccess = run.terminal.error !== undefined && run.terminal.result === undefined; + if (scenario === 'incomplete') checks.incompleteNamed = run.terminal.error?.message.includes('compaction continuation ended before prompt completion') === true || run.terminal.error?.data?.code === 'OPENCODE_COMPACTION_CONTINUATION_INCOMPLETE'; + if (values['require-continuation'] && scenario === 'incomplete-outstanding') { + checks.unknownToolNotCommitted = run.terminal.error?.data?.phase === 'tool_outstanding' && (run.terminal.error.data.continuation as Record | undefined)?.toolResultsCommitted === false; + } + if (values['require-write-abort'] && ['write-disconnect', 'write-missing-terminal'].includes(scenario)) checks.abortedBeforeTerminal = abortCount === 1 && run.openCodeEvents.some(event => event.path.endsWith('/abort') && event.phase === 'response-complete' && BigInt(event.atNs) <= BigInt(run.terminalReceivedAtNs)); + results.push({ scenario, passed: Object.values(checks).every(Boolean), checks, evidence }); + } catch (error) { + results.push({ scenario, passed: false, checks: {}, evidence: join(out, scenario), error: String(error) }); + } +} +if (values['require-continuation']) { + const evidence = join(out, 'native-continuation.json'); + try { + const data = incomplete?.terminal.error?.data; + const cursor = data?.continuation as Record | undefined; + const capabilities = incomplete?.initialize.result?.agentCapabilities as Record | undefined; + const extensions = capabilities?._meta as Record | undefined; + if (capabilities?.loadSession !== true || extensions?.['com.open-design.nativeSessionContinue']?.version !== 1 || + data?.kind !== 'opencode_continuation_incomplete' || data.code !== 'OPENCODE_COMPACTION_CONTINUATION_INCOMPLETE' || + data.runtime !== 'opencode' || data.phase !== 'post_tool_resume' || data.retryable !== false || + cursor?.version !== 1 || cursor.toolResultsCommitted !== true || typeof data.openCodeSessionId !== 'string') { + throw new Error('target binary lacks the explicit continuation capability and committed-tool error contract'); + } + const resumed = await runVelaContract({ binary, directory: join(out, 'native-continuation'), scenario: 'complete', loadSessionId: data.openCodeSessionId, continuation: cursor }); + await writeFile(evidence, `${JSON.stringify(resumed, null, 2)}\n`); + const checks = { + sameDurableSession: resumed.session.result?.openCodeSessionId === data.openCodeSessionId, + noNewSession: !resumed.sent.some(frame => frame.method === 'session/new') && !resumed.openCodeRequests.some(request => request.method === 'POST' && request.path === '/session'), + noPromptReplay: !resumed.sent.some(frame => frame.method === 'session/prompt') && !resumed.openCodeRequests.some(request => request.path.endsWith('/prompt_async')), + continueOnce: resumed.openCodeRequests.filter(request => request.path.endsWith('/continue')).length === 1, + finalModelStep: resumed.terminal.result?.stopReason === 'end_turn', + terminalOnce: resumed.received.filter(frame => frame.id === 4).length === 1, + }; + results.push({ scenario: 'native-continuation', passed: Object.values(checks).every(Boolean), checks, evidence }); + } catch (error) { results.push({ scenario: 'native-continuation', passed: false, checks: {}, evidence, error: String(error) }); } +} +const binaryUnchanged = sha256 === await binaryHash(); +const report = { schemaVersion: 1, scope: 'real-vela-synthetic-opencode', capturedAt: new Date().toISOString(), + machine: { hostname: hostname(), platform: platform(), arch: arch(), osRelease: release() }, + binary: { path: binary, version, sha256, unchangedDuringSuite: binaryUnchanged }, requireWriteAbort: values['require-write-abort'], requireContinuation: values['require-continuation'], + passed: binaryUnchanged && results.every(result => result.passed), results, + realCompactionRuns: 0, packageAcceptance: 'not-performed', postReleaseSampling: 'not-performed' }; +await writeFile(join(out, 'report.json'), `${JSON.stringify(report, null, 2)}\n`); +console.log(JSON.stringify(report, null, 2)); +if (!report.passed) process.exitCode = 1; diff --git a/e2e/tests/amr/vela-contract.test.ts b/e2e/tests/amr/vela-contract.test.ts new file mode 100644 index 0000000000..50391f5a8b --- /dev/null +++ b/e2e/tests/amr/vela-contract.test.ts @@ -0,0 +1,107 @@ +import { execFileSync } from 'node:child_process'; +import { EventEmitter } from 'node:events'; +import { mkdtemp, rm } from 'node:fs/promises'; +import { createRequire } from 'node:module'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { PassThrough } from 'node:stream'; +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; + +import { T } from '@/timeouts'; +import { runVelaContract } from '@/amr/vela-contract'; +import { attachAcpSession } from '../../../apps/daemon/src/agent-protocol/acp/session.ts'; +import { summarizeRunToolProgress } from '../../../apps/daemon/src/run-diagnostics.ts'; +import { decideSafeRunRetry } from '../../../apps/daemon/src/run-retry-policy.ts'; + +// This resolves the exact dependency used by packaging; no developer PATH fallback. +const requirePack = createRequire(new URL('../../../tools/pack/package.json', import.meta.url)); +const resolveBinary = () => (requirePack('@powerformer/vela-cli') as { resolveVelaCliBin: (input: { strict: boolean }) => string }).resolveVelaCliBin({ strict: true }); + +class ReplayChild extends EventEmitter { + stdin = new PassThrough(); + stdout = new PassThrough(); + stderr = new PassThrough(); + kill() { return true; } +} + +// Replay the *actual* binary transcript through the current daemon consumer. +function consume(result: Awaited>) { + const child = new ReplayChild(); + const events: Array<{ event: string; data: unknown }> = []; + const session = attachAcpSession({ child: child as never, prompt: 'contract', model: 'contract-model', + send: (event, data) => events.push({ event, data }) }); + for (const frame of result.received) { + // Host cancellation precedes the agent acknowledgement in the real flow. + if (frame.result?.stopReason === 'cancelled') session.abort(); + child.stdout.write(`${JSON.stringify(frame)}\n`); + } + const completed = session.completedSuccessfully(); + const fatal = session.hasFatalError(); + session.abort(); + return { events, completed, fatal, progress: summarizeRunToolProgress(events) }; +} + +describe.skipIf(process.platform === 'win32')('Vela package dependency / daemon wire contract', () => { + let root: string; + let binary: string; + let failed = false; + afterEach(({ task }) => { failed ||= task.result?.state === 'fail'; }); + beforeAll(async () => { root = await mkdtemp(join(tmpdir(), 'od-vela-contract-')); binary = resolveBinary(); }); + afterAll(async () => { + if (failed) console.error(`Vela contract scratch retained: ${root}`); + else if (root) await rm(root, { recursive: true, force: true }); + }, T.long); + + it('[P1] runs the exact package pin rather than trusting initialize agentInfo', () => { + const pin = (requirePack('./package.json') as { optionalDependencies: Record }).optionalDependencies['@powerformer/vela-cli']; + expect(execFileSync(binary, ['--version'], { encoding: 'utf8', timeout: 10_000 }).trim()).toBe(pin); + }, T.long); + + it('[P1] accepts a compaction continuation only after its terminal model step', async () => { + const result = await runVelaContract({ binary, directory: join(root, 'complete'), scenario: 'complete' }); + expect(result.terminal.result?.stopReason).toBe('end_turn'); + const consumed = consume(result); + expect(consumed.completed).toBe(true); + expect(consumed.fatal).toBe(false); + expect(consumed.progress).toMatchObject({ toolCallSeen: true, toolResultSent: true, hasOutstandingTool: false }); + expect(result.openCodeRequests.filter(request => request.path.endsWith('/prompt_async'))).toHaveLength(1); + }, T.long); + + it('[P1] preserves 0.0.35 incomplete protection and never turns committed tools into an automatic full replay', async () => { + const result = await runVelaContract({ binary, directory: join(root, 'incomplete'), scenario: 'incomplete' }); + expect(result.terminal.result).toBeUndefined(); + expect(result.terminal.error).toMatchObject({ data: { kind: 'opencode_prompt_error', runtime: 'opencode', phase: 'event_stream', lastToolStatus: 'completed' } }); + expect(result.terminal.error?.message).toContain('compaction continuation ended before prompt completion'); + const consumed = consume(result); + expect(consumed.completed).toBe(false); + expect(consumed.fatal).toBe(true); + expect(consumed.progress.toolResultSent).toBe(true); + expect(decideSafeRunRetry({ result: 'failed', attemptCount: 0, failure: { failure_category: 'upstream_unavailable', failure_detail: 'stream_disconnected', retryable: true }, sideEffects: { toolCallSeen: true, artifactWriteSeen: true } }).shouldRetry).toBe(false); + expect(result.sent.filter(frame => frame.method === 'session/prompt')).toHaveLength(1); + expect(result.openCodeRequests.filter(request => request.path.endsWith('/prompt_async'))).toHaveLength(1); + }, T.long); + + it('[P1] cancels an outstanding Write without claiming success or resending it', async () => { + const result = await runVelaContract({ binary, directory: join(root, 'cancel'), scenario: 'write-stall', cancel: true }); + expect(result.terminal.result?.stopReason).toBe('cancelled'); + expect(result.openCodeRequests.filter(request => request.path.endsWith('/abort'))).toHaveLength(1); + expect(result.openCodeRequests.filter(request => request.path.endsWith('/prompt_async'))).toHaveLength(1); + const consumed = consume(result); + expect(consumed.completed).toBe(false); + expect(consumed.progress.hasOutstandingTool).toBe(true); + }, T.long); + + it('[P1] does not let a completed tool hide an outstanding parallel Write', async () => { + const result = await runVelaContract({ binary, directory: join(root, 'outstanding'), scenario: 'incomplete-outstanding' }); + expect(result.terminal.error).toBeDefined(); + expect(result.terminal.result).toBeUndefined(); + expect(consume(result).progress).toMatchObject({ toolCallSeen: true, toolResultSent: false, hasOutstandingTool: true }); + }, T.long); + + it('[P1] keeps unknown runtime errors unsuccessful', async () => { + const result = await runVelaContract({ binary, directory: join(root, 'unknown'), scenario: 'unknown-error' }); + expect(result.terminal.error?.data?.kind).toBe('opencode_prompt_error'); + expect(result.terminal.result).toBeUndefined(); + expect(consume(result)).toMatchObject({ completed: false, fatal: true }); + }, T.long); +}); diff --git a/scripts/guard.ts b/scripts/guard.ts index 5c58911250..0ab91d918d 100644 --- a/scripts/guard.ts +++ b/scripts/guard.ts @@ -35,6 +35,8 @@ const allowedE2eScripts = new Set([ // Explicit opt-in local daemon acceptance; not part of hermetic CI test discovery. "e2e/scripts/syntax-acceptance.ts", "e2e/scripts/visual-report.ts", + // Cross-version real Vela / synthetic OpenCode protocol acceptance. + "e2e/scripts/vela-contract.ts", ]); function toRepositoryPath(filePath: string): string { diff --git a/tools/pack/src/index.ts b/tools/pack/src/index.ts index 292983d01e..d79c7f75ae 100644 --- a/tools/pack/src/index.ts +++ b/tools/pack/src/index.ts @@ -114,6 +114,17 @@ function addWinLifecycleOptions(command: CacCommand) { const cli = cac("tools-pack"); +cli.command('verify-runtime', 'Verify installed prerelease Vela/OpenCode identity against a release manifest') + .option('--resources ', 'installed package Resources directory') + .option('--manifest ', 'release platform manifest JSON') + .option('--expected-vela ', 'exact Vela version from the reviewed dependency pin') + .option('--expected-opencode ', 'exact OpenCode version from the reviewed Vela release') + .option('--json', 'print JSON evidence (also the default)') + .action(async (options: { resources: string; manifest: string; expectedVela: string; expectedOpencode: string }) => { + const { verifyPackagedRuntime } = await import('./resources/runtime-verification.js'); + printJson(await verifyPackagedRuntime({ ...options, expectedOpenCode: options.expectedOpencode })); + }); + addMacBuildOptions(addSharedOptions(cli.command("mac ", "Mac packaging commands: build|install|start|stop|logs|uninstall|cleanup|inspect"))).action( async (action: string, options: CliOptions) => { const config = resolveToolPackConfig("mac", options); diff --git a/tools/pack/src/resources/runtime-verification.ts b/tools/pack/src/resources/runtime-verification.ts new file mode 100644 index 0000000000..766e404cf4 --- /dev/null +++ b/tools/pack/src/resources/runtime-verification.ts @@ -0,0 +1,93 @@ +import { execFile } from 'node:child_process'; +import { createHash } from 'node:crypto'; +import { createReadStream } from 'node:fs'; +import { readFile, realpath, stat } from 'node:fs/promises'; +import { arch, hostname, platform, release } from 'node:os'; +import { isAbsolute, join, relative, resolve } from 'node:path'; +import { promisify } from 'node:util'; + +const exec = promisify(execFile); +type Host = { platform: string; arch: string }; +type RuntimeVerificationInput = { + resources: string; + manifest: string; + expectedVela: string; + expectedOpenCode: string; + host?: Host; + runVersion?: (binary: string) => Promise; +}; + +function object(value: unknown, label: string): Record { + if (!value || typeof value !== 'object' || Array.isArray(value)) throw new Error(`invalid ${label}`); + return value as Record; +} +function text(value: unknown, label: string): string { + if (typeof value !== 'string' || !value.trim()) throw new Error(`missing ${label}`); + return value.trim(); +} +async function digest(path: string): Promise { + const hash = createHash('sha256'); + for await (const chunk of createReadStream(path)) hash.update(chunk); + return hash.digest('hex'); +} +async function packageFile(root: string, path: string): Promise { + const real = await realpath(path); + const rel = relative(root, real); + if (rel === '..' || rel.startsWith('../') || rel.startsWith('..\\') || isAbsolute(rel)) { + throw new Error(`package file resolves outside selected resources: ${path}`); + } + if (!(await stat(real)).isFile()) throw new Error(`package file is not a regular file: ${path}`); + return real; +} + +/** Inspect installed bytes. This is deliberately separate from runtime regression acceptance. */ +export async function verifyPackagedRuntime(input: RuntimeVerificationInput) { + const expectedVela = text(input.expectedVela, 'expected Vela version'); + const expectedOpenCode = text(input.expectedOpenCode, 'expected OpenCode version'); + const root = await realpath(resolve(text(input.resources, 'resources directory'))); + const manifestPath = await realpath(resolve(text(input.manifest, 'release manifest path'))); + const manifestBytes = await readFile(manifestPath); + const manifest = object(JSON.parse(manifestBytes.toString('utf8')), 'release manifest'); + const configPath = await packageFile(root, join(root, 'open-design-config.json')); + const configBytes = await readFile(configPath); + const config = object(JSON.parse(configBytes.toString('utf8')), 'packaged config'); + const version = text(manifest.releaseVersion, 'manifest releaseVersion'); + if (config.appVersion !== version) throw new Error(`packaged app version ${String(config.appVersion)} does not match manifest ${version}`); + const channel = text(manifest.channel, 'manifest channel'); + if (channel !== 'prerelease' || !/^\d+\.\d+\.\d+-prerelease\.\d+$/.test(version)) { + throw new Error('runtime regression verification requires a prerelease manifest and version'); + } + const host = input.host ?? { platform: platform(), arch: arch() }; + const platformKey = `${({ darwin: 'mac', win32: 'win', linux: 'linux' } as Record)[host.platform] ?? 'unsupported'}_${host.arch}`; + if (!['mac_arm64', 'mac_x64', 'win_x64', 'linux_x64'].includes(platformKey) || manifest.platformKey !== platformKey) { + throw new Error(`manifest platform ${String(manifest.platformKey)} does not match host ${platformKey}`); + } + const commit = text(object(manifest.github, 'manifest github').commit, 'release commit'); + if (!/^[a-f0-9]{40}$/.test(commit)) throw new Error('release commit must be an exact SHA'); + const suffix = host.platform === 'win32' ? '.exe' : ''; + // Resolve both before probing either: never fall back to a developer binary on PATH. + const vela = await packageFile(root, join(root, 'open-design/bin', `vela${suffix}`)); + const opencode = await packageFile(root, join(root, 'open-design/bin/libexec/opencode', `opencode${suffix}`)); + const runVersion = input.runVersion ?? (async (binary: string) => { + const result = await exec(binary, ['--version'], { timeout: 10_000, maxBuffer: 64 * 1024, windowsHide: true, encoding: 'utf8' }); + return result.stdout; + }); + async function inspect(binary: string, expected: string, label: string) { + const sha256 = await digest(binary); + const actual = (await runVersion(binary)).trim(); + if (sha256 !== await digest(binary)) throw new Error(`${label} binary changed during verification`); + if (actual !== expected) throw new Error(`${label} version ${JSON.stringify(actual)} does not match expected ${expected}`); + return { path: binary, version: actual, sha256, size: (await stat(binary)).size }; + } + const binaries = { vela: await inspect(vela, expectedVela, 'Vela'), opencode: await inspect(opencode, expectedOpenCode, 'OpenCode') }; + return { + schemaVersion: 1, + scope: 'binary-identity-only', + capturedAt: new Date().toISOString(), + machine: { hostname: hostname(), ...host, osRelease: release() }, + resources: root, + release: { version, channel, platformKey, commit, manifestPath, manifestSha256: createHash('sha256').update(manifestBytes).digest('hex') }, + configSha256: createHash('sha256').update(configBytes).digest('hex'), + binaries, + }; +} diff --git a/tools/pack/tests/resources/runtime-verification.test.ts b/tools/pack/tests/resources/runtime-verification.test.ts new file mode 100644 index 0000000000..29a0f75605 --- /dev/null +++ b/tools/pack/tests/resources/runtime-verification.test.ts @@ -0,0 +1,77 @@ +import { mkdtemp, mkdir, realpath, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, describe, expect, it, vi } from 'vitest'; + +import { verifyPackagedRuntime } from '@/resources/runtime-verification.js'; + +const roots: string[] = []; +afterEach(async () => { await Promise.all(roots.splice(0).map(root => rm(root, { recursive: true, force: true }))); }); + +async function fixture() { + const root = await mkdtemp(join(tmpdir(), 'od-runtime-verification-')); + roots.push(root); + const resources = join(await realpath(root), 'resources'); + await mkdir(join(resources, 'open-design/bin/libexec/opencode'), { recursive: true }); + await writeFile(join(resources, 'open-design-config.json'), JSON.stringify({ appVersion: '0.22.0-prerelease.19' })); + await writeFile(join(resources, 'open-design/bin/vela'), 'vela fixture'); + await writeFile(join(resources, 'open-design/bin/libexec/opencode/opencode'), 'opencode fixture'); + const manifest = join(root, 'mac_arm64.json'); + await writeFile(manifest, JSON.stringify({ channel: 'prerelease', releaseVersion: '0.22.0-prerelease.19', platformKey: 'mac_arm64', github: { commit: 'a'.repeat(40) } })); + const runVersion = vi.fn(async (binary: string): Promise => binary.endsWith('/vela') ? '0.0.35\n' : '0.0.0--202609020336\n'); + return { root, resources, manifest, runVersion, expectedVela: '0.0.35', expectedOpenCode: '0.0.0--202609020336', host: { platform: 'darwin', arch: 'arm64' } as const }; +} + +describe('packaged runtime identity', () => { + it('records exact installed bytes, versions, host and release provenance without claiming regression acceptance', async () => { + const input = await fixture(); + const result = await verifyPackagedRuntime(input); + expect(result.release).toMatchObject({ channel: 'prerelease', version: '0.22.0-prerelease.19', commit: 'a'.repeat(40) }); + expect(result.binaries.vela).toMatchObject({ version: '0.0.35', sha256: expect.stringMatching(/^[a-f0-9]{64}$/) }); + expect(result.binaries.opencode.version).toBe('0.0.0--202609020336'); + expect(input.runVersion.mock.calls.map(([binary]) => binary)).toEqual([ + join(input.resources, 'open-design/bin/vela'), join(input.resources, 'open-design/bin/libexec/opencode/opencode'), + ]); + expect(result.scope).toBe('binary-identity-only'); + }); + it('rejects an app/manifest mismatch before executing a binary', async () => { + const input = await fixture(); + await writeFile(join(input.resources, 'open-design-config.json'), JSON.stringify({ appVersion: '0.22.0-prerelease.18' })); + await expect(verifyPackagedRuntime(input)).rejects.toThrow('app version'); + expect(input.runVersion).not.toHaveBeenCalled(); + }); + it('rejects a manifest for another platform', async () => { + const input = await fixture(); + await expect(verifyPackagedRuntime({ ...input, host: { platform: 'linux', arch: 'x64' } })).rejects.toThrow('platform'); + expect(input.runVersion).not.toHaveBeenCalled(); + }); + it('rejects a wrong binary version, including a misleading substring', async () => { + const input = await fixture(); + input.runVersion.mockResolvedValue('0.0.350'); + await expect(verifyPackagedRuntime(input)).rejects.toThrow('Vela version'); + }); + it('rejects a missing companion instead of finding OpenCode on PATH', async () => { + const input = await fixture(); + await rm(join(input.resources, 'open-design/bin/libexec/opencode/opencode')); + await expect(verifyPackagedRuntime(input)).rejects.toThrow(); + }); + it('rejects binaries that escape the selected package through symlinks', async () => { + const input = await fixture(); + const binary = join(input.resources, 'open-design/bin/vela'); + await rm(binary); + await writeFile(join(input.root, 'outside-vela'), 'foreign'); + await symlink(join(input.root, 'outside-vela'), binary); + await expect(verifyPackagedRuntime(input)).rejects.toThrow('outside'); + expect(input.runVersion).not.toHaveBeenCalled(); + }); + it('rejects a binary that changes while its version is probed', async () => { + const input = await fixture(); + input.runVersion.mockImplementation(async binary => { await writeFile(binary, 'changed'); return '0.0.35'; }); + await expect(verifyPackagedRuntime(input)).rejects.toThrow('changed'); + }); + it('rejects missing explicit target versions', async () => { + const input = await fixture(); + await expect(verifyPackagedRuntime({ ...input, expectedOpenCode: '' })).rejects.toThrow('expected OpenCode'); + expect(input.runVersion).not.toHaveBeenCalled(); + }); +});