Files
open-design/e2e/tests/scripts/check-cross-app-imports.test.ts
14fcc411cd feat(ci): confidence-tiered scope rule table with merge-queue shadow evaluation (#6029)
* chore(scripts): make root scripts/ test-free and purify the guard entrypoint

Root scripts/ is now source-only, enforced by a new "scripts test-free"
guard check (any *.test.{ts,tsx,js,jsx,mjs,cjs} under scripts/ fails
pnpm guard). The root guard script becomes a pure tsx ./scripts/guard.ts
invocation instead of chaining a hand-maintained node --test file list.

Script behavior-contract coverage follows the boundary e2e/AGENTS.md
already declares for e2e/tests/scripts/: the fork PR approval security
tests migrate there (converted to Vitest, 24 tests passing), and the
remaining script unit tests are deleted — five of them (batch-design-
system-test, check-design-system-package-quality, plugin-previews-diff,
plugin-previews-gc, migrate-to-plugins/example) were orphans no runner
ever executed; the guard-chained checker tests covered logic whose
repo-wide enforcement remains in guard.ts checks. Coverage with a real
test claim gets rebuilt in e2e/tests/scripts/ on demand.

* test(ci): pin characterization goldens for scopes.ts scope detection

Golden tests run scripts/scopes.ts as a subprocess exactly the way ci.yml
does, with gh stubbed through the existing OPEN_DESIGN_GH_NODE_SCRIPT seam
(hermetic stub materialized per run), and pin the complete 20-key output
plan for a matrix of changed-file sets across pull_request,
workflow_dispatch (hot/full), and merge_group events.

They deliberately pin current quirks (skills/*.md arming daemon+web without
the ui-critical fallback; e2e/ui/*.md arming ui_p0 with workspace validation
off; dispatch-hot skipping the workspace_validation re-derivation) so the
upcoming rule-table restructuring can prove behavior preservation
byte-for-byte.

* refactor(ci): restructure scopes.ts into a confidence-tiered rule table

Scope detection becomes data plus a pure evaluator: every changed file is
classified by an additive rule table (union of matched effects, minimum
confidence across matched rules), and an evaluation context brings a trust
threshold — files below it, or matching no rule, escalate fail-closed to
the full radius. Today's behavior is exactly the degenerate case of this
structure (all rules medium, certain set empty): pull_request/hot believes
everything, the merge queue believes nothing and stays full. The
characterization goldens pass unchanged, byte-for-byte.

The exempt surface becomes an explicit empty-radius rule instead of a
special case, so exemptions can be split, rated, and promoted per rule.
Promotion to "certain" requires a guard field naming the enforcing check
(enforced by a rule-table invariant test).

New offline entry points for rule iteration: `plan --context
pr|merge-queue|full --files ...` evaluates without any GitHub environment
and prints { plan, trace }; `rules` lists the table. Every evaluation now
returns a decision trace (per-rule hit counts, per-file escalations, and
an ifTrustAll shadow plan that shows what medium-confidence rules would
have allowed — the evidence stream for future certain-tier promotions);
github-output mode logs it and writes a step-summary table without
touching the frozen 20-key output contract.

The legacy dispatch-hot asymmetry (no workspace_validation re-derivation)
is preserved behind an explicit option and pinned by the goldens.

* feat(ci): evaluate merge-group union diffs at the certain threshold with shadow trace

The merge queue now resolves the queued group's union diff
(compare base_sha...head_sha from the merge_group event payload) and
evaluates it through the rule table at the "certain" trust threshold.
While the certain rule set is empty every file escalates and the queue
keeps running everything — the goldens pin this byte-for-byte — but the
evaluation's ifTrustAll shadow column now records, for every queued
group, what medium-confidence rules would have allowed: the evidence
stream that future certain-tier promotions will cite.

Queue lanes derive from the evaluated outputs (ci_mode stays reported as
"full", but lanes are no longer unconditionally forced), so promoting a
rule to "certain" later is a pure rule-table change with no further
structural work.

Fail-safe behavior is explicit and tested: resolution errors and empty
resolutions fall open to the full plan without failing the scopes job
(queue throughput must never depend on this evaluation), while
pull_request resolution failures keep failing the run as before.

Changed-file resolution no longer filters out removed files (manual hot
runs previously did): deleting a runtime source file must trigger the
same validation scopes as editing it. This is the only behavior change,
and it is strictly fail-closed.

* fix(ci): preserve guard contracts and fail closed on compare limits

Restore the eight guard behavior suites in the e2e script lane and make merge-group compare resolution treat GitHub's 300-file ceiling as incomplete.

Generated-By: looper 0.11.0 (runner=fixer, agent=codex)

* fix(ci): enforce extension-agnostic script test boundary

Match every suffix after .test. and cover TypeScript, MTS, and non-test filenames in the e2e script suite.

Generated-By: looper 0.11.0 (runner=fixer, agent=codex)

* fix(ci): route script contract tests to e2e

Ensure changes under e2e/tests/scripts arm E2E Vitest with the same daemon and web scope effects as their root script counterparts, and pin the behavior with a PR golden.\n\nGenerated-By: looper 0.11.0 (runner=fixer, agent=codex)

---------

Co-authored-by: Looper <looper@noreply.github.com>
2026-07-24 05:18:39 +00:00

243 lines
8.6 KiB
TypeScript

import assert from "node:assert/strict";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { test } from "vitest";
import {
type AppDirectoryRegistry,
collectCrossAppImportViolationsFromSource,
isCrossAppImportSourceFile,
loadAppDirectoryRegistry,
} from "../../../scripts/check-cross-app-imports.ts";
const registry: AppDirectoryRegistry = {
packageNameByDirectory: new Map([
["daemon", "@open-design/daemon"],
["web", "@open-design/web"],
]),
};
test("cross-app import check rejects web importing daemon src via relative path", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/web/src/providers/daemon.ts",
"import { spawnAgent } from '../../../daemon/src/server.ts';",
registry,
);
assert.equal(violations.length, 1);
assert.equal(violations[0]?.targetApp, "daemon");
assert.equal(violations[0]?.lineNumber, 1);
});
test("cross-app import check rejects another app's package name", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/web/src/runtime/todos.ts",
[
"import type { RunRecord } from '@open-design/daemon/src/server.ts';",
"const mod = await import('@open-design/daemon');",
"const legacy = require('@open-design/daemon/dist/cli.js');",
].join("\n"),
registry,
);
assert.deepEqual(
violations.map((violation) => violation.lineNumber),
[1, 2, 3],
);
assert.ok(violations.every((violation) => violation.targetApp === "daemon"));
});
test("cross-app import check rejects another app's package name in require.resolve", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/web/src/setup-runtime.ts",
"const daemonManifest = require.resolve('@open-design/daemon/package.json');",
registry,
);
assert.equal(violations.length, 1);
assert.equal(violations[0]?.specifier, "@open-design/daemon/package.json");
assert.equal(violations[0]?.targetApp, "daemon");
assert.equal(violations[0]?.lineNumber, 1);
});
test("cross-app import check rejects createRequire-based cross-app resolution", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/web/src/setup-runtime.ts",
[
"import { createRequire, createRequire as makeRequire } from 'node:module';",
"import * as nodeModule from 'node:module';",
"const directDaemonManifest = createRequire(import.meta.url).resolve('@open-design/daemon/package.json');",
"const aliasDaemonManifest = makeRequire(import.meta.url).resolve('@open-design/daemon/package.json');",
"const nodeRequire = nodeModule.createRequire(import.meta.url);",
"const daemonCli = nodeRequire('@open-design/daemon/dist/cli.js');",
"const daemonPackageJson = nodeRequire.resolve('@open-design/daemon/package.json');",
].join("\n"),
registry,
);
assert.deepEqual(
violations.map((violation) => violation.specifier),
[
"@open-design/daemon/package.json",
"@open-design/daemon/package.json",
"@open-design/daemon/dist/cli.js",
"@open-design/daemon/package.json",
],
);
assert.deepEqual(
violations.map((violation) => violation.lineNumber),
[3, 4, 6, 7],
);
assert.ok(violations.every((violation) => violation.targetApp === "daemon"));
});
test("cross-app import check rejects CommonJS node:module namespace createRequire resolution", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/web/src/setup-runtime.cjs",
[
'const moduleApi = require("node:module");',
"const nodeRequire = moduleApi.createRequire(__filename);",
'nodeRequire.resolve("@open-design/daemon/package.json");',
].join("\n"),
registry,
);
assert.equal(violations.length, 1);
assert.equal(violations[0]?.specifier, "@open-design/daemon/package.json");
assert.equal(violations[0]?.targetApp, "daemon");
assert.equal(violations[0]?.lineNumber, 3);
});
test("cross-app import check rejects cross-app imports from app-owned mjs entrypoints", () => {
assert.equal(isCrossAppImportSourceFile("entry.js"), true);
assert.equal(isCrossAppImportSourceFile("entry.cjs"), true);
assert.equal(isCrossAppImportSourceFile("postcss.config.mjs"), true);
assert.equal(isCrossAppImportSourceFile("types.d.ts"), true);
assert.equal(isCrossAppImportSourceFile("package.json"), false);
const violations = collectCrossAppImportViolationsFromSource(
"apps/web/postcss.config.mjs",
"import { startDaemon } from '@open-design/daemon/src/server.ts';",
registry,
);
assert.equal(violations.length, 1);
assert.equal(violations[0]?.targetApp, "daemon");
assert.equal(violations[0]?.lineNumber, 1);
});
test("cross-app import check rejects export-from and apps/ rooted specifiers", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/daemon/src/index.ts",
["export { FileViewer } from 'apps/web/src/components/FileViewer.tsx';", "import 'apps/web/src/index.css';"].join(
"\n",
),
registry,
);
assert.deepEqual(
violations.map((violation) => violation.targetApp),
["web", "web"],
);
});
test("cross-app import check allows packages, same-app relatives, and externals", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/web/src/components/ChatPane.tsx",
[
"import { Button } from '@open-design/components';",
"import type { ChatRunEvent } from '@open-design/contracts';",
"import { latestTodoWriteInputFromMessages } from '../runtime/todos.ts';",
"import path from 'node:path';",
"import React from 'react';",
].join("\n"),
registry,
);
assert.deepEqual(violations, []);
});
test("cross-app import check ignores quoted snippets and comments", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/web/tests/import-boundary-fixture.test.ts",
[
"const snippet = \"import { x } from '@open-design/daemon/src/server.ts';\";",
"// import { y } from '@open-design/daemon/src/server.ts';",
"/*",
"require('@open-design/daemon/dist/cli.js');",
"*/",
].join("\n"),
registry,
);
assert.deepEqual(violations, []);
});
test("cross-app import check ignores relatives that escape apps/ without hitting a registered app", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/daemon/tests/mcp-extract-refs.test.ts",
"const refs = extractRelativeRefs('@import \"../../shared.css\";', 'a/b/c/file.css', 'text/css');",
registry,
);
assert.deepEqual(violations, []);
});
test("cross-app import check allows allowlisted packaged -> desktop main export", () => {
const violations = collectCrossAppImportViolationsFromSource(
"apps/packaged/src/index.ts",
"import { applyOsLocaleSwitch, createSplashWindow } from '@open-design/desktop/main';",
{ packageNameByDirectory: new Map([["packaged", "@open-design/packaged"], ["desktop", "@open-design/desktop"]]) },
);
assert.deepEqual(violations, []);
});
test("cross-app import check ignores files outside apps/", () => {
const violations = collectCrossAppImportViolationsFromSource(
"e2e/tests/dialog/stop-reconciles-message.test.ts",
"import { something } from '../../../apps/daemon/src/server.ts';",
registry,
);
assert.deepEqual(violations, []);
});
test("app registry loading fails loudly when an app manifest is malformed", async () => {
const appsRoot = await mkdtemp(path.join(os.tmpdir(), "open-design-apps-"));
const appRoot = path.join(appsRoot, "web");
const manifestPath = path.join(appRoot, "package.json");
await mkdir(appRoot);
await writeFile(manifestPath, "{ invalid json", "utf8");
await assert.rejects(
loadAppDirectoryRegistry(appsRoot),
(error) =>
error instanceof Error &&
error.message.includes(`Failed to load app package manifest at ${manifestPath}:`) &&
error.message.includes("JSON"),
);
await rm(appsRoot, { force: true, recursive: true });
});
test("app registry loading rejects parseable app manifests without a package name", async () => {
const appsRoot = await mkdtemp(path.join(os.tmpdir(), "open-design-apps-"));
const appRoot = path.join(appsRoot, "web");
const manifestPath = path.join(appRoot, "package.json");
await mkdir(appRoot);
await writeFile(manifestPath, "{}", "utf8");
await assert.rejects(
loadAppDirectoryRegistry(appsRoot),
(error) =>
error instanceof Error &&
error.message === `Failed to load app package manifest at ${manifestPath}: package name must be a non-empty string`,
);
await rm(appsRoot, { force: true, recursive: true });
});