Commit Graph
4 Commits
Author SHA1 Message Date
SnapOtterandGitHub f1ec3beaf7 test: settle 202 jobs instead of returning without asserting (#652)
A 202 means the sync window expired while the job was still running. Tests treated it as a terminal pass: `if (isAsyncFallback(res)) return;` checked the envelope and returned, asserting nothing about the outcome and leaving the job running into the next test, which is the leak cancelAcceptedJobAndWait exists to prevent.

Because the window only expires under load, coverage tracked runner load. On CI 44 tests took this path and verified nothing; the same tests on a dev machine asserted in full (one measured 6.4s locally against 31s on CI).

settleAsyncFallback waits for a terminal state and asserts the job finished, and that a failure carries a message rather than being a crash. A clean failure stays valid, since the exotic-format fixtures are meant to be rejected. All 82 call sites moved over.

per-fork-env no longer floors SYNC_WAIT_MS, so forcing it to 0 drives every request through its 202 path. 570 tests were validated that way and matched their normal-window results exactly.

The 29-34s band dropped from 44 tests (23.4% of test time) to 6 (3.0%). Total test time rose 5.8% and CI wall went 12.8 to 13.1 min: the forks were doing real work during that wait, so this buys determinism, not speed. Per-shard totals unchanged at 9903 tests, 9435 passed, 468 skipped.
2026-07-27 12:16:48 +08:00
SnapOtterandGitHub bbad953e79 fix(nightly): de-flake matrix timeouts, exclude browser tool from fuzz (#349)
Systematic round - attack the failure classes, not one bug at a time:

- Flakiness (Coverage/Extended Matrix flipped green<->red): the generated
  format-matrix conversion tests hardcode a 30s/60s per-test timeout that the
  jobs' VITEST_TEST_TIMEOUT can't override. Heavy conversions under coverage
  instrumentation / full matrix intermittently exceed it. Raise to 120s.
- Schemathesis: html-to-image returns 503 when its headless browser isn't
  present (always, in the fuzz env) - same expected-unavailable class as the
  AI 501s. Exclude it from the fuzz (it can't be exercised without a browser).
- Baseline regen: bump timeout 60->120 min so it can finish now that the
  route sweep (#347) lets the visual specs pass instead of failing+retrying.
2026-06-25 00:14:45 +08:00
SnapOtter 5ffa1d55ea Merge branch 'worktree-test+suite-overhaul-and-real-fixtures' into chore/consolidate-v2.0.0
# Conflicts:
#	tests/integration/generated/settings-matrix.test.ts
#	tests/integration/platform/api.test.ts
#	tests/integration/platform/concurrent.test.ts
#	tests/integration/platform/factory-multi-input.test.ts
#	tests/integration/security/adversarial-comprehensive.test.ts
#	tests/integration/security/adversarial-coverage-gaps.test.ts
#	tests/integration/security/adversarial-extended.test.ts
#	tests/integration/security/adversarial-final-gaps.test.ts
#	tests/integration/security/adversarial-matrix.test.ts
#	tests/integration/security/adversarial-security.test.ts
#	tests/integration/security/adversarial.test.ts
#	tests/integration/tools/image/color-adjustments.test.ts
2026-06-21 02:18:53 +08:00
SnapOtter 1727a7a73e test: reorganize flat test files into purpose-based subdirectories (phase 6)
Group 245 flat integration tests and 25 loose unit tests into
discoverable subdirectories per spec section 6:

  integration/tools/{image,video,audio,document,data}/  (156 files)
  integration/platform/                                  (64 files)
  integration/generated/                                 (14 files)
  integration/security/                                  (10 files)

  unit/security/     (8 files, new subdir)
  unit/api/          (7 files moved in)
  unit/web/          (3 files moved in)
  unit/shared/       (6 files moved in)
  unit/image-engine/ (1 file moved in)

All moves via git mv (history preserved). Relative imports repaired
for both depth levels (platform/generated/security = +1, tools/ = +2):
static from-imports, dynamic import() calls, vi.mock() paths,
import.meta.dirname joins, and __dirname joins.

Vitest discovery unchanged (no test.include in config, recursive glob
matches subdirs, shard-by-hash unaffected). test-server.ts and
tool-route-drift.test.ts stay at integration root. fixtures/ untouched.

Parity gate: 13189 passing test names before = 13189 after (0 dropped).
2026-06-20 05:07:46 +08:00