test(nightly): stabilize the exhaustive nightly suite (#345)

Triaged the nightly failures (all pre-existing, unrelated to the analytics
work) and fixed the ones with clear root causes:

- video-speed: a 1s tiny.mp4 sped up 2x rounds to ~0.75s, flaking the +/-25%
  duration assertion under heavy CI load. Use the 8s hero.mp4 (still 44.1kHz)
  so rounding is negligible. Verified locally.
- Extended Matrix + Coverage timeouts: full-matrix / coverage-instrumented runs
  starve the heavy media tests under 4 forks at the 30s default. Make maxForks
  env-overridable (VITEST_MAX_FORKS) and run those jobs with 2 forks + a 300s
  timeout so format-matrix conversions and qr-generate stop timing out.
- Device Matrix visual baselines: the update-visual-baselines workflow could
  not start the app ('failed to create database') because it never provisioned
  Postgres/Redis. Add the same services block the e2e jobs use.
- Docker E2E: a container pnpm install network blip exits 254. Add fetch
  retries + a longer network timeout (frozen-lockfile already passes locally).
- Cross-browser: the home page is the tool catalog now (no dropzone), and the
  tool routes moved to /<section>/<toolId>. Point the upload test at a real
  tool page and fix the stale single-segment routes (/resize -> /image/resize,
  etc.).

The flaky/timeout and cross-browser fixes can only be confirmed by the nightly
(they are load- and browser-specific); a fresh nightly run will verify.
This commit is contained in:
SnapOtter
2026-06-24 18:23:23 +08:00
committed by GitHub
parent 8f4235d2c6
commit 33dfcecd6a
6 changed files with 74 additions and 17 deletions
+12
View File
@@ -273,6 +273,12 @@ jobs:
FULL_MATRIX: "1"
FUZZ: "1"
FUZZ_RUNS: "50"
# Full-matrix tests iterate every format x tool; with 4 forks the heavy
# media conversions starve and hit the 30s default. Fewer forks (more
# CPU each) plus a generous timeout keeps them from flaking.
VITEST_MAX_FORKS: "2"
VITEST_TEST_TIMEOUT: "300000"
VITEST_HOOK_TIMEOUT: "120000"
api-fuzz:
name: Schemathesis API Fuzz
@@ -361,6 +367,12 @@ jobs:
- uses: ./.github/actions/setup
- name: Run tests with coverage
run: pnpm vitest run --coverage tests/unit/ tests/integration/
# Coverage instrumentation slows every test; fewer forks + a longer
# timeout keep slow media tests (e.g. qr-generate) from timing out.
env:
VITEST_MAX_FORKS: "2"
VITEST_TEST_TIMEOUT: "300000"
VITEST_HOOK_TIMEOUT: "120000"
- name: Upload coverage artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
@@ -12,6 +12,32 @@ jobs:
name: Regenerate linux visual baselines
runs-on: ubuntu-latest
timeout-minutes: 60
# The e2e webServer (playwright.config.ts) creates an isolated database via
# tests/e2e-pg-create-db.cjs, so it needs Postgres + Redis on localhost.
# Without these services the webServer fails with "failed to create database".
services:
postgres:
image: postgres:17-alpine
env:
POSTGRES_USER: snapotter
POSTGRES_PASSWORD: snapotter
POSTGRES_DB: snapotter
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U snapotter"
--health-interval 5s
--health-timeout 3s
--health-retries 10
redis:
image: redis:8-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 3s
--health-retries 10
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: ./.github/actions/setup