Nightly has been red since 07-28; per-PR CI and main are green. Two investigations traced all five failing classes to #649: extended-matrix required AI bundles it never installs (removed), nightly SYSTEM_DEPS drifted from ci.yml (added libreoffice + a doc-binaries composite for pandoc/pdfcpu), the generated-case classifier only skipped ffmpeg (widened to pdfcpu/soffice/pandoc + excluded repo-audit specs from the lean docker image), a settings spec capped loginAttemptLimit and 429-cascaded the serial bucket (restore via API), and type-to-search refused keystrokes under a route announcer's programmatic focus (guard added). A nightly dispatch on the branch confirmed all five classes green.
The scheduled Nightly had been red for over a week across nearly every job. This
root-causes and fixes each one. All were pre-existing: missing CI provisioning,
specs that drifted as the app grew, a job too heavy for its timeout, and a fuzz
that was never configured for file-upload endpoints. None came from the recent
security merge.
- Coverage + Docker Container E2E: install tesseract and its language packs so
the built-in Fast OCR tests stop throwing spawn ENOENT; gate two repo-file and
release-workflow tests that cannot run inside the slimmed container image.
- E2E (Full, Serial, Cross-Browser, Device Matrix): refresh specs that drifted
behind the app (tool renames, the now admin-only Tools tab, dropped About copy,
locator collisions scoped to the right region). One real product fix rode
along: /config/auth was refetched six times per tool-page load, so cache it
behind a single shared fetch, dropping the tool page from 13 to 8 API calls.
- Extended Matrix + Fuzz: shard the integration suite four ways so the full
format x tool matrix plus property fuzz fits its budget instead of overrunning
the 90-minute ceiling every night.
- Schemathesis: exclude the tools with bespoke handlers that process
synchronously in-request (they hang the fuzz on adversarial input) and suppress
Hypothesis's data-generation health checks, which fire because file-upload
endpoints reject the fuzzer's random bytes. not_a_server_error still runs on
every generated case (5000+ per run).
- Stabilize two long-tail flakes: raise the avif matrix per-test cap from 240s to
600s, and assert toHaveCount(0) on the deleted user row so a transient success
toast no longer trips a strict-mode violation.
Verified end to end: the full Nightly workflow is green on this branch (all 14
jobs), and PR CI is green.
Lands five integrated branches: pipeline templates (#355), analytics opt-out (#354), 83 conversion presets bringing the catalog to 240 tools (#356), self-hosted positioning (#353), and e2e modernization (#351).
Integration fixes: aligned stale web analytics tests with the opt-out/allow-list model, closed 3 CodeQL incomplete-sanitization alerts in the i18n generator, resolved settings/index/docs/format-matrix conflicts, and corrected tool counts to 240.
Closes the "e2e never runs in CI" hole. Adds per-PR e2e smoke gate,
nightly full-suite workflows, parallel vitest forks (per-fork DBs),
Playwright parallel/serial/visual projects against production builds,
metadata-generated test suites (drift guards, hostile inputs, format
matrix, pairwise settings, property-based fuzz), Stryker mutation
testing, Schemathesis API fuzz, coverage ratchet, and fixes for three
session-poisoning bugs that caused 200+ serial-bucket failures.
Bug fix included: favicon/split/bulk-rename could hang clients forever
when ZIP streaming failed after reply.hijack().
Users can no longer customize the app name or logo. The branding API
endpoints, permission, frontend UI, env vars (APP_NAME, MAX_LOGO_SIZE_KB),
and all related tests are removed. Includes a migration to clean up
branding data from existing databases.
1. passport-photo 404 vs 501: add base route at /api/v1/tools/passport-photo
that returns 501 FEATURE_NOT_INSTALLED when the AI bundle is missing,
matching other AI tools. The /generate sub-route is Sharp-only (no
sidecar) so it correctly skips the isToolInstalled guard.
2. AuthGuard analytics consent race: don't evaluate shouldShowConsent()
until analyticsConfig has been fetched (guard on analyticsConfig !== null).
Prevents redirect to /analytics-consent before config is loaded.
3. Fragile sidebar Settings selector: add openSettings(page) helper to
E2E helpers that checks sidebar visibility with fallback to button role.
Replace all 134 occurrences of page.locator("aside").getByText("Settings")
across 18 test files.
Add ~500 new E2E tests and ~300 new integration tests covering:
- 24 new GUI E2E specs: navigation, responsive layout, keyboard shortcuts,
tool UI for all 35 non-AI tools, batch/pipeline workflows, settings/RBAC,
visual regression, accessibility, and performance budgets
- 3 new E2E-Docker specs: batch workflows, advanced pipelines, cross-format
- 1 new adversarial integration test: memory pressure, corrupted files,
unicode filenames, extreme dimensions, pipeline/batch edge cases
- 29 expanded integration test files: HEIC/HEIF input, large files, parameter
boundaries, batch processing, format edge cases across all tools
- Cross-format matrix expanded: 641 tests covering every tool x 18 formats
- AI bridge unit tests expanded: lifecycle, tool modules, error propagation
- Unit test gaps filled: analytics, tool-registry, web stores
Also fixes:
- vitest.config.ts: exclude e2e-docs and e2e-landing from Vitest runner
- AI E2E specs: add sidecar health check to skip gracefully when Python
AI backend is not running instead of timing out
