Follow-up to a full re-audit of the 2.0 tree. Most prior findings were already
fixed; this closes the ones that were not:
- SAML assertion replay: validateInResponseTo ifPresent plus a Redis-backed
CacheProvider, so a captured signed assertion cannot be replayed. ifPresent
keeps IdP-initiated SSO working.
- MFA login challenge burned after 5 wrong TOTP codes.
- api_keys.key_prefix indexed; the per-request lookup was a full table scan.
- MAX_AI_JOBS_PER_USER caps a user's in-flight single-file AI jobs (the AI pool
runs at concurrency 1). Batch and pipeline AI stay uncapped.
- MAX_WORKSPACE_SIZE_GB enforced instead of being dead config.
- SUBPROCESS_MEMORY_LIMIT_MB (default off) for the native media and doc engines;
not applied to the AI sidecar.
- SVG sanitizer closes unquoted and whitespace-prefixed javascript: hrefs and
the animateTransform/animateMotion/handler/mpath elements.
- Windows-style paths stripped from error output to match the Sentry scrubber.
- Postgres and Redis compose services get cap_drop plus pids_limit and cpus.
- .env.example ships MAX_SVG_SIZE_MB=50 (0 disabled the cap).
Adds security-focused unit and integration tests. typecheck, biome, and the
full unit and integration suites pass.
The download route sets Content-Length from a stat and then streams the
object; when the stat size exceeds the bytes the stream yields (#590
"cause 2"), the client hangs on keep-alive framing waiting for a tail
that never arrives. Both send paths now run through a backpressure-safe
byte-counting Transform that resets the socket on a shortfall, so the
download fails at once instead of hanging. Adds a real-socket regression
test at the generic download route, the coverage gap #590 named.
Refs #590
Co-authored-by: harshjainnn <170849281+harshjainnn@users.noreply.github.com>
Centralize role-authority enforcement across user management, role management, configuration import, SCIM, GDPR, and MFA mutations. Add regression coverage for delegated custom roles and protect higher-privilege accounts from reset, deletion, or takeover.
Defers the onboarding usage survey to the instance's first successful processing (the worker writes a one-time onboarding.firstProcessedAt marker and the overlay gates on it), so it reaches engaged users instead of first-landing visitors.
Replaces the two questions telemetry already answers (modality preference from tool_used, install method from instance_started) with what it can't infer: prior tool, self-host motivation, and discovery source.
Adds feedback_prompt_shown and feedback_prompt_dismissed on all five feedback surfaces (usage survey, per-job prompt, admin install card, global nav dialog, search-miss) so skip and completion rates are measurable, not just submissions. New survey strings translated into all 20 non-English locales.
Five fixes to the PostHog event stream, from an audit of what we actually collect versus what's flowing in. Each one is test-first.
## What changed
**Silenced the `$opt_in` noise.** `initAnalytics` called `opt_in_capturing()` on every page load to clear a stale opt-out flag, and posthog-js emits an `$opt_in` event on every call. That was 10k+ events a month (up to 55 per user) carrying no signal: analytics is on by default with an admin opt-out, so there is no per-user consent to record. Both call sites now pass `captureEventName: false`.
**Split the onboarding survey out of `feedback_submitted`.** The onboarding usage survey rode the same event as real feedback, so about 93% of "feedback" was actually onboarding profiling. It now emits `onboarding_survey_submitted`, so feedback metrics mean feedback again.
**Set `pipeline_executed.file_count`.** It was declared in the properties interface but never populated. A pure `pipelineExecutedProps` helper now derives it (batch size for a batch run, else 1) and is shared by the success and failure paths, which also drops a duplicated payload.
**Tracked OIDC login failures.** All six OIDC callback failure branches bumped the Prometheus counter and wrote an audit log but never emitted `auth_login_failed`. A `recordOidcFailure` helper mirrors the password path.
**Added `TELEMETRY.md`.** A contributor-facing event dictionary: every event, its properties, where it fires, and the privacy invariants, with the allowlists as source of truth. A drift test fails if any `ANALYTICS_EVENTS` value goes undocumented.
I left the published telemetry guide (`apps/docs/guide/telemetry.md`) alone. It is high-level and still accurate, and editing it would pull in the 21-locale stale-gate for no gain.
## Verification
- Unit (63 tests): `analytics-events`, `telemetry-doc-drift`, `api/analytics`, `web/analytics`, `worker.behavior`
- Integration (41 tests): `oidc-auth`, `feedback`
- Full typecheck across all 9 workspaces
- Biome clean on the changed files
All green locally.
AI sidecar failures reached Sentry as 'Error: Error': the scrubber type-onlys plain Errors and the tool wrappers threw them from result.error. The bridge now exports toSidecarError(), wrapping the sidecar reason in a SafeError (memory-allocation text classifies as operational, the rest as bug); all 14 wrappers use it, plus the dispatcher crash/stdin/spawn rejection paths and parseStdoutJson. toBgRemovalError from #535 delegates to the shared helper.
On the web side, DOMExceptions report their specific name via err.name, so the NATIVE_ERRORS allowlist dropped the whole family's browser-authored messages. It now carries the full WebIDL DOMException name table; messages still pass through url/path redaction.
Bridge-mocking test files switched to importOriginal passthrough mocks.
Stackless uncaught errors reached Sentry as a bare Error with no frames and collapsed into one ungroupable issue. beforeSend now fingerprints frameless events by safe identity (name, code, one-way hash of the message) so distinct crashes separate without leaking PII. Only frameless events are touched; an upstream fingerprint is never overridden.
BullMQ raises UnrecoverableError when a job loses its lock (a stall), e.g. a heavy upscale under memory pressure. We never throw it ourselves, so classifyError now treats it as operational (one warning per hour) instead of a bug. ReplyError stays a bug.
RealESRGAN's enhance() and rembg's remove() run in one opaque call, so the
progress bar froze at 30% for the whole inference. Add a time-based
heartbeat that advances the bar in a background thread while the model runs
and stops when it returns, so the bar moves instead of freezing. Verified
end to end on a GPU box (forced CPU): a 55s upscale emitted 26 steady ticks
then completed; background removal too.
Fixes#591
Add a one-line notice to the upscale and background-removal settings that
heavy AI runs much slower without a GPU and a large image can take minutes,
so a CPU-only self-hoster isn't caught off guard by a slow run. Translated
into all 21 locales.
Refs #591
Send X-Accel-Buffering: no on file download responses so nginx and
compatible reverse proxies stream them through instead of buffering, the
usual cause of a self-hosted download that "starts but never finishes".
The app already delivers exactly Content-Length bytes; a new real-socket
test proves it for both the collated PDF and the multi-file ZIP.
Refs #590
Adds a Rounded Crop image tool for logo, favicon, and app-icon work. It masks the framed square to a rounded rectangle (with a corner-radius control) or an iOS-style squircle, reusing circle-crop's zoom/offset framing, border ring, background fill, and output-size options. Includes translations across all 21 locales.
Closes#601
When a PDF has no text layer (scanned or image-only), pdf-to-text now
returns a 422 that points at the OCR tool instead of a silent empty file,
and text downloads carry charset=utf-8 so UTF-8 Arabic renders correctly
when the .txt is viewed inline.
Fixes#589
The library file-storage helpers joined FILES_STORAGE_PATH with a database stored_name and never checked containment, so a crafted name could read or delete files outside the storage root after a malicious 1.x SQLite import (which copies stored_name verbatim). Add assertSafeStoredName() and apply it in every helper that resolves a stored name to a path, matching the containment guard object-storage already uses.
Reported by Alpesh Bhagwatkar.
Replace the job timeout message that hardcoded "background-removal" for
every tool with a tool-agnostic one that sets the CPU-vs-GPU expectation,
the usual reason heavy AI times out on modest hardware. The client-side
SSE stall message gets the same treatment. Both stay under friendlyError's
280-char limit so the guidance reaches the user instead of collapsing to a
generic "Processing failed".
Refs #591
The job cancel endpoint authenticated the caller but never checked that the job belonged to them, so any authenticated user could cancel another user's job by ID. Load the job's owner and allow cancellation only for the owner or a caller with files:all; return 404 for missing and non-owned jobs alike. Extract the route into a shared registerJobRoutes() so the ownership check is covered by tests.
Reported by Alpesh Bhagwatkar.
Empty the hardcoded page-range default in remove/split/extract PDF tools (remove-pages defaulted to "2,4-6", out of range for any PDF under 6 pages) and disable submit until a range is entered. Reject password-protected PDFs up front for PDF-only tools with guidance to unlock first, instead of failing cryptically in the qpdf worker. Adds integration + e2e coverage.
Pin the compose Postgres healthchecks to POSTGRES_DB (pg_isready was
defaulting to the username, silently reporting healthy while spamming
FATAL logs when USER and DB differ), and make docker/wait-for-postgres.mjs
log the target host and error code instead of a silent retry loop. Adds a
change-me note next to the default password in README and Docker Hub.
Refs #592
Add -movflags +faststart for mp4/mov/m4v output from stabilize-video so
the stabilized result streams and previews progressively instead of
appearing broken or corrupted (moov atom was landing after mdat).
Fixes#588
Two OCR-install fixes surfaced while verifying the accurate-OCR (v3 runtime) path end to end:
- Installer timeout must be a safe integer, not a performance.now() float. With the default INSTALL_MAX_MS this failed every accurate-OCR install via the app right after the download (masked by the unpublished runtime; CI drives install_runtime.py directly so it never surfaced). Fixed via remainingInstallerTimeoutMs().
- Classify an absent or forbidden runtime index (401/403/404/410) as OcrRuntimeNotPublishedError with a clear "Fast OCR still works" message instead of a raw HTTP 404, without retrying.
Refs #552
Closes#565. Wires the fileId/saveMode pair into the ocr, erase-object, remove-background, background-replace, and blur-background submitters so the library save-mode selector works for them; remove-background's two-phase effects route now auto-saves the final composite instead of the transparent intermediate.
Adds an opt-in High Quality mode to the Object Eraser, backed by a new inpaint-hq feature bundle (Stable Diffusion 1.5 inpainting via diffusers). The default fast LaMa path is unchanged. Both arch archives are published to deepsafe/feature-bundles and the manifest carries their real sha256/sizes.
Verified end to end: a fresh container pulls the bundle from HuggingFace, checksum-verifies it, extracts torch/diffusers plus the fp16 model, and the HQ sidecar erases a large object with a plausible fill.
Refs #141
Exports a pure buildRedirects() from generate-redirects.mjs and adds a unit test asserting the committed apps/landing/public/_redirects matches it, so tool additions can't silently leave the generated redirects stale (see #573).
Fixes#568. Adds a real --color-ring token (ink orange #A85518 light / #F0A766 dark) and sweeps all 57 focus-indicator occurrences onto it: soft opacity rings blended to 1.2-1.7:1, border-only indicators sat at 2.6-3.0:1, and four focus:ring-ring sites referenced a token that never existed. Sponsor button keeps pink via pink-700; range sliders and the file list gain their missing keyboard indicators; landing skip link and form borders hardened; the palette contrast guard pins the ring at 3:1 in both themes.
Fixes#557. Vivid fill, ink label: brand #E07832 stays on fills while primary-foreground flips to #1A1814 (5.83:1); new theme-aware ink tokens carry orange, destructive, and success text roles; opacity-modified text purged; landing, demo, and the docs fund button retuned. Guarded by a CSS-parsing unit contrast test, rebuilt axe baselines with zero contrast entries, a new landing axe smoke, and fully regenerated darwin visual baselines.
The landing Playwright suite ran in no CI workflow, so six specs had drifted red on main. Five subpages navigation tests asserted bare paths while the site emits trailing-slash URLs (format: directory), and one asserted a localized tool-detail page that is English-only by design. Fix the assertions and rewrite the tool test to the real invariant, then add a test-e2e-landing job gated on a new landing path filter so the suite runs on landing-relevant PRs and can't silently rot again.
The custom nav cluster (theme toggle + Fund + GitHub Star) rendered inline at
every width, overriding VitePress's responsive collapse: a horizontal scrollbar
at 768-959px and off-screen clipping of the buttons on 1280-1366px laptops.
Show the custom cluster only at >=1440px where it fits, defer to VitePress's
native nav below that, anchor the flyout menu to the start edge in RTL, and drop
the redundant "Home" nav link so the nav fits at 768px.
Closes#556
Editing a file from the library used to silently supersede it: the worker auto-saved every result as a new version and the leaf-only listing hid the original, which read as a destructive overwrite. Tool pages now show a per-edit choice for library-sourced files. The default saves the result as an independent new file and keeps the original; picking overwrite keeps the old superseding-version behavior.
The client sends a saveMode multipart field next to fileId, validated with a 400 on unknown values, and autoSaveToLibrary branches on it. Every hand-written route that honors fileId parses the field the same way as the factory. The review panel shows where an auto-saved result went instead of offering a second, duplicate save. Tools whose route or submitter ignores fileId keep the selector hidden via a shared unsupported-tools set, and the choice resets to the non-destructive default whenever a new file is staged.
Closes#495
The docs e2e suite clicked navbar and sidebar controls before Vue hydrated the
multi-locale bundle, so the clicks were swallowed. That raced the Pagefind
search open (filed as #551), the appearance toggle, and the homepage and
sidebar navigation tests. The old search tests also matched an input
placeholder the config overrides, so they failed against a working build.
Add a waitForHydration helper (gates on #app.__vue_app__, set inside Vue's
app.mount()) and an openDocsSearch helper, and route the affected tests through
them. Search itself was never broken; this change is test-only. Docs e2e suite
is green (43/43).
Closes#551
Astro's getRelativeLocaleUrl lowercases the locale segment by default, so landing links and hreflang for zh-CN, zh-TW, and pt-BR were emitted lowercase and 404 on case-sensitive Cloudflare Pages. Pin the casing at the localizeHref chokepoint with normalizeLocale: false, add an e2e hreflang casing guard, and add a deploy-time check that blocks the build if any lowercased locale path leaks into the output.
Closes#554
The Convert Audio tool promised configurable bitrate, sample rate, and channel count, but only format and bitrate were exposed. Adds an optional sampleRate setting (8000 to 96000 Hz, omitted = preserve source) wired through the Zod schema, the FFmpeg -ar flag, the standalone settings panel, and the pipeline builder controls.
Impossible combinations fail loudly instead of degrading silently: MP3 + 96000 Hz is rejected (libmp3lame caps at 48 kHz), and MP3 bitrates above the encoder ceiling at low rates (64 kbps at 8 kHz, 160 kbps at 16/22.05 kHz) are rejected rather than clamped. The UI offers only legal combinations and sanitizes stored pipeline settings on load.
Docs updated in English plus all 20 localized pages with refreshed i18n_source_hash stamps; two new UI strings added to all 21 locales.
Fixes#558
Tool-detail pages (/tools/<section>/<tool>/) and the /self-hosted pages are
built only in English, with no per-locale route, so a locale-prefixed link
404s in the static build. Add an enOnlyHref() helper and use it for those
links in Footer, Navbar, HeroSearch, and ToolGrid so localized pages point at
the English pages that actually exist. Adds an e2e guard asserting localized
pages emit un-prefixed URLs for those routes.
On mobile browsers 100vh is the tall viewport (URL bar retracted), so the
full-height shells (h-screen + overflow-hidden) rendered their bottom strip
below the visible area with no way to scroll to it. After an upload the tool
page's bottom control is the "Process" peek bar, so it landed off-screen on
every tool.
Switch the fixed-height shells from h-screen (100vh) to h-dvh (100dvh), which
tracks the visible viewport. AppLayout covers every tool page; the same sweep
covers the editor, login, change-password, 404, and the error/loading screens.
On desktop dvh equals vh, so desktop is unchanged. Adds a source-level guard
(tests/unit/web/app-shell-viewport.test.ts) since headless Chromium has no URL
bar and cannot reproduce the bug.
Comprehensive telemetry quality improvements across Sentry and PostHog, grounded in an audit of the live data plus current best-practice research.
Sentry: job_id/instance_id tags, operational fingerprinting, PII-safe settings context on bug events, web tag population + extension-noise filtering, an early-crash buffer, http status/method kept on breadcrumbs, and a gated-off-by-default performance-tracing re-enable (tracesSampler that zeroes db/redis/queue-poll root spans + drops the Redis integration) with worker job spans and canonical-host cron monitors.
PostHog: history_change SPA pageviews, instance_id super property for fleet rollups, enriched tool_used (formats, byte sizes, is_batch, execution_hint, real error_kind taxonomy), the previously-dead result_saved/batch_processed/ai_bundle_prompted events fired, search click-through, editor + Automate authoring + auth instrumentation, a before_send PII boundary, and minimal opt-in landing-site pageviews.
A genuine pdf.js load failure (corrupt or password-protected file) was swallowed by the same catch that silences teardown rejections, leaving a blank canvas that looks like it is still loading. The two cases are now distinguished by the load effect's cancelled flag, and a real failure renders a clear message pointing at Unlock PDF for encrypted files. New loadFailed string in all 21 locales.
Item 4 of #478.
Both PyPI onnxruntime flavors unpack into the same site-packages directory, so a bundle carrying the CPU build (transcription, via faster-whisper) overwrote the GPU build's native libraries during install while the stale onnxruntime_gpu dist-info kept claiming otherwise. Every ONNX-backed tool then silently ran on CPU.
The installer now reconciles the flavor before the venv merge and the GPU build always wins, in both install orders; reinstalling any GPU bundle repairs a previously clobbered venv. gpu.py's warning now says exactly that. Build-side, build-bundle.sh gains the same reconcile and verify-bundle-compatibility.sh layers bundles through the real installer merge and asserts a single flavor.
Verified live on an RTX 4070 against the published bundles: reproduced the clobber with the stock installer, then confirmed both the prevention and repair paths with the patched one.
Fixes#490
The password policy toggles (uppercase, lowercase, digit, special) can all be switched off in Settings -> Security, but the minimum-length input clamped at 4, so homelab admins couldn't deliberately allow short passwords. The API never enforced a floor; only the UI did. Lower the input floor to 1 and pin it with a test.
Closes#136
Defer the overall deadline until the first fetch is dispatched (the deferred-deadline pattern the sibling 20ms test already uses), so a loaded CI runner cannot abort before fetch and randomly fail Unit Tests.
Briefly retry Postgres and Redis connectivity at boot (waitForService, DB_STARTUP_TIMEOUT_MS) so an ordered-but-not-yet-ready dependency recovers cleanly instead of crash-looping.
removeBackground failures wrap in a SafeError so the specific reason survives the Sentry scrubber; the OOM lighter-model fallback and bridge SafeError passthrough are preserved.
Every worker tool error now carries an input_format tag (file extension only, never the filename) for triage, derived once at the worker error path and added to the scrubber allowlist.
Postgres auth (28xxx), permission (42501), resource (class 53), and operator-intervention (class 57) failures now classify as operational via a cause-chain walk, not bug. pg query bugs (e.g. 42601) stay bugs.
Route both Python exit paths through pythonExitError so the reason survives the scrubber; OOM/segfault stay operational and keep "out of memory" for the lighter-model fallback.
Wrap convert and gif-tools process functions so a Sharp .toBuffer() failure carries an authored SafeError title (and the original as cause) rather than a scrubbed "Error: Error".
Route every doc_* helper JSON.parse through a guarded helper; non-JSON stdout now yields a diagnosable SafeError with the raw output in the cause instead of a bare SyntaxError.
Fixes#533, found while working on #529/#531.
OIDC and SAML logins hard-blocked on the MFA policy with zero check of whether the user actually enrolled TOTP, and no challenge step at all. Once an admin turned on an MFA-required policy, every SSO user was permanently locked out regardless of enrollment status.
- Extract the post-auth MFA decision (challenge / enrollment-required / proceed) into a shared, unit-tested function so OIDC and SAML can't independently diverge again
- An already-enrolled user now gets a real challenge (reusing the existing, auth-method-agnostic MFA completion flow) instead of being blocked
- An unenrolled user under a required policy gets a distinct, correctly mapped error instead of the old generic one
- Fix a real fail-open regression caught in review: a transient DB error during the enrollment-status check could have silently skipped MFA entirely for an enrolled user; now it fails closed and logs
- Strip the one-time challenge token from the URL after consuming it
Fixes#529 (opened investigating #515).
Setting MFA policy to "required"/"admins only" saved regardless of whether the mfa enterprise feature was licensed, and there was no enrollment UI at all, so any instance that flipped the toggle locked every unenrolled user out with no way back in. The login page and Settings save also both collapsed the resulting error into a generic message, hiding the real reason.
- Reject saving mfaPolicy to admins_only/required server-side unless mfa is licensed
- Surface the specific server error on login and on a failed settings save instead of a generic fallback
- Add a self-service two-factor authentication enrollment flow (QR code, manual entry, recovery codes, verify, disable) so a licensed admin can actually satisfy the policy before it's enforced
- Fix a pending-enrollment dead end, silent error swallowing in verify/disable, and a silent clipboard-copy failure on the recovery codes screen
- Add the integration test that actually proves the fix: a real login attempt returns 403 MFA_ENROLLMENT_REQUIRED
Add a proportion chip row (Free, Original, 1:1, 4:3, 3:2, 16:9, 3:4, 9:16) to the Resize tool's Custom tab. Picking a ratio locks width and height so editing one recomputes the other, and prefills the largest box of that ratio that fits the source so it never upscales. Free stays the default, preserving existing behavior. Replaces the previously non-functional lock-aspect button. Frontend only, no backend or schema change; adds strings to all 21 locales.
