After #249 merged, main's CI is red on failures that predate this work (they
were cache-masked on the old branch). This applies only the missing fixes onto
current main (reverts nothing from #246/#248; saml/user-files are already fixed
on main):
- remove 8 dead landing unit tests importing @landing/app and @landing/components
React paths deleted in the Astro migration
- format tool-factory.ts and json-xml.ts to clear the api lint errors
- accept a 202 async fallback in the image-enhancement large-image test (it
exceeds the sync window on slower CI runners)
- relax the pipeline no-file assertion to match 'No image file provided' or
'No file provided'
* fix(media): mux container-correct codecs in video tools
Video tools hardcoded H.264 (and AAC) while keeping the input's container extension, so a .webm input produced an invalid file (ffmpeg exit 234: H.264 cannot be muxed into WebM). Add shared videoEncodeArgsForContainer/audioEncodeArgsForContainer helpers (vp9+opus for webm, theora+vorbis for ogv, h264+aac otherwise) and apply them across 14 tools; re-encode audio to AAC in burn-subtitles (forced mp4). Adds a webm regression test for change-fps.
* fix(eraser): recover Object Eraser when its progress SSE drops
The eraser used a bespoke EventSource with no recovery, so a dropped SSE left the UI stuck at ~25% forever even though the backend job had finished and saved its result. Add a resilient subscription (reconnect on tab refocus, which replays the cached terminal frame; 5-minute stall timeout) mirroring the standard processor's PR #203/#204 recovery.
* feat(ui): rename the Documents modality to PDF and Data to Files
Updates modality display names, the home-page tabs, the tool-page breadcrumb, and the homePage.documents/data + modalities labels across all 21 locales. URL slugs are unchanged for link stability.
* feat(compress-pdf): add quality and target-size compression modes
Mirror the image Compress tool: a quality slider (1-100) and a target file size, replacing the screen/ebook/printer preset. Adds gsCompressPdfQuality to doc-engine (quality maps to image downsample DPI, the dominant size lever for PDFs); target-size binary-searches the DPI for the highest quality under the target. The frontend reuses the shared CompressControls component, so no new translation strings are needed.
* feat(ocr-pdf): show the PDF preview and extracted text side by side
ocr-pdf fell back to the image viewer, which cannot render a PDF, so the right pane showed 'Preview not available' and the extracted text was only a download. It now uses a custom results view (custom-results display mode) rendering the input PDF via pdf.js (DocumentView gains an inputOnly prop, since the tool's output is a .txt) next to the extracted OCR text, with a copy button.
* feat(ui): link the modality breadcrumb to its tools tab
The modality segment of the tool breadcrumb (PDF, Image, Video, Audio, Files) is now a link to /?modality=<tab>. The home page reads the param, activates the matching tab, and cleans the URL, so it returns to the existing Tools page filtered to that modality without a new page. Handles the file modality whose tab key is 'data'.
* feat(circle-crop): add zoom/offset framing, border, background, and output size
Upgrade the circle-crop tool from a bare centered crop into a framing and
styling tool. New settings (all backward-compatible with the old empty
payload):
- zoom (1-5x) plus offsetX/offsetY (0-1) to control how tight the circle is
and where it sits in the source image
- borderWidth (0-200px) plus borderColor for an optional ring
- background: transparent (clear corners) or a hex fill
- outputSize for a square output; omitted keeps native size
The settings panel gains an inline draggable circular preview that mirrors
the framing live, a zoom slider, a border slider with color, a
transparent/color background toggle, and an output-size field. Adds an
integration test covering output size, border, and a solid background.
* feat(image-tools): flesh out five thin tools (gif-webp, histogram, favicon, color-palette, lqip)
Tier A of the image-tool depth pass. Each of these was as bare as the old
circle-crop (empty settings, opaque or invisible output). Now:
- gif-webp: quality, lossless, and resize-percent controls; shows before/after size
- histogram: returns full per-channel bins + stats; the settings panel renders an
inline interactive histogram with R/G/B/Luma toggles, linear/log scale, and a
mean/median/stdev readout (server PNG still downloadable)
- favicon: background fill, padding, corner-radius, theme color, and a per-size
checklist, with a live preview grid; the route applies the styling and honors
the size filter
- color-palette: count (2-16) and hex/rgb/hsl format controls, median-cut
extraction, a palette strip, and CSS/JSON export
- lqip-placeholder: blur/pixelate/solid strategies, format and quality; the
output panel now surfaces the data URI with copy plus HTML/CSS snippets and a
preview (previously the deliverable was never shown)
Also expose resultPayload from useToolProcessor so a tool can render the route's
extra result fields (histogram bins, lqip data URI) in its own panel. Updates the
five integration tests to cover the new settings.
* feat(image-tools): deepen five thin tools (duotone, vignette, pixelate, background-replace, blur-background)
Tier B of the image-tool depth pass.
- duotone: preset palettes, an intensity slider that blends the duotone with
the original, and a true live duotone preview (a self-contained grayscale +
lighten/darken overlay so the pane filter cannot wash it out)
- vignette: radius, softness, roundness, and center-x/y controls driving a
rebuilt radial gradient, with a matching live overlay
- pixelate: a selection mode that exposes the route's region support via a
draggable box over the image plus width/height sliders, so a face or plate
can be pixelated in isolation
- background-replace: gradient backgrounds, edge feather, and webp output on top
of the existing solid color; now shown before/after
- blur-background: edge feather and webp output; now shown before/after
The live previews for duotone and vignette needed onImageStyle to mount the
overlay branch in image-viewer. The duotone intensity blend and both AI tools'
edge feather were rewritten to splice the alpha channel through raw buffers;
joinChannel did not reliably re-tag the merged channel as alpha and a
raw-without-encoder buffer broke the next decode. Updates the five integration
tests.
* fix(data): rename Files modality to Data + 20 Data-tool bug fixes (#247)
* fix(ui): restore the Data modality name (revert Files rename)
The 'file' modality reverts to the 'Data' label in modality.ts, the home-page tab, and the tools.data + documentsAndFiles i18n keys across all 21 locales. The separate Documents to PDF rename is kept. The URL slug was already /data, so name and slug realign; the tool breadcrumb follows modality.ts automatically.
* fix(create-zip): require at least two files before enabling submit
create-zip enabled its submit button with a single file, but the backend rejects fewer than two files ('Zipping needs at least two files'), producing a 422 error. Gate the button on files.length >= 2 to match the sibling merge-csvs tool. Found during the Data-modality QA sweep.
* fix(data): resolve 17 bugs found in a deeper Data-tool review
Crashes (threw an internal error on otherwise-valid input):
- csv-json: a primitive JSON array like [1,2,3] threw "Unable to serialize"; now a clear error.
- json-xml: a null or primitive JSON root crashed the XML builder; now a clear 4xx.
- yaml-json: an empty or comment-only YAML returned undefined and threw on Buffer.from; now emits null.
Data loss / wrong output:
- csv-json: nested objects rendered as "[object Object]" (now serialized to JSON); heterogeneous objects dropped columns (now the union of all keys).
- xml-to-csv: leaked fast-xml-parser markers ("@_" on attributes, "#text") into CSV headers (now cleaned); a single-record XML failed to tabulate (now a 1-row table); heterogeneous records dropped columns (now the union of all keys).
- csv-excel: xlsx date cells were rendered in the server timezone via Date.toString (now ISO 8601, round-trippable).
- create-zip and extract-zip: filename/basename collisions overwrote zip entries and silently lost a file; dedup now checks generated names and guarantees uniqueness.
- chart-maker: negative values produced invalid/degenerate SVG that Sharp silently dropped; now rejected with a clear message.
Empty output / validation:
- split-csv: a header-only CSV produced an empty zip; now errors with "No data rows to split".
- extract-zip: a directory-only zip produced an empty zip; now errors with "No extractable files found".
- create-zip and merge-csvs: a single-file request fell through to the worker and returned 422; the factory now supports minInputs and returns 400 pre-enqueue.
UI:
- review-panel: the result card showed "Saved +X%" when the output grew; the savings row now appears only when the file is actually smaller (Original/Processed sizes always shown).
Found via two adversarial code-review passes over the 10 Data routes. All 24 fix + regression checks pass against a fresh Docker stack on :1359.
* fix(data): clean 400 for unsafe-zip entries; drop header on split keepHeader=false
- tool-factory: add an opt-in preValidate hook that runs after input prep and
before enqueue. Throwing InputValidationError there returns its statusCode
(400) instead of the worker's generic 422. BullMQ loses the error class across
the job boundary, so InputValidationErrors thrown in the worker cannot be
mapped to their status; pre-enqueue validation can.
- extract-zip: validate entry paths via preValidate, rejecting path-traversal
and absolute-path archives (and unreadable/corrupt zips) with a clear 400. The
processV2 guards remain as defense-in-depth for the pipeline/batch path.
- split-csv: keepHeader=false now drops the header (parts contain only data
rows) instead of keeping it as the first data row of part-1.
Verified against a fresh Docker stack: unsafe / absolute / corrupt zips -> 400,
normal zip still 200; split keepHeader=false drops the header while true repeats
it in each part. No regressions across 51 fix + scenario checks.
* feat(image-tools): deepen image-pad and sprite-sheet, fix sprite-sheet multi-file submit
Tier C of the image-tool depth pass.
- image-pad: a custom W:H ratio alongside the presets, a background mode
(solid color, transparent, or an Instagram-style blurred cover fill), and an
extra padding margin. The settings panel gains a real live preview of the
padded canvas (it previously declared live-preview but rendered nothing) via
onImageStyle + onImageOverlay.
- sprite-sheet: PNG/WebP/JPEG output with a quality control, and the coordinate
map it already computes is now returned and surfaced as Copy CSS (per-frame
background-position rules) and Copy JSON exports.
Also fix a pre-existing sprite-sheet bug: with more than one image the panel
called processAllFiles, fanning out to the per-file batch route (422). It now
calls processFiles, which packs all images into a single sheet request (it is a
MULTI_FILE tool). Updates both integration tests.
* fix(media): preserve source sample rate after loudnorm (#243)
ffmpeg's loudnorm filter runs internally at 192 kHz and emits 192 kHz
unless the chain resamples back. normalize-audio and video-loudnorm
therefore produced 192 kHz output (4.3x larger files) regardless of the
input rate. Append aresample to restore the input's sample rate.
runMediaTool now exposes the input audio sample rate to its args callback.
* fix(color-palette): collapse solid-color images to one swatch
The median-cut bucket selector started bestRange at -1, so a uniform bucket
(range 0) still satisfied the > comparison and kept splitting, yielding N
identical swatches for a solid-color image. Start at 0 so only buckets with
real color spread are split.
* fix(lint): annotate implicit-any lets in saml and user-files
biome noImplicitAnyLet flagged the bare let in saml.ts (profile) and user-files.ts (stream); add derived type annotations (type-only, no behavior change). Latent on main via the turbo lint cache; surfaced when the Data changes busted the apps/api lint cache.
Resolved conflict in worker.ts: kept remote refactored worker
(v2 process, scratch paths, extra outputs, metrics) and re-applied
the auto-save removal from the feature branch.
* feat(tracing): add OpenTelemetry dependencies and --import preload flag
* feat(enterprise): add distributed_tracing feature gate
* feat(tracing): add SDK bootstrap with enterprise gating
* fix(tracing): correct test coverage for enterprise-unavailable path and prevent double-init
Test 2 now mocks @snapotter/enterprise to throw an import error, exercising
the catch block in the preload. Test 3 imports with no endpoint so the preload
is a no-op, avoiding leaked SDK from double-initialization. Added idempotency
guard to initTracing() as a safety net.
* feat(tracing): add Pino trace mixin and shared logger
When OTel tracing is active, every Pino log line now includes traceId,
spanId, and traceFlags fields for log-to-trace correlation. The mixin
is a no-op when no SDK is registered (community users).
* feat(tracing): add _otel to ToolJobData and inject trace context at enqueue
Add optional _otel carrier field to ToolJobData for W3C trace context
propagation across BullMQ job boundaries. When an active OTel span exists,
propagation.inject() writes traceparent/tracestate into the job data before
queue.add(). When no SDK is registered (community edition), the carrier
stays empty and _otel remains undefined -- zero overhead.
* feat(tracing): extract trace context and create spans in BullMQ worker
* feat(tracing): inject trace context into Python sidecar calls
* feat(tracing): add trace context extraction to Python sidecar
* feat(tracing): add shutdownTracing to graceful shutdown sequence
* feat(tracing): enrich HTTP spans with tool_id and user_id attributes
* docs: add OpenTelemetry env var documentation to .env.example
* test(tracing): add lifecycle integration tests for trace propagation
* fix(tracing): inject trace context into pipeline and batch flow jobs
* fix(tracing): add sidecar.execute Node-side span and remove unnecessary comment
Wraps PythonDispatcher.run() with a sidecar.execute span on the Node
side so traces show the full round-trip (Node span -> Python span).
Also removes an obvious comment from logger.ts.
- Remove obsolete fullscreen redirect test (route deleted in Task 3)
- Update dropzone test strings to match new i18n values
- Fix categories count in i18n locale test (12 -> 24)
- Add chart-maker to no-dropzone exempt list in tool-registry test
- Auto-fix import ordering (Biome) in popular tools route and landing config
Zero client JS, 165 static pages, self-hosted fonts, Otter Orange
design system. Includes tool SEO data for all 157 tools, enterprise
page, and updated e2e landing tests.
Move user resolution logic (match by externalId, auto-link by email,
auto-create with user limit check) into a shared module that both
OIDC and SAML callbacks can use. Includes sanitizeUsername and
findUniqueUsername helpers. Preserves all existing OIDC behavior
and audit events.
- Use /opt/venv directly when --entrypoint bash bypasses entrypoint.sh
- Use sys.executable for all pip calls (not bare pip)
- Override entrypoint in CI workflow to avoid startup banner
- Fix Biome formatting (template literals, try/catch blocks)
Add isToolAuditEnabled() helper that checks the auditToolOperations
DB setting (off by default) or falls back to the enterprise
audit_export feature flag. The createToolRoute factory now emits a
TOOL_EXECUTED audit entry on successful tool execution when enabled,
using a fire-and-forget pattern so a failed audit write never blocks
the tool response.
Static data-driven test covering 90 tools with 551 variations. For each
tool that has configurable settings, every enum value, boundary number,
and boolean flag is exercised through the HTTP endpoint. The test asserts
no 500s occur and validates response shape for 200/202 results.
Coverage spans image (29 tools), video (20), audio (15), PDF/document
(21), and data (5) modalities.
Closes#71. Several error paths in the worker could leak internal
filesystem paths (/tmp/workspace, /data/ai/venv, /app) through SSE
frames, resultPayload objects, and Redis batch-error lists. The
existing stripInternalPaths call at worker.ts line 345 only covered
the single-file processToolJob catch block.
Wrapped 6 additional call sites:
- processPipelineStep: prevError from DB and catch errorMsg
- processPipelineFinalize: composed errorMsg reaching SSE,
recordChildOutcome, and resultPayload
- processBatchChild: catch error reaching recordChildOutcome
and resultPayload
- processBatchFinalize: manifest errorMsg from DB rows
- recordChildOutcome (batch-progress.ts): defense-in-depth
strip before Redis rpush
Added 10 unit tests for stripInternalPaths covering /tmp, /data,
/app, /opt, /home, /workspace, multi-path messages, safe passthrough,
and pipeline-step wrapping.
On 1.x-to-2.0 upgrade the /data/ai/venv already existed with stale
packages, so the entrypoint skipped the cp from /opt/venv and 2.0
Python tools failed with ImportError (#85).
Dockerfile now writes a SHA-256 of `pip freeze` to /opt/venv/.venv-version
at build time. The entrypoint compares that stamp against the copy on the
volume; on mismatch it does a clean nuke + re-copy and resets
installed.json so the UI correctly shows bundles as needing reinstall.
Models in /data/ai/models survive the refresh, so bundle reinstall only
reruns pip (model downloads are idempotent).
Backward compatible: if the image has no stamp (old build), the
entrypoint falls through to the existing skip-if-exists behavior.
