Closes#17, #18, #19, #31, #32, #33, #34
Format preservation (#17, #18, #19):
- Add resolveOutputFormat to rotate, resize, text-overlay, watermark-text,
border, replace-color, blur-faces, upscale, erase-object, restore-photo
- Alpha-aware fallback: border with corner radius/shadow and replace-color
with makeTransparent fall back to PNG for non-alpha formats (JPEG)
- Python sidecar tools (blur-faces, upscale, erase-object) now convert
PNG output back to input format, matching restore-photo/colorize pattern
- Upscale and erase-object default to "auto" format detection instead of PNG
Dispatcher stability (#31, #32):
- Add gc.collect() and torch.cuda.empty_cache() after each dispatcher request
- Add configurable max_requests (default 50) for periodic dispatcher restart
- Add exponential backoff to dispatcher crash recovery in bridge.ts
- Circuit breaker: 5 crashes within 60s permanently disables dispatcher
- Reset crash counter on successful dispatcher startup
Health & security (#33, #34):
- Export getDispatcherStatus() from @snapotter/ai with running/ready/failed/
gpu/pid/consecutiveCrashes fields
- Admin health endpoint now includes full dispatcher status
- Add pip-audit job to CI workflow for Python dependency scanning
- Bump APP_VERSION to 1.15.11 (was hardcoded at 1.15.9, causing
health endpoint to report wrong version in Docker images)
- Fix cpu_fallback_packages() splitting --index-url into separate
pip install arguments, breaking torch install on CPU-only amd64
Code fixes:
- Sidebar state bleed: reset file store on HomePage mount
- restore-photo: raise error instead of silently skipping colorize
when DDColor model missing
- PaddleOCR OOM: cap input images to 2048px before OCR inference
- Torch CPU optimization: use --index-url .../whl/cpu on CPU nodes
Test fixes:
- upscale: add exact:true to scale factor button locators
- smart-crop: add exact:true to "Pad to square" locator
- colorize: use regex for model button names (Best/Balanced/Fast)
- enhance-faces: use .first() for ambiguous percentage display
- passport-photo: fix DPI locator, .or() compound, generate fallback
- people: update maxUsers assertions for unlimited (0) default
- automate: "Save Pipeline" → "Save" matching actual button text
- tools.test: add resize to Sharp mock chain for OCR tests
- Add enable_mkldnn=False to PaddleOCR constructor to bypass PaddlePaddle
3.3+ OneDNN/PIR crash on CPU-only systems
- Add 25MP and 75% max-reduction guard to seam carving with clear error
messages instead of silent timeout/crash
- Replace barcode/QR AVIF test fixtures with actual scannable codes
(old fixtures did not contain real barcodes)
- Add Cloudflare Pages deployment for landing page (snapotter.com) and
docs (docs.snapotter.com)
- Create deploy-landing.yml and update deploy-docs.yml workflows
- Update CI to ignore apps/landing/** paths
- Fix logo transparency (remove white background) across all apps
- Recreate social-preview.png with SnapOtter branding
- Update all docs URLs from GitHub Pages to docs.snapotter.com
- Update VitePress config: light theme default, fix llms.txt paths
- Add .vitepress/cache/ and .env.* to gitignore
When model is set to "auto", CodeFormer failure previously threw an
error telling users to manually switch to GFPGAN. Now it falls back
to GFPGAN automatically, matching the graceful degradation pattern
already used in OCR.
1. split batch 404: register split tool in batch registry via
registerToolProcessFn() so /api/v1/tools/split/batch works
2. CodeFormer crash: inference_app() expects a file path, not a numpy
array. Save to temp file before calling, read result back.
3. OCR fallback chain: fix case-sensitive "Segmentation fault" match
that prevented PaddleOCR crash from triggering Tesseract fallback.
Also add "process crashed" check. Upgrade ARM paddlepaddle to >=3.2.1.
4. blur-faces large images: downscale to 1920px max before MediaPipe
detection, scale coordinates back. Also add rotation retry for
portrait-oriented images where BlazeFace misses faces. Applied to
detect_faces.py, enhance_faces.py, and restore.py.
5. color-adjustments tool ID: fix mismatch in index.ts registration
array (was "color-adjustments", should be "adjust-colors").
When the Python dispatcher crashes and bridge.ts retries via per-request
spawning, the shim from dispatcher.py isn't loaded. basicsr then fails
importing torchvision.transforms.functional_tensor (removed in v0.17).
Adding the shim directly to both scripts ensures they work regardless
of whether they run through the dispatcher or standalone.
- Add safe_onnx_session() to gpu.py with graceful CUDA EP → CPU fallback
- Replace bare ort.InferenceSession() calls across colorize, restore, inpaint, remove_bg
- Add libcublas-12-6 to production Dockerfile for ONNX Runtime CUDA EP
- Add skipIfFeatureNotInstalled guards to remove-bg, blur-faces, smart-crop, ocr, noise-removal e2e specs
- Add AI tool install prompt detection in tools-all.spec.ts
- Add smart-crop to PYTHON_SIDECAR_TOOLS so frontend shows install prompt correctly
- Create Dockerfile.test.dockerignore to include tests/ in test image builds
- Add libheif-examples and exiftool to Dockerfile.test for HEIC and metadata tests
- Regenerate visual regression baselines for Docker/Linux and skip on non-Docker platforms
- Add 8 new E2E specs for AI tools (upscale, enhance-faces, colorize,
restore-photo, erase-object, smart-crop, passport-photo, red-eye-removal)
closing all HIGH/MEDIUM coverage gaps from the test matrix audit
- Fix ensureAiDirs() crash on non-Docker environments by gating on
isDockerEnvironment() — prevents ENOENT when /data doesn't exist
- Bump torch 2.6.0→2.7.0 and torchvision 0.21.0→0.22.0 in feature
manifest for broader Python version compatibility
- Add Python 3.14 version guard warning in install_feature.py
- Remove duplicate torchvision shims from upscale.py and enhance_faces.py
(dispatcher.py already handles this at startup)
- Remove orphaned tools.batch i18n key and dead pipeline-builder filter
- Regenerate 4 visual regression baselines for current UI state
- Add data-testid to passport-photo generate button for E2E testability
The torchvision compatibility shim for basicsr 1.4.2 was missing the
parent-package binding and only proxied a single attribute, causing
upscale and enhance-faces to fail at import time. The fix adds a
__getattr__ proxy for all attributes, binds the shim to the parent
package, and installs it in the dispatcher at startup for defense-in-depth.
Also removes unused anyInstalling variable, redundant `as any` cast,
and applies Biome formatting fixes across the codebase.
- Fix NameError in restore.py: face enhancement loop used undefined
variable `i`, now uses enumerate()
- Fix gpu.py ONNX fallback: previous smoke-test with empty bytes
always raised, making GPU detection unreachable via the ONNX path.
Now uses nvidia-smi hardware check after confirming CUDA EP is
compiled in — works on Linux, Windows, and gracefully fails on macOS
- Fix cpu_fallback_packages stripping CUDA-specific index URLs when
replacing paddlepaddle-gpu with paddlepaddle for CPU-only systems
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pin torch==2.6.0+cu126 and torchvision==0.21.0+cu126 in feature
manifest to prevent NCCL symbol mismatch on CUDA 12.6 base images
- Move lpips after torch in install order to prevent wrong version
resolution from PyPI
- Add einops to upscale-enhance common deps (required by SCUNet)
- Update cpu_fallback_packages to handle multi-package CUDA torch
entries on amd64 without GPU
- Fix gpu.py ONNX CUDA detection: replace hardcoded .so path with
cross-platform session smoke-test
- Fix os.dup(1) crashes on Windows in upscale, enhance_faces, and
noise_removal by wrapping in try/except with sys.stderr fallback
- Guard top-level numpy/cv2 imports in colorize.py and restore.py
with helpful error messages
- Add weights_only=False fallback for torch.load in noise_removal
- Fix integration tests to accept 501 for uninstalled AI features
and 422 for missing system tools (exiftool, libheif)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add tool-specific suffix to output filenames so downloads don't overwrite originals (batch & single-tool routes)
- Skip deleting shared models when uninstalling a bundle that shares models with another installed bundle
- Auto-detect NVIDIA GPU and swap GPU-only pip packages (onnxruntime-gpu, paddlepaddle-gpu) for CPU equivalents
- Refactor docker-compose with YAML anchors and explicit cpu/gpu profiles
- Add libheif-plugin-x265 to Dockerfile
- Fix install-all queue logic to handle concurrent individual installs and clear stale errors
- Unify playwright docker config to use same test dir with API_URL env var
- Fix flaky e2e selectors, rename Strip Metadata → Remove Metadata, handle collage custom dropzone, improve fallback test image generation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The on-demand feature download system stores models at /data/ai/models/
(set via MODELS_PATH env var), but all Python scripts hardcoded
/opt/models/ as the base path. Each script now reads MODELS_PATH and
falls back to /opt/models for backward compatibility.
Check installed.json before exec()-ing AI scripts so that requests
for uninstalled feature bundles return a structured error instead
of crashing with an ImportError. Also sets U2NET_HOME to the
bundled model directory when present.
Reads the feature manifest, installs pip packages (common + arch-specific),
downloads models in parallel with atomic rename, and writes installed.json.
Includes disk space pre-check, NCCL conflict handling, retry logic, and
progress reporting via stderr JSON lines. Also updates the feature route
to pass manifestPath and modelsDir as CLI arguments.
- Added model mismatch warnings in colorize, enhance-faces, and upscale routes.
- Improved error handling in colorize, enhance_faces, remove_bg, restore, and upscale scripts with detailed logging.
- Updated Dockerfile to align NCCL versions for compatibility.
- Introduced a new full tool audit script to test all tools for functionality and GPU usage.
- Created Playwright E2E tests for GPU-dependent tools to ensure proper functionality and performance.
- Replace [object Object] errors with readable messages across all 20+ API
routes by normalizing Zod validation errors to strings (formatZodErrors)
- Add parseApiError() on frontend to defensively handle any details type
- Add global Fastify error handler with full stack traces in logs
- Fix image-to-pdf auth: Object.entries(headers) → headers.forEach()
- Fix passport-photo: safeParse + formatZodErrors, safe error extraction
- Fix OCR silent fallbacks: log exception type/message when falling back,
include actual engine used in API response and Docker logs
- Fix split tool: process all uploaded images, combine into ZIP with
subfolders per image
- Fix batch support for blur-faces, strip-metadata, edit-metadata,
vectorize: add processAllFiles branch for multi-file uploads
- Docker: LOG_LEVEL=debug, PYTHONWARNINGS=default for visibility
- Add Playwright e2e tests verifying all fixes against Docker container
Run both short-range and full-range MediaPipe models and merge results,
then apply non-maximum suppression to remove duplicate bounding boxes.
Fixes missed faces in group photos where the single-model loop exited
early after the first positive detection.
Three fixes to ensure zero network access after docker pull:
1. rembg model allowlist: validate model parameter against the 7
pre-downloaded models, preventing rembg from attempting to download
unknown models via a raw API call.
2. GFPGAN/CodeFormer auxiliary models: pre-download facexlib's
detection_Resnet50_Final.pth and parsing_parsenet.pth at build time.
These were previously downloaded on first use via basicsr. Symlinks
in /app/gfpgan/weights/ ensure codeformer-pip also finds them.
3. OpenCV colorize models: pre-download the prototxt, caffemodel, and
pts_in_hull.npy so the lightweight OpenCV colorizer fallback works
in addition to the primary DDColor method.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
MediaPipe >= 0.10.30 removed the mp.solutions namespace. This broke
face blur, face enhance, red-eye removal, and photo restoration for
users running newer mediapipe versions (closes#43).
All 5 Python scripts that use mediapipe now try the legacy mp.solutions
API first and fall back to the new mp.tasks API on AttributeError.
Model files (blaze_face_short_range.task, face_landmarker.task) are
pre-downloaded during Docker build into /opt/models/mediapipe/ so the
image works fully airgapped. Local dev auto-downloads to .models/.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
- Old API (mp.solutions.face_mesh) for Docker with mediapipe < 0.10.30
- New API (mp.tasks.vision.FaceLandmarker) for newer mediapipe >= 0.10.30
- Auto-downloads face_landmarker.task model on first use with new API
- Extracted shared landmark index constants and key point extraction
* feat(passport-photo): add passport specs database and tool constants
* feat(passport-photo): add MediaPipe FaceMesh landmark detection script
* feat(passport-photo): add TypeScript bridge for face landmark detection
* feat(passport-photo): add API routes with analyze and generate endpoints
* fix(passport-photo): accept landmarks from request body and fix pixel coordinate conversion
- Generate endpoint now accepts landmarks + imageWidth/imageHeight in request body
instead of re-running AI face detection (makes generate phase instant)
- Fixed bug where normalized landmark coordinates (0-1) were used directly
as pixel values in crop computation - now properly multiplied by imgW/imgH
- Fixed same bug in pipeline process function
* feat(passport-photo): add UI component with live preview and compliance overlay
---------
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
enhance_faces.py relied on implicit PyTorch auto-detection for both
GFPGAN and CodeFormer, bypassing the centralized gpu.py module.
inpaint.py queried ort.get_available_providers() directly, which
reports compiled-in backends rather than actual hardware.
Both tools now go through gpu.py so STIRLING_GPU=false correctly
forces CPU across every AI tool.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Add comprehensive photo restoration tool that chains multiple AI models:
- Scratch/tear/spot detection via morphological analysis (top-hat/black-hat transforms)
- Damage inpainting via LaMa ONNX model (reuses existing infrastructure)
- Face enhancement via CodeFormer ONNX (~377MB, from facefusion/models-3.0.0)
- Noise reduction via OpenCV NLMeans in LAB color space
- Optional B&W auto-colorization via DDColor (reuses existing model)
Settings: 3 restoration modes (Light/Auto/Heavy), individual feature toggles
for scratch removal, face enhancement (with fidelity slider), denoising
(with strength slider), and auto-colorize. Before/after comparison view.
Handles HEIC, HEIF, and all standard formats. Batch processing supported.
No new Python dependencies - reuses onnxruntime, cv2, mediapipe, PIL.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
* feat(shared): add enhance-faces tool definition and i18n strings
* feat(ai): add face enhancement script with GFPGAN and CodeFormer support
Detects faces via MediaPipe dual-model approach, then enhances using
GFPGAN (proven) or CodeFormer (via codeformer-pip) with auto fallback.
Supports strength-based alpha blending with original image.
* feat(ai): add TypeScript bridge for face enhancement
* feat(api): add enhance-faces route with GFPGAN/CodeFormer support
* feat(web): add enhance-faces settings component and register in tool registry
* feat(docker): add CodeFormer dependency and model download
- Add codeformer-pip to both CPU and GPU requirements
- Download CodeFormer model (~375MB) at Docker build time
- Add CodeFormer to smoke test verification
* fix(enhance-faces): address code review findings
- Skip alpha blend for CodeFormer (strength already applied via fidelity weight)
- Hide "only enhance main face" checkbox when Best (CodeFormer) is selected
- Fix sensitivity slider labels (swap More/Fewer faces to match actual behavior)
- Register EnhanceFacesControls in pipeline step settings
- Remove model names from user-facing descriptions
* fix(enhance-faces): fix CodeFormer integration and Docker setup
- Add codeformer-pip install to Dockerfile with --no-deps to avoid numpy 2.x conflict
- Re-pin numpy==1.26.4 after codeformer-pip install
- Pin codeformer-pip==0.0.4 in requirements files
- Broaden auto-mode fallback to catch any Exception from CodeFormer
---------
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
* feat(noise-removal): register tool in shared constants and i18n
* feat(noise-removal): add SCUNet and NAFNet model architectures
* feat(noise-removal): add Python denoising engine with 4 quality tiers
* feat(noise-removal): add TypeScript bridge for Python sidecar
* feat(noise-removal): add frontend settings with 4-tier selector
* feat(noise-removal): register in tool registry and pipeline
* feat(noise-removal): add Fastify API route with Zod validation
* feat(noise-removal): add SCUNet and NAFNet model downloads to Docker build
* test(noise-removal): add to e2e tool page rendering tests
* test(noise-removal): add integration tests for API endpoint
* style: fix biome formatting and import ordering
* fix(noise-removal): use correct model download URLs
NAFNet model is hosted on HuggingFace, not GitHub releases.
Also align SCUNet URL to use the KAIR releases (same as Docker build).
* fix(noise-removal): remove emojis from tier selector, simplify labels
Drop emoji icons from Quick/Balanced/Quality/Maximum buttons. Replace
technical algorithm names with plain descriptions users can understand.
---------
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Add AI-powered photo colorization that converts B&W/grayscale images to
full color using DDColor (ICCV 2023 dual-decoder architecture) via ONNX
Runtime. Includes model selection (Auto/DDColor/Classic), adjustable color
intensity, batch processing, before/after preview, and full HEIC/HEIF support.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Update erase-object pipeline, eraser canvas, and inpainting Python script.
Add LaMa model download script and update Dockerfile for model support.
Update multi-file tool routes for consistency.
Replace the confusing 2-mode smart crop with a clear 3-mode system:
- Subject Focus: Sharp attention/entropy saliency crop with social media presets
- Face Focus: MediaPipe face detection with headshot framing presets
- Auto Trim: Border removal with optional pad-to-square
Adds detectFaces() to AI package, face preset constants, backward
compatibility for old mode names, and comprehensive integration tests.
- Pin PaddlePaddle to 3.0.0 on ARM64 to fix segfault in PIR inference
engine (3.1+ crashes on aarch64 Debian Bookworm)
- Fix text extraction for PaddleOCR 3.4.x result format (rec_texts)
- Add Node.js-level fallback chain (best -> balanced -> fast) when
Python subprocess crashes
- Add multi-image OCR: processes all uploaded files sequentially with
per-file progress and filename headers in combined output
- Convert input images to PNG via Sharp before OCR so HEIC, AVIF, WebP,
TIFF all work transparently
- Implement real auto-detect language using Tesseract multi-lang script
detection (analyzes Unicode ranges for Hangul, CJK, Kana, Latin)
- Default enhance to off (hurts clean digital images)
- Replace Auto/AI/Fast buttons with Fast/Balanced/Best (consistent with other tools)
- Rename "Denoise" to "Noise Reduction" with explanatory subtitle
- Change output format from 3 buttons to dropdown with all formats (PNG, JPG, WebP, AVIF, TIFF, GIF, HEIC, HEIF)
- Add HEIC/HEIF input decoding (was missing unlike other tools)
- Add HEIC/HEIF/AVIF output conversion via Sharp and heif-enc
- Generate browser-compatible WebP preview for non-previewable output formats
- Fix torchvision compatibility shim so Real-ESRGAN actually loads (was silently falling back to Lanczos)
- Fix denoise crash: Image.fromarray() instead of type(img).fromarray()
- Redirect stdout for entire AI pipeline to prevent library messages corrupting JSON output
- Add GFPGAN model download for face enhancement
- Use batch endpoint for multi-file uploads (enables Download All ZIP)
- Fix multi-image: process selected file, not always first
- Fix progress bar: asymptotic fill prevents visual stalling
- Fix slider: write results to captured index, not current selection
- Add model selection (Auto/AI/Fast), face enhancement, denoise
- Add output format (PNG/JPEG/WebP) with quality control
- Add Upscale All for sequential batch processing with queue
- More granular Python progress stages for smoother UX
- Fix temp file leak: clean up preprocessed image in finally block
- Log warning instead of silently swallowing preprocessing failures
- Simplify auto_detect_language to honest default (was a stub that
wasted time loading a model but always returned "en")
Ultra quality (People only):
- BiRefNet-matting ONNX (928MB) for true alpha matting with per-pixel
transparency on hair wisps and fine edges
- Custom rembg session class, zero new Python dependencies
- Model pre-downloaded in Docker build for immediate availability
Quality tier labels: Fast / HD / Max / Ultra (shorter, fits 4-col grid)
Adds a new "Ultra" quality tier for People subject type that uses
BiRefNet-matting (ONNX, 928MB) for true alpha matting instead of
binary segmentation. Produces per-pixel transparency for hair wisps
and fine edges that standard models miss.
- Custom rembg session class loads BiRefNet-matting ONNX from GitHub releases
- Zero new Python dependencies (reuses existing onnxruntime)
- Model pre-downloaded in Docker build alongside existing models
- Ultra option only visible when subject is People
- Falls back to Best when switching to Products/General
Remove Background:
- Two-phase flow: AI removes bg once, then effects adjust instantly
- Blur background effect with real-time CSS preview (portrait mode)
- Drop shadow effect with opacity control
- Gradient backgrounds with presets, custom colors, and angle
- Custom background image upload (including HEIC/HEIF)
- Solid color backgrounds moved from Python to Node.js/Sharp
- Effects-only API endpoint for instant re-renders without AI re-run
- HEIC/HEIF input support (decoded before passing to Python/rembg)
- Passport/ID photo checkbox defaults ON for People subject
- Before/after slider preserved when no effects active
- 15 comprehensive Playwright e2e tests
Color Tools:
- Consolidated 4 tools (brightness-contrast, saturation, color-channels,
color-effects) into single "Adjust Colors" tool
- Added exposure, temperature, tint, hue, sharpness controls
- SVG filter-based live preview for all adjustments
- Backward-compatible URL redirects from old tool paths
Other fixes:
- Favicon tool: download button instead of auto-download
- Batch processing: HEIC filename extension fix
- File store: processedFilename field for proper batch downloads
Replace the Python seam-carving library with caire (esimov/caire v1.5.0),
a Go-based content-aware resize engine that is faster and supports both
shrinking and enlarging via seam insertion.
- Add Go builder stage in Dockerfile to compile caire from source
- Rewrite seam-carving.ts to call caire via execFile (no Python sidecar)
- Remove content-aware-resize from PYTHON_SIDECAR_TOOLS (60s timeout)
- Add new options: blur radius, edge sensitivity, square mode, face detection
- Move content-aware toggle below standard resize in UI (subtler placement)
- Rename "Don't enlarge" to "Limit to original size" with hover tooltip
- Add smooth progress bar for medium-duration tools
- Delete seam_carve.py and remove seam-carving pip dependency
- Update integration tests and visual regression screenshots
SnapOtter