1. split batch 404: register split tool in batch registry via
registerToolProcessFn() so /api/v1/tools/split/batch works
2. CodeFormer crash: inference_app() expects a file path, not a numpy
array. Save to temp file before calling, read result back.
3. OCR fallback chain: fix case-sensitive "Segmentation fault" match
that prevented PaddleOCR crash from triggering Tesseract fallback.
Also add "process crashed" check. Upgrade ARM paddlepaddle to >=3.2.1.
4. blur-faces large images: downscale to 1920px max before MediaPipe
detection, scale coordinates back. Also add rotation retry for
portrait-oriented images where BlazeFace misses faces. Applied to
detect_faces.py, enhance_faces.py, and restore.py.
5. color-adjustments tool ID: fix mismatch in index.ts registration
array (was "color-adjustments", should be "adjust-colors").
When the Python dispatcher crashes and bridge.ts retries via per-request
spawning, the shim from dispatcher.py isn't loaded. basicsr then fails
importing torchvision.transforms.functional_tensor (removed in v0.17).
Adding the shim directly to both scripts ensures they work regardless
of whether they run through the dispatcher or standalone.
- Add safe_onnx_session() to gpu.py with graceful CUDA EP → CPU fallback
- Replace bare ort.InferenceSession() calls across colorize, restore, inpaint, remove_bg
- Add libcublas-12-6 to production Dockerfile for ONNX Runtime CUDA EP
- Add skipIfFeatureNotInstalled guards to remove-bg, blur-faces, smart-crop, ocr, noise-removal e2e specs
- Add AI tool install prompt detection in tools-all.spec.ts
- Add smart-crop to PYTHON_SIDECAR_TOOLS so frontend shows install prompt correctly
- Create Dockerfile.test.dockerignore to include tests/ in test image builds
- Add libheif-examples and exiftool to Dockerfile.test for HEIC and metadata tests
- Regenerate visual regression baselines for Docker/Linux and skip on non-Docker platforms
- Add 8 new E2E specs for AI tools (upscale, enhance-faces, colorize,
restore-photo, erase-object, smart-crop, passport-photo, red-eye-removal)
closing all HIGH/MEDIUM coverage gaps from the test matrix audit
- Fix ensureAiDirs() crash on non-Docker environments by gating on
isDockerEnvironment() — prevents ENOENT when /data doesn't exist
- Bump torch 2.6.0→2.7.0 and torchvision 0.21.0→0.22.0 in feature
manifest for broader Python version compatibility
- Add Python 3.14 version guard warning in install_feature.py
- Remove duplicate torchvision shims from upscale.py and enhance_faces.py
(dispatcher.py already handles this at startup)
- Remove orphaned tools.batch i18n key and dead pipeline-builder filter
- Regenerate 4 visual regression baselines for current UI state
- Add data-testid to passport-photo generate button for E2E testability
The torchvision compatibility shim for basicsr 1.4.2 was missing the
parent-package binding and only proxied a single attribute, causing
upscale and enhance-faces to fail at import time. The fix adds a
__getattr__ proxy for all attributes, binds the shim to the parent
package, and installs it in the dispatcher at startup for defense-in-depth.
Also removes unused anyInstalling variable, redundant `as any` cast,
and applies Biome formatting fixes across the codebase.
- Fix NameError in restore.py: face enhancement loop used undefined
variable `i`, now uses enumerate()
- Fix gpu.py ONNX fallback: previous smoke-test with empty bytes
always raised, making GPU detection unreachable via the ONNX path.
Now uses nvidia-smi hardware check after confirming CUDA EP is
compiled in — works on Linux, Windows, and gracefully fails on macOS
- Fix cpu_fallback_packages stripping CUDA-specific index URLs when
replacing paddlepaddle-gpu with paddlepaddle for CPU-only systems
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pin torch==2.6.0+cu126 and torchvision==0.21.0+cu126 in feature
manifest to prevent NCCL symbol mismatch on CUDA 12.6 base images
- Move lpips after torch in install order to prevent wrong version
resolution from PyPI
- Add einops to upscale-enhance common deps (required by SCUNet)
- Update cpu_fallback_packages to handle multi-package CUDA torch
entries on amd64 without GPU
- Fix gpu.py ONNX CUDA detection: replace hardcoded .so path with
cross-platform session smoke-test
- Fix os.dup(1) crashes on Windows in upscale, enhance_faces, and
noise_removal by wrapping in try/except with sys.stderr fallback
- Guard top-level numpy/cv2 imports in colorize.py and restore.py
with helpful error messages
- Add weights_only=False fallback for torch.load in noise_removal
- Fix integration tests to accept 501 for uninstalled AI features
and 422 for missing system tools (exiftool, libheif)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add tool-specific suffix to output filenames so downloads don't overwrite originals (batch & single-tool routes)
- Skip deleting shared models when uninstalling a bundle that shares models with another installed bundle
- Auto-detect NVIDIA GPU and swap GPU-only pip packages (onnxruntime-gpu, paddlepaddle-gpu) for CPU equivalents
- Refactor docker-compose with YAML anchors and explicit cpu/gpu profiles
- Add libheif-plugin-x265 to Dockerfile
- Fix install-all queue logic to handle concurrent individual installs and clear stale errors
- Unify playwright docker config to use same test dir with API_URL env var
- Fix flaky e2e selectors, rename Strip Metadata → Remove Metadata, handle collage custom dropzone, improve fallback test image generation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The on-demand feature download system stores models at /data/ai/models/
(set via MODELS_PATH env var), but all Python scripts hardcoded
/opt/models/ as the base path. Each script now reads MODELS_PATH and
falls back to /opt/models for backward compatibility.
Check installed.json before exec()-ing AI scripts so that requests
for uninstalled feature bundles return a structured error instead
of crashing with an ImportError. Also sets U2NET_HOME to the
bundled model directory when present.
Reads the feature manifest, installs pip packages (common + arch-specific),
downloads models in parallel with atomic rename, and writes installed.json.
Includes disk space pre-check, NCCL conflict handling, retry logic, and
progress reporting via stderr JSON lines. Also updates the feature route
to pass manifestPath and modelsDir as CLI arguments.
- Added model mismatch warnings in colorize, enhance-faces, and upscale routes.
- Improved error handling in colorize, enhance_faces, remove_bg, restore, and upscale scripts with detailed logging.
- Updated Dockerfile to align NCCL versions for compatibility.
- Introduced a new full tool audit script to test all tools for functionality and GPU usage.
- Created Playwright E2E tests for GPU-dependent tools to ensure proper functionality and performance.
- Replace [object Object] errors with readable messages across all 20+ API
routes by normalizing Zod validation errors to strings (formatZodErrors)
- Add parseApiError() on frontend to defensively handle any details type
- Add global Fastify error handler with full stack traces in logs
- Fix image-to-pdf auth: Object.entries(headers) → headers.forEach()
- Fix passport-photo: safeParse + formatZodErrors, safe error extraction
- Fix OCR silent fallbacks: log exception type/message when falling back,
include actual engine used in API response and Docker logs
- Fix split tool: process all uploaded images, combine into ZIP with
subfolders per image
- Fix batch support for blur-faces, strip-metadata, edit-metadata,
vectorize: add processAllFiles branch for multi-file uploads
- Docker: LOG_LEVEL=debug, PYTHONWARNINGS=default for visibility
- Add Playwright e2e tests verifying all fixes against Docker container
Run both short-range and full-range MediaPipe models and merge results,
then apply non-maximum suppression to remove duplicate bounding boxes.
Fixes missed faces in group photos where the single-model loop exited
early after the first positive detection.
Three fixes to ensure zero network access after docker pull:
1. rembg model allowlist: validate model parameter against the 7
pre-downloaded models, preventing rembg from attempting to download
unknown models via a raw API call.
2. GFPGAN/CodeFormer auxiliary models: pre-download facexlib's
detection_Resnet50_Final.pth and parsing_parsenet.pth at build time.
These were previously downloaded on first use via basicsr. Symlinks
in /app/gfpgan/weights/ ensure codeformer-pip also finds them.
3. OpenCV colorize models: pre-download the prototxt, caffemodel, and
pts_in_hull.npy so the lightweight OpenCV colorizer fallback works
in addition to the primary DDColor method.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
MediaPipe >= 0.10.30 removed the mp.solutions namespace. This broke
face blur, face enhance, red-eye removal, and photo restoration for
users running newer mediapipe versions (closes#43).
All 5 Python scripts that use mediapipe now try the legacy mp.solutions
API first and fall back to the new mp.tasks API on AttributeError.
Model files (blaze_face_short_range.task, face_landmarker.task) are
pre-downloaded during Docker build into /opt/models/mediapipe/ so the
image works fully airgapped. Local dev auto-downloads to .models/.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
- Old API (mp.solutions.face_mesh) for Docker with mediapipe < 0.10.30
- New API (mp.tasks.vision.FaceLandmarker) for newer mediapipe >= 0.10.30
- Auto-downloads face_landmarker.task model on first use with new API
- Extracted shared landmark index constants and key point extraction
* feat(passport-photo): add passport specs database and tool constants
* feat(passport-photo): add MediaPipe FaceMesh landmark detection script
* feat(passport-photo): add TypeScript bridge for face landmark detection
* feat(passport-photo): add API routes with analyze and generate endpoints
* fix(passport-photo): accept landmarks from request body and fix pixel coordinate conversion
- Generate endpoint now accepts landmarks + imageWidth/imageHeight in request body
instead of re-running AI face detection (makes generate phase instant)
- Fixed bug where normalized landmark coordinates (0-1) were used directly
as pixel values in crop computation - now properly multiplied by imgW/imgH
- Fixed same bug in pipeline process function
* feat(passport-photo): add UI component with live preview and compliance overlay
---------
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
enhance_faces.py relied on implicit PyTorch auto-detection for both
GFPGAN and CodeFormer, bypassing the centralized gpu.py module.
inpaint.py queried ort.get_available_providers() directly, which
reports compiled-in backends rather than actual hardware.
Both tools now go through gpu.py so STIRLING_GPU=false correctly
forces CPU across every AI tool.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Add comprehensive photo restoration tool that chains multiple AI models:
- Scratch/tear/spot detection via morphological analysis (top-hat/black-hat transforms)
- Damage inpainting via LaMa ONNX model (reuses existing infrastructure)
- Face enhancement via CodeFormer ONNX (~377MB, from facefusion/models-3.0.0)
- Noise reduction via OpenCV NLMeans in LAB color space
- Optional B&W auto-colorization via DDColor (reuses existing model)
Settings: 3 restoration modes (Light/Auto/Heavy), individual feature toggles
for scratch removal, face enhancement (with fidelity slider), denoising
(with strength slider), and auto-colorize. Before/after comparison view.
Handles HEIC, HEIF, and all standard formats. Batch processing supported.
No new Python dependencies - reuses onnxruntime, cv2, mediapipe, PIL.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
* feat(shared): add enhance-faces tool definition and i18n strings
* feat(ai): add face enhancement script with GFPGAN and CodeFormer support
Detects faces via MediaPipe dual-model approach, then enhances using
GFPGAN (proven) or CodeFormer (via codeformer-pip) with auto fallback.
Supports strength-based alpha blending with original image.
* feat(ai): add TypeScript bridge for face enhancement
* feat(api): add enhance-faces route with GFPGAN/CodeFormer support
* feat(web): add enhance-faces settings component and register in tool registry
* feat(docker): add CodeFormer dependency and model download
- Add codeformer-pip to both CPU and GPU requirements
- Download CodeFormer model (~375MB) at Docker build time
- Add CodeFormer to smoke test verification
* fix(enhance-faces): address code review findings
- Skip alpha blend for CodeFormer (strength already applied via fidelity weight)
- Hide "only enhance main face" checkbox when Best (CodeFormer) is selected
- Fix sensitivity slider labels (swap More/Fewer faces to match actual behavior)
- Register EnhanceFacesControls in pipeline step settings
- Remove model names from user-facing descriptions
* fix(enhance-faces): fix CodeFormer integration and Docker setup
- Add codeformer-pip install to Dockerfile with --no-deps to avoid numpy 2.x conflict
- Re-pin numpy==1.26.4 after codeformer-pip install
- Pin codeformer-pip==0.0.4 in requirements files
- Broaden auto-mode fallback to catch any Exception from CodeFormer
---------
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
* feat(noise-removal): register tool in shared constants and i18n
* feat(noise-removal): add SCUNet and NAFNet model architectures
* feat(noise-removal): add Python denoising engine with 4 quality tiers
* feat(noise-removal): add TypeScript bridge for Python sidecar
* feat(noise-removal): add frontend settings with 4-tier selector
* feat(noise-removal): register in tool registry and pipeline
* feat(noise-removal): add Fastify API route with Zod validation
* feat(noise-removal): add SCUNet and NAFNet model downloads to Docker build
* test(noise-removal): add to e2e tool page rendering tests
* test(noise-removal): add integration tests for API endpoint
* style: fix biome formatting and import ordering
* fix(noise-removal): use correct model download URLs
NAFNet model is hosted on HuggingFace, not GitHub releases.
Also align SCUNet URL to use the KAIR releases (same as Docker build).
* fix(noise-removal): remove emojis from tier selector, simplify labels
Drop emoji icons from Quick/Balanced/Quality/Maximum buttons. Replace
technical algorithm names with plain descriptions users can understand.
---------
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Add AI-powered photo colorization that converts B&W/grayscale images to
full color using DDColor (ICCV 2023 dual-decoder architecture) via ONNX
Runtime. Includes model selection (Auto/DDColor/Classic), adjustable color
intensity, batch processing, before/after preview, and full HEIC/HEIF support.
Co-authored-by: stirling-image <stirling-image@users.noreply.github.com>
Update erase-object pipeline, eraser canvas, and inpainting Python script.
Add LaMa model download script and update Dockerfile for model support.
Update multi-file tool routes for consistency.
Replace the confusing 2-mode smart crop with a clear 3-mode system:
- Subject Focus: Sharp attention/entropy saliency crop with social media presets
- Face Focus: MediaPipe face detection with headshot framing presets
- Auto Trim: Border removal with optional pad-to-square
Adds detectFaces() to AI package, face preset constants, backward
compatibility for old mode names, and comprehensive integration tests.
- Pin PaddlePaddle to 3.0.0 on ARM64 to fix segfault in PIR inference
engine (3.1+ crashes on aarch64 Debian Bookworm)
- Fix text extraction for PaddleOCR 3.4.x result format (rec_texts)
- Add Node.js-level fallback chain (best -> balanced -> fast) when
Python subprocess crashes
- Add multi-image OCR: processes all uploaded files sequentially with
per-file progress and filename headers in combined output
- Convert input images to PNG via Sharp before OCR so HEIC, AVIF, WebP,
TIFF all work transparently
- Implement real auto-detect language using Tesseract multi-lang script
detection (analyzes Unicode ranges for Hangul, CJK, Kana, Latin)
- Default enhance to off (hurts clean digital images)
- Replace Auto/AI/Fast buttons with Fast/Balanced/Best (consistent with other tools)
- Rename "Denoise" to "Noise Reduction" with explanatory subtitle
- Change output format from 3 buttons to dropdown with all formats (PNG, JPG, WebP, AVIF, TIFF, GIF, HEIC, HEIF)
- Add HEIC/HEIF input decoding (was missing unlike other tools)
- Add HEIC/HEIF/AVIF output conversion via Sharp and heif-enc
- Generate browser-compatible WebP preview for non-previewable output formats
- Fix torchvision compatibility shim so Real-ESRGAN actually loads (was silently falling back to Lanczos)
- Fix denoise crash: Image.fromarray() instead of type(img).fromarray()
- Redirect stdout for entire AI pipeline to prevent library messages corrupting JSON output
- Add GFPGAN model download for face enhancement
- Use batch endpoint for multi-file uploads (enables Download All ZIP)
- Fix multi-image: process selected file, not always first
- Fix progress bar: asymptotic fill prevents visual stalling
- Fix slider: write results to captured index, not current selection
- Add model selection (Auto/AI/Fast), face enhancement, denoise
- Add output format (PNG/JPEG/WebP) with quality control
- Add Upscale All for sequential batch processing with queue
- More granular Python progress stages for smoother UX
- Fix temp file leak: clean up preprocessed image in finally block
- Log warning instead of silently swallowing preprocessing failures
- Simplify auto_detect_language to honest default (was a stub that
wasted time loading a model but always returned "en")
Ultra quality (People only):
- BiRefNet-matting ONNX (928MB) for true alpha matting with per-pixel
transparency on hair wisps and fine edges
- Custom rembg session class, zero new Python dependencies
- Model pre-downloaded in Docker build for immediate availability
Quality tier labels: Fast / HD / Max / Ultra (shorter, fits 4-col grid)
Adds a new "Ultra" quality tier for People subject type that uses
BiRefNet-matting (ONNX, 928MB) for true alpha matting instead of
binary segmentation. Produces per-pixel transparency for hair wisps
and fine edges that standard models miss.
- Custom rembg session class loads BiRefNet-matting ONNX from GitHub releases
- Zero new Python dependencies (reuses existing onnxruntime)
- Model pre-downloaded in Docker build alongside existing models
- Ultra option only visible when subject is People
- Falls back to Best when switching to Products/General
Remove Background:
- Two-phase flow: AI removes bg once, then effects adjust instantly
- Blur background effect with real-time CSS preview (portrait mode)
- Drop shadow effect with opacity control
- Gradient backgrounds with presets, custom colors, and angle
- Custom background image upload (including HEIC/HEIF)
- Solid color backgrounds moved from Python to Node.js/Sharp
- Effects-only API endpoint for instant re-renders without AI re-run
- HEIC/HEIF input support (decoded before passing to Python/rembg)
- Passport/ID photo checkbox defaults ON for People subject
- Before/after slider preserved when no effects active
- 15 comprehensive Playwright e2e tests
Color Tools:
- Consolidated 4 tools (brightness-contrast, saturation, color-channels,
color-effects) into single "Adjust Colors" tool
- Added exposure, temperature, tint, hue, sharpness controls
- SVG filter-based live preview for all adjustments
- Backward-compatible URL redirects from old tool paths
Other fixes:
- Favicon tool: download button instead of auto-download
- Batch processing: HEIC filename extension fix
- File store: processedFilename field for proper batch downloads
Replace the Python seam-carving library with caire (esimov/caire v1.5.0),
a Go-based content-aware resize engine that is faster and supports both
shrinking and enlarging via seam insertion.
- Add Go builder stage in Dockerfile to compile caire from source
- Rewrite seam-carving.ts to call caire via execFile (no Python sidecar)
- Remove content-aware-resize from PYTHON_SIDECAR_TOOLS (60s timeout)
- Add new options: blur radius, edge sensitivity, square mode, face detection
- Move content-aware toggle below standard resize in UI (subtler placement)
- Rename "Don't enlarge" to "Limit to original size" with hover tooltip
- Add smooth progress bar for medium-duration tools
- Delete seam_carve.py and remove seam-carving pip dependency
- Update integration tests and visual regression screenshots
Merge CPU, CUDA, and lite Docker images into a single unified image.
One tag (latest) works on all platforms: amd64 (NVIDIA CUDA) and arm64 (CPU).
GPU auto-detected at runtime. All ML models and packages baked in.
Key changes:
- Platform-conditional Dockerfile (nvidia/cuda on amd64, node on arm64)
- tini as PID 1 for proper signal handling
- Fix FILES_STORAGE_PATH data loss bug
- Fix RealESRGAN upscaler (was broken, always fell back to Lanczos)
- Fix PaddleOCR language codes and stdout corruption
- Simplified CI/CD (single build, single tag)
- Expanded model pre-download with verification
- Shutdown timeout, improved health endpoint
- Remove unused lama-cleaner
- Replace OpenCV Haar Cascades with MediaPipe for face detection, using
short-range model first with full-range fallback for better accuracy
- Add auto-orient to remove-background route for EXIF-rotated photos
- Change default background removal model from u2net to birefnet-general-lite
- Fix flaky test by setting SQLite busy_timeout before journal_mode pragma
Co-authored-by: Siddharth Kumar Sah <siddharth123sk@gmail.com>
onnxruntime-gpu reports CUDAExecutionProvider as "available" just
because the library was compiled with CUDA support, even on machines
with no GPU. This made gpu_available() return True incorrectly,
causing upscale.py to try torch.device("cuda") and fall back to
Lanczos instead of running Real-ESRGAN on CPU.
torch.cuda.is_available() actually probes the hardware. Use it as
the single source of truth for GPU detection.
Verified: CUDA image on Apple Silicon (no GPU) now correctly reports
gpu: false and all AI tools run on CPU without crashes.
The STIRLING_GPU=true env var was baked into the :cuda Dockerfile,
which made gpu_available() return True without checking actual
hardware. On machines without a GPU, this would crash upscale.py
(torch.device("cuda") fails) and ocr.py (PaddleOCR use_gpu=True).
Fix: the env var can only disable GPU (set to false/0), never
force-enable it. Hardware detection always runs. Removed the
baked env var from the Dockerfile since it adds no value now.
ashim-hq