SnapOtter/docs/superpowers/specs/2026-05-13-restore-photo-quality-overhaul-design.md

Restore Photo Quality Overhaul

Date: 2026-05-13 Status: Draft Scope: Fix critical quality issues in the restore-photo pipeline, informed by diagnostic testing on real damaged photos Supersedes: docs/specs/2026-04-15-restore-photo-design.md (approved but unimplemented; this spec is scoped to the subset of changes backed by diagnostic evidence)

---

Diagnostic Evidence

Full pipeline was run on 4 sample images with intermediate outputs saved at each step. Key findings:

Image	Pixels	Mask Coverage	Actual Damage	Result
images2.jpg	290x174	68.7%	~3%	Faces erased, gray blobs
images.jpg	188x267	30.6%	~1%	Face structure altered
woman-baby1.webp	400x277	23.2%	~6%	Baby face plasticky, scratches remain
ai-old-photo...webp	768x513	6.4%	~5%	Reasonable but scratches still visible

Root causes identified (ranked by impact):

1. Scratch detection massively over-detects on small images (4-15x worse than reality)
2. LaMa at 512x512 can't fully remove scratches even when correctly detected
3. CodeFormer over-smooths small faces (< 80px)
4. NLMeans default strength too high (40 maps to h=7.8, over-smooths compressed images)

Diagnostic outputs saved at /tmp/restore-diagnostic/ with per-step intermediate images.

---

Changes

1. Scratch Detection Rewrite (restore.py :: detect_scratches)

Current: 4 angles (0/45/90/135), 3 kernel sizes, fixed threshold per mode, no filtering.

New:

• 8 angles: 0, 22.5, 45, 67.5, 90, 112.5, 135, 157.5 degrees
• Rotation-matrix kernels: cv2.getRotationMatrix2D + cv2.warpAffine for accurate arbitrary-angle line kernels (replaces hard-coded pixel patterns)
• Image-size-adaptive kernel count: for images with shortest side < 300px, use only 2 kernel sizes (max kernel capped at base_dim // 15). For >= 300px, use 3 sizes as before
• Pre-filtering: light bilateral filter (d=5, sigmaColor=50, sigmaSpace=50) before CLAHE to suppress JPEG/WebP compression artifacts
• Adaptive threshold: use cv2.THRESH_BINARY + cv2.THRESH_OTSU per combined morphological response map instead of a fixed threshold. This auto-adapts to each image's contrast characteristics. If Otsu selects a threshold < 60, skip detection entirely (image has no significant scratches; low Otsu = unimodal histogram = no damage signal)
• Connected component filtering: after initial detection, run cv2.connectedComponentsWithStats. Keep a component if:
  • Area >= 20 pixels AND (elongation >= 2.5 OR area >= 200 pixels)
  • Elongation = max(bbox_w, bbox_h) / max(min(bbox_w, bbox_h), 1)
  • Reject components with area > 5% of total image area (too large to be a scratch)
• Coverage cap: after filtering, compute total mask coverage. If it exceeds 15%, compute the morphological response intensity at the 85th percentile of masked pixels and re-threshold at that intensity, keeping only the strongest 15% of detections. Log a warning. This prevents the catastrophic "erase everything" failure mode
• Remove sensitivity parameter: no more light/medium/heavy. The adaptive Otsu threshold + filtering handles all cases

Removed: _make_line_kernel() (replaced by _make_line_kernel_rotated())

2. LaMa Inpainting Resolution Fix (restore.py :: inpaint_damage)

Current: Resize entire image to 512x512, process, resize back. Destroys detail and distorts aspect ratio.

New two-path approach:

Small images (both dimensions <= 512px):

• Pad image to 512x512 using cv2.copyMakeBorder with BORDER_REFLECT_101 (no resize, no aspect distortion)
• Pad mask identically (padded area = mask 0, no damage)
• Run single-pass LaMa at 512x512
• Crop result back to original dimensions
• Feathered composite with original

Large images (any dimension > 512px):

• Process in overlapping 512x512 tiles (stride = 384, overlap = 128)
• For edge tiles that extend past the image boundary: pad with BORDER_REFLECT_101
• Only process tiles where mask has > 0 damaged pixels (skip clean tiles)
• Blend tiles using raised-cosine window to eliminate seams
• Feathered composite with original (only replace masked areas)

Improved mask dilation:

• Current: MORPH_ELLIPSE(3,3), 1 iteration = 3px dilation
• New: MORPH_ELLIPSE(5,5), 2 iterations = ~8px dilation
• This ensures scratch edges are fully covered for cleaner inpainting

Improved feathering:

• Current: feather_r = max(3, min(w,h) // 200) = 3px for most images
• New: feather_r = max(5, min(w,h) // 100) = at least 5px, scales better

3. Face Enhancement Guard (restore.py :: enhance_faces)

Current: Minimum face size 24x24px. CodeFormer runs at full strength on all faces.

New:

• Minimum face size raised to 48x48px (skip smaller faces entirely)
• For faces 48-120px: clamp fidelity to max(fidelity, 0.85) to prevent over-smoothing
• For faces > 120px: use user's fidelity setting as-is

4. Lower Default Denoise Strength

Current: denoiseStrength default = 40, maps to NLMeans h = 7.8

New: denoiseStrength default = 25, maps to NLMeans h = 6.0. Still removes grain but doesn't wash out small compressed images. Users can increase manually.

5. Remove Mode System

API: Remove mode: "auto" | "light" | "heavy" from Zod schema. Zod strips unknown fields by default, so old clients sending mode will work (it's silently ignored).

Python: Remove mode parameter from main(). The adaptive Otsu threshold replaces the fixed threshold that mode controlled.

Frontend: Remove 3-button mode selector from RestorePhotoControls.

6. Add Colorize Strength Control

API: Add colorizeStrength: z.number().min(0).max(100).default(85) to Zod schema.

Python: colorize_bw() already accepts intensity parameter. Pass colorizeStrength / 100 from main.

Frontend: Add slider (0-100, step 5) below Auto-Colorize checkbox, visible when colorize is checked. Same nested styling as Face Fidelity slider.

i18n: Add colorizeStrength: "Colorize Strength" to en.ts.

7. TypeScript Bridge Update (restoration.ts)

• Remove mode from RestorePhotoOptions
• Add colorizeStrength?: number

---

Files Changed

File	Change
packages/ai/python/restore.py	Detection rewrite, tiled LaMa, face guard, remove mode, colorize strength passthrough
packages/ai/src/restoration.ts	Remove mode, add colorizeStrength to RestorePhotoOptions
apps/api/src/routes/tools/restore-photo.ts	Remove mode from both Zod schemas, add colorizeStrength, lower denoiseStrength default
apps/web/src/components/tools/restore-photo-settings.tsx	Remove mode selector, add colorize strength slider, update default denoise to 25
packages/shared/src/i18n/en.ts	Add colorizeStrength key
tests/unit/ai/restoration.test.ts	Remove mode tests, add colorizeStrength tests
tests/integration/restore-photo.test.ts	Remove mode tests, add colorizeStrength validation
tests/e2e/restore-photo.spec.ts	Remove mode UI test, add colorize strength slider test

---

What This Spec Does NOT Include

The following items from the earlier approved spec (2026-04-15) are deferred. They add model downloads and complexity without diagnostic evidence that they're needed:

• Stable Diffusion ONNX inpainting (~1.7 GB): LaMa with tiled inference at native resolution should be sufficient. If quality is still lacking after this fix, SD-inpainting can be added as a follow-up
• NAFNet neural denoising (~68 MB): NLMeans at lower strength is adequate. NAFNet is a future upgrade
• CT2 colorization (~850 MB): DDColor with adjustable strength is good enough
• GFPGAN fallback: CodeFormer with the face size guard should handle all cases
• Real-ESRGAN super-resolution step: Separate from restoration quality; can be added independently
• New superResolution setting: Deferred with the SR step

These can each be added incrementally in future PRs if testing shows they're needed.

---

Testing

Unit (Vitest)

• API schema rejects mode field gracefully (stripped, not errored)
• API schema accepts colorizeStrength with default 85
• denoiseStrength default is 25
• colorizeStrength out of range (101, -1) rejected

Integration (Vitest)

• POST with colorizeStrength: 50 accepted
• POST with old mode: "auto" doesn't error (backward compat)

E2E (Playwright)

• Mode selector buttons no longer present
• Colorize strength slider appears when Auto-Colorize is checked
• Default denoise strength shows 25

Manual verification

• Run pipeline on all 4 diagnostic sample images
• Compare before/after at each step
• Verify: mask coverage < 15% on all samples, scratches removed, faces natural, no blurriness

---

Success Criteria

The fix is successful when all 4 diagnostic images produce visibly better results than current:

1. images2.jpg: faces preserved (not erased), scratches reduced
2. images.jpg: face unchanged (minimal actual damage detected)
3. woman-baby1.webp: baby face natural (not plasticky), scratches removed
4. ai-old-photo...webp: diagonal scratch fully removed, face natural