feat: add Phase 4 AI tools with Python bridge and 6 new tools

Add Python bridge (packages/ai/src/bridge.ts) that calls Python scripts
via child_process with venv-first fallback to system python3. Implements
6 AI-powered tools:

- Remove Background: rembg-based with U2-Net/IS-Net models
- Image Upscaling: Real-ESRGAN with Lanczos fallback
- OCR/Text Extraction: Tesseract + PaddleOCR engines
- Face/PII Blur: MediaPipe face detection with configurable blur
- Object Eraser: LaMa inpainting with mask-based input
- Smart Crop: Sharp attention-based entropy cropping (no Python needed)

Each tool includes: Python script, TypeScript wrapper, API route,
and React settings component. All Python scripts handle ImportError
gracefully with clear installation messages.
This commit is contained in:
Siddharth Kumar Sah
2026-03-22 04:31:49 +08:00
parent a8cc611eb2
commit 5524939b6f
30 changed files with 1880 additions and 2 deletions
+33
View File
@@ -0,0 +1,33 @@
import { z } from "zod";
import sharp from "sharp";
import type { FastifyInstance } from "fastify";
import { createToolRoute } from "../tool-factory.js";
const settingsSchema = z.object({
width: z.number().int().positive(),
height: z.number().int().positive(),
});
/**
* Smart crop using Sharp's attention-based strategy.
* Uses entropy/saliency detection to find the most interesting region.
* No Python needed.
*/
export function registerSmartCrop(app: FastifyInstance) {
createToolRoute(app, {
toolId: "smart-crop",
settingsSchema,
process: async (inputBuffer, settings, filename) => {
const result = await sharp(inputBuffer)
.resize(settings.width, settings.height, {
fit: "cover",
position: sharp.strategy.attention,
})
.png()
.toBuffer();
const outputFilename = filename.replace(/\.[^.]+$/, "") + "_smartcrop.png";
return { buffer: result, filename: outputFilename, contentType: "image/png" };
},
});
}