* fix(api): prevent a crash when an over-limit upload stream has no consumer yet busboy's "limit" handler destroyed the file stream with an error but never attached its own error listener, relying entirely on whatever consumes part.file downstream to do so. On a fast enough connection (or a fully buffered body, e.g. Fastify inject()), busboy can process enough bytes to hit the size limit before the route handler's receiveUpload() call has attached its own stream listener, leaving the resulting "error" event with zero listeners -- which crashes the whole process by default in Node. Surfaced by tonight's FULL_MATRIX+FUZZ integration run (880 uncaught exceptions, all the same root cause). Reproduces deterministically in isolation; unrelated to this release's actual code delta (file untouched since PR #413, well before the baseline QA pass). Fix: attach a baseline no-op error listener the moment the stream is created, guaranteeing at least one listener always exists. EventEmitter delivers "error" to every registered listener, so the real consumer's own error handling is unaffected. * fix(ai-bundles): rebuild upscale-enhance and photo-restoration to reconcile scipy ABI upscale-enhance and photo-restoration both depend on codeformer-pip, whose transitive closure (basicsr -> realesrgan -> gfpgan) pulls in an unpinned scipy. Both bundles were last built ~June 18-19, before PR #437 added the manifest's `constraints` array (numpy==1.26.4, scipy==1.12.0, etc.) to pin exactly this kind of dependency during bundle builds. Only the ocr bundle was rebuilt after that fix landed. install_feature.py has no pip install step -- it's a raw tarfile extraction with no cross-bundle conflict resolution, so installing OCR alongside either stale bundle left three incompatible scipy versions' files mixed in the same site-packages directory (a compiled _rotation.*.so from one release next to Python files expecting a different release's API), breaking the `upscale` tool and OCR's higher-quality tiers with an ImportError. Rebuilt both bundles for amd64-gpu and arm64-cpu from the current manifest, verified scipy/scikit-learn/scikit-image/pandas all resolve to the pinned versions in the tarballs themselves, then verified end-to-end on real hardware (Mac arm64 CPU and ubuntu_gpu .248 RTX 4070): installing all affected bundles together now yields exactly one version of each constrained package, `upscale` produces correct output, and OCR's balanced/best tiers correctly use PaddleOCR-GPU instead of erroring out. Published the rebuilt tarballs to the public deepsafe/feature-bundles HuggingFace repo and updated this manifest's sha256/sizes to match. Also adds verify-bundle-compatibility.sh: verify-bundle.sh checks each bundle in isolation (a fresh venv per bundle), which is exactly why this shipped twice -- nothing ever checked that bundles built at different times agree once layered into the one shared venv real installs use. The new script installs every bundle for an arch into one venv and asserts each constrained package has exactly one, correct version. Known follow-up (not fixed here, needs separate discussion): uninstalling a bundle only removes its downloaded model weights, never the site-packages it added, so existing installations that already hit this bug have no clean self-service fix via uninstall+reinstall -- they need a full AI-venv wipe. * fix(docker): bake a real rate limit default for the all-in-one one-liner The documented single-container `docker run` install had RATE_LIMIT_PER_MIN=0 (effectively unlimited, ~50k/min) baked in, since only docker-compose.yml carried a hardened override. A self-hoster following the one-liner path got no meaningful throttling anywhere, including auth-adjacent routes with no dedicated per-route limit. Bakes a generous-but-real 1000/min default into the Dockerfile, raises both compose files' fallback to match so the two documented install paths converge on the same posture, and updates the Zod schema default plus docs that quoted the old value. * fix(api): boot log undercounted tool routes by the conversion-preset total The "Tool routes: N active" line logged before registerConversionPresets(app) ran, so it only ever reported the base 158 tools, 83 short of the real 241-tool total. Presets have to register after the base loop (they delegate to each base tool's own processV2), so the fix moves the log line to after that call and has registerConversionPresets return its count instead of reordering the dependency. * fix(ai): forward {info}/{warning} stderr JSON instead of dropping it The dispatcher stderr parser only recognized {ready} and {progress,stage} shaped JSON lines; anything else that parsed as valid JSON (like ocr.py's GPU-to-tesseract downgrade notice, an {"info": ...} line) matched neither branch and fell through silently, never reaching docker logs. Adds explicit {info}/{warning} handling that forwards to console.log/console.warn, same as the existing [prefix]-tagged non-JSON path. * fix(api): fall back to a lower OCR tier when PaddleOCR itself is unusable ocr.ts already retries lower quality tiers on a crashed dispatcher, but the condition only matched crash-style messages (segfault, exited unexpectedly). ocr.py's own ImportError/exception handlers already produce messages telling the caller to use a lower tier (e.g. on the scipy ABI conflict class of bug), but nothing ever acted on them, so a broken PaddleOCR hard-failed with 422 instead of degrading to Tesseract like ocr-pdf effectively does. Broadens the retry condition to also catch PaddleOCR-engine-unusable messages. Note: ocr-pdf's tesseract-only behavior turned out to be an unrelated, pre-existing, deliberate design choice (PaddleOCR segfaults on rasterized PDF pages on arm64), not a graceful-fallback mechanism to copy -- the two tools weren't actually solving the same problem, so this fixes ocr.ts's own gap rather than trying to mirror ocr-pdf.
7.0 KiB
description
| description |
|---|
| Local development setup, commands, code conventions, and how to add a new tool to SnapOtter. |
Developer guide
How to set up a local development environment and contribute code to SnapOtter.
Prerequisites
- Node.js 22+
- pnpm 9+ (
corepack enable && corepack prepare pnpm@latest --activate) - Docker (required for local Postgres + Redis, container builds, and AI features)
- Git
Python 3.10+ is only needed if you are working on the AI/ML sidecar (background removal, upscaling, OCR).
Setup
git clone https://github.com/snapotter-hq/snapotter.git
cd snapotter
docker compose -f docker-compose.dev.yml up -d # start Postgres + Redis
pnpm install
pnpm dev
This starts two dev servers:
| Service | URL | Notes |
|---|---|---|
| Frontend | http://localhost:1349 | Vite dev server, proxies /api |
| Backend | http://localhost:13490 | Fastify API (accessed via proxy) |
Open http://localhost:1349 in your browser. Login with admin / admin. You will be prompted to change the password on first login.
Project structure
apps/
api/ Fastify backend
web/ Vite + React frontend
docs/ VitePress documentation (this site)
packages/
shared/ Constants, types, i18n strings
image-engine/ Sharp-based image operations
media-engine/ FFmpeg spawn + progress parsing
doc-engine/ qpdf, LibreOffice, ghostscript wrappers
ai/ Python sidecar bridge for ML models
tests/
unit/ Vitest unit tests
integration/ Vitest integration tests (full API)
e2e/ Playwright end-to-end specs
fixtures/ Small test images
Commands
pnpm dev # start frontend + backend
pnpm build # build all workspaces
pnpm typecheck # TypeScript check across monorepo
pnpm lint # Biome lint + format check
pnpm lint:fix # auto-fix lint + format
pnpm test # unit + integration tests
pnpm test:unit # unit tests only
pnpm test:integration # integration tests only
pnpm test:e2e # Playwright e2e tests
pnpm test:coverage # tests with coverage report
Code conventions
- Double quotes, semicolons, 2-space indentation (enforced by Biome)
- ES modules in all workspaces
- Conventional commits for semantic-release
- Zod for all API input validation
- No modifications to Biome, TypeScript, or editor config files. Fix the code, not the linter.
Database
PostgreSQL 17 via Drizzle ORM (pg-core). Local dev requires Postgres and Redis running - start them with:
docker compose -f docker-compose.dev.yml up -d
This gives you Postgres on port 5432 and Redis on port 6379. Then generate and apply migrations:
cd apps/api
npx drizzle-kit generate # generate a migration from schema changes
npx drizzle-kit migrate # apply pending migrations
Schema is defined in apps/api/src/db/schema.ts. Tables: users, sessions, settings, jobs, apiKeys, pipelines, teams, userFiles, roles, auditLog.
Adding a new tool
Every tool follows the same pattern. Here is a minimal example.
1. Backend route
Create apps/api/src/routes/tools/my-tool.ts:
import { z } from "zod";
import type { FastifyInstance } from "fastify";
import { createToolRoute } from "../tool-factory.js";
const settingsSchema = z.object({
intensity: z.number().min(0).max(100).default(50),
});
export function registerMyTool(app: FastifyInstance) {
createToolRoute(app, {
toolId: "my-tool",
settingsSchema,
async process(inputBuffer, settings, filename) {
// Use sharp or other libraries to process the image
const sharp = (await import("sharp")).default;
const result = await sharp(inputBuffer)
// ... your processing logic
.toBuffer();
return {
buffer: result,
filename: filename.replace(/\.[^.]+$/, ".png"),
contentType: "image/png",
};
},
});
}
Then register it in apps/api/src/routes/tools/index.ts.
2. Frontend settings component
Create apps/web/src/components/tools/my-tool-settings.tsx:
import { useState } from "react";
import { useToolProcessor } from "@/hooks/use-tool-processor";
import { useFileStore } from "@/stores/file-store";
export function MyToolSettings() {
const { files } = useFileStore();
const { processFiles, processing, error, downloadUrl } =
useToolProcessor("my-tool");
const [intensity, setIntensity] = useState(50);
const handleProcess = () => {
processFiles(files, { intensity });
};
return (
<div className="space-y-4">
{/* your controls here */}
<button
type="button"
onClick={handleProcess}
disabled={files.length === 0 || processing}
data-testid="my-tool-submit"
className="w-full py-2.5 rounded-lg bg-primary text-primary-foreground font-medium disabled:opacity-50"
>
Process
</button>
</div>
);
}
Then register it in the frontend tool registry at apps/web/src/lib/tool-registry.tsx:
// Add the lazy import
const MyToolSettings = lazy(() =>
import("@/components/tools/my-tool-settings").then((m) => ({
default: m.MyToolSettings,
})),
);
// Add to the toolRegistry Map
["my-tool", { displayMode: "before-after", Settings: MyToolSettings }],
Display modes: "side-by-side", "before-after", "live-preview", "no-comparison", "interactive-crop", "interactive-eraser", "no-dropzone".
3. i18n entry
Add to packages/shared/src/i18n/en.ts:
"my-tool": {
name: "My Tool",
description: "Short description of what this tool does",
},
4. Tests
Add a data-testid attribute to your action button (as shown above) so e2e tests can target it reliably.
Docker builds
Build the full production image locally:
docker build -f docker/Dockerfile -t snapotter:latest .
Use BuildKit cache mounts for faster rebuilds:
DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile -t snapotter:latest .
Environment variables
See the Configuration guide for the full list. Key ones for development:
| Variable | Default | Description |
|---|---|---|
AUTH_ENABLED |
true |
Enable/disable authentication |
DEFAULT_USERNAME |
admin |
Default admin username |
DEFAULT_PASSWORD |
admin |
Default admin password |
SKIP_MUST_CHANGE_PASSWORD |
false |
Skip forced password change (CI/dev only) |
RATE_LIMIT_PER_MIN |
1000 |
API rate limit per minute (0 = disabled) |
MAX_UPLOAD_SIZE_MB |
100 |
Maximum upload size in MB (0 = unlimited) |