mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
SnapOtterandGitHub
17726ae59d
* docs: rebrand from image-only to multi-modality across docs and metadata SnapOtter expanded from image-only to 157 tools across 5 modalities (image, video, audio, document/PDF, data). Update all product-level copy, metadata, and i18n that still framed it as an image-only tool. - README, package.json, root llms.txt: multi-modality framing, 157 tools - OpenAPI info + tags, generated /llms.txt tagline (docs.ts) - VitePress docs site: hero, getting-started, architecture, security, deployment, configuration, developer, supported-formats - i18n: 10 product keys across all 21 locales (hero, app description, privacy notes, AI features, progress messages, getting-started) - web/demo/landing meta + privacy copy, COMMUNITY_GUIDE, .env.example Stale tool counts (53/50+/52/70+/35) corrected to 157 throughout. Database/container deployment claims left unchanged (out of scope). * docs: fix stale post-rebrand test assertions and README language list - tests/e2e-docs/homepage.spec.ts: assert the current docs homepage (file toolkit, 157 tools, 5 modalities) instead of the old image-only strings - tests/unit/api/docs-route.test.ts: sync the reproduced llms.txt tagline with docs.ts - README.md: 21 languages with the correct list (add Swedish and Chinese Traditional, drop Czech which is not supported) * docs: correct 2.0 architecture references (Postgres 17 + Redis 8, 3-container stack) The docs and metadata still described the 1.x stack (SQLite, single container, p-queue). Update them to the current 2.0 reality. - README: replace the broken single-container `docker run` quick-start with the real Docker Compose stack (app + Postgres 17 + Redis 8); fix the "no Redis, no Postgres" feature bullet - package.json: description no longer claims a single container - apps/docs: rewrite database.md for Postgres; configuration.md DB_PATH -> DATABASE_URL + REDIS_URL; architecture.md SQLite/p-queue/better-sqlite3 -> Postgres/BullMQ/pg and add media-engine + doc-engine; developer/security/deployment/docker-tags/getting-started/contributing compose examples now include postgres + redis; index.md + api/ai.md AI count 16 -> 19 - SECURITY.md: Drizzle (SQLite) -> (PostgreSQL) - landing: enterprise/FeatureHighlights single-container wording; TrustSignals/ToolGrid 150+ -> 157 (dynamic); Pricing/FAQ 15 -> 19 AI tools * docs(api): document all video, audio, document, and data tool endpoints in OpenAPI The spec covered only image tools; the Scalar UI and the generated /llms.txt and /llms-full.txt inherited that gap. Add the 104 missing tool endpoints so the API docs match the code. - Video: 29 endpoints (most long/async; auto-subtitles is AI) - Audio: 17 (transcribe-audio is AI) - Document/PDF: 36 (ocr-pdf is AI; conversions are long/async) - Data: 10 - Image: 12 newer tools (background-replace, blur-background AI; histogram/lqip-placeholder/sprite-sheet custom responses; barcode-generate uses a JSON body) Each schema is derived from the tool's Zod validator and executionHint (fast -> 200, long -> 202+SSE, AI adds 501 FeatureNotInstalledError, multi-file inputs as arrays), referencing the existing shared schemas. Tool path entries: 64 -> 168. Spec parses as valid YAML with no duplicate paths and only known $refs.
121 lines
7.7 KiB
Markdown
121 lines
7.7 KiB
Markdown
---
|
|
description: Monorepo structure, app and package architecture, request lifecycle, and resource footprint of SnapOtter.
|
|
---
|
|
|
|
# Architecture
|
|
|
|
SnapOtter is a monorepo managed with pnpm workspaces and Turborepo. It deploys as a 3-container Docker Compose stack: the SnapOtter app image, PostgreSQL 17, and Redis 8.
|
|
|
|
## Project structure
|
|
|
|
```
|
|
snapotter/
|
|
├── apps/
|
|
│ ├── api/ # Fastify backend
|
|
│ ├── web/ # React + Vite frontend
|
|
│ └── docs/ # This VitePress site
|
|
├── packages/
|
|
│ ├── image-engine/ # Sharp-based image operations
|
|
│ ├── media-engine/ # FFmpeg spawn + progress parsing
|
|
│ ├── doc-engine/ # qpdf, LibreOffice, ghostscript wrappers
|
|
│ ├── ai/ # Python AI model bridge
|
|
│ └── shared/ # Types, constants, i18n
|
|
└── docker/ # Dockerfile and Compose config
|
|
```
|
|
|
|
## Packages
|
|
|
|
### `@snapotter/image-engine`
|
|
|
|
The core image processing library built on [Sharp](https://sharp.pixelplumbing.com/). It handles all non-AI operations: resize, crop, rotate, flip, convert, compress, strip metadata, and color adjustments (brightness, contrast, saturation, grayscale, sepia, invert, color channels).
|
|
|
|
This package has no network dependencies and runs entirely in-process.
|
|
|
|
### `@snapotter/ai`
|
|
|
|
A bridge layer that calls Python scripts for ML operations. On first use, the bridge starts a persistent Python dispatcher process that pre-imports heavy libraries (PIL, NumPy, MediaPipe, rembg) so subsequent AI calls skip the import overhead. If the dispatcher is not yet ready, the bridge falls back to spawning a fresh Python subprocess per request.
|
|
|
|
**Models are not pre-loaded.** Each tool script loads its model weights from disk at request time and discards them when the request finishes. See [Resource footprint](#resource-footprint) for the full memory profile.
|
|
|
|
Supported operations: background removal (rembg/BiRefNet), upscaling (RealESRGAN), face blur (MediaPipe), face enhancement (GFPGAN/CodeFormer), object erasing (LaMa ONNX), OCR (PaddleOCR/Tesseract), colorization (DDColor), noise removal, red eye removal, photo restoration, passport photo generation, transparency fixing (BiRefNet HR-matting), and content-aware resize (Go caire binary).
|
|
|
|
Python scripts live in `packages/ai/python/`. The Docker image pre-downloads all model weights during the build so the container works fully offline.
|
|
|
|
### `@snapotter/shared`
|
|
|
|
Shared TypeScript types, constants (like `APP_VERSION` and tool definitions), and i18n translation strings used by both the frontend and backend.
|
|
|
|
## Applications
|
|
|
|
### API (`apps/api`)
|
|
|
|
A Fastify v5 server exposing 157 tool routes across five modalities (image, video, audio, document, data) that handles:
|
|
- File uploads, temporary workspace management, and persistent file storage
|
|
- User file library with version chains (`user_files` table) -- each processed result links back to its source file and records which tool was applied, with auto-generated thumbnails for the Files page
|
|
- Tool execution (routes each tool request to the image engine or AI bridge)
|
|
- Pipeline orchestration (chaining multiple tools sequentially)
|
|
- Batch processing with concurrency control via BullMQ job queues (pools: image, media, ai, docs, system)
|
|
- User authentication, RBAC (admin/user roles with a full permission set), API key management, and rate limiting
|
|
- Teams management -- admin-only CRUD; users are assigned to a team via the `team` field on their profile
|
|
- Runtime settings -- a key-value store in the `settings` table that controls `disabledTools`, `enableExperimentalTools`, `loginAttemptLimit`, and other operational knobs without redeploying
|
|
- Custom branding -- logo upload endpoint; the uploaded image is stored at `data/branding/logo.png` and served to the frontend
|
|
- Swagger/OpenAPI documentation at `/api/docs`
|
|
- Serving the built frontend as a SPA in production
|
|
|
|
Key dependencies: Fastify, Drizzle ORM (pg-core, node-postgres), Sharp, BullMQ, ioredis, Zod for validation.
|
|
|
|
The server handles graceful shutdown on SIGTERM/SIGINT: it drains HTTP connections, stops BullMQ workers, shuts down the Python dispatcher, and closes the database connection.
|
|
|
|
### Web (`apps/web`)
|
|
|
|
A React 19 single-page app built with Vite. Uses Zustand for state management, Tailwind CSS v4 for styling, and Lucide for icons. Communicates with the API over REST and SSE (for progress tracking).
|
|
|
|
Pages include a tool workspace, a Files page for managing persistent uploads and results, an automation/pipeline builder, and an admin settings panel.
|
|
|
|
The built frontend gets served by the Fastify backend in production, so there is no separate web server in the Docker container.
|
|
|
|
### Docs (`apps/docs`)
|
|
|
|
This VitePress site. Deployed to Cloudflare Pages automatically on push to `main`.
|
|
|
|
## How a request flows
|
|
|
|
1. The user picks a tool in the web UI and uploads a file.
|
|
2. The frontend sends a multipart POST to `/api/v1/tools/:toolId` with the file and settings.
|
|
3. The API route validates the input with Zod, then dispatches processing.
|
|
4. For standard tools, the job is enqueued to the appropriate BullMQ pool (image, media, or docs based on modality). The in-process BullMQ worker auto-orients the image based on EXIF metadata, runs the tool's process function, and returns the result.
|
|
5. For AI tools, the TypeScript bridge sends a request to the persistent Python dispatcher (or spawns a fresh subprocess as fallback), waits for it to finish, and reads the output file.
|
|
6. Job progress is persisted to the `jobs` table in PostgreSQL so state survives container restarts. Real-time updates are delivered via SSE at `/api/v1/jobs/:jobId/progress`.
|
|
7. The API returns a `jobId` and `downloadUrl`. The user downloads the processed file from `/api/v1/download/:jobId/:filename`.
|
|
|
|
For pipelines, the API feeds the output of each step as input to the next, running them sequentially.
|
|
|
|
For batch processing, the API uses BullMQ flows with per-step child jobs and returns a ZIP file with all processed files.
|
|
|
|
## Resource footprint
|
|
|
|
SnapOtter is designed for low idle memory use. Nothing is preloaded or kept warm at startup.
|
|
|
|
### At idle
|
|
|
|
The Node.js/Fastify process, PostgreSQL, and Redis are running. Typical idle RAM is **~200-300 MB** across all three containers (Node.js process, Postgres, and Redis). No Python process, no model weights in memory.
|
|
|
|
### What starts, and when
|
|
|
|
| Component | Starts when | Memory while active |
|
|
|-----------|-------------|---------------------|
|
|
| Fastify server + Postgres + Redis | Container start | ~200-300 MB total |
|
|
| BullMQ workers | Container start (in-process) | One worker per pool (image, media, ai, docs, system) |
|
|
| Python dispatcher | First AI tool request | Python interpreter + pre-imported libraries (PIL, NumPy, MediaPipe, rembg) - no model weights |
|
|
| AI model weights | During the specific tool's request | Loaded from disk, freed when the request finishes |
|
|
|
|
### Model loading
|
|
|
|
All model weight files (totalling several GB) sit on disk in `/opt/models/` at all times. Each AI tool script loads only its own model(s) into memory for the duration of a request, then releases them. Some scripts explicitly call `del model` and `torch.cuda.empty_cache()` after inference to ensure memory is returned immediately.
|
|
|
|
There is no model cache between requests. Running the same AI tool back-to-back reloads the model each time. This keeps idle memory near zero at the cost of a model-load delay on every AI request.
|
|
|
|
### First AI request cold start
|
|
|
|
The Python dispatcher is not running when the container starts. The first AI request triggers two things in parallel: the dispatcher starts warming up in the background, and the request itself falls back to a one-off Python subprocess spawn. Once the dispatcher signals ready, all subsequent AI requests use it directly and skip the subprocess spawn cost.
|