External/SnapOtter
1
0
Fork 0
mirror of https://github.com/snapotter-hq/SnapOtter.git synced 2026-08-03 07:46:42 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix: PDF tool QA sweep - library auto-save versioning, AI fileId threading, modality polish (#251)

Browse Source 
* fix(pdf): never enlarge on compress, honor redact case, hide same-format convert

- compress-pdf: guard both modes so output is never larger than the input; low-DPI scans could be upsampled and grow. Falls back to the original bytes.

- doc_redact.py: caseSensitive=true now filters PyMuPDF's case-insensitive search to exact-case hits, so the toggle works instead of always over-redacting.

- convert-{document,presentation,spreadsheet}: omit the input's own format from the output dropdown; the backend already rejects same-format conversions.

Verified end-to-end against an isolated Docker stack during a full visual QA sweep of all 37 PDF tools.

* fix(ui): show real multi-file preview thumbnails per modality

The bottom multi-file preview strip rendered a raw <img src=blobUrl> for every file, so audio/video/PDF inputs showed a broken-image icon plus the filename. ThumbnailStrip now branches on FileEntry.previewKind: images use <img> (icon fallback on error), video shows a captured first frame, PDF shows a pdf.js page-1 render, and audio/other show a type icon + extension. Fixes the multi-file preview across all modalities.

Verified in the browser for image/PDF/audio/video.

* fix(modality): make pipeline, batch validation, save/upload, previews & UI modality-aware

The app grew up image-only; several paths still assumed image. They now dispatch on the tool/file modality (image/video/audio/document/file):

- pipeline /execute + /batch: validate+decode input via inputHandlerFor(modality) instead of validateImageBuffer, so PDF/audio/video/data pipelines work (were rejected 'Invalid image').

- batch: non-image inputs now get per-modality validation (ffprobe/qpdf) before the worker instead of passing through unchecked.

- files /upload, user-files /save-result + /thumbnail: accept non-image files (MIME from extension; video-poster / pdf-first-page thumbnails).

- postprocess CONTENT_TYPE_TO_EXT: cover video/audio/pdf/text/zip so output extensions are corrected for all modalities.

- worker pipeline-finalize: attach result payload to the complete SSE event so the sync-window-timeout fallback still delivers a download.

- frontend: batch-ZIP blob MIME by extension (not svg-only); modality-neutral fallback labels/filenames; 'smaller file' not 'smaller image'.

Found via a codebase-wide image-only-assumption audit. Verified: PDF/audio/video pipelines + batch now work; image paths unchanged. canBrowserPreview kept image-only by design (non-image is rendered by dedicated displayMode viewers).

* fix(pipeline): generate a modality-aware preview for pipeline results

processPipelineFinalize now derives the output content type from its extension and runs generatePreview (video poster / pdf first page / image thumb), sets previewRef on the result, and surfaces previewUrl in the /execute sync response and the SSE complete event (via buildLegacyResultPayload). Pipeline outputs get a preview like single-tool results instead of always returning previewUrl: undefined.

Verified: PDF pipeline -> previewUrl returns a valid PNG first-page render; png pipeline correctly has no previewUrl; audio/video/multi-step pipelines all 200.

* fix(worker): auto-save a new library version when processing a library file

The worker hardcoded savedFileId = undefined ('No auto-save') even though the whole versioning feature was wired around it: the frontend sends fileId for library files and reads result.savedFileId, tool-factory threads fileId into ToolJobData, and autoSaveToLibrary implements the new-version save -- but the worker never called it (dead code from the tool-first-workflow merge). processToolJob now calls autoSaveToLibrary with data.fileId; without a fileId it is a no-op, so tool-first uploads are unchanged.

Verified: processing a library PDF with fileId creates version 2 (parent linked, toolChain appended, savedFileId returned); processing without fileId saves nothing.

* fix(library): ownership check + modality-aware dimensions in autoSaveToLibrary

- Only create a new version when the requester owns the parent (parent.userId === opts.userId); prevents versioning another user's file via a known fileId.

- Dimensions are modality-aware: sharp for images, ffprobe (probeMedia) for video, null for audio/document. Previously sharp-only, so non-image versions always got null dims.

* fix(ai): thread fileId + real userId through the 16 AI tool routes

AI custom routes parsed neither the fileId multipart field nor the authenticated user (they hardcoded userId: null), so processing a library file via an AI tool never created a new version, and AI jobs were unattributed. Each route now parses fileId like clientJobId and passes getAuthUser(request)?.id as userId to enqueueToolJob.

Verified: ocr-pdf on a library PDF creates a new version (v2); the ownership check still denies cross-user versioning.
This commit is contained in:
SnapOtter
2026-06-16 15:48:07 +08:00
committed by GitHub
parent d50e8e42a7
commit 08961fcc89
32 changed files with 580 additions and 131 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/api/src/jobs/postprocess.ts
+49 -8
View File
@@ -7,7 +7,10 @@
* keeps its own workspace-based preview write until Task 8 converts it.
*/
import { randomUUID } from "node:crypto";
import { extname } from "node:path";
import { mkdir, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { extname, join } from "node:path";
import { probeMedia } from "@snapotter/media-engine";
import { eq } from "drizzle-orm";
import sharp from "sharp";
import { db, schema } from "../db/index.js";
@@ -39,6 +42,23 @@ export const CONTENT_TYPE_TO_EXT: Record<string, string> = {
"image/vnd.ms-dds": ".dds",
"image/x-dpx": ".dpx",
"image/fits": ".fits",
// Video
"video/mp4": ".mp4",
"video/webm": ".webm",
"video/quicktime": ".mov",
// Audio
"audio/mpeg": ".mp3",
"audio/wav": ".wav",
"audio/flac": ".flac",
"audio/ogg": ".ogg",
"audio/aac": ".aac",
// Document / data
"application/pdf": ".pdf",
"text/plain": ".txt",
"text/csv": ".csv",
"application/json": ".json",
"application/xml": ".xml",
"application/zip": ".zip",
};
// ── Build output filename ──────────────────────────────────────
@@ -177,21 +197,42 @@ export async function autoSaveToLibrary(opts: AutoSaveOpts): Promise<string | un
.from(schema.userFiles)
.where(eq(schema.userFiles.id, opts.fileId));
if (!parent) return undefined;
// Only version a file the requester owns; never create a version on
// another user's file via a known fileId.
if (parent.userId !== opts.userId) return undefined;
const newVersion = parent.version + 1;
const parentChain: string[] = parent.toolChain ?? [];
const newToolChain = [...parentChain, opts.toolId];
const storedName = await saveFile(opts.buffer, opts.outName);
// Get image dimensions from the processed output
// Output dimensions: sharp for images, ffprobe for video; null where N/A
// (audio/document have no pixel dimensions). Non-critical, best-effort.
let width: number | null = null;
let height: number | null = null;
try {
const meta = await sharp(opts.buffer).metadata();
width = meta.width ?? null;
height = meta.height ?? null;
} catch {
// dimensions are non-critical
if (opts.contentType.startsWith("image/")) {
try {
const meta = await sharp(opts.buffer).metadata();
width = meta.width ?? null;
height = meta.height ?? null;
} catch {
// dimensions are non-critical
}
} else if (opts.contentType.startsWith("video/")) {
const probeDir = join(tmpdir(), `autosave-probe-${randomUUID()}`);
try {
await mkdir(probeDir, { recursive: true });
const probePath = join(probeDir, "input");
await writeFile(probePath, opts.buffer);
const info = await probeMedia(probePath);
const v = info.streams.find((s) => s.type === "video");
width = v?.width ?? null;
height = v?.height ?? null;
} catch {
// dimensions are non-critical
} finally {
await rm(probeDir, { recursive: true, force: true }).catch(() => {});
}
}
const newId = randomUUID();
apps/api/src/jobs/worker.ts
+67 -14
View File
@@ -44,7 +44,7 @@ import { hasAiJobHandler, runAiToolJob } from "./ai-handlers.js";
import { recordChildOutcome } from "./batch-progress.js";
import { registerCancelable, unregisterCancelable } from "./cancel.js";
import { createBullMQConnection } from "./connection.js";
import { buildOutputName, generatePreview } from "./postprocess.js";
import { autoSaveToLibrary, buildOutputName, generatePreview } from "./postprocess.js";
import { runSystemJob } from "./system-jobs.js";
import { POOLS, type Pool, queueName, type ToolJobData, type ToolJobResult } from "./types.js";
@@ -267,8 +267,18 @@ async function processToolJob(job: Job<ToolJobData>): Promise<ToolJobResult> {
// Generate preview for non-browser-previewable formats
const previewRef = await generatePreview(resultBuffer, resultContentType, jobId, inputBuffer);
// No auto-save -- users save to library explicitly via the UI
const savedFileId: string | undefined = undefined;
// Auto-save a new version when the input came from the user's library
// (data.fileId is set by tool-factory when the upload referenced a
// library file). Without a fileId this is a no-op, so tool-first uploads
// are not auto-saved.
const savedFileId = await autoSaveToLibrary({
fileId: data.fileId,
userId: data.userId,
buffer: resultBuffer,
outName,
contentType: resultContentType,
toolId: data.toolId,
});
const durationMs = Date.now() - startTime;
@@ -495,6 +505,37 @@ async function processPipelineStep(job: Job<ToolJobData>): Promise<ToolJobResult
* When part of a pipeline-batch (parentId is set), also records the
* child outcome for batch progress tracking.
*/
/** Best-effort content type from a filename extension (for preview dispatch). */
function contentTypeForFilename(name: string): string {
const ext = name.split(".").pop()?.toLowerCase() ?? "";
const map: Record<string, string> = {
pdf: "application/pdf",
mp4: "video/mp4",
webm: "video/webm",
mov: "video/quicktime",
mkv: "video/x-matroska",
avi: "video/x-msvideo",
mp3: "audio/mpeg",
wav: "audio/wav",
ogg: "audio/ogg",
flac: "audio/flac",
m4a: "audio/mp4",
aac: "audio/aac",
png: "image/png",
jpg: "image/jpeg",
jpeg: "image/jpeg",
gif: "image/gif",
webp: "image/webp",
avif: "image/avif",
svg: "image/svg+xml",
bmp: "image/bmp",
tiff: "image/tiff",
heic: "image/heic",
heif: "image/heif",
};
return map[ext] ?? "application/octet-stream";
}
async function processPipelineFinalize(job: Job<ToolJobData>): Promise<ToolJobResult> {
const data = job.data;
const totalSteps = data.totalSteps ?? 0;
@@ -582,6 +623,12 @@ async function processPipelineFinalize(job: Job<ToolJobData>): Promise<ToolJobRe
const parentKey = `outputs/${data.jobId}/${outFilename}`;
await putObject(parentKey, lastOutputBuffer);
// Modality-aware preview of the final output (video poster / pdf first page /
// image thumb) so the pipeline result carries a previewUrl like single-tool
// results do. Content type is derived from the output extension.
const contentType = contentTypeForFilename(outFilename);
const previewRef = await generatePreview(lastOutputBuffer, contentType, data.jobId);
await db
.update(schema.jobs)
.set({
@@ -593,11 +640,27 @@ async function processPipelineFinalize(job: Job<ToolJobData>): Promise<ToolJobRe
})
.where(eq(schema.jobs.id, data.jobId));
const result: ToolJobResult = {
outputRefs: [parentKey],
filename: outFilename,
contentType,
originalSize: firstBytesIn,
processedSize: lastBytesOut,
previewRef,
resultPayload: {
stepsCompleted: totalSteps,
steps,
},
};
updateSingleFileProgress({
jobId: progressJobId,
phase: "complete",
percent: 100,
stage: "complete",
// Carry the full result (incl. previewUrl) so the SSE fallback path
// (sync-window timeout) still delivers a downloadable output.
result: buildLegacyResultPayload(result, data.jobId),
});
// Batch progress (pipeline-batch only)
@@ -605,17 +668,7 @@ async function processPipelineFinalize(job: Job<ToolJobData>): Promise<ToolJobRe
await recordChildOutcome(data.parentId, data.totalFiles, outFilename);
}
return {
outputRefs: [parentKey],
filename: outFilename,
contentType: "application/octet-stream",
originalSize: firstBytesIn,
processedSize: lastBytesOut,
resultPayload: {
stepsCompleted: totalSteps,
steps,
},
};
return result;
}
// ── Batch child handler ───────────────────────────────────────
apps/api/src/routes/batch.ts
+2 -2
View File
@@ -101,7 +101,7 @@ export async function registerBatchRoutes(app: FastifyInstance): Promise<void> {
if (buffer.length > 0) {
files.push({
buffer,
filename: sanitizeFilename(part.filename ?? "image"),
filename: sanitizeFilename(part.filename ?? "file"),
});
}
} else if (part.fieldname === "settings") {
@@ -121,7 +121,7 @@ export async function registerBatchRoutes(app: FastifyInstance): Promise<void> {
}
if (files.length === 0) {
return reply.status(400).send({ error: "No image files provided" });
return reply.status(400).send({ error: "No files provided" });
}
// Enforce batch size limit
apps/api/src/routes/files.ts
+6 -9
View File
@@ -52,16 +52,12 @@ export async function fileRoutes(app: FastifyInstance): Promise<void> {
// Skip empty parts (e.g. empty file field)
if (buffer.length === 0) continue;
// Validate the image (pass filename for extension-based format detection)
const validation = await validateImageBuffer(buffer, part.filename);
if (!validation.valid) {
return reply.status(400).send({
error: `Invalid file "${part.filename}": ${validation.reason}`,
});
}
// Try image validation; non-image files are accepted with format from extension
const validation = await validateImageBuffer(buffer, part.filename).catch(() => null);
const isValidImage = validation?.valid === true;
// Sanitize SVG uploads to prevent XXE, SSRF, and script injection
const safeBuffer = isSvgBuffer(buffer) ? sanitizeSvg(buffer) : buffer;
const safeBuffer = isValidImage && isSvgBuffer(buffer) ? sanitizeSvg(buffer) : buffer;
// Sanitize filename (canonical; do NOT re-sanitize downstream)
const safeName = sanitizeFilename(part.filename ?? "upload");
@@ -69,10 +65,11 @@ export async function fileRoutes(app: FastifyInstance): Promise<void> {
// Write to object storage uploads prefix
await putObject(`uploads/${jobId}/${safeName}`, safeBuffer);
const fileExt = safeName.split(".").pop()?.toLowerCase() ?? "";
uploadedFiles.push({
name: safeName,
size: safeBuffer.length,
format: validation.format,
format: isValidImage ? validation.format : fileExt,
});
}
apps/api/src/routes/pipeline.ts
+7 -4
View File
@@ -205,7 +205,7 @@ export async function registerPipelineRoutes(app: FastifyInstance): Promise<void
*/
app.post("/api/v1/pipeline/execute", async (request: FastifyRequest, reply: FastifyReply) => {
let fileBuffer: Buffer | null = null;
let filename = "image";
let filename = "file";
let pipelineRaw: string | null = null;
let clientJobId: string | null = null;
@@ -219,7 +219,7 @@ export async function registerPipelineRoutes(app: FastifyInstance): Promise<void
chunks.push(chunk);
}
fileBuffer = Buffer.concat(chunks);
filename = sanitizeFilename(part.filename ?? "image");
filename = sanitizeFilename(part.filename ?? "file");
} else if (part.fieldname === "pipeline") {
pipelineRaw = part.value as string;
} else if (part.fieldname === "clientJobId") {
@@ -501,6 +501,9 @@ export async function registerPipelineRoutes(app: FastifyInstance): Promise<void
return reply.send({
jobId,
downloadUrl: `/api/v1/download/${jobId}/${encodeURIComponent(result.filename)}`,
previewUrl: result.previewRef
? `/api/v1/download/${jobId}/${result.previewRef.split("/").pop()}`
: undefined,
originalSize,
processedSize: result.processedSize,
stepsCompleted: result.resultPayload?.stepsCompleted ?? parsedSteps.length,
@@ -686,7 +689,7 @@ export async function registerPipelineRoutes(app: FastifyInstance): Promise<void
if (buffer.length > 0) {
files.push({
buffer,
filename: sanitizeFilename(part.filename ?? "image"),
filename: sanitizeFilename(part.filename ?? "file"),
});
}
} else if (part.fieldname === "pipeline") {
@@ -706,7 +709,7 @@ export async function registerPipelineRoutes(app: FastifyInstance): Promise<void
}
if (files.length === 0) {
return reply.status(400).send({ error: "No image files provided" });
return reply.status(400).send({ error: "No files provided" });
}
// Enforce batch size limit
apps/api/src/routes/tool-factory.ts
+3 -3
View File
@@ -230,7 +230,7 @@ export function createToolRoute<T>(app: FastifyInstance, config: ToolRouteConfig
const jobId = randomUUID();
const maxInputs = config.maxInputs ?? 1;
const minInputs = config.minInputs ?? 1;
let filename = "image";
let filename = "file";
let settingsRaw: string | null = null;
let fileId: string | null = null;
let clientJobId: string | null = null;
@@ -318,7 +318,7 @@ export function createToolRoute<T>(app: FastifyInstance, config: ToolRouteConfig
// Require at least one file
if (received.length === 0) {
return reply.status(400).send({ error: "No image file provided" });
return reply.status(400).send({ error: "No file provided" });
}
// Require the tool's minimum number of files (e.g. create-zip / merge-csvs
@@ -590,7 +590,7 @@ export function createToolRoute<T>(app: FastifyInstance, config: ToolRouteConfig
is_ai_tool: getBundleForTool(config.toolId) !== null,
error_code: err instanceof Error ? err.constructor.name : "UnknownError",
error_message:
err instanceof Error ? err.message.slice(0, 200) : "Image processing failed",
err instanceof Error ? err.message.slice(0, 200) : "Processing failed",
});
return reply.status(422).send({
error: "Processing failed",
apps/api/src/routes/tools/ai-canvas-expand.ts
+7 -1
View File
@@ -19,6 +19,7 @@ import { decodeHeic, encodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { resolveOutputFormat } from "../../lib/output-format.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const settingsSchema = z.object({
@@ -144,11 +145,13 @@ export function registerAiCanvasExpand(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -165,6 +168,8 @@ export function registerAiCanvasExpand(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -241,12 +246,13 @@ export function registerAiCanvasExpand(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/auto-subtitles.ts
+7 -1
View File
@@ -12,6 +12,7 @@ import { formatZodErrors, stripInternalPaths } from "../../lib/errors.js";
import { isToolInstalled } from "../../lib/feature-status.js";
import { type TranscriptSegment, toSrt, toVtt } from "../../lib/subtitle-format.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
const settingsSchema = z.object({
language: z
@@ -102,10 +103,12 @@ export function registerAutoSubtitles(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let filename = "video";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -122,6 +125,8 @@ export function registerAutoSubtitles(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -155,12 +160,13 @@ export function registerAutoSubtitles(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/background-replace.ts
+7 -1
View File
@@ -14,6 +14,7 @@ import { validateImageBuffer } from "../../lib/file-validation.js";
import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.js";
import { decodeHeic } from "../../lib/heic-converter.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
const HEX_RE = /^#[0-9a-fA-F]{6}$/;
@@ -123,11 +124,13 @@ export function registerBackgroundReplace(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -144,6 +147,8 @@ export function registerBackgroundReplace(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -216,12 +221,13 @@ export function registerBackgroundReplace(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/blur-background.ts
+7 -1
View File
@@ -14,6 +14,7 @@ import { validateImageBuffer } from "../../lib/file-validation.js";
import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.js";
import { decodeHeic } from "../../lib/heic-converter.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
const settingsSchema = z.object({
intensity: z.number().int().min(1).max(100).default(50),
@@ -93,11 +94,13 @@ export function registerBlurBackground(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -114,6 +117,8 @@ export function registerBlurBackground(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -186,12 +191,13 @@ export function registerBlurBackground(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/blur-faces.ts
+7 -1
View File
@@ -18,6 +18,7 @@ import { decodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { resolveOutputFormat } from "../../lib/output-format.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const settingsSchema = z.object({
@@ -77,11 +78,13 @@ export function registerBlurFaces(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -98,6 +101,8 @@ export function registerBlurFaces(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -161,12 +166,13 @@ export function registerBlurFaces(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/colorize.ts
+7 -1
View File
@@ -18,6 +18,7 @@ import { decodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { resolveOutputFormat } from "../../lib/output-format.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const settingsSchema = z.object({
@@ -78,11 +79,13 @@ export function registerColorize(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -99,6 +102,8 @@ export function registerColorize(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -162,12 +167,13 @@ export function registerColorize(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/compress-pdf.ts
+7
View File
@@ -66,6 +66,13 @@ export function registerCompressPdf(app: FastifyInstance) {
await gsCompressPdfQuality(inPath, outPath, qualityToDpi(settings.quality ?? 75));
}
// A "Compress" tool must never enlarge the file. If re-encoding produced
// something at least as large as the original (common for already
// compressed or low-DPI scanned PDFs), keep the original bytes instead.
if ((await stat(outPath)).size >= input.buffer.length) {
await writeFile(outPath, input.buffer);
}
ctx.report(95, "Done");
return {
scratchPath: outPath,
apps/api/src/routes/tools/enhance-faces.ts
+7 -1
View File
@@ -16,6 +16,7 @@ import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.j
import { decodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const settingsSchema = z.object({
@@ -70,11 +71,13 @@ export function registerEnhanceFaces(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -91,6 +94,8 @@ export function registerEnhanceFaces(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -154,12 +159,13 @@ export function registerEnhanceFaces(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/erase-object.ts
+7 -1
View File
@@ -15,6 +15,7 @@ import { decodeHeic, encodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { resolveOutputFormat } from "../../lib/output-format.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
const settingsSchema = z.object({
format: z
@@ -45,11 +46,13 @@ export function registerEraseObject(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let imageBuffer: Buffer | null = null;
let maskBuffer: Buffer | null = null;
let filename = "image";
let clientJobId: string | null = null;
let fileId: string | null = null;
let format = "png";
let quality = 95;
let imageKey: string | null = null;
@@ -72,6 +75,8 @@ export function registerEraseObject(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
} else if (part.fieldname === "format") {
format = (part.value as string) || "png";
} else if (part.fieldname === "quality") {
@@ -157,12 +162,13 @@ export function registerEraseObject(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [imageKey, maskKey],
filename,
settings: { format, quality },
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/noise-removal.ts
+7 -1
View File
@@ -16,6 +16,7 @@ import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.j
import { decodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const settingsSchema = z.object({
@@ -81,11 +82,13 @@ export function registerNoiseRemoval(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -102,6 +105,8 @@ export function registerNoiseRemoval(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -165,12 +170,13 @@ export function registerNoiseRemoval(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings: parsed,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/ocr-pdf.ts
+7 -1
View File
@@ -10,6 +10,7 @@ import { enqueueToolJob } from "../../jobs/enqueue.js";
import { formatZodErrors, stripInternalPaths } from "../../lib/errors.js";
import { isToolInstalled } from "../../lib/feature-status.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
const settingsSchema = z.object({
quality: z.enum(["fast", "balanced", "best"]).default("balanced"),
@@ -69,10 +70,12 @@ export function registerOcrPdf(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let filename = "document.pdf";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -89,6 +92,8 @@ export function registerOcrPdf(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -122,12 +127,13 @@ export function registerOcrPdf(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/red-eye-removal.ts
+7 -1
View File
@@ -16,6 +16,7 @@ import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.j
import { decodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const settingsSchema = z.object({
@@ -71,11 +72,13 @@ export function registerRedEyeRemoval(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -92,6 +95,8 @@ export function registerRedEyeRemoval(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -155,12 +160,13 @@ export function registerRedEyeRemoval(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/remove-background.ts
+7 -1
View File
@@ -22,6 +22,7 @@ import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.j
import { decodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const settingsSchema = z.object({
@@ -105,11 +106,13 @@ export function registerRemoveBackground(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -126,6 +129,8 @@ export function registerRemoveBackground(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -207,12 +212,13 @@ export function registerRemoveBackground(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/restore-photo.ts
+7 -1
View File
@@ -18,6 +18,7 @@ import { decodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { resolveOutputFormat } from "../../lib/output-format.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const settingsSchema = z.object({
@@ -94,11 +95,13 @@ export function registerRestorePhoto(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -115,6 +118,8 @@ export function registerRestorePhoto(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -185,12 +190,13 @@ export function registerRestorePhoto(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId: "restore-photo",
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/transcribe-audio.ts
+7 -1
View File
@@ -11,6 +11,7 @@ import { formatZodErrors, stripInternalPaths } from "../../lib/errors.js";
import { isToolInstalled } from "../../lib/feature-status.js";
import { type TranscriptSegment, toSrt, toVtt } from "../../lib/subtitle-format.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
const settingsSchema = z.object({
language: z
@@ -92,10 +93,12 @@ export function registerTranscribeAudio(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let filename = "audio";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -112,6 +115,8 @@ export function registerTranscribeAudio(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -145,12 +150,13 @@ export function registerTranscribeAudio(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/transparency-fixer.ts
+7 -1
View File
@@ -17,6 +17,7 @@ import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.j
import { decodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const TOOL_ID = "transparency-fixer";
@@ -164,11 +165,13 @@ export function registerTransparencyFixer(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -185,6 +188,8 @@ export function registerTransparencyFixer(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -252,12 +257,13 @@ export function registerTransparencyFixer(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId: TOOL_ID,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/tools/upscale.ts
+7 -1
View File
@@ -19,6 +19,7 @@ import { decodeHeic, encodeHeic } from "../../lib/heic-converter.js";
import { getObjectBuffer, putObject } from "../../lib/object-storage.js";
import { resolveOutputFormat } from "../../lib/output-format.js";
import { receiveUpload } from "../../lib/upload-stream.js";
import { getAuthUser } from "../../plugins/auth.js";
import { registerToolProcessFn } from "../tool-factory.js";
const settingsSchema = z.object({
@@ -128,11 +129,13 @@ export function registerUpscale(app: FastifyInstance) {
});
}
const userId = getAuthUser(request)?.id ?? null;
const jobId = randomUUID();
let fileBuffer: Buffer | null = null;
let filename = "image";
let settingsRaw: string | null = null;
let clientJobId: string | null = null;
let fileId: string | null = null;
let inputKey: string | null = null;
try {
@@ -149,6 +152,8 @@ export function registerUpscale(app: FastifyInstance) {
if (/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(raw)) {
clientJobId = raw;
}
} else if (part.fieldname === "fileId") {
fileId = part.value as string;
}
}
} catch (err) {
@@ -214,12 +219,13 @@ export function registerUpscale(app: FastifyInstance) {
await enqueueToolJob({
jobId,
toolId,
userId: null,
userId,
pool: "ai",
inputRefs: [inputKey],
filename,
settings,
clientJobId: clientJobId ?? undefined,
fileId: fileId ?? undefined,
kind: "ai-tool",
});
apps/api/src/routes/user-files.ts
+49 -10
View File
@@ -32,6 +32,7 @@ import { sanitizeFilename } from "../lib/filename.js";
import { decodeToSharpCompat, needsCliDecode } from "../lib/format-decoders.js";
import { decodeHeic } from "../lib/heic-converter.js";
import { isSvgBuffer, sanitizeSvg } from "../lib/svg-sanitize.js";
import { pdfFirstPagePreview, videoPosterPreview } from "../modality/preview.js";
import { hasEffectivePermission } from "../permissions.js";
import { getAuthUser, requireAuth } from "../plugins/auth.js";
@@ -62,6 +63,20 @@ function extToMime(ext: string): string {
tiff: "image/tiff",
tif: "image/tiff",
avif: "image/avif",
mp4: "video/mp4",
webm: "video/webm",
mov: "video/quicktime",
mp3: "audio/mpeg",
wav: "audio/wav",
flac: "audio/flac",
ogg: "audio/ogg",
aac: "audio/aac",
pdf: "application/pdf",
txt: "text/plain",
csv: "text/csv",
json: "application/json",
xml: "application/xml",
zip: "application/zip",
};
return map[clean] ?? "application/octet-stream";
}
@@ -480,6 +495,34 @@ export async function userFileRoutes(app: FastifyInstance): Promise<void> {
try {
const rawBuffer = await readStoredFile(file.storedName);
// Video thumbnail via ffmpeg poster frame
if (file.mimeType.startsWith("video/")) {
const poster = await videoPosterPreview(rawBuffer);
if (!poster) {
return reply.status(422).send({ error: "Could not generate thumbnail" });
}
saveThumbnail(file.storedName, poster).catch(() => {});
return reply
.header("Content-Type", "image/webp")
.header("Cache-Control", "public, max-age=86400, immutable")
.send(poster);
}
// PDF thumbnail via ghostscript first page
if (file.mimeType === "application/pdf") {
const page = await pdfFirstPagePreview(rawBuffer);
if (!page) {
return reply.status(422).send({ error: "Could not generate thumbnail" });
}
saveThumbnail(file.storedName, page).catch(() => {});
return reply
.header("Content-Type", "image/png")
.header("Cache-Control", "public, max-age=86400, immutable")
.send(page);
}
// Image thumbnail (existing path)
const validation = await validateImageBuffer(rawBuffer, file.originalName);
let decoded: Buffer<ArrayBuffer> = Buffer.from(rawBuffer);
if (validation.valid && validation.format === "heif") {
@@ -671,13 +714,9 @@ export async function userFileRoutes(app: FastifyInstance): Promise<void> {
return reply.status(400).send({ error: "parentId is required" });
}
// Validate the image
const validation = await validateImageBuffer(fileBuffer, filename);
if (!validation.valid) {
return reply.status(400).send({
error: `Invalid file: ${validation.reason}`,
});
}
// Try image validation; non-image outputs from trusted tools are accepted
const validation = await validateImageBuffer(fileBuffer, filename).catch(() => null);
const isValidImage = validation?.valid === true;
// Look up the parent to compute the next version and carry forward the tool chain
const [parent] = await db
@@ -700,7 +739,7 @@ export async function userFileRoutes(app: FastifyInstance): Promise<void> {
const baseName = parent.originalName.replace(/\.[^.]+$/, "");
const resultName = `${baseName}${ext}`;
const mimeType = formatToMime(validation.format) || extToMime(ext);
const mimeType = isValidImage ? formatToMime(validation.format) : extToMime(ext);
// Sanitize SVG results to prevent XXE, SSRF, and script injection
const safeResultBuffer = isSvgBuffer(fileBuffer) ? sanitizeSvg(fileBuffer) : fileBuffer;
@@ -727,8 +766,8 @@ export async function userFileRoutes(app: FastifyInstance): Promise<void> {
storedName,
mimeType,
size: fileSize,
width: validation.width,
height: validation.height,
width: isValidImage ? validation.width : null,
height: isValidImage ? validation.height : null,
version: nextVersion,
parentId,
toolChain: newChain,