mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
fix(api): gate every tool endpoint and stop ZIP streams failing quietly (#646)
Three defects from #645, all of which let the server report something that was not true. Tool access was enforced per route, so it drifted. createToolRoute calls requireToolAccess and the factory tools were fine, but all 45 hand-written routes had to remember the same call and none of them did. A role without tools:use could run image-to-pdf, svg-to-raster, erase-object, favicon, qr-generate, upscale, sign-pdf and the rest. The issue described this as affecting two routes; it was every one of them. The check now lives in a single preHandler keyed off the tool the router matched, so it covers sub-paths (/batch, /info, /preview, /analyze, /inspect) and any route added later without that route opting in. Ids no tool claims stay unresolved, which keeps an unknown or misfiled tool a 404 rather than telling an unauthorized caller which ids exist. Resolution reads request.routeOptions.url, the pattern the app itself registered, rather than parsing request.url a second time. find-my-way decodes before matching, so an independent parse disagrees with the router and the router wins: `/api/v1/tools/image/%66avicon` ran favicon while the gate saw no tool at all. Absolute-form request targets slipped it the same way. Taking the router's own answer removes the disagreement. A ZIP stream that failed after the 200 headers were out called reply.raw.end(). On a chunked response that is indistinguishable from success, so a client kept an archive with no central directory believing it whole. Worse, a source stream that errored had no listener: the request hung until it timed out and the error surfaced as unhandled. A poisoned-storage probe reproduced both. The socket is destroyed instead, and every source stream is listened to. svg-to-raster additionally ran its append loop past the hijack with no try/catch, where a throw leaves Fastify logging and walking away with the socket neither ended nor destroyed. pdf-to-image is fixed alongside the other two: it shipped in #643 with the destroy half but not the listener, so it hung the same way. A zero-byte upload was dropped during parsing. The client pairs results with its own file list by index, so every later result shifted onto the wrong file: one document's output was presented as another's, under another's name, while the file that actually converted was marked "not found in batch results". Empty parts now keep their slot and fail in place with a reason. Two tests in adversarial-extended.test.ts asserted the old zero-byte behavior, including a comment that batch "silently skips zero-byte parts". They now pin the replacement: still rejected, nothing processed, but the caller is told which files were empty instead of being told it sent none. A guard walks the whole catalog and fails if any of the 241 tools answers anything but 403 for a role without tools:use, so a tool cannot escape the gate by being registered in a shape nobody thought to sample. Fixes #645
This commit is contained in:
@@ -53,6 +53,7 @@ import { registerMfa } from "./plugins/mfa.js";
|
||||
import { oidcRoutes } from "./plugins/oidc.js";
|
||||
import { registerSaml } from "./plugins/saml.js";
|
||||
import { registerStatic } from "./plugins/static.js";
|
||||
import { toolAccessMiddleware } from "./plugins/tool-access.js";
|
||||
import { registerUpload } from "./plugins/upload.js";
|
||||
import { adminOpsRoutes } from "./routes/admin-ops.js";
|
||||
import { analyticsRoutes } from "./routes/analytics.js";
|
||||
@@ -456,6 +457,11 @@ await preferencesRoutes(app);
|
||||
// Auth middleware (must be registered before routes it protects)
|
||||
await authMiddleware(app);
|
||||
|
||||
// Tool access gate (after auth so request.user is populated). Covers every
|
||||
// per-tool endpoint including the hand-written routes, which each used to
|
||||
// have to remember the check themselves.
|
||||
await toolAccessMiddleware(app);
|
||||
|
||||
// Per-user rate limiting (after auth so request.user is populated)
|
||||
await registerPerUserRateLimit(app);
|
||||
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
import { TOOLS, toolSection } from "@snapotter/shared";
|
||||
import type { FastifyInstance } from "fastify";
|
||||
import { requireToolAccess } from "../permissions.js";
|
||||
|
||||
const TOOLS_PREFIX = "/api/v1/tools/";
|
||||
|
||||
/** toolId -> section, so a route resolves without scanning the catalog. */
|
||||
const SECTION_BY_TOOL_ID = new Map(TOOLS.map((tool) => [tool.id, toolSection(tool)]));
|
||||
|
||||
/**
|
||||
* The tool a request targets, read from the route the router actually matched
|
||||
* rather than from the raw URL.
|
||||
*
|
||||
* This distinction is the whole point. find-my-way percent-decodes before
|
||||
* matching, so `/api/v1/tools/image/%66avicon` runs the favicon handler while
|
||||
* a second, independent parse of the raw string sees no tool at all and waves
|
||||
* it through. Taking the router's own answer means the gate and the router
|
||||
* cannot disagree. `routeUrl` is the registered pattern, so it is already
|
||||
* canonical for static routes; the one parametric tool route
|
||||
* (`:section/:toolId/batch`) fills in from params, which Fastify has likewise
|
||||
* already decoded.
|
||||
*
|
||||
* Null means "not a per-tool endpoint": the catalog listings, and ids no tool
|
||||
* claims. Leaving those alone keeps an unknown or misfiled tool a 404 rather
|
||||
* than telling an unauthorized caller which ids exist.
|
||||
*/
|
||||
export function toolIdFromRoute(
|
||||
routeUrl: string | undefined,
|
||||
params?: { section?: string; toolId?: string },
|
||||
): string | null {
|
||||
if (!routeUrl?.startsWith(TOOLS_PREFIX)) return null;
|
||||
const [patternSection, patternToolId] = routeUrl.slice(TOOLS_PREFIX.length).split("/");
|
||||
if (!patternSection || !patternToolId) return null;
|
||||
const section = patternSection === ":section" ? params?.section : patternSection;
|
||||
const toolId = patternToolId === ":toolId" ? params?.toolId : patternToolId;
|
||||
if (!section || !toolId) return null;
|
||||
return SECTION_BY_TOOL_ID.get(toolId) === section ? toolId : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Enforce tool access for every per-tool endpoint from one place.
|
||||
*
|
||||
* createToolRoute gates the tools it builds, but the 45 hand-written routes
|
||||
* each had to remember the same call and none of them did, so a role without
|
||||
* `tools:use` could run image-to-pdf, svg-to-raster, erase-object, favicon and
|
||||
* the rest (issue #645). Per-route calls are exactly what drifted, so the
|
||||
* check keys off the matched route and therefore covers sub-paths (/batch,
|
||||
* /info, /preview, /analyze, /inspect) and any route added later without it
|
||||
* having to opt in.
|
||||
*
|
||||
* Must be registered after the auth middleware, which populates request.user.
|
||||
* The remaining per-route calls stay as defense in depth.
|
||||
*/
|
||||
export async function toolAccessMiddleware(app: FastifyInstance): Promise<void> {
|
||||
app.addHook("preHandler", async (request, reply) => {
|
||||
const toolId = toolIdFromRoute(
|
||||
request.routeOptions?.url,
|
||||
request.params as { section?: string; toolId?: string } | undefined,
|
||||
);
|
||||
if (!toolId) return;
|
||||
if (!(await requireToolAccess(request, reply, toolId))) return reply;
|
||||
});
|
||||
}
|
||||
@@ -138,7 +138,11 @@ export async function registerBatchRoutes(app: FastifyInstance): Promise<void> {
|
||||
413,
|
||||
);
|
||||
}
|
||||
if (file.size > 0) files.push({ kind: "path", ...file });
|
||||
// Empty parts keep their slot: the client maps results back
|
||||
// onto its own file list by index, so dropping one here
|
||||
// would label a converted file with a different file's name
|
||||
// (issue #645). They fail in place in the loop below.
|
||||
files.push({ kind: "path", ...file });
|
||||
} else {
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of part.file) {
|
||||
@@ -154,14 +158,12 @@ export async function registerBatchRoutes(app: FastifyInstance): Promise<void> {
|
||||
chunks.push(chunk);
|
||||
}
|
||||
const buffer = Buffer.concat(chunks);
|
||||
if (buffer.length > 0) {
|
||||
files.push({
|
||||
kind: "buffer",
|
||||
buffer,
|
||||
filename: sanitizeFilename(part.filename ?? "file"),
|
||||
size: buffer.length,
|
||||
});
|
||||
}
|
||||
files.push({
|
||||
kind: "buffer",
|
||||
buffer,
|
||||
filename: sanitizeFilename(part.filename ?? "file"),
|
||||
size: buffer.length,
|
||||
});
|
||||
}
|
||||
filePartIndex++;
|
||||
} else if (part.fieldname === "settings") {
|
||||
@@ -285,6 +287,18 @@ export async function registerBatchRoutes(app: FastifyInstance): Promise<void> {
|
||||
const childId = `${parentId}-f${flowChildIndex}`;
|
||||
let key = `uploads/${childId}/${processFilename}`;
|
||||
|
||||
// Reported against the slot it arrived in, so every later result
|
||||
// stays paired with the file it came from.
|
||||
if (file.size === 0) {
|
||||
preFailures.push({
|
||||
originalIndex: i,
|
||||
filename: file.filename,
|
||||
error: "File is empty",
|
||||
});
|
||||
if (file.kind === "path") await rm(file.path, { force: true }).catch(() => {});
|
||||
continue;
|
||||
}
|
||||
|
||||
if (file.kind === "path") {
|
||||
try {
|
||||
await storeValidatedOcrPdf(file, key, {
|
||||
@@ -573,12 +587,20 @@ export async function registerBatchRoutes(app: FastifyInstance): Promise<void> {
|
||||
|
||||
const archive = archiver("zip", { zlib: { level: 5 } });
|
||||
|
||||
archive.on("error", (err) => {
|
||||
request.log.error({ err }, "Archiver error during batch processing");
|
||||
if (!reply.raw.writableEnded) {
|
||||
reply.raw.end();
|
||||
}
|
||||
});
|
||||
// The 200 headers are already out, so nothing here can change the
|
||||
// status. Destroy the socket rather than end() it: a clean end on a
|
||||
// chunked response is indistinguishable from success, so the client
|
||||
// would keep a ZIP with no central directory believing it complete.
|
||||
let streamFailed = false;
|
||||
const failStream = (err: Error, msg: string) => {
|
||||
if (streamFailed) return;
|
||||
streamFailed = true;
|
||||
request.log.error({ err, jobId: parentId }, msg);
|
||||
archive.abort();
|
||||
reply.raw.destroy(err);
|
||||
};
|
||||
|
||||
archive.on("error", (err) => failStream(err, "Archiver error during batch processing"));
|
||||
|
||||
archive.pipe(reply.raw);
|
||||
|
||||
@@ -587,16 +609,22 @@ export async function registerBatchRoutes(app: FastifyInstance): Promise<void> {
|
||||
for (const entry of successEntries) {
|
||||
if (!entry.outputRef) continue;
|
||||
const stream = await getObjectStream(entry.outputRef);
|
||||
// A backend that resolves the stream and only then fails (local
|
||||
// createReadStream emitting ENOENT) never rejects here, so the
|
||||
// catch below cannot see it. Without this listener that error is
|
||||
// unhandled and the request hangs instead of terminating.
|
||||
stream.on("error", (err: Error) =>
|
||||
failStream(err, "Object stream error during batch processing"),
|
||||
);
|
||||
archive.append(stream, { name: entry.filename });
|
||||
}
|
||||
|
||||
await archive.finalize();
|
||||
} catch (err) {
|
||||
request.log.error({ err }, "Failed to stream ZIP entries during batch processing");
|
||||
archive.abort();
|
||||
if (!reply.raw.writableEnded) {
|
||||
reply.raw.end();
|
||||
}
|
||||
failStream(
|
||||
err instanceof Error ? err : new Error(String(err)),
|
||||
"Failed to stream ZIP entries during batch processing",
|
||||
);
|
||||
}
|
||||
} finally {
|
||||
request.raw.removeListener("aborted", abortIngress);
|
||||
|
||||
@@ -591,7 +591,10 @@ export function registerPdfToImageRoute(
|
||||
// Destroy the socket rather than end() it: a clean end on a chunked
|
||||
// response is indistinguishable from success, and the client would keep
|
||||
// a truncated ZIP believing it complete.
|
||||
let streamFailed = false;
|
||||
const failStream = (err: Error, msg: string) => {
|
||||
if (streamFailed) return;
|
||||
streamFailed = true;
|
||||
request.log.error({ err, jobId }, msg);
|
||||
archive.abort();
|
||||
reply.raw.destroy(err);
|
||||
@@ -603,7 +606,14 @@ export function registerPdfToImageRoute(
|
||||
try {
|
||||
for (const entry of results) {
|
||||
if (!entry) continue;
|
||||
archive.append(await getObjectStream(entry.key), { name: entry.filename });
|
||||
const stream = await getObjectStream(entry.key);
|
||||
// The local backend resolves the stream and only then emits ENOENT,
|
||||
// so the catch below never sees it. Without this listener the error
|
||||
// is unhandled and the request hangs instead of terminating.
|
||||
stream.on("error", (err: Error) =>
|
||||
failStream(err, "Object stream error during PDF batch processing"),
|
||||
);
|
||||
archive.append(stream, { name: entry.filename });
|
||||
}
|
||||
await archive.finalize();
|
||||
} catch (err) {
|
||||
|
||||
@@ -130,13 +130,14 @@ export function registerSvgToRasterRoute(
|
||||
for await (const chunk of part.file) {
|
||||
chunks.push(chunk);
|
||||
}
|
||||
const buf = Buffer.concat(chunks);
|
||||
if (buf.length > 0) {
|
||||
files.push({
|
||||
buffer: buf,
|
||||
filename: sanitizeFilename(part.filename ?? "output"),
|
||||
});
|
||||
}
|
||||
// Empty parts keep their slot: the client maps results back onto its
|
||||
// own file list by index, so dropping one would label a converted
|
||||
// file with a different file's name (issue #645). The per-file loop
|
||||
// below fails it in place.
|
||||
files.push({
|
||||
buffer: Buffer.concat(chunks),
|
||||
filename: sanitizeFilename(part.filename ?? "output"),
|
||||
});
|
||||
} else if (part.fieldname === "settings") {
|
||||
settingsRaw = part.value as string;
|
||||
} else if (part.fieldname === "clientJobId") {
|
||||
@@ -165,7 +166,13 @@ export function registerSvgToRasterRoute(
|
||||
|
||||
if (opts.accept) {
|
||||
const accept = opts.accept;
|
||||
const invalid = files.find((file) => !matchesAccept(file.filename, accept));
|
||||
// Empty parts are exempt: they now keep their slot rather than being
|
||||
// dropped (issue #645), and a nameless one would otherwise fail this
|
||||
// whole-request check and take every valid file down with it. They fail
|
||||
// on their own below, where the reason reaches the file it belongs to.
|
||||
const invalid = files.find(
|
||||
(file) => file.buffer.length > 0 && !matchesAccept(file.filename, accept),
|
||||
);
|
||||
if (invalid) {
|
||||
return reply.status(400).send({
|
||||
error: "File is not a valid SVG. This tool only accepts SVG files.",
|
||||
@@ -337,22 +344,41 @@ export function registerSvgToRasterRoute(
|
||||
|
||||
const archive = archiver("zip", { zlib: { level: 5 } });
|
||||
|
||||
archive.on("error", (err) => {
|
||||
request.log.error({ err }, "Archiver error during SVG batch processing");
|
||||
if (!reply.raw.writableEnded) {
|
||||
reply.raw.end();
|
||||
}
|
||||
});
|
||||
// Headers are already out, so this cannot change the status. Destroying
|
||||
// the socket is the only way to tell the client the archive is incomplete:
|
||||
// ending it cleanly reads as success and hands back a ZIP with no central
|
||||
// directory. Entries here are in-memory buffers, so unlike the other batch
|
||||
// routes there is no storage read to fail, but an archiver fault still has
|
||||
// to reach the client.
|
||||
let streamFailed = false;
|
||||
const failStream = (err: Error, msg: string) => {
|
||||
if (streamFailed) return;
|
||||
streamFailed = true;
|
||||
request.log.error({ err, jobId }, msg);
|
||||
archive.abort();
|
||||
reply.raw.destroy(err);
|
||||
};
|
||||
|
||||
archive.on("error", (err) => failStream(err, "Archiver error during SVG batch processing"));
|
||||
|
||||
archive.pipe(reply.raw);
|
||||
|
||||
for (const result of results) {
|
||||
if (result) {
|
||||
archive.append(result.buffer, { name: result.filename });
|
||||
// A throw past this point cannot be answered: the reply is hijacked, so
|
||||
// Fastify logs and walks away, leaving the socket neither ended nor
|
||||
// destroyed and the client waiting forever.
|
||||
try {
|
||||
for (const result of results) {
|
||||
if (result) {
|
||||
archive.append(result.buffer, { name: result.filename });
|
||||
}
|
||||
}
|
||||
await archive.finalize();
|
||||
} catch (err) {
|
||||
failStream(
|
||||
err instanceof Error ? err : new Error(String(err)),
|
||||
"Failed to finalize ZIP during SVG batch processing",
|
||||
);
|
||||
}
|
||||
|
||||
await archive.finalize();
|
||||
});
|
||||
|
||||
// --- Single-file endpoint ---
|
||||
|
||||
Reference in New Issue
Block a user