fix: PDF tool QA sweep - library auto-save versioning, AI fileId threading, modality polish (#251)

* fix(pdf): never enlarge on compress, honor redact case, hide same-format convert - compress-pdf: guard both modes so output is never larger than the input; low-DPI scans could be upsampled and grow. Falls back to the original bytes. - doc_redact.py: caseSensitive=true now filters PyMuPDF's case-insensitive search to exact-case hits, so the toggle works instead of always over-redacting. - convert-{document,presentation,spreadsheet}: omit the input's own format from the output dropdown; the backend already rejects same-format conversions. Verified end-to-end against an isolated Docker stack during a full visual QA sweep of all 37 PDF tools. * fix(ui): show real multi-file preview thumbnails per modality The bottom multi-file preview strip rendered a raw <img src=blobUrl> for every file, so audio/video/PDF inputs showed a broken-image icon plus the filename. ThumbnailStrip now branches on FileEntry.previewKind: images use <img> (icon fallback on error), video shows a captured first frame, PDF shows a pdf.js page-1 render, and audio/other show a type icon + extension. Fixes the multi-file preview across all modalities. Verified in the browser for image/PDF/audio/video. * fix(modality): make pipeline, batch validation, save/upload, previews & UI modality-aware The app grew up image-only; several paths still assumed image. They now dispatch on the tool/file modality (image/video/audio/document/file): - pipeline /execute + /batch: validate+decode input via inputHandlerFor(modality) instead of validateImageBuffer, so PDF/audio/video/data pipelines work (were rejected 'Invalid image'). - batch: non-image inputs now get per-modality validation (ffprobe/qpdf) before the worker instead of passing through unchecked. - files /upload, user-files /save-result + /thumbnail: accept non-image files (MIME from extension; video-poster / pdf-first-page thumbnails). - postprocess CONTENT_TYPE_TO_EXT: cover video/audio/pdf/text/zip so output extensions are corrected for all modalities. - worker pipeline-finalize: attach result payload to the complete SSE event so the sync-window-timeout fallback still delivers a download. - frontend: batch-ZIP blob MIME by extension (not svg-only); modality-neutral fallback labels/filenames; 'smaller file' not 'smaller image'. Found via a codebase-wide image-only-assumption audit. Verified: PDF/audio/video pipelines + batch now work; image paths unchanged. canBrowserPreview kept image-only by design (non-image is rendered by dedicated displayMode viewers). * fix(pipeline): generate a modality-aware preview for pipeline results processPipelineFinalize now derives the output content type from its extension and runs generatePreview (video poster / pdf first page / image thumb), sets previewRef on the result, and surfaces previewUrl in the /execute sync response and the SSE complete event (via buildLegacyResultPayload). Pipeline outputs get a preview like single-tool results instead of always returning previewUrl: undefined. Verified: PDF pipeline -> previewUrl returns a valid PNG first-page render; png pipeline correctly has no previewUrl; audio/video/multi-step pipelines all 200. * fix(worker): auto-save a new library version when processing a library file The worker hardcoded savedFileId = undefined ('No auto-save') even though the whole versioning feature was wired around it: the frontend sends fileId for library files and reads result.savedFileId, tool-factory threads fileId into ToolJobData, and autoSaveToLibrary implements the new-version save -- but the worker never called it (dead code from the tool-first-workflow merge). processToolJob now calls autoSaveToLibrary with data.fileId; without a fileId it is a no-op, so tool-first uploads are unchanged. Verified: processing a library PDF with fileId creates version 2 (parent linked, toolChain appended, savedFileId returned); processing without fileId saves nothing. * fix(library): ownership check + modality-aware dimensions in autoSaveToLibrary - Only create a new version when the requester owns the parent (parent.userId === opts.userId); prevents versioning another user's file via a known fileId. - Dimensions are modality-aware: sharp for images, ffprobe (probeMedia) for video, null for audio/document. Previously sharp-only, so non-image versions always got null dims. * fix(ai): thread fileId + real userId through the 16 AI tool routes AI custom routes parsed neither the fileId multipart field nor the authenticated user (they hardcoded userId: null), so processing a library file via an AI tool never created a new version, and AI jobs were unattributed. Each route now parses fileId like clientJobId and passes getAuthUser(request)?.id as userId to enqueueToolJob. Verified: ocr-pdf on a library PDF creates a new version (v2); the ownership check still denies cross-user versioning.
2026-08-03 07:46:42 +02:00 · 2026-06-16 15:48:07 +08:00
parent d50e8e42a7
commit 08961fcc89
32 changed files with 580 additions and 131 deletions
@@ -6,6 +6,11 @@ import { ThumbnailStrip } from "@/components/common/thumbnail-strip";
 import { useTranslation } from "@/contexts/i18n-context";
 import { useFileStore } from "@/stores/file-store";

+/**
+ * Formats that browsers can render in <img> tags.
+ * Intentionally image-only: consumers (BeforeAfterSlider, ImageViewer) render via <img>.
+ * Video/audio/PDF outputs should use dedicated viewer components instead.
+ */
 const BROWSER_PREVIEWABLE_EXTS = new Set([
  "jpg",
  "jpeg",
@@ -1,52 +1,176 @@
 import {
-  AudioLines,
  CheckCircle2,
  File as FileIcon,
  FileText,
  Film,
  Loader2,
+  Music,
  XCircle,
 } from "lucide-react";
-import { useEffect, useRef } from "react";
+import { useEffect, useRef, useState } from "react";
 import type { FileEntry, PreviewKind } from "@/stores/file-store";

 const BROWSER_IMG_EXTS = new Set(["jpg", "jpeg", "png", "gif", "webp", "svg", "bmp", "avif"]);
+const THUMB_W = 104;
+const THUMB_H = 76;

-/**
- * A renderable <img> source for the thumbnail, or null when the entry has no
- * image to show (e.g. audio/video/document originals). Processed previews and
- * processed image outputs are always real images, so they win when present.
- */
-function thumbnailImageSrc(entry: FileEntry): string | null {
+/** Image URL to show directly in an <img>, or null when the entry has no
+ *  browser-renderable image (audio/video/PDF inputs get a generated thumb or
+ *  a modality icon instead of a broken <img>). */
+function imageThumbSrc(entry: FileEntry): string | null {
  if (entry.processedPreviewUrl) return entry.processedPreviewUrl;
  if (entry.processedUrl) {
-    if (entry.processedUrl.startsWith("blob:")) return entry.processedUrl;
-    const ext = decodeURIComponent(entry.processedUrl).split(".").pop()?.toLowerCase() ?? "";
+    const ext =
+      decodeURIComponent(entry.processedUrl).split("?")[0].split(".").pop()?.toLowerCase() ?? "";
    if (BROWSER_IMG_EXTS.has(ext)) return entry.processedUrl;
+    if (entry.processedUrl.startsWith("blob:") && entry.previewKind === "image")
+      return entry.processedUrl;
  }
-  // The original blob only renders as an image for image-modality files;
-  // pointing an <img> at an audio/video/pdf blob just shows a broken icon.
  if (entry.previewKind === "image") return entry.blobUrl;
  return null;
 }

-const PLACEHOLDER_ICON: Record<Exclude<PreviewKind, "image">, typeof FileIcon> = {
-  audio: AudioLines,
-  video: Film,
-  document: FileText,
-  none: FileIcon,
-};
+/** Grab the first frame of a video into a small JPEG data URL. Returns a
+ *  cancel fn; calls back with null on any decode/timeout failure. */
+function captureVideoFrame(src: string, onDone: (url: string | null) => void): () => void {
+  const video = document.createElement("video");
+  video.muted = true;
+  video.playsInline = true;
+  video.preload = "auto";
+  let settled = false;
+  const timer = setTimeout(() => settle(null), 5000);
+  function settle(url: string | null) {
+    if (settled) return;
+    settled = true;
+    clearTimeout(timer);
+    video.removeEventListener("loadeddata", onLoaded);
+    video.removeEventListener("seeked", onSeeked);
+    video.removeEventListener("error", onErr);
+    video.removeAttribute("src");
+    video.load();
+    onDone(url);
+  }
+  function onLoaded() {
+    const d = video.duration;
+    const t = Number.isFinite(d) && d > 0 ? Math.min(0.1, d / 2) : 0;
+    try {
+      video.currentTime = t;
+    } catch {
+      settle(null);
+    }
+  }
+  function onSeeked() {
+    const w = video.videoWidth;
+    const h = video.videoHeight;
+    if (!w || !h) return settle(null);
+    try {
+      const scale = Math.min(THUMB_W / w, THUMB_H / h, 1);
+      const canvas = document.createElement("canvas");
+      canvas.width = Math.max(1, Math.round(w * scale));
+      canvas.height = Math.max(1, Math.round(h * scale));
+      const ctx = canvas.getContext("2d");
+      if (!ctx) return settle(null);
+      ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
+      settle(canvas.toDataURL("image/jpeg", 0.7));
+    } catch {
+      settle(null);
+    }
+  }
+  function onErr() {
+    settle(null);
+  }
+  video.addEventListener("loadeddata", onLoaded);
+  video.addEventListener("seeked", onSeeked);
+  video.addEventListener("error", onErr);
+  video.src = src;
+  return () => settle(null);
+}

-/** Icon + format label shown when a file has no image thumbnail. */
-function ThumbnailPlaceholder({ entry }: { entry: FileEntry }) {
-  const kind = entry.previewKind === "image" ? "none" : entry.previewKind;
-  const Icon = PLACEHOLDER_ICON[kind];
-  const ext = (entry.file.name.split(".").pop() ?? "").toUpperCase().slice(0, 4);
+/** Render page 1 of a PDF into a small JPEG data URL (pdf.js, lazy-imported). */
+async function renderPdfThumb(file: File, onDone: (url: string | null) => void): Promise<void> {
+  try {
+    const pdfjs = await import("pdfjs-dist");
+    pdfjs.GlobalWorkerOptions.workerSrc = new URL(
+      "pdfjs-dist/build/pdf.worker.min.mjs",
+      import.meta.url,
+    ).href;
+    const loadingTask = pdfjs.getDocument({ data: new Uint8Array(await file.arrayBuffer()) });
+    const doc = await loadingTask.promise;
+    const page = await doc.getPage(1);
+    const base = page.getViewport({ scale: 1 });
+    const scale = Math.min(THUMB_W / base.width, THUMB_H / base.height, 2);
+    const viewport = page.getViewport({ scale });
+    const canvas = document.createElement("canvas");
+    canvas.width = Math.ceil(viewport.width);
+    canvas.height = Math.ceil(viewport.height);
+    await page.render({ canvas, viewport }).promise;
+    onDone(canvas.toDataURL("image/jpeg", 0.8));
+    loadingTask.destroy();
+  } catch {
+    onDone(null);
+  }
+}
+
+/** Lazily produce a thumbnail data URL for video/PDF entries. */
+function useGeneratedThumb(entry: FileEntry): string | null {
+  const { previewKind, blobUrl, file } = entry;
+  const isPdf = file.name.toLowerCase().endsWith(".pdf");
+  const [thumb, setThumb] = useState<string | null>(null);
+  useEffect(() => {
+    let active = true;
+    setThumb(null);
+    if (previewKind === "video") {
+      const cancel = captureVideoFrame(blobUrl, (u) => {
+        if (active) setThumb(u);
+      });
+      return () => {
+        active = false;
+        cancel();
+      };
+    }
+    if (previewKind === "document" && isPdf) {
+      renderPdfThumb(file, (u) => {
+        if (active) setThumb(u);
+      });
+    }
+    return () => {
+      active = false;
+    };
+  }, [previewKind, blobUrl, isPdf, file]);
+  return thumb;
+}
+
+function ModalityIcon({ kind }: { kind: PreviewKind }) {
+  const cls = "h-4 w-4 text-muted-foreground";
+  if (kind === "video") return <Film className={cls} />;
+  if (kind === "audio") return <Music className={cls} />;
+  if (kind === "document") return <FileText className={cls} />;
+  return <FileIcon className={cls} />;
+}
+
+/** Per-tile content: a real image/thumbnail when available, else a modality
+ *  icon + extension label (never a broken <img>). */
+function Thumb({ entry }: { entry: FileEntry }) {
+  const generated = useGeneratedThumb(entry);
+  const [imgError, setImgError] = useState(false);
+  const src = entry.previewKind === "image" ? imageThumbSrc(entry) : generated;
+  if (src && !imgError) {
+    return (
+      <img
+        src={src}
+        alt={entry.file.name}
+        className="w-full h-full object-cover"
+        draggable={false}
+        onError={() => setImgError(true)}
+      />
+    );
+  }
+  const ext = entry.file.name.split(".").pop()?.toUpperCase().slice(0, 4) ?? "";
  return (
    <div className="w-full h-full flex flex-col items-center justify-center gap-0.5 bg-muted">
-      <Icon className="h-4 w-4 text-muted-foreground" />
+      <ModalityIcon kind={entry.previewKind} />
      {ext && (
-        <span className="text-[8px] font-semibold leading-none text-muted-foreground">{ext}</span>
+        <span className="text-[7px] leading-none font-medium text-muted-foreground">{ext}</span>
      )}
    </div>
  );
@@ -80,7 +204,6 @@ export function ThumbnailStrip({ entries, selectedIndex, onSelect }: ThumbnailSt
        const isSelected = i === selectedIndex;
        const isCompleted = entry.status === "completed";
        const isFailed = entry.status === "failed";
-        const imgSrc = thumbnailImageSrc(entry);
        return (
          <button
            key={entry.file.name}
@@ -99,15 +222,8 @@ export function ThumbnailStrip({ entries, selectedIndex, onSelect }: ThumbnailSt
              <div className="w-full h-full flex items-center justify-center bg-muted">
                <Loader2 className="h-3.5 w-3.5 text-muted-foreground animate-spin" />
              </div>
-            ) : imgSrc ? (
-              <img
-                src={imgSrc}
-                alt={entry.file.name}
-                className="w-full h-full object-cover"
-                draggable={false}
-              />
            ) : (
-              <ThumbnailPlaceholder entry={entry} />
+              <Thumb entry={entry} />
            )}
            {isCompleted && (
              <div className="absolute -top-0.5 -right-0.5 w-3.5 h-3.5 bg-green-500 rounded-full flex items-center justify-center">
@@ -7,6 +7,13 @@ import { useFileStore } from "@/stores/file-store";

 type DocFormat = "docx" | "odt" | "rtf" | "txt";

+const ALL_FORMATS: { value: DocFormat; label: string }[] = [
+  { value: "docx", label: "DOCX" },
+  { value: "odt", label: "ODT" },
+  { value: "rtf", label: "RTF" },
+  { value: "txt", label: "TXT" },
+];
+
 export function ConvertDocumentSettings() {
  const { t } = useTranslation();
  const s = t.toolSettings["convert-document"];
@@ -16,11 +23,20 @@ export function ConvertDocumentSettings() {

  const [outFormat, setOutFormat] = useState<DocFormat>("odt");

+  // Never offer the input's own format as a target. LibreOffice rejects a
+  // same-format conversion ("already in that format"), so drop it from the
+  // options and keep the current selection valid.
+  const inputExt = files[0]?.name.split(".").pop()?.toLowerCase();
+  const formats = ALL_FORMATS.filter((f) => f.value !== inputExt);
+  const selected = formats.some((f) => f.value === outFormat)
+    ? outFormat
+    : (formats[0]?.value ?? outFormat);
+
  const hasFile = files.length > 0;
  const hasMultiple = files.length > 1;

  const handleProcess = () => {
-    const settings = { format: outFormat };
+    const settings = { format: selected };
    if (hasMultiple) {
      processAllFiles(files, settings);
    } else {
@@ -36,14 +52,15 @@ export function ConvertDocumentSettings() {
        </label>
        <select
          id="cd-format"
-          value={outFormat}
+          value={selected}
          onChange={(e) => setOutFormat(e.target.value as DocFormat)}
          className="w-full mt-0.5 px-2 py-1.5 rounded border border-border bg-background text-sm text-foreground"
        >
-          <option value="docx">DOCX</option>
-          <option value="odt">ODT</option>
-          <option value="rtf">RTF</option>
-          <option value="txt">TXT</option>
+          {formats.map((f) => (
+            <option key={f.value} value={f.value}>
+              {f.label}
+            </option>
+          ))}
        </select>
      </div>

@@ -7,6 +7,11 @@ import { useFileStore } from "@/stores/file-store";

 type PresFormat = "pptx" | "odp";

+const ALL_FORMATS: { value: PresFormat; label: string }[] = [
+  { value: "pptx", label: "PPTX" },
+  { value: "odp", label: "ODP" },
+];
+
 export function ConvertPresentationSettings() {
  const { t } = useTranslation();
  const s = t.toolSettings["convert-presentation"];
@@ -16,11 +21,20 @@ export function ConvertPresentationSettings() {

  const [outFormat, setOutFormat] = useState<PresFormat>("odp");

+  // Never offer the input's own format as a target. LibreOffice rejects a
+  // same-format conversion ("already in that format"), so drop it from the
+  // options and keep the current selection valid.
+  const inputExt = files[0]?.name.split(".").pop()?.toLowerCase();
+  const formats = ALL_FORMATS.filter((f) => f.value !== inputExt);
+  const selected = formats.some((f) => f.value === outFormat)
+    ? outFormat
+    : (formats[0]?.value ?? outFormat);
+
  const hasFile = files.length > 0;
  const hasMultiple = files.length > 1;

  const handleProcess = () => {
-    const settings = { format: outFormat };
+    const settings = { format: selected };
    if (hasMultiple) {
      processAllFiles(files, settings);
    } else {
@@ -36,12 +50,15 @@ export function ConvertPresentationSettings() {
        </label>
        <select
          id="cp-format"
-          value={outFormat}
+          value={selected}
          onChange={(e) => setOutFormat(e.target.value as PresFormat)}
          className="w-full mt-0.5 px-2 py-1.5 rounded border border-border bg-background text-sm text-foreground"
        >
-          <option value="pptx">PPTX</option>
-          <option value="odp">ODP</option>
+          {formats.map((f) => (
+            <option key={f.value} value={f.value}>
+              {f.label}
+            </option>
+          ))}
        </select>
      </div>

@@ -7,6 +7,12 @@ import { useFileStore } from "@/stores/file-store";

 type SheetFormat = "xlsx" | "ods" | "csv";

+const ALL_FORMATS: { value: SheetFormat; label: string }[] = [
+  { value: "xlsx", label: "XLSX" },
+  { value: "ods", label: "ODS" },
+  { value: "csv", label: "CSV" },
+];
+
 export function ConvertSpreadsheetSettings() {
  const { t } = useTranslation();
  const s = t.toolSettings["convert-spreadsheet"];
@@ -16,11 +22,20 @@ export function ConvertSpreadsheetSettings() {

  const [outFormat, setOutFormat] = useState<SheetFormat>("ods");

+  // Never offer the input's own format as a target. LibreOffice rejects a
+  // same-format conversion ("already in that format"), so drop it from the
+  // options and keep the current selection valid.
+  const inputExt = files[0]?.name.split(".").pop()?.toLowerCase();
+  const formats = ALL_FORMATS.filter((f) => f.value !== inputExt);
+  const selected = formats.some((f) => f.value === outFormat)
+    ? outFormat
+    : (formats[0]?.value ?? outFormat);
+
  const hasFile = files.length > 0;
  const hasMultiple = files.length > 1;

  const handleProcess = () => {
-    const settings = { format: outFormat };
+    const settings = { format: selected };
    if (hasMultiple) {
      processAllFiles(files, settings);
    } else {
@@ -36,13 +51,15 @@ export function ConvertSpreadsheetSettings() {
        </label>
        <select
          id="cs-format"
-          value={outFormat}
+          value={selected}
          onChange={(e) => setOutFormat(e.target.value as SheetFormat)}
          className="w-full mt-0.5 px-2 py-1.5 rounded border border-border bg-background text-sm text-foreground"
        >
-          <option value="xlsx">XLSX</option>
-          <option value="ods">ODS</option>
-          <option value="csv">CSV</option>
+          {formats.map((f) => (
+            <option key={f.value} value={f.value}>
+              {f.label}
+            </option>
+          ))}
        </select>
      </div>

@@ -38,6 +38,34 @@ const LONG_RUNNING_TOOLS = new Set<string>(["content-aware-resize", "ai-canvas-e
 const UPLOAD_WEIGHT = 15;
 const SSE_STALL_TIMEOUT_MS = 300_000;

+/** Extension to MIME type for batch ZIP blob construction. Falls back to undefined (generic). */
+const MIME_BY_EXT: Record<string, string> = {
+  png: "image/png",
+  jpg: "image/jpeg",
+  jpeg: "image/jpeg",
+  gif: "image/gif",
+  webp: "image/webp",
+  avif: "image/avif",
+  svg: "image/svg+xml",
+  mp4: "video/mp4",
+  webm: "video/webm",
+  mov: "video/quicktime",
+  ogv: "video/ogg",
+  mp3: "audio/mpeg",
+  wav: "audio/wav",
+  ogg: "audio/ogg",
+  flac: "audio/flac",
+  m4a: "audio/mp4",
+  aac: "audio/aac",
+  pdf: "application/pdf",
+  txt: "text/plain",
+  csv: "text/csv",
+  json: "application/json",
+  xml: "application/xml",
+  html: "text/html",
+  zip: "application/zip",
+};
+
 export function useToolProcessor(toolId: string) {
  const { t } = useTranslation();
  const {
@@ -116,7 +144,7 @@ export function useToolProcessor(toolId: string) {
              if (elapsedRef.current) clearInterval(elapsedRef.current);
              clearActiveJob();
              setError(
-                "Processing timed out with no progress for 5 minutes. Try again or use a smaller image.",
+                "Processing timed out with no progress for 5 minutes. Try again or use a smaller file.",
              );
              setProcessing(false);
              setProgress(IDLE_PROGRESS);
@@ -264,7 +292,7 @@ export function useToolProcessor(toolId: string) {
            error: "Processing timed out",
          });
          setError(
-            "Processing timed out with no progress for 5 minutes. Try again or use a smaller image.",
+            "Processing timed out with no progress for 5 minutes. Try again or use a smaller file.",
          );
          setProcessing(false);
          setProgress(IDLE_PROGRESS);
@@ -604,7 +632,8 @@ export function useToolProcessor(toolId: string) {
        for (let i = 0; i < entries.length; i++) {
          const processedName = fileResults[String(i)];
          if (processedName && extracted[processedName]) {
-            const blobType = processedName.endsWith(".svg") ? "image/svg+xml" : undefined;
+            const ext = processedName.split(".").pop()?.toLowerCase() ?? "";
+            const blobType = MIME_BY_EXT[ext];
            const blob = new Blob(
              [extracted[processedName] as BlobPart],
              blobType ? { type: blobType } : undefined,
@@ -66,7 +66,12 @@ const NonNativePreview = lazy(() =>
  })),
 );

-/** Formats that browsers can render in <img> tags. */
+/**
+ * Formats that browsers can render in <img> tags.
+ * Intentionally image-only: all consumers (BeforeAfterSlider, SideBySideComparison,
+ * ImageViewer) render via <img>. Video/audio/PDF processed outputs are handled by
+ * dedicated display-mode branches (media-player, document) before this check runs.
+ */
 const BROWSER_PREVIEWABLE_EXTS = new Set([
  "jpg",
  "jpeg",
@@ -505,7 +510,7 @@ export function ToolPage() {
    const url = URL.createObjectURL(batchZipBlob);
    const a = document.createElement("a");
    a.href = url;
-    a.download = batchZipFilename ?? "processed-images.zip";
+    a.download = batchZipFilename ?? "processed-files.zip";
    a.click();
    URL.revokeObjectURL(url);
  }, [batchZipBlob, batchZipFilename]);
@@ -584,9 +589,9 @@ export function ToolPage() {
  const processedFileName =
    currentEntry?.processedFilename ??
    (processedUrl
-      ? decodeURIComponent(processedUrl.split("/").pop() ?? "processed-image")
-      : "processed-image");
-  const processedFileType = processedFileName.split(".").pop()?.toUpperCase() || "IMAGE";
+      ? decodeURIComponent(processedUrl.split("/").pop() ?? "processed-file")
+      : "processed-file");
+  const processedFileType = processedFileName.split(".").pop()?.toUpperCase() || "FILE";
  const isProcessedPreviewable = processedUrl
    ? canBrowserPreview(processedUrl, currentEntry?.processedFilename ?? processedFileName)
    : false;