mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
fix: harden install queue/dispatcher lifecycle and repair review-sweep regressions (#395)
Fixes 15 defects found by a max-effort multi-agent review of the last 6 merged PRs (#388, #390, #391, #392, #393, #394), all adversarially verified before fixing. Install queue + dispatcher (the serious cluster): - features.ts: finalize the installer child exactly once. A failed spawn fires both "error" and "close", and the second event released the file lock and active slot that pump() had just handed to the next queued bundle, letting two pip processes write the same venv concurrently. Outcome recording now happens before pump() so the next bundle's first progress frame cannot race the previous install's bookkeeping. - feature-status.ts: keep failed-install errors in a per-bundle map instead of the single progress slot. With the queue auto-starting the next install, the slot was overwritten within seconds and a failed install vanished without ever surfacing to GET /features. - bridge.ts: scope child lifecycle per process (stopped-children set + request generation tags) instead of an instance-wide shuttingDown flag that the next spawn reset. A stale SIGTERMed child's late close event could record a phantom crash (5 of which permanently disable the dispatcher), null out the freshly spawned child, and reject the new child's pending requests. The request-timeout kill path still counts as a real crash. - install_feature.py: the pre-write disk re-check measured ai_dir's filesystem even when budgeting the cross-filesystem copy that lands on the venv's disk; now each budget is checked against the filesystem the bytes actually land on, so ENOSPC cannot strike mid-write and leave site-packages half overwritten. Behavior regressions: - embed-subtitles: preserve pre-existing subtitle tracks (0:s?) and MKV attachments (0:t?) that the -map 0:v:0/0:a? rewrite silently dropped; data streams stay unmapped on purpose (the actual MPEG remux fix). The new subtitle maps first so the language tag hits the right stream. - usage-survey-overlay: fail closed when the settings fetch fails; the fail-open path rendered the blocking survey against an unhealthy API and soft-locked admins, the lock-out class #392 fixed. - features-store: queued bundles poll instead of each holding an SSE connection (Install All could pin 7 EventSources and exhaust the browser's 6-per-origin HTTP/1.1 limit, hanging the whole app); listenToProgress closes any prior stream and stops any poll before subscribing; installAll skips bundles already installing or queued. Contracts, tests, i18n: - openapi.yaml: add "queued" to the features status enum and document downloadBytes/installedBytes (Schemathesis conformance). - feature-lifecycle e2e: queue transcription (~0.5 GB) instead of ocr (~6 GB) and give the test a budget that covers both install drains (the stacked waits exceeded the old 900s timeout). - docker-compose.qa.yml: parameterize the host port (QA_APP_PORT) so QA_PROJECT_NAME concurrent stacks can actually bind. - compare + watermark-image: restore per-input error attribution ("Invalid first/second image", "Invalid watermark image") lost in the shared-handler migration. - ai-features-section: the "{size} on disk" suffix now goes through i18n; key added to all 21 locales. - watermark-image + content-aware-resize: migrate to the shared inputHandlerFor("image") chain like compare/vectorize/compose, fixing drift in the inline copies (no SVG sanitize, no RAW extension hint, no AVIF probe). Verified: typecheck across 9 workspaces, Biome clean on all changed files, 584 targeted unit tests and 249 integration tests green (including real-ffmpeg embed-subtitles runs). One unit test updated to the new poll-while-queued contract with a single-EventSource assertion. Claude-Session: https://claude.ai/code/session_017mR1HiHaf3a1BmUtrHX4j3
This commit is contained in:
@@ -257,9 +257,18 @@ export function getInstallingBundle(): {
|
|||||||
let currentProgress: {
|
let currentProgress: {
|
||||||
bundleId: string;
|
bundleId: string;
|
||||||
progress: { percent: number; stage: string } | null;
|
progress: { percent: number; stage: string } | null;
|
||||||
error: string | null;
|
|
||||||
} | null = null;
|
} | null = null;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Failed-install errors keyed by bundle. Errors live outside the single
|
||||||
|
* progress slot because the queue pump starts the next install immediately
|
||||||
|
* after a failure: if the error sat in the slot, the next bundle's first
|
||||||
|
* progress frame would overwrite it and the failure would never surface to
|
||||||
|
* GET /features. An entry clears when a new install of the same bundle
|
||||||
|
* starts (its first setInstallProgress call with a null error).
|
||||||
|
*/
|
||||||
|
const installErrors = new Map<string, string>();
|
||||||
|
|
||||||
export function setInstallProgress(
|
export function setInstallProgress(
|
||||||
bundleId: string | null,
|
bundleId: string | null,
|
||||||
progress: { percent: number; stage: string } | null,
|
progress: { percent: number; stage: string } | null,
|
||||||
@@ -267,9 +276,20 @@ export function setInstallProgress(
|
|||||||
): void {
|
): void {
|
||||||
if (!bundleId) {
|
if (!bundleId) {
|
||||||
currentProgress = null;
|
currentProgress = null;
|
||||||
|
installErrors.clear();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
currentProgress = { bundleId, progress, error };
|
if (error !== null) {
|
||||||
|
installErrors.set(bundleId, error);
|
||||||
|
if (currentProgress?.bundleId === bundleId) currentProgress = null;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
installErrors.delete(bundleId);
|
||||||
|
if (progress === null) {
|
||||||
|
if (currentProgress?.bundleId === bundleId) currentProgress = null;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
currentProgress = { bundleId, progress };
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Manifest reading ────────────────────────────────────────────────────
|
// ── Manifest reading ────────────────────────────────────────────────────
|
||||||
@@ -519,14 +539,15 @@ export function getFeatureStates(): FeatureBundleState[] {
|
|||||||
let error: string | null = null;
|
let error: string | null = null;
|
||||||
let progress: { percent: number; stage: string } | null = null;
|
let progress: { percent: number; stage: string } | null = null;
|
||||||
|
|
||||||
|
const installError = installErrors.get(bundle.id) ?? null;
|
||||||
if (lock && lock.bundleId === bundle.id) {
|
if (lock && lock.bundleId === bundle.id) {
|
||||||
status = "installing";
|
status = "installing";
|
||||||
if (currentProgress && currentProgress.bundleId === bundle.id) {
|
if (currentProgress && currentProgress.bundleId === bundle.id) {
|
||||||
progress = currentProgress.progress;
|
progress = currentProgress.progress;
|
||||||
if (currentProgress.error) {
|
|
||||||
status = "error";
|
|
||||||
error = currentProgress.error;
|
|
||||||
}
|
}
|
||||||
|
if (installError) {
|
||||||
|
status = "error";
|
||||||
|
error = installError;
|
||||||
}
|
}
|
||||||
} else if (installedBundle) {
|
} else if (installedBundle) {
|
||||||
// Verify model files exist and are properly sized
|
// Verify model files exist and are properly sized
|
||||||
@@ -540,9 +561,9 @@ export function getFeatureStates(): FeatureBundleState[] {
|
|||||||
} else if (queuedIds.has(bundle.id)) {
|
} else if (queuedIds.has(bundle.id)) {
|
||||||
// Waiting behind the active install in the server-side queue.
|
// Waiting behind the active install in the server-side queue.
|
||||||
status = "queued";
|
status = "queued";
|
||||||
} else if (currentProgress?.bundleId === bundle.id && currentProgress.error) {
|
} else if (installError) {
|
||||||
status = "error";
|
status = "error";
|
||||||
error = currentProgress.error;
|
error = installError;
|
||||||
}
|
}
|
||||||
|
|
||||||
const archive = manifest?.bundles[bundle.id]?.archives?.[arch];
|
const archive = manifest?.bundles[bundle.id]?.archives?.[arch];
|
||||||
|
|||||||
@@ -6450,11 +6450,22 @@ paths:
|
|||||||
description: Human-readable description of the feature bundle
|
description: Human-readable description of the feature bundle
|
||||||
status:
|
status:
|
||||||
type: string
|
type: string
|
||||||
enum: [installed, not_installed, installing, error]
|
enum: [installed, not_installed, installing, queued, error]
|
||||||
installedVersion:
|
installedVersion:
|
||||||
type: [string, "null"]
|
type: [string, "null"]
|
||||||
estimatedSize:
|
estimatedSize:
|
||||||
type: string
|
type: string
|
||||||
|
downloadBytes:
|
||||||
|
type: [integer, "null"]
|
||||||
|
description: >-
|
||||||
|
Compressed archive size for this host's
|
||||||
|
architecture, when the manifest records it. Null in
|
||||||
|
native (non-Docker) mode or when unmeasured.
|
||||||
|
installedBytes:
|
||||||
|
type: [integer, "null"]
|
||||||
|
description: >-
|
||||||
|
Extracted on-disk size for this host's
|
||||||
|
architecture, when the manifest records it.
|
||||||
enablesTools:
|
enablesTools:
|
||||||
type: array
|
type: array
|
||||||
items:
|
items:
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ export async function verifyPassword(password: string, stored: string): Promise<
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Fixed-cost hash used to equalize login timing for usernames that don't
|
* Fixed-cost hash used to equalize login timing for usernames that don't
|
||||||
* exist (or have no local password). Computed once per process and cached --
|
* exist (or have no local password). Computed once per process and cached;
|
||||||
* without this, a login attempt for an unknown username returns as soon as
|
* without this, a login attempt for an unknown username returns as soon as
|
||||||
* the user lookup misses, while a wrong password for a real user waits on a
|
* the user lookup misses, while a wrong password for a real user waits on a
|
||||||
* full scrypt run. That gap is a timing side-channel an attacker can use to
|
* full scrypt run. That gap is a timing side-channel an attacker can use to
|
||||||
|
|||||||
@@ -74,6 +74,11 @@ function startInstall(bundleId: string, jobId: string): void {
|
|||||||
const installStartTime = Date.now();
|
const installStartTime = Date.now();
|
||||||
|
|
||||||
void (async () => {
|
void (async () => {
|
||||||
|
// Mark the install as started right away: a zero-percent frame claims the
|
||||||
|
// progress slot and clears any stale error left by a previous failed
|
||||||
|
// attempt of this bundle, so a retry never shows the old failure.
|
||||||
|
setInstallProgress(bundleId, { percent: 0, stage: "" }, null);
|
||||||
|
|
||||||
// Hold the venv lock across the whole install so no AI tool job loads
|
// Hold the venv lock across the whole install so no AI tool job loads
|
||||||
// native libs from the venv while pip is rewriting them (that segfaults
|
// native libs from the venv while pip is rewriting them (that segfaults
|
||||||
// the sidecar). This awaits any in-flight AI job before the installer
|
// the sidecar). This awaits any in-flight AI job before the installer
|
||||||
@@ -87,6 +92,21 @@ function startInstall(bundleId: string, jobId: string): void {
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// A failed spawn fires BOTH "error" and "close", so both handlers funnel
|
||||||
|
// their teardown through this once-guard. Without it the second event
|
||||||
|
// would release the file lock and active slot that pump() just handed to
|
||||||
|
// the next queued bundle, letting two installers run into the same venv
|
||||||
|
// at once (the corruption the lock exists to prevent).
|
||||||
|
let finalized = false;
|
||||||
|
const finalizeOnce = (): boolean => {
|
||||||
|
if (finalized) return false;
|
||||||
|
finalized = true;
|
||||||
|
releaseVenvOnce();
|
||||||
|
releaseInstallLock();
|
||||||
|
clearActive();
|
||||||
|
return true;
|
||||||
|
};
|
||||||
|
|
||||||
const child = spawn(pythonPath, [scriptPath, bundleId, manifestPath, modelsDir], {
|
const child = spawn(pythonPath, [scriptPath, bundleId, manifestPath, modelsDir], {
|
||||||
stdio: ["ignore", "pipe", "pipe"],
|
stdio: ["ignore", "pipe", "pipe"],
|
||||||
env: {
|
env: {
|
||||||
@@ -139,15 +159,12 @@ function startInstall(bundleId: string, jobId: string): void {
|
|||||||
});
|
});
|
||||||
|
|
||||||
child.on("close", (code) => {
|
child.on("close", (code) => {
|
||||||
releaseVenvOnce();
|
if (!finalizeOnce()) return;
|
||||||
releaseInstallLock();
|
|
||||||
clearActive();
|
|
||||||
pump();
|
|
||||||
|
|
||||||
if (code === 0) {
|
if (code === 0) {
|
||||||
invalidateCache();
|
invalidateCache();
|
||||||
shutdownDispatcher();
|
shutdownDispatcher();
|
||||||
setInstallProgress(null, null, null);
|
setInstallProgress(bundleId, null, null);
|
||||||
updateSingleFileProgress({ jobId, phase: "complete", percent: 100, stage: "Complete" });
|
updateSingleFileProgress({ jobId, phase: "complete", percent: 100, stage: "Complete" });
|
||||||
trackEvent(ANALYTICS_EVENTS.AI_BUNDLE_ACTION, {
|
trackEvent(ANALYTICS_EVENTS.AI_BUNDLE_ACTION, {
|
||||||
bundle_id: bundleId,
|
bundle_id: bundleId,
|
||||||
@@ -195,16 +212,19 @@ function startInstall(bundleId: string, jobId: string): void {
|
|||||||
setInstallProgress(bundleId, null, errorMsg);
|
setInstallProgress(bundleId, null, errorMsg);
|
||||||
updateSingleFileProgress({ jobId, phase: "failed", percent: 0, error: errorMsg });
|
updateSingleFileProgress({ jobId, phase: "failed", percent: 0, error: errorMsg });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Record the outcome BEFORE starting the next install, so the next
|
||||||
|
// bundle's first progress frame cannot race with (or be wiped by) this
|
||||||
|
// install's completion/failure bookkeeping.
|
||||||
|
pump();
|
||||||
});
|
});
|
||||||
|
|
||||||
child.on("error", (err) => {
|
child.on("error", (err) => {
|
||||||
releaseVenvOnce();
|
if (!finalizeOnce()) return;
|
||||||
releaseInstallLock();
|
|
||||||
clearActive();
|
|
||||||
pump();
|
|
||||||
const errorMsg = `Failed to spawn install process: ${err.message}`;
|
const errorMsg = `Failed to spawn install process: ${err.message}`;
|
||||||
setInstallProgress(bundleId, null, errorMsg);
|
setInstallProgress(bundleId, null, errorMsg);
|
||||||
updateSingleFileProgress({ jobId, phase: "failed", percent: 0, error: errorMsg });
|
updateSingleFileProgress({ jobId, phase: "failed", percent: 0, error: errorMsg });
|
||||||
|
pump();
|
||||||
});
|
});
|
||||||
})();
|
})();
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -48,16 +48,25 @@ export function registerCompare(app: FastifyInstance) {
|
|||||||
|
|
||||||
try {
|
try {
|
||||||
const imageHandler = inputHandlerFor("image");
|
const imageHandler = inputHandlerFor("image");
|
||||||
bufferA = (
|
// Attribute validation failures to the specific upload: with two
|
||||||
await imageHandler.prepare(bufferA, filenameA, {
|
// inputs, a bare "Invalid image: ..." does not tell the user which of
|
||||||
scratchDir: tmpdir(),
|
// their files was rejected. Restores the route's pre-migration
|
||||||
})
|
// "Invalid first/second image: ..." message contract.
|
||||||
).buffer;
|
const prepareInput = async (buffer: Buffer, filename: string, which: string) => {
|
||||||
bufferB = (
|
try {
|
||||||
await imageHandler.prepare(bufferB, filenameB, {
|
return (await imageHandler.prepare(buffer, filename, { scratchDir: tmpdir() })).buffer;
|
||||||
scratchDir: tmpdir(),
|
} catch (err) {
|
||||||
})
|
if (err instanceof InputValidationError) {
|
||||||
).buffer;
|
const message = err.message.startsWith("Invalid image")
|
||||||
|
? err.message.replace("Invalid image", `Invalid ${which} image`)
|
||||||
|
: `${err.message} (${which} image)`;
|
||||||
|
throw new InputValidationError(message, err.statusCode, err.details);
|
||||||
|
}
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
bufferA = await prepareInput(bufferA, filenameA, "first");
|
||||||
|
bufferB = await prepareInput(bufferB, filenameB, "second");
|
||||||
|
|
||||||
// Normalize both to same size for comparison
|
// Normalize both to same size for comparison
|
||||||
const metaA = await sharp(bufferA).metadata();
|
const metaA = await sharp(bufferA).metadata();
|
||||||
|
|||||||
@@ -12,6 +12,8 @@ import { sanitizeFilename } from "../../lib/filename.js";
|
|||||||
import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.js";
|
import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.js";
|
||||||
import { decodeHeic } from "../../lib/heic-converter.js";
|
import { decodeHeic } from "../../lib/heic-converter.js";
|
||||||
import { putObject } from "../../lib/object-storage.js";
|
import { putObject } from "../../lib/object-storage.js";
|
||||||
|
import { InputValidationError } from "../../modality/contract.js";
|
||||||
|
import { inputHandlerFor } from "../../modality/input-handler.js";
|
||||||
import { registerToolProcessFn } from "../tool-factory.js";
|
import { registerToolProcessFn } from "../tool-factory.js";
|
||||||
|
|
||||||
const settingsSchema = z.object({
|
const settingsSchema = z.object({
|
||||||
@@ -59,38 +61,26 @@ export function registerContentAwareResize(app: FastifyInstance) {
|
|||||||
return reply.status(400).send({ error: "No image file provided" });
|
return reply.status(400).send({ error: "No image file provided" });
|
||||||
}
|
}
|
||||||
|
|
||||||
const validation = await validateImageBuffer(fileBuffer, filename);
|
// Shared image input chain (validate, HEIC/RAW decode with filename
|
||||||
if (!validation.valid) {
|
// rewrite, SVG sanitize, AVIF probe, autoOrient): the same handler the
|
||||||
return reply.status(400).send({ error: `Invalid image: ${validation.reason}` });
|
// factory-based image routes use, replacing an inline copy that had
|
||||||
}
|
// already drifted (it never sanitized SVG or passed the file extension
|
||||||
|
// to the RAW decoder).
|
||||||
// Decode HEIC/HEIF input (caire can't read HEIF containers)
|
|
||||||
if (validation.format === "heif") {
|
|
||||||
try {
|
try {
|
||||||
fileBuffer = await decodeHeic(fileBuffer);
|
const prepared = await inputHandlerFor("image").prepare(fileBuffer, filename, {
|
||||||
const ext = filename.match(/\.[^.]+$/)?.[0];
|
scratchDir: tmpdir(),
|
||||||
if (ext) filename = `${filename.slice(0, -ext.length)}.png`;
|
});
|
||||||
|
fileBuffer = prepared.buffer;
|
||||||
|
filename = prepared.filename;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
|
if (err instanceof InputValidationError) {
|
||||||
|
return reply.status(err.statusCode).send({ error: err.message, details: err.details });
|
||||||
|
}
|
||||||
return reply.status(422).send({
|
return reply.status(422).send({
|
||||||
error: "Failed to decode HEIC/HEIF file",
|
error: "Failed to prepare image",
|
||||||
details: friendlyError(err instanceof Error ? err.message : String(err)),
|
details: friendlyError(err instanceof Error ? err.message : String(err)),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
|
||||||
|
|
||||||
// Decode CLI-decoded formats (RAW, TGA, PSD, EXR, HDR)
|
|
||||||
if (needsCliDecode(validation.format)) {
|
|
||||||
try {
|
|
||||||
fileBuffer = await decodeToSharpCompat(fileBuffer, validation.format);
|
|
||||||
const ext = filename.match(/\.[^.]+$/)?.[0];
|
|
||||||
if (ext) filename = `${filename.slice(0, -ext.length)}.png`;
|
|
||||||
} catch (err) {
|
|
||||||
return reply.status(422).send({
|
|
||||||
error: `Failed to decode ${validation.format} file`,
|
|
||||||
details: friendlyError(err instanceof Error ? err.message : String(err)),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Validate settings
|
// Validate settings
|
||||||
let settings: Settings;
|
let settings: Settings;
|
||||||
@@ -124,9 +114,6 @@ export function registerContentAwareResize(app: FastifyInstance) {
|
|||||||
"Starting content-aware resize",
|
"Starting content-aware resize",
|
||||||
);
|
);
|
||||||
|
|
||||||
// Auto-orient to fix EXIF rotation before seam carving
|
|
||||||
fileBuffer = await autoOrient(fileBuffer);
|
|
||||||
|
|
||||||
const jobId = randomUUID();
|
const jobId = randomUUID();
|
||||||
const scratchDir = join(tmpdir(), "snapotter-scratch", jobId);
|
const scratchDir = join(tmpdir(), "snapotter-scratch", jobId);
|
||||||
await mkdir(scratchDir, { recursive: true });
|
await mkdir(scratchDir, { recursive: true });
|
||||||
|
|||||||
@@ -64,8 +64,19 @@ export function registerEmbedSubtitles(app: FastifyInstance) {
|
|||||||
"0:v:0",
|
"0:v:0",
|
||||||
"-map",
|
"-map",
|
||||||
"0:a?",
|
"0:a?",
|
||||||
|
// The new subtitle is mapped BEFORE the source's existing subtitle
|
||||||
|
// tracks so it is always output subtitle stream s:0, which the
|
||||||
|
// language tag below targets.
|
||||||
"-map",
|
"-map",
|
||||||
"1:0",
|
"1:0",
|
||||||
|
// Preserve subtitle tracks already in the source (re-encoded to the
|
||||||
|
// container's text codec, matching the old `-map 0` behavior).
|
||||||
|
// Data streams stay unmapped on purpose: MPEG data tracks such as
|
||||||
|
// teletext cannot be remuxed into mp4/mkv and fail the whole job.
|
||||||
|
"-map",
|
||||||
|
"0:s?",
|
||||||
|
// MKV can carry attachment streams (embedded fonts); MP4 cannot.
|
||||||
|
...(toMp4 ? [] : ["-map", "0:t?"]),
|
||||||
...(reencodeInputStreams
|
...(reencodeInputStreams
|
||||||
? [...videoEncodeArgsForContainer(outExt), ...audioEncodeArgsForContainer(outExt)]
|
? [...videoEncodeArgsForContainer(outExt), ...audioEncodeArgsForContainer(outExt)]
|
||||||
: ["-c:v", "copy", "-c:a", "copy"]),
|
: ["-c:v", "copy", "-c:a", "copy"]),
|
||||||
|
|||||||
@@ -1,16 +1,14 @@
|
|||||||
import { randomUUID } from "node:crypto";
|
import { randomUUID } from "node:crypto";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
import type { FastifyInstance } from "fastify";
|
import type { FastifyInstance } from "fastify";
|
||||||
import sharp from "sharp";
|
import sharp from "sharp";
|
||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
import { autoOrient } from "../../lib/auto-orient.js";
|
|
||||||
import { formatZodErrors } from "../../lib/errors.js";
|
import { formatZodErrors } from "../../lib/errors.js";
|
||||||
import { validateImageBuffer } from "../../lib/file-validation.js";
|
|
||||||
import { sanitizeFilename } from "../../lib/filename.js";
|
import { sanitizeFilename } from "../../lib/filename.js";
|
||||||
import { decodeToSharpCompat, needsCliDecode } from "../../lib/format-decoders.js";
|
|
||||||
import { decodeHeic } from "../../lib/heic-converter.js";
|
|
||||||
import { putObject } from "../../lib/object-storage.js";
|
import { putObject } from "../../lib/object-storage.js";
|
||||||
import { resolveOutputFormat } from "../../lib/output-format.js";
|
import { resolveOutputFormat } from "../../lib/output-format.js";
|
||||||
import { decompressSvgz, sanitizeSvg } from "../../lib/svg-sanitize.js";
|
import { InputValidationError } from "../../modality/contract.js";
|
||||||
|
import { inputHandlerFor } from "../../modality/input-handler.js";
|
||||||
|
|
||||||
const settingsSchema = z.object({
|
const settingsSchema = z.object({
|
||||||
position: z
|
position: z
|
||||||
@@ -81,86 +79,35 @@ export function registerWatermarkImage(app: FastifyInstance) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const valMain = await validateImageBuffer(mainBuffer, filename);
|
// Shared image input chain (validate, HEIC/RAW decode, SVG sanitize,
|
||||||
if (!valMain.valid) {
|
// AVIF probe, autoOrient), the same handler compare/vectorize/compose
|
||||||
return reply.status(400).send({ error: `Invalid image: ${valMain.reason}` });
|
// use. It also rewrites the filename extension after a decode so the
|
||||||
}
|
// output name and resolveOutputFormat below stay consistent.
|
||||||
if (valMain.format === "heif") {
|
const imageHandler = inputHandlerFor("image");
|
||||||
try {
|
const preparedMain = await imageHandler.prepare(mainBuffer, filename, {
|
||||||
mainBuffer = await decodeHeic(mainBuffer);
|
scratchDir: tmpdir(),
|
||||||
} catch (err) {
|
|
||||||
return reply.status(422).send({
|
|
||||||
error: "Failed to decode HEIC file. Ensure libheif-examples is installed.",
|
|
||||||
details: err instanceof Error ? err.message : String(err),
|
|
||||||
});
|
});
|
||||||
}
|
mainBuffer = preparedMain.buffer;
|
||||||
}
|
filename = preparedMain.filename;
|
||||||
if (needsCliDecode(valMain.format)) {
|
|
||||||
try {
|
|
||||||
const ext = filename.split(".").pop()?.toLowerCase();
|
|
||||||
mainBuffer = await decodeToSharpCompat(mainBuffer, valMain.format, ext);
|
|
||||||
} catch {
|
|
||||||
try {
|
|
||||||
await sharp(mainBuffer).metadata();
|
|
||||||
} catch (err) {
|
|
||||||
return reply.status(422).send({
|
|
||||||
error: `Failed to decode ${valMain.format.toUpperCase()} file`,
|
|
||||||
details: err instanceof Error ? err.message : String(err),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (valMain.format === "svg") {
|
|
||||||
try {
|
|
||||||
mainBuffer = decompressSvgz(mainBuffer);
|
|
||||||
mainBuffer = sanitizeSvg(mainBuffer);
|
|
||||||
} catch (err) {
|
|
||||||
return reply.status(400).send({
|
|
||||||
error: err instanceof Error ? err.message : "Invalid SVG",
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
mainBuffer = await autoOrient(mainBuffer);
|
|
||||||
|
|
||||||
const valWm = await validateImageBuffer(watermarkBuffer, watermarkFilename);
|
|
||||||
if (!valWm.valid) {
|
|
||||||
return reply.status(400).send({ error: `Invalid watermark image: ${valWm.reason}` });
|
|
||||||
}
|
|
||||||
if (valWm.format === "heif") {
|
|
||||||
try {
|
try {
|
||||||
watermarkBuffer = await decodeHeic(watermarkBuffer);
|
watermarkBuffer = (
|
||||||
|
await imageHandler.prepare(watermarkBuffer, watermarkFilename, {
|
||||||
|
scratchDir: tmpdir(),
|
||||||
|
})
|
||||||
|
).buffer;
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
return reply.status(422).send({
|
// Attribute validation failures to the watermark upload so the user
|
||||||
error: "Failed to decode watermark (HEIC). Ensure libheif-examples is installed.",
|
// knows which of the two files was rejected, preserving the route's
|
||||||
details: err instanceof Error ? err.message : String(err),
|
// established "Invalid watermark image: ..." message contract.
|
||||||
});
|
if (err instanceof InputValidationError) {
|
||||||
|
const message = err.message.startsWith("Invalid image")
|
||||||
|
? err.message.replace("Invalid image", "Invalid watermark image")
|
||||||
|
: `${err.message} (watermark)`;
|
||||||
|
throw new InputValidationError(message, err.statusCode, err.details);
|
||||||
}
|
}
|
||||||
|
throw err;
|
||||||
}
|
}
|
||||||
if (needsCliDecode(valWm.format)) {
|
|
||||||
try {
|
|
||||||
watermarkBuffer = await decodeToSharpCompat(watermarkBuffer, valWm.format);
|
|
||||||
} catch {
|
|
||||||
try {
|
|
||||||
await sharp(watermarkBuffer).metadata();
|
|
||||||
} catch (err) {
|
|
||||||
return reply.status(422).send({
|
|
||||||
error: `Failed to decode watermark (${valWm.format.toUpperCase()})`,
|
|
||||||
details: err instanceof Error ? err.message : String(err),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (valWm.format === "svg") {
|
|
||||||
try {
|
|
||||||
watermarkBuffer = decompressSvgz(watermarkBuffer);
|
|
||||||
watermarkBuffer = sanitizeSvg(watermarkBuffer);
|
|
||||||
} catch (err) {
|
|
||||||
return reply.status(400).send({
|
|
||||||
error: err instanceof Error ? err.message : "Invalid SVG (watermark)",
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
watermarkBuffer = await autoOrient(watermarkBuffer);
|
|
||||||
|
|
||||||
const mainImage = sharp(mainBuffer);
|
const mainImage = sharp(mainBuffer);
|
||||||
const mainMeta = await mainImage.metadata();
|
const mainMeta = await mainImage.metadata();
|
||||||
@@ -242,6 +189,9 @@ export function registerWatermarkImage(app: FastifyInstance) {
|
|||||||
processedSize: result.length,
|
processedSize: result.length,
|
||||||
});
|
});
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
|
if (err instanceof InputValidationError) {
|
||||||
|
return reply.status(err.statusCode).send({ error: err.message, details: err.details });
|
||||||
|
}
|
||||||
return reply.status(422).send({
|
return reply.status(422).send({
|
||||||
error: "Processing failed",
|
error: "Processing failed",
|
||||||
details: err instanceof Error ? err.message : "Image processing failed",
|
details: err instanceof Error ? err.message : "Image processing failed",
|
||||||
|
|||||||
@@ -67,7 +67,13 @@ export function UsageSurveyOverlay() {
|
|||||||
if (!eligibleAuthState || !eligibleRoute) return;
|
if (!eligibleAuthState || !eligibleRoute) return;
|
||||||
apiGet<{ settings: Record<string, string> }>("/v1/settings")
|
apiGet<{ settings: Record<string, string> }>("/v1/settings")
|
||||||
.then((data) => setSettings(data.settings))
|
.then((data) => setSettings(data.settings))
|
||||||
.catch(() => setSettings({}));
|
.catch(() => {
|
||||||
|
// Fail closed: without settings we cannot know whether the admin
|
||||||
|
// already answered, and showing the full-screen overlay while the
|
||||||
|
// API is unhealthy would soft-lock them (the dismiss/continue
|
||||||
|
// writes would fail against the same unhealthy API). Skipping the
|
||||||
|
// survey for this load is the cheap, recoverable outcome.
|
||||||
|
});
|
||||||
}, [eligibleAuthState, eligibleRoute]);
|
}, [eligibleAuthState, eligibleRoute]);
|
||||||
|
|
||||||
const visible =
|
const visible =
|
||||||
|
|||||||
@@ -285,7 +285,12 @@ function BundleCard({
|
|||||||
<p className="text-xs text-muted-foreground">
|
<p className="text-xs text-muted-foreground">
|
||||||
{bundle.description} (~
|
{bundle.description} (~
|
||||||
{bundle.downloadBytes ? formatFileSize(bundle.downloadBytes) : bundle.estimatedSize}
|
{bundle.downloadBytes ? formatFileSize(bundle.downloadBytes) : bundle.estimatedSize}
|
||||||
{bundle.installedBytes ? `, ${formatFileSize(bundle.installedBytes)} on disk` : ""})
|
{bundle.installedBytes
|
||||||
|
? `, ${format(t.settings.aiFeatures.sizeOnDisk, {
|
||||||
|
size: formatFileSize(bundle.installedBytes),
|
||||||
|
})}`
|
||||||
|
: ""}
|
||||||
|
)
|
||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
<div className="flex items-center gap-3 shrink-0 ms-4">
|
<div className="flex items-center gap-3 shrink-0 ms-4">
|
||||||
|
|||||||
@@ -96,6 +96,14 @@ export const useFeaturesStore = create<FeaturesState>((set, get) => {
|
|||||||
maybeFinishInstallAll();
|
maybeFinishInstallAll();
|
||||||
};
|
};
|
||||||
|
|
||||||
|
const stopPolling = (bundleId: string) => {
|
||||||
|
const ref = pollRefs[bundleId];
|
||||||
|
if (ref) {
|
||||||
|
clearInterval(ref);
|
||||||
|
delete pollRefs[bundleId];
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
const startPolling = (bundleId: string) => {
|
const startPolling = (bundleId: string) => {
|
||||||
if (pollRefs[bundleId]) return;
|
if (pollRefs[bundleId]) return;
|
||||||
pollRefs[bundleId] = setInterval(async () => {
|
pollRefs[bundleId] = setInterval(async () => {
|
||||||
@@ -126,8 +134,7 @@ export const useFeaturesStore = create<FeaturesState>((set, get) => {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Terminal: installed / error / not_installed.
|
// Terminal: installed / error / not_installed.
|
||||||
clearInterval(pollRefs[bundleId]);
|
stopPolling(bundleId);
|
||||||
delete pollRefs[bundleId];
|
|
||||||
stopTracking(bundleId);
|
stopTracking(bundleId);
|
||||||
onInstallSettled(
|
onInstallSettled(
|
||||||
bundleId,
|
bundleId,
|
||||||
@@ -138,6 +145,13 @@ export const useFeaturesStore = create<FeaturesState>((set, get) => {
|
|||||||
};
|
};
|
||||||
|
|
||||||
const listenToProgress = (bundleId: string, jobId: string) => {
|
const listenToProgress = (bundleId: string, jobId: string) => {
|
||||||
|
// One tracker per bundle: close any previous stream and stop any poll so
|
||||||
|
// a re-POST (which the server dedups to the same job) can never leave two
|
||||||
|
// live subscriptions whose terminal events each run the settle logic.
|
||||||
|
esRefs[bundleId]?.close();
|
||||||
|
delete esRefs[bundleId];
|
||||||
|
stopPolling(bundleId);
|
||||||
|
|
||||||
const es = new EventSource(`/api/v1/jobs/${jobId}/progress`);
|
const es = new EventSource(`/api/v1/jobs/${jobId}/progress`);
|
||||||
esRefs[bundleId] = es;
|
esRefs[bundleId] = es;
|
||||||
|
|
||||||
@@ -277,7 +291,7 @@ export const useFeaturesStore = create<FeaturesState>((set, get) => {
|
|||||||
},
|
},
|
||||||
|
|
||||||
installBundle: async (bundleId: string) => {
|
installBundle: async (bundleId: string) => {
|
||||||
// Always POST immediately -- the server owns the queue now, so a POST is
|
// Always POST immediately: the server owns the queue now, so a POST is
|
||||||
// durable even if this tab closes. Optimistically show "installing"; the
|
// durable even if this tab closes. Optimistically show "installing"; the
|
||||||
// response tells us whether it actually started or got queued.
|
// response tells us whether it actually started or got queued.
|
||||||
const errors = { ...get().errors };
|
const errors = { ...get().errors };
|
||||||
@@ -294,16 +308,21 @@ export const useFeaturesStore = create<FeaturesState>((set, get) => {
|
|||||||
{},
|
{},
|
||||||
);
|
);
|
||||||
if (result.queued) {
|
if (result.queued) {
|
||||||
// Server queued it behind an active install; show the pill instead of
|
// Server queued it behind an active install; show the pill and poll
|
||||||
// a progress bar until the first progress frame arrives.
|
// for the transition instead of holding an SSE connection open. An
|
||||||
|
// EventSource per queued bundle would let Install All pin up to 7
|
||||||
|
// connections for the whole run, exhausting the browser's
|
||||||
|
// per-origin limit on HTTP/1.1 and starving every other request.
|
||||||
const installing = { ...get().installing };
|
const installing = { ...get().installing };
|
||||||
delete installing[bundleId];
|
delete installing[bundleId];
|
||||||
set({
|
set({
|
||||||
installing,
|
installing,
|
||||||
queued: get().queued.includes(bundleId) ? get().queued : [...get().queued, bundleId],
|
queued: get().queued.includes(bundleId) ? get().queued : [...get().queued, bundleId],
|
||||||
});
|
});
|
||||||
}
|
startPolling(bundleId);
|
||||||
|
} else {
|
||||||
listenToProgress(bundleId, result.jobId);
|
listenToProgress(bundleId, result.jobId);
|
||||||
|
}
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
stopTracking(bundleId);
|
stopTracking(bundleId);
|
||||||
|
|
||||||
@@ -344,7 +363,12 @@ export const useFeaturesStore = create<FeaturesState>((set, get) => {
|
|||||||
},
|
},
|
||||||
|
|
||||||
installAll: async () => {
|
installAll: async () => {
|
||||||
const pending = get().bundles.filter((b) => b.status !== "installed");
|
// Skip bundles the server is already installing or holding in its
|
||||||
|
// queue: re-POSTing them just dedups server-side and used to leave a
|
||||||
|
// second progress subscription racing the first one's terminal events.
|
||||||
|
const pending = get().bundles.filter(
|
||||||
|
(b) => b.status !== "installed" && b.status !== "installing" && b.status !== "queued",
|
||||||
|
);
|
||||||
if (pending.length === 0) return;
|
if (pending.length === 0) return;
|
||||||
|
|
||||||
// Mark every pending bundle up front so the run never looks "drained"
|
// Mark every pending bundle up front so the run never looks "drained"
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ def detect_arch() -> str:
|
|||||||
Only two archive variants are currently published to the bundle repo:
|
Only two archive variants are currently published to the bundle repo:
|
||||||
'amd64-gpu' and 'arm64-cpu' (see deepsafe/feature-bundles). There is no
|
'amd64-gpu' and 'arm64-cpu' (see deepsafe/feature-bundles). There is no
|
||||||
CPU-only amd64 variant yet, so amd64 hosts always resolve to 'amd64-gpu'
|
CPU-only amd64 variant yet, so amd64 hosts always resolve to 'amd64-gpu'
|
||||||
even when no GPU is present -- this downloads working CUDA-capable
|
even when no GPU is present: this downloads working CUDA-capable
|
||||||
packages, just larger than a CPU-only host strictly needs. Do not change
|
packages, just larger than a CPU-only host strictly needs. Do not change
|
||||||
this to branch on GPU presence without first publishing an 'amd64-cpu'
|
this to branch on GPU presence without first publishing an 'amd64-cpu'
|
||||||
archive for every bundle; requesting a key that doesn't exist in the
|
archive for every bundle; requesting a key that doesn't exist in the
|
||||||
@@ -97,7 +97,7 @@ def estimate_extracted(compressed: int, extracted: int) -> int:
|
|||||||
extractedSize (0), the budget would otherwise collapse to just the
|
extractedSize (0), the budget would otherwise collapse to just the
|
||||||
compressed size and under-reserve for the extracted payload; fall back to a
|
compressed size and under-reserve for the extracted payload; fall back to a
|
||||||
conservative 3x of compressed (measured extracted/compressed ratios reach
|
conservative 3x of compressed (measured extracted/compressed ratios reach
|
||||||
~3x). This is only the early sanity bail -- the accurate guard is the
|
~3x). This is only the early sanity bail; the accurate guard is the
|
||||||
real-on-disk re-check just before the destructive venv write."""
|
real-on-disk re-check just before the destructive venv write."""
|
||||||
return extracted if extracted > 0 else compressed * 3
|
return extracted if extracted > 0 else compressed * 3
|
||||||
|
|
||||||
@@ -499,22 +499,23 @@ def main() -> None:
|
|||||||
# -- Disk re-check before the first destructive venv write --
|
# -- Disk re-check before the first destructive venv write --
|
||||||
# The upfront check ran before the download and used an estimate; now the
|
# The upfront check ran before the download and used an estimate; now the
|
||||||
# payload is really on disk, so measure it and verify there's room to place
|
# payload is really on disk, so measure it and verify there's room to place
|
||||||
# it before we start writing into the venv. On the same filesystem the move
|
# it before we start writing into the venv. Running here (after the
|
||||||
# is a rename (no extra space needed, just a safety floor); across
|
# local/remote branches merge) also covers the offline-import path, which
|
||||||
# filesystems it is a copy that transiently needs the payload's size again.
|
# skipped the upfront check entirely. Each budget is checked against the
|
||||||
# Running here (after the local/remote branches merge) also covers the
|
# filesystem the bytes actually land on: when the venv lives on a
|
||||||
# offline-import path, which skipped the upfront check entirely.
|
# different filesystem than staging, the site-packages payload is COPIED
|
||||||
staging_real = dir_size(staging_dir)
|
# onto the venv's disk, so that disk (not ai_dir's) must hold it. Models
|
||||||
if same_filesystem(staging_dir, venv_path):
|
# stay under ai_dir either way, moving by rename.
|
||||||
recheck_needed = 1024 ** 3 # 1 GB floor for fixups / installed.json / slack
|
disk_floor = 1024 ** 3 # 1 GB for fixups / installed.json / slack
|
||||||
else:
|
staging_sp = os.path.join(staging_dir, "site-packages")
|
||||||
recheck_needed = staging_real + 1024 ** 3
|
if not same_filesystem(staging_dir, venv_path):
|
||||||
check_disk_space(ai_dir, recheck_needed)
|
sp_bytes = dir_size(staging_sp) if os.path.isdir(staging_sp) else 0
|
||||||
|
check_disk_space(venv_path, sp_bytes + disk_floor)
|
||||||
|
check_disk_space(ai_dir, disk_floor)
|
||||||
|
|
||||||
# -- Move site-packages --
|
# -- Move site-packages --
|
||||||
emit_progress(92, "Installing packages...")
|
emit_progress(92, "Installing packages...")
|
||||||
site_packages_dir = get_site_packages_dir(venv_path)
|
site_packages_dir = get_site_packages_dir(venv_path)
|
||||||
staging_sp = os.path.join(staging_dir, "site-packages")
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
if os.path.isdir(staging_sp) and site_packages_dir:
|
if os.path.isdir(staging_sp) and site_packages_dir:
|
||||||
|
|||||||
+63
-34
@@ -105,6 +105,8 @@ interface PendingRequest {
|
|||||||
reject: (err: Error) => void;
|
reject: (err: Error) => void;
|
||||||
onProgress?: ProgressCallback;
|
onProgress?: ProgressCallback;
|
||||||
stderrLines: string[];
|
stderrLines: string[];
|
||||||
|
/** Which child generation this request was written to (see startChild). */
|
||||||
|
generation: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Crash recovery constants
|
// Crash recovery constants
|
||||||
@@ -133,11 +135,24 @@ export class PythonDispatcher {
|
|||||||
private childFailed = false;
|
private childFailed = false;
|
||||||
private gpuAvail = false;
|
private gpuAvail = false;
|
||||||
private pending = new Map<string, PendingRequest>();
|
private pending = new Map<string, PendingRequest>();
|
||||||
private stdoutBuf = "";
|
|
||||||
private crashes = 0;
|
private crashes = 0;
|
||||||
private lastCrashTs = 0;
|
private lastCrashTs = 0;
|
||||||
private backoffEnd = 0;
|
private backoffEnd = 0;
|
||||||
private shuttingDown = false;
|
/**
|
||||||
|
* Monotonic child counter. Each spawned child and every request written to
|
||||||
|
* it carry the generation current at spawn time, so a stale child's late
|
||||||
|
* close/error events (SIGTERM delivery can lag a replacement spawn) only
|
||||||
|
* ever touch their own generation's pending requests.
|
||||||
|
*/
|
||||||
|
private generation = 0;
|
||||||
|
/**
|
||||||
|
* Children we SIGTERMed on purpose (shutdown/reload). Tracked per child
|
||||||
|
* rather than as an instance-wide flag: an instance flag reset by the next
|
||||||
|
* spawn would let the stale child's close event record a phantom crash and
|
||||||
|
* null out the fresh child. The request-timeout kill path deliberately does
|
||||||
|
* NOT add to this set, so a genuinely hung script still counts as a crash.
|
||||||
|
*/
|
||||||
|
private stoppedChildren = new WeakSet<ChildProcess>();
|
||||||
|
|
||||||
constructor(opts: { profile: "ai" | "docs" }) {
|
constructor(opts: { profile: "ai" | "docs" }) {
|
||||||
this.profile = opts.profile;
|
this.profile = opts.profile;
|
||||||
@@ -173,9 +188,19 @@ export class PythonDispatcher {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Reject and drop the pending requests written to one child generation. */
|
||||||
|
private rejectPendingForGeneration(generation: number, message: string): void {
|
||||||
|
for (const [id, req] of this.pending.entries()) {
|
||||||
|
if (req.generation !== generation) continue;
|
||||||
|
req.reject(new Error(message));
|
||||||
|
this.pending.delete(id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private startChild(): ChildProcess | null {
|
private startChild(): ChildProcess | null {
|
||||||
if (this.childFailed) return null;
|
if (this.childFailed) return null;
|
||||||
this.shuttingDown = false;
|
this.generation++;
|
||||||
|
const gen = this.generation;
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const proc = spawn(getPythonPath(), [resolve(PYTHON_DIR, "dispatcher.py")], {
|
const proc = spawn(getPythonPath(), [resolve(PYTHON_DIR, "dispatcher.py")], {
|
||||||
@@ -188,23 +213,23 @@ export class PythonDispatcher {
|
|||||||
console.error(
|
console.error(
|
||||||
`[bridge] Dispatcher stdin pipe broken (${err.code}), rejecting pending requests`,
|
`[bridge] Dispatcher stdin pipe broken (${err.code}), rejecting pending requests`,
|
||||||
);
|
);
|
||||||
for (const [id, req] of this.pending.entries()) {
|
this.rejectPendingForGeneration(gen, "Python dispatcher stdin closed unexpectedly");
|
||||||
req.reject(new Error("Python dispatcher stdin closed unexpectedly"));
|
// An intentional shutdown() ends stdin then SIGTERMs the child,
|
||||||
this.pending.delete(id);
|
// which can surface here as an EPIPE/ERR_STREAM_DESTROYED. That is
|
||||||
}
|
// not a crash: counting it would let repeated legitimate restarts
|
||||||
// An intentional shutdown() ends stdin then SIGTERMs the child, which
|
// (shutdownDispatcher() runs after every AI bundle install) trip
|
||||||
// can surface here as an EPIPE/ERR_STREAM_DESTROYED. That is not a
|
// the crash limit and permanently disable the dispatcher. Guard
|
||||||
// crash -- counting it would let repeated legitimate restarts (e.g.
|
// mirrors the "close" handler below.
|
||||||
// shutdownDispatcher() on every AI bundle install) trip the crash
|
if (!this.stoppedChildren.has(proc)) this.recordCrash();
|
||||||
// limit and permanently disable the dispatcher. Guard mirrors the
|
if (this.child === proc) {
|
||||||
// "close" handler below.
|
|
||||||
if (!this.shuttingDown) this.recordCrash();
|
|
||||||
this.child = null;
|
this.child = null;
|
||||||
this.childReady = false;
|
this.childReady = false;
|
||||||
}
|
}
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
let stderrBuf = "";
|
let stderrBuf = "";
|
||||||
|
let stdoutBuf = "";
|
||||||
|
|
||||||
proc.stderr?.on("data", (chunk: Buffer) => {
|
proc.stderr?.on("data", (chunk: Buffer) => {
|
||||||
stderrBuf += chunk.toString();
|
stderrBuf += chunk.toString();
|
||||||
@@ -218,21 +243,24 @@ export class PythonDispatcher {
|
|||||||
try {
|
try {
|
||||||
const parsed = JSON.parse(trimmed);
|
const parsed = JSON.parse(trimmed);
|
||||||
|
|
||||||
// Readiness signal
|
// Readiness signal. Ignore it from a superseded child so a stale
|
||||||
|
// process can't mark a not-yet-ready replacement as ready.
|
||||||
if (parsed.ready === true) {
|
if (parsed.ready === true) {
|
||||||
|
if (this.child === proc) {
|
||||||
this.childReady = true;
|
this.childReady = true;
|
||||||
this.gpuAvail = parsed.gpu === true;
|
this.gpuAvail = parsed.gpu === true;
|
||||||
this.crashes = 0;
|
this.crashes = 0;
|
||||||
console.log(`[bridge] Python dispatcher ready (GPU: ${parsed.gpu === true})`);
|
console.log(`[bridge] Python dispatcher ready (GPU: ${parsed.gpu === true})`);
|
||||||
|
}
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Progress event - route to the currently active request
|
// Progress event - route to the currently active request
|
||||||
if (typeof parsed.progress === "number" && typeof parsed.stage === "string") {
|
if (typeof parsed.progress === "number" && typeof parsed.stage === "string") {
|
||||||
// Progress goes to all pending requests (only one should be active at a time
|
// Progress goes to this child's pending requests (only one
|
||||||
// since Python processes synchronously)
|
// should be active at a time since Python processes synchronously)
|
||||||
for (const req of this.pending.values()) {
|
for (const req of this.pending.values()) {
|
||||||
req.onProgress?.(parsed.progress, parsed.stage);
|
if (req.generation === gen) req.onProgress?.(parsed.progress, parsed.stage);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} catch {
|
} catch {
|
||||||
@@ -242,16 +270,16 @@ export class PythonDispatcher {
|
|||||||
console.log(`[python] ${trimmed}`);
|
console.log(`[python] ${trimmed}`);
|
||||||
}
|
}
|
||||||
for (const req of this.pending.values()) {
|
for (const req of this.pending.values()) {
|
||||||
req.stderrLines.push(trimmed);
|
if (req.generation === gen) req.stderrLines.push(trimmed);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
proc.stdout?.on("data", (chunk: Buffer) => {
|
proc.stdout?.on("data", (chunk: Buffer) => {
|
||||||
this.stdoutBuf += chunk.toString();
|
stdoutBuf += chunk.toString();
|
||||||
const lines = this.stdoutBuf.split("\n");
|
const lines = stdoutBuf.split("\n");
|
||||||
this.stdoutBuf = lines.pop() ?? "";
|
stdoutBuf = lines.pop() ?? "";
|
||||||
|
|
||||||
for (const line of lines) {
|
for (const line of lines) {
|
||||||
const trimmed = line.trim();
|
const trimmed = line.trim();
|
||||||
@@ -292,29 +320,27 @@ export class PythonDispatcher {
|
|||||||
console.error(`[bridge] Dispatcher error: ${err.message} (code: ${err.code})`);
|
console.error(`[bridge] Dispatcher error: ${err.message} (code: ${err.code})`);
|
||||||
if (err.code === "ENOENT") {
|
if (err.code === "ENOENT") {
|
||||||
this.childFailed = true;
|
this.childFailed = true;
|
||||||
} else if (!this.shuttingDown) {
|
} else if (!this.stoppedChildren.has(proc)) {
|
||||||
// Skip crash accounting when we initiated the teardown (shutdown()
|
// Skip crash accounting when we initiated the teardown (shutdown()
|
||||||
// sets shuttingDown before killing the child); mirrors "close".
|
// marks the child stopped before killing it); mirrors "close".
|
||||||
this.recordCrash();
|
this.recordCrash();
|
||||||
}
|
}
|
||||||
for (const [id, req] of this.pending.entries()) {
|
this.rejectPendingForGeneration(gen, extractPythonError(err));
|
||||||
req.reject(new Error(extractPythonError(err)));
|
if (this.child === proc) {
|
||||||
this.pending.delete(id);
|
|
||||||
}
|
|
||||||
this.child = null;
|
this.child = null;
|
||||||
this.childReady = false;
|
this.childReady = false;
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
proc.on("close", (code) => {
|
proc.on("close", (code) => {
|
||||||
for (const [id, req] of this.pending.entries()) {
|
this.rejectPendingForGeneration(gen, "Python dispatcher exited unexpectedly");
|
||||||
req.reject(new Error("Python dispatcher exited unexpectedly"));
|
if (code !== 0 && !this.stoppedChildren.has(proc)) {
|
||||||
this.pending.delete(id);
|
|
||||||
}
|
|
||||||
if (code !== 0 && !this.shuttingDown) {
|
|
||||||
this.recordCrash();
|
this.recordCrash();
|
||||||
}
|
}
|
||||||
|
if (this.child === proc) {
|
||||||
this.child = null;
|
this.child = null;
|
||||||
this.childReady = false;
|
this.childReady = false;
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
return proc;
|
return proc;
|
||||||
@@ -378,6 +404,9 @@ export class PythonDispatcher {
|
|||||||
reject: wrappedReject,
|
reject: wrappedReject,
|
||||||
onProgress: options.onProgress,
|
onProgress: options.onProgress,
|
||||||
stderrLines: [],
|
stderrLines: [],
|
||||||
|
// getChild() above either reused or just spawned the child this
|
||||||
|
// request is written to, so the current generation is its generation.
|
||||||
|
generation: this.generation,
|
||||||
});
|
});
|
||||||
|
|
||||||
const msg: Record<string, unknown> = { id, script: scriptName.replace(".py", ""), args };
|
const msg: Record<string, unknown> = { id, script: scriptName.replace(".py", ""), args };
|
||||||
@@ -541,7 +570,7 @@ export class PythonDispatcher {
|
|||||||
*/
|
*/
|
||||||
shutdown(): void {
|
shutdown(): void {
|
||||||
if (this.child && !this.child.killed) {
|
if (this.child && !this.child.killed) {
|
||||||
this.shuttingDown = true;
|
this.stoppedChildren.add(this.child);
|
||||||
this.child.stdin?.end();
|
this.child.stdin?.end();
|
||||||
this.child.kill("SIGTERM");
|
this.child.kill("SIGTERM");
|
||||||
this.child = null;
|
this.child = null;
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ import type {
|
|||||||
} from "../types.js";
|
} from "../types.js";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Above this pixel count, CLAHE is skipped in applyCorrections() -- see the
|
* Above this pixel count, CLAHE is skipped in applyCorrections(); see the
|
||||||
* comment at its call site for why.
|
* comment at its call site for why.
|
||||||
*/
|
*/
|
||||||
const MAX_CLAHE_PIXELS = 16_000_000;
|
const MAX_CLAHE_PIXELS = 16_000_000;
|
||||||
@@ -230,7 +230,7 @@ export function applyCorrections(
|
|||||||
|
|
||||||
let result = image;
|
let result = image;
|
||||||
// Tracks whether .clahe() actually ran (not just whether the toggle allowed
|
// Tracks whether .clahe() actually ran (not just whether the toggle allowed
|
||||||
// it) -- Step 5 below applies a compensation boost keyed off this, and it
|
// it). Step 5 below applies a compensation boost keyed off this, and it
|
||||||
// needs to stay correct now that CLAHE can also be skipped by image size.
|
// needs to stay correct now that CLAHE can also be skipped by image size.
|
||||||
let claheApplied = false;
|
let claheApplied = false;
|
||||||
|
|
||||||
@@ -238,7 +238,7 @@ export function applyCorrections(
|
|||||||
// maxSlope must be an integer (Sharp requirement); skip for tiny images.
|
// maxSlope must be an integer (Sharp requirement); skip for tiny images.
|
||||||
// CLAHE's cost scales with total pixel count regardless of tile size (tile
|
// CLAHE's cost scales with total pixel count regardless of tile size (tile
|
||||||
// size only bounds granularity, not the per-pixel histogram/interpolation
|
// size only bounds granularity, not the per-pixel histogram/interpolation
|
||||||
// work), so it's also skipped above MAX_CLAHE_PIXELS -- a 5504x3672 (20MP)
|
// work), so it's also skipped above MAX_CLAHE_PIXELS: a 5504x3672 (20MP)
|
||||||
// real-world RAW photo measured 40+ seconds in this step alone versus ~1s
|
// real-world RAW photo measured 40+ seconds in this step alone versus ~1s
|
||||||
// for every other correction combined. The other six corrections below
|
// for every other correction combined. The other six corrections below
|
||||||
// still apply at full resolution regardless of size.
|
// still apply at full resolution regardless of size.
|
||||||
|
|||||||
@@ -3475,6 +3475,7 @@ export const ar: TranslationKeys = {
|
|||||||
description: "إدارة حزم نماذج AI لمعالجة الملفات المتقدمة.",
|
description: "إدارة حزم نماذج AI لمعالجة الملفات المتقدمة.",
|
||||||
installAll: "تثبيت الكل",
|
installAll: "تثبيت الكل",
|
||||||
diskUsage: "استخدام القرص: {size}",
|
diskUsage: "استخدام القرص: {size}",
|
||||||
|
sizeOnDisk: "{size} على القرص",
|
||||||
installed: "مُثبَّت",
|
installed: "مُثبَّت",
|
||||||
notInstalled: "غير مُثبَّت",
|
notInstalled: "غير مُثبَّت",
|
||||||
queued: "في قائمة الانتظار",
|
queued: "في قائمة الانتظار",
|
||||||
|
|||||||
@@ -3512,6 +3512,7 @@ export const de: TranslationKeys = {
|
|||||||
description: "AI-Modellpakete für erweiterte Dateiverarbeitung verwalten.",
|
description: "AI-Modellpakete für erweiterte Dateiverarbeitung verwalten.",
|
||||||
installAll: "Alle installieren",
|
installAll: "Alle installieren",
|
||||||
diskUsage: "Speicherplatznutzung: {size}",
|
diskUsage: "Speicherplatznutzung: {size}",
|
||||||
|
sizeOnDisk: "{size} auf der Festplatte",
|
||||||
installed: "Installiert",
|
installed: "Installiert",
|
||||||
notInstalled: "Nicht installiert",
|
notInstalled: "Nicht installiert",
|
||||||
queued: "In Warteschlange",
|
queued: "In Warteschlange",
|
||||||
|
|||||||
@@ -3425,6 +3425,7 @@ export const en = {
|
|||||||
description: "Manage AI model bundles for advanced file processing.",
|
description: "Manage AI model bundles for advanced file processing.",
|
||||||
installAll: "Install All",
|
installAll: "Install All",
|
||||||
diskUsage: "Disk usage: {size}",
|
diskUsage: "Disk usage: {size}",
|
||||||
|
sizeOnDisk: "{size} on disk",
|
||||||
installed: "Installed",
|
installed: "Installed",
|
||||||
notInstalled: "Not installed",
|
notInstalled: "Not installed",
|
||||||
queued: "Queued",
|
queued: "Queued",
|
||||||
|
|||||||
@@ -3492,6 +3492,7 @@ export const es: TranslationKeys = {
|
|||||||
description: "Gestiona paquetes de modelos AI para procesamiento avanzado de archivos.",
|
description: "Gestiona paquetes de modelos AI para procesamiento avanzado de archivos.",
|
||||||
installAll: "Instalar todo",
|
installAll: "Instalar todo",
|
||||||
diskUsage: "Uso del disco: {size}",
|
diskUsage: "Uso del disco: {size}",
|
||||||
|
sizeOnDisk: "{size} en disco",
|
||||||
installed: "Instalado",
|
installed: "Instalado",
|
||||||
notInstalled: "No instalado",
|
notInstalled: "No instalado",
|
||||||
queued: "En cola",
|
queued: "En cola",
|
||||||
|
|||||||
@@ -3516,6 +3516,7 @@ export const fr: TranslationKeys = {
|
|||||||
description: "Gérez les paquets de modèles AI pour le traitement avancé de fichiers.",
|
description: "Gérez les paquets de modèles AI pour le traitement avancé de fichiers.",
|
||||||
installAll: "Tout installer",
|
installAll: "Tout installer",
|
||||||
diskUsage: "Utilisation du disque : {size}",
|
diskUsage: "Utilisation du disque : {size}",
|
||||||
|
sizeOnDisk: "{size} sur le disque",
|
||||||
installed: "Installé",
|
installed: "Installé",
|
||||||
notInstalled: "Non installé",
|
notInstalled: "Non installé",
|
||||||
queued: "En file d'attente",
|
queued: "En file d'attente",
|
||||||
|
|||||||
@@ -3303,6 +3303,7 @@ export const hi: TranslationKeys = {
|
|||||||
description: "उन्नत फ़ाइल प्रोसेसिंग के लिए AI मॉडल बंडल प्रबंधित करें।",
|
description: "उन्नत फ़ाइल प्रोसेसिंग के लिए AI मॉडल बंडल प्रबंधित करें।",
|
||||||
installAll: "सभी इंस्टॉल करें",
|
installAll: "सभी इंस्टॉल करें",
|
||||||
diskUsage: "डिस्क उपयोग: {size}",
|
diskUsage: "डिस्क उपयोग: {size}",
|
||||||
|
sizeOnDisk: "डिस्क पर {size}",
|
||||||
installed: "इंस्टॉल किया गया",
|
installed: "इंस्टॉल किया गया",
|
||||||
notInstalled: "इंस्टॉल नहीं है",
|
notInstalled: "इंस्टॉल नहीं है",
|
||||||
queued: "कतार में",
|
queued: "कतार में",
|
||||||
|
|||||||
@@ -3494,6 +3494,7 @@ export const id: TranslationKeys = {
|
|||||||
description: "Kelola bundel model AI untuk pemrosesan file tingkat lanjut.",
|
description: "Kelola bundel model AI untuk pemrosesan file tingkat lanjut.",
|
||||||
installAll: "Instal Semua",
|
installAll: "Instal Semua",
|
||||||
diskUsage: "Penggunaan disk: {size}",
|
diskUsage: "Penggunaan disk: {size}",
|
||||||
|
sizeOnDisk: "{size} di disk",
|
||||||
installed: "Terinstal",
|
installed: "Terinstal",
|
||||||
notInstalled: "Belum terinstal",
|
notInstalled: "Belum terinstal",
|
||||||
queued: "Dalam antrean",
|
queued: "Dalam antrean",
|
||||||
|
|||||||
@@ -3506,6 +3506,7 @@ export const it: TranslationKeys = {
|
|||||||
description: "Gestisci pacchetti di modelli di IA per l'elaborazione avanzata dei file.",
|
description: "Gestisci pacchetti di modelli di IA per l'elaborazione avanzata dei file.",
|
||||||
installAll: "Installa tutto",
|
installAll: "Installa tutto",
|
||||||
diskUsage: "Utilizzo del disco: {size}",
|
diskUsage: "Utilizzo del disco: {size}",
|
||||||
|
sizeOnDisk: "{size} su disco",
|
||||||
installed: "Installato",
|
installed: "Installato",
|
||||||
notInstalled: "Non installato",
|
notInstalled: "Non installato",
|
||||||
queued: "Code",
|
queued: "Code",
|
||||||
|
|||||||
@@ -3445,6 +3445,7 @@ export const ja: TranslationKeys = {
|
|||||||
description: "高度なファイル処理のためのAIモデルバンドルを管理します。",
|
description: "高度なファイル処理のためのAIモデルバンドルを管理します。",
|
||||||
installAll: "すべてインストール",
|
installAll: "すべてインストール",
|
||||||
diskUsage: "ディスク使用量: {size}",
|
diskUsage: "ディスク使用量: {size}",
|
||||||
|
sizeOnDisk: "ディスク上 {size}",
|
||||||
installed: "インストール済み",
|
installed: "インストール済み",
|
||||||
notInstalled: "未インストール",
|
notInstalled: "未インストール",
|
||||||
queued: "待機中",
|
queued: "待機中",
|
||||||
|
|||||||
@@ -3429,6 +3429,7 @@ export const ko: TranslationKeys = {
|
|||||||
description: "고급 파일 처리를 위한 AI 모델 번들을 관리합니다.",
|
description: "고급 파일 처리를 위한 AI 모델 번들을 관리합니다.",
|
||||||
installAll: "모두 설치",
|
installAll: "모두 설치",
|
||||||
diskUsage: "디스크 사용량: {size}",
|
diskUsage: "디스크 사용량: {size}",
|
||||||
|
sizeOnDisk: "디스크에 {size}",
|
||||||
installed: "설치됨",
|
installed: "설치됨",
|
||||||
notInstalled: "설치되지 않음",
|
notInstalled: "설치되지 않음",
|
||||||
queued: "대기 중",
|
queued: "대기 중",
|
||||||
|
|||||||
@@ -3502,6 +3502,7 @@ export const nl: TranslationKeys = {
|
|||||||
description: "Beheer AI-modelbundels voor geavanceerde bestandsverwerking.",
|
description: "Beheer AI-modelbundels voor geavanceerde bestandsverwerking.",
|
||||||
installAll: "Alles installeren",
|
installAll: "Alles installeren",
|
||||||
diskUsage: "Schijfgebruik: {size}",
|
diskUsage: "Schijfgebruik: {size}",
|
||||||
|
sizeOnDisk: "{size} op schijf",
|
||||||
installed: "Geïnstalleerd",
|
installed: "Geïnstalleerd",
|
||||||
notInstalled: "Niet geïnstalleerd",
|
notInstalled: "Niet geïnstalleerd",
|
||||||
queued: "In wachtrij",
|
queued: "In wachtrij",
|
||||||
|
|||||||
@@ -3507,6 +3507,7 @@ export const pl: TranslationKeys = {
|
|||||||
description: "Zarządzanie pakietami modeli AI do zaawansowanego przetwarzania plików.",
|
description: "Zarządzanie pakietami modeli AI do zaawansowanego przetwarzania plików.",
|
||||||
installAll: "Zainstaluj wszystkie",
|
installAll: "Zainstaluj wszystkie",
|
||||||
diskUsage: "Wykorzystanie dysku: {size}",
|
diskUsage: "Wykorzystanie dysku: {size}",
|
||||||
|
sizeOnDisk: "{size} na dysku",
|
||||||
installed: "Zainstalowane",
|
installed: "Zainstalowane",
|
||||||
notInstalled: "Niezainstalowane",
|
notInstalled: "Niezainstalowane",
|
||||||
queued: "W kolejce",
|
queued: "W kolejce",
|
||||||
|
|||||||
@@ -3500,6 +3500,7 @@ export const ptBR: TranslationKeys = {
|
|||||||
description: "Gerencie pacotes de modelos de IA para processamento avançado de arquivos.",
|
description: "Gerencie pacotes de modelos de IA para processamento avançado de arquivos.",
|
||||||
installAll: "Instalar tudo",
|
installAll: "Instalar tudo",
|
||||||
diskUsage: "Uso do disco: {size}",
|
diskUsage: "Uso do disco: {size}",
|
||||||
|
sizeOnDisk: "{size} em disco",
|
||||||
installed: "Instalado",
|
installed: "Instalado",
|
||||||
notInstalled: "Não instalado",
|
notInstalled: "Não instalado",
|
||||||
queued: "Na fila",
|
queued: "Na fila",
|
||||||
|
|||||||
@@ -3497,6 +3497,7 @@ export const ru: TranslationKeys = {
|
|||||||
description: "Управление пакетами AI-моделей для расширенной обработки файлов.",
|
description: "Управление пакетами AI-моделей для расширенной обработки файлов.",
|
||||||
installAll: "Установить все",
|
installAll: "Установить все",
|
||||||
diskUsage: "Использование диска: {size}",
|
diskUsage: "Использование диска: {size}",
|
||||||
|
sizeOnDisk: "{size} на диске",
|
||||||
installed: "Установлено",
|
installed: "Установлено",
|
||||||
notInstalled: "Не установлено",
|
notInstalled: "Не установлено",
|
||||||
queued: "В очереди",
|
queued: "В очереди",
|
||||||
|
|||||||
@@ -3492,6 +3492,7 @@ export const sv: TranslationKeys = {
|
|||||||
description: "Hantera AI-modellpaket för avancerad filbehandling.",
|
description: "Hantera AI-modellpaket för avancerad filbehandling.",
|
||||||
installAll: "Installera alla",
|
installAll: "Installera alla",
|
||||||
diskUsage: "Diskanvändning: {size}",
|
diskUsage: "Diskanvändning: {size}",
|
||||||
|
sizeOnDisk: "{size} på disk",
|
||||||
installed: "Installerad",
|
installed: "Installerad",
|
||||||
notInstalled: "Inte installerad",
|
notInstalled: "Inte installerad",
|
||||||
queued: "I kö",
|
queued: "I kö",
|
||||||
|
|||||||
@@ -3456,6 +3456,7 @@ export const th: TranslationKeys = {
|
|||||||
description: "จัดการชุดโมเดล AI สำหรับการประมวลผลไฟล์ขั้นสูง",
|
description: "จัดการชุดโมเดล AI สำหรับการประมวลผลไฟล์ขั้นสูง",
|
||||||
installAll: "ติดตั้งทั้งหมด",
|
installAll: "ติดตั้งทั้งหมด",
|
||||||
diskUsage: "การใช้พื้นที่ดิสก์: {size}",
|
diskUsage: "การใช้พื้นที่ดิสก์: {size}",
|
||||||
|
sizeOnDisk: "{size} บนดิสก์",
|
||||||
installed: "ติดตั้งแล้ว",
|
installed: "ติดตั้งแล้ว",
|
||||||
notInstalled: "ยังไม่ได้ติดตั้ง",
|
notInstalled: "ยังไม่ได้ติดตั้ง",
|
||||||
queued: "อยู่ในคิว",
|
queued: "อยู่ในคิว",
|
||||||
|
|||||||
@@ -3500,6 +3500,7 @@ export const tr: TranslationKeys = {
|
|||||||
description: "Gelişmiş dosya işleme için AI model paketlerini yönetin.",
|
description: "Gelişmiş dosya işleme için AI model paketlerini yönetin.",
|
||||||
installAll: "Tümünü Kur",
|
installAll: "Tümünü Kur",
|
||||||
diskUsage: "Disk kullanımı: {size}",
|
diskUsage: "Disk kullanımı: {size}",
|
||||||
|
sizeOnDisk: "diskte {size}",
|
||||||
installed: "Kurulu",
|
installed: "Kurulu",
|
||||||
notInstalled: "Kurulu değil",
|
notInstalled: "Kurulu değil",
|
||||||
queued: "Sırada",
|
queued: "Sırada",
|
||||||
|
|||||||
@@ -3497,6 +3497,7 @@ export const uk: TranslationKeys = {
|
|||||||
description: "Керування пакетами AI-моделей для розширеної обробки файлів.",
|
description: "Керування пакетами AI-моделей для розширеної обробки файлів.",
|
||||||
installAll: "Встановити все",
|
installAll: "Встановити все",
|
||||||
diskUsage: "Використання диска: {size}",
|
diskUsage: "Використання диска: {size}",
|
||||||
|
sizeOnDisk: "{size} на диску",
|
||||||
installed: "Встановлено",
|
installed: "Встановлено",
|
||||||
notInstalled: "Не встановлено",
|
notInstalled: "Не встановлено",
|
||||||
queued: "У черзі",
|
queued: "У черзі",
|
||||||
|
|||||||
@@ -3492,6 +3492,7 @@ export const vi: TranslationKeys = {
|
|||||||
description: "Quản lý các gói mô hình AI cho xử lý tệp nâng cao.",
|
description: "Quản lý các gói mô hình AI cho xử lý tệp nâng cao.",
|
||||||
installAll: "Cài đặt tất cả",
|
installAll: "Cài đặt tất cả",
|
||||||
diskUsage: "Dung lượng đĩa: {size}",
|
diskUsage: "Dung lượng đĩa: {size}",
|
||||||
|
sizeOnDisk: "{size} trên ổ đĩa",
|
||||||
installed: "Đã cài đặt",
|
installed: "Đã cài đặt",
|
||||||
notInstalled: "Chưa cài đặt",
|
notInstalled: "Chưa cài đặt",
|
||||||
queued: "Đang chờ",
|
queued: "Đang chờ",
|
||||||
|
|||||||
@@ -3243,6 +3243,7 @@ export const zhCN: TranslationKeys = {
|
|||||||
description: "管理用于高级文件处理的 AI 模型包。",
|
description: "管理用于高级文件处理的 AI 模型包。",
|
||||||
installAll: "全部安装",
|
installAll: "全部安装",
|
||||||
diskUsage: "磁盘使用量:{size}",
|
diskUsage: "磁盘使用量:{size}",
|
||||||
|
sizeOnDisk: "占用磁盘 {size}",
|
||||||
installed: "已安装",
|
installed: "已安装",
|
||||||
notInstalled: "未安装",
|
notInstalled: "未安装",
|
||||||
queued: "排队中",
|
queued: "排队中",
|
||||||
|
|||||||
@@ -3241,6 +3241,7 @@ export const zhTW: TranslationKeys = {
|
|||||||
description: "管理用於進階檔案處理的AI模型套件。",
|
description: "管理用於進階檔案處理的AI模型套件。",
|
||||||
installAll: "全部安裝",
|
installAll: "全部安裝",
|
||||||
diskUsage: "磁碟使用量:{size}",
|
diskUsage: "磁碟使用量:{size}",
|
||||||
|
sizeOnDisk: "佔用磁碟 {size}",
|
||||||
installed: "已安裝",
|
installed: "已安裝",
|
||||||
notInstalled: "未安裝",
|
notInstalled: "未安裝",
|
||||||
queued: "等待中",
|
queued: "等待中",
|
||||||
|
|||||||
@@ -316,24 +316,33 @@ test.describe("Install lifecycle - face-detection", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test("install of different bundle while one is active is queued (202)", async ({ request }) => {
|
test("install of different bundle while one is active is queued (202)", async ({ request }) => {
|
||||||
test.setTimeout(900_000);
|
// The drain below waits out two full installs, each with the helper's
|
||||||
|
// 600s default budget, so the test timeout must cover both plus slack.
|
||||||
|
test.setTimeout(1_500_000);
|
||||||
const status = await getBundleStatus(request, "face-detection");
|
const status = await getBundleStatus(request, "face-detection");
|
||||||
if (status !== "installing") return; // fast install already finished; nothing to assert
|
if (status !== "installing") return; // fast install already finished; nothing to assert
|
||||||
|
|
||||||
const headers = await authHeaders(request);
|
const headers = await authHeaders(request);
|
||||||
const res = await request.post(`${API}/api/v1/admin/features/ocr/install`, { headers });
|
// Queue transcription: the smallest bundle (~0.5 GB archive) that no
|
||||||
|
// other test here depends on. There is no cancel API for a queued
|
||||||
|
// install, so the drain below really downloads it; queueing ocr here
|
||||||
|
// used to pull a ~6 GB archive just to assert queued=true.
|
||||||
|
const res = await request.post(`${API}/api/v1/admin/features/transcription/install`, {
|
||||||
|
headers,
|
||||||
|
});
|
||||||
expect(res.status()).toBe(202);
|
expect(res.status()).toBe(202);
|
||||||
expect((await res.json()).queued).toBe(true);
|
expect((await res.json()).queued).toBe(true);
|
||||||
|
|
||||||
const ocr = await getBundle(request, "ocr");
|
const queuedBundle = await getBundle(request, "transcription");
|
||||||
expect(["queued", "installing"]).toContain(ocr.status);
|
expect(["queued", "installing"]).toContain(queuedBundle.status);
|
||||||
|
|
||||||
// ocr auto-starts once face-detection finishes. Drain and uninstall it so
|
// transcription auto-starts once face-detection finishes. Drain and
|
||||||
// the rest of the serial suite runs against a clean lock/state.
|
// uninstall it so the rest of the serial suite runs against a clean
|
||||||
|
// lock/state.
|
||||||
await waitForInstallComplete(request, "face-detection");
|
await waitForInstallComplete(request, "face-detection");
|
||||||
await waitForInstallComplete(request, "ocr", 900_000);
|
await waitForInstallComplete(request, "transcription");
|
||||||
await request.post(`${API}/api/v1/admin/features/ocr/uninstall`, { headers });
|
await request.post(`${API}/api/v1/admin/features/transcription/uninstall`, { headers });
|
||||||
expect(await getBundleStatus(request, "ocr")).toBe("not_installed");
|
expect(await getBundleStatus(request, "transcription")).toBe("not_installed");
|
||||||
});
|
});
|
||||||
|
|
||||||
test("after install completes, status is installed with version", async ({ request }) => {
|
test("after install completes, status is installed with version", async ({ request }) => {
|
||||||
|
|||||||
@@ -112,7 +112,7 @@ describe("Login failures", () => {
|
|||||||
// hash (scrypt), while "wrong password for a real user" always pays the
|
// hash (scrypt), while "wrong password for a real user" always pays the
|
||||||
// scrypt cost. That gap lets an attacker enumerate valid usernames purely
|
// scrypt cost. That gap lets an attacker enumerate valid usernames purely
|
||||||
// from response timing even though the status code and body are identical.
|
// from response timing even though the status code and body are identical.
|
||||||
// See getDummyHash() in apps/api/src/plugins/auth.ts -- it equalizes cost
|
// See getDummyHash() in apps/api/src/plugins/auth.ts; it equalizes cost
|
||||||
// by running verifyPassword against a dummy hash on the unknown-user path.
|
// by running verifyPassword against a dummy hash on the unknown-user path.
|
||||||
const SAMPLES = 10;
|
const SAMPLES = 10;
|
||||||
const median = (values: number[]) => {
|
const median = (values: number[]) => {
|
||||||
|
|||||||
@@ -5,7 +5,7 @@
|
|||||||
* The installer child process (spawn) and the venv lock (@snapotter/ai) are
|
* The installer child process (spawn) and the venv lock (@snapotter/ai) are
|
||||||
* mocked so no real Python runs: spawn returns a controllable fake child we can
|
* mocked so no real Python runs: spawn returns a controllable fake child we can
|
||||||
* drive with emit("close"). This lets us assert the route contract
|
* drive with emit("close"). This lets us assert the route contract
|
||||||
* deterministically -- a second concurrent install is queued (202 { queued:
|
* deterministically: a second concurrent install is queued (202 { queued:
|
||||||
* true }) instead of rejected, the next queued bundle auto-starts when the
|
* true }) instead of rejected, the next queued bundle auto-starts when the
|
||||||
* running one finishes, and a bundle queued while an import holds the lock
|
* running one finishes, and a bundle queued while an import holds the lock
|
||||||
* starts once the import route releases it.
|
* starts once the import route releases it.
|
||||||
|
|||||||
@@ -135,7 +135,7 @@ describe("extract-zip (pure JS, no skipIf)", () => {
|
|||||||
// Same binary-patch technique as the "../evil.txt" case above, but with a
|
// Same binary-patch technique as the "../evil.txt" case above, but with a
|
||||||
// multi-segment ".." path to make sure the segment-splitting check
|
// multi-segment ".." path to make sure the segment-splitting check
|
||||||
// (name.split(/[/\\]/).some(s => s === "..")) isn't fooled by extra
|
// (name.split(/[/\\]/).some(s => s === "..")) isn't fooled by extra
|
||||||
// "../" hops -- only the first ".." segment mattering would be a bug.
|
// "../" hops; only the first ".." segment mattering would be a bug.
|
||||||
const zip = new AdmZip();
|
const zip = new AdmZip();
|
||||||
const traversalName = "../../../etc/passphrase-lol";
|
const traversalName = "../../../etc/passphrase-lol";
|
||||||
const placeholder = "X".repeat(traversalName.length);
|
const placeholder = "X".repeat(traversalName.length);
|
||||||
@@ -157,7 +157,7 @@ describe("extract-zip (pure JS, no skipIf)", () => {
|
|||||||
}, 30_000);
|
}, 30_000);
|
||||||
|
|
||||||
it("rejects a zip with an absolute Unix path entry (/etc/passwd) with 400", async () => {
|
it("rejects a zip with an absolute Unix path entry (/etc/passwd) with 400", async () => {
|
||||||
// Absolute paths bypass ".." detection entirely -- extract-zip must reject
|
// Absolute paths bypass ".." detection entirely; extract-zip must reject
|
||||||
// entry names starting with "/" on their own (preValidate's
|
// entry names starting with "/" on their own (preValidate's
|
||||||
// name.startsWith("/") branch), independent of the ".." segment check.
|
// name.startsWith("/") branch), independent of the ".." segment check.
|
||||||
const zip = new AdmZip();
|
const zip = new AdmZip();
|
||||||
|
|||||||
@@ -640,7 +640,7 @@ async function main() {
|
|||||||
format: ext,
|
format: ext,
|
||||||
status: statusCode,
|
status: statusCode,
|
||||||
outputOk: false,
|
outputOk: false,
|
||||||
note: `BUG: custom body route returned ${statusCode} -- ${body.slice(0, 300)}`,
|
note: `BUG: custom body route returned ${statusCode}. ${body.slice(0, 300)}`,
|
||||||
};
|
};
|
||||||
results.push(r);
|
results.push(r);
|
||||||
bugs.push(r);
|
bugs.push(r);
|
||||||
@@ -673,7 +673,7 @@ async function main() {
|
|||||||
outputOk: verification.ok,
|
outputOk: verification.ok,
|
||||||
note: verification.ok
|
note: verification.ok
|
||||||
? `pass: ${verification.detail}`
|
? `pass: ${verification.detail}`
|
||||||
: `BUG: corrupt success -- ${verification.detail}`,
|
: `BUG: corrupt success. ${verification.detail}`,
|
||||||
};
|
};
|
||||||
results.push(r);
|
results.push(r);
|
||||||
if (verification.ok) {
|
if (verification.ok) {
|
||||||
@@ -691,7 +691,7 @@ async function main() {
|
|||||||
format: ext,
|
format: ext,
|
||||||
status: "network-error",
|
status: "network-error",
|
||||||
outputOk: false,
|
outputOk: false,
|
||||||
note: `BUG: custom body request failed -- ${msg.slice(0, 200)}`,
|
note: `BUG: custom body request failed. ${msg.slice(0, 200)}`,
|
||||||
};
|
};
|
||||||
results.push(r);
|
results.push(r);
|
||||||
bugs.push(r);
|
bugs.push(r);
|
||||||
@@ -817,7 +817,7 @@ async function main() {
|
|||||||
} catch {}
|
} catch {}
|
||||||
const msg = parsed.error || parsed.details || body.slice(0, 200);
|
const msg = parsed.error || parsed.details || body.slice(0, 200);
|
||||||
const fullMsg =
|
const fullMsg =
|
||||||
[parsed.error, parsed.details].filter(Boolean).join(" -- ") || body.slice(0, 300);
|
[parsed.error, parsed.details].filter(Boolean).join(" | ") || body.slice(0, 300);
|
||||||
|
|
||||||
// Check if this format is in the tool's own acceptedInputs
|
// Check if this format is in the tool's own acceptedInputs
|
||||||
const isSelfFormat = tool.acceptedInputs.includes(ext);
|
const isSelfFormat = tool.acceptedInputs.includes(ext);
|
||||||
@@ -993,7 +993,7 @@ async function main() {
|
|||||||
outputOk: verification.ok,
|
outputOk: verification.ok,
|
||||||
note: verification.ok
|
note: verification.ok
|
||||||
? `pass: ${verification.detail}`
|
? `pass: ${verification.detail}`
|
||||||
: `BUG: corrupt success -- ${verification.detail}`,
|
: `BUG: corrupt success. ${verification.detail}`,
|
||||||
};
|
};
|
||||||
results.push(r);
|
results.push(r);
|
||||||
|
|
||||||
|
|||||||
@@ -8,8 +8,10 @@
|
|||||||
# Project/container names default to snapotter-qa. Two sessions on the same host
|
# Project/container names default to snapotter-qa. Two sessions on the same host
|
||||||
# running this file verbatim at the same time will silently steal each other's
|
# running this file verbatim at the same time will silently steal each other's
|
||||||
# container (last `up` wins, no error) since container_name is fixed rather than
|
# container (last `up` wins, no error) since container_name is fixed rather than
|
||||||
# derived from the project name. If you need a second concurrent stack, override:
|
# derived from the project name. A second concurrent stack also needs its own
|
||||||
# QA_PROJECT_NAME=snapotter-qa-2 docker compose -f tests/qa/docker-compose.qa.yml up -d
|
# host port (the default stack holds 13499), so override both:
|
||||||
|
# QA_PROJECT_NAME=snapotter-qa-2 QA_APP_PORT=13498 \
|
||||||
|
# docker compose -f tests/qa/docker-compose.qa.yml up -d
|
||||||
name: ${QA_PROJECT_NAME:-snapotter-qa}
|
name: ${QA_PROJECT_NAME:-snapotter-qa}
|
||||||
|
|
||||||
services:
|
services:
|
||||||
@@ -17,7 +19,7 @@ services:
|
|||||||
image: snapotter/snapotter:latest
|
image: snapotter/snapotter:latest
|
||||||
container_name: ${QA_PROJECT_NAME:-snapotter-qa}
|
container_name: ${QA_PROJECT_NAME:-snapotter-qa}
|
||||||
ports:
|
ports:
|
||||||
- "13499:1349"
|
- "${QA_APP_PORT:-13499}:1349"
|
||||||
volumes:
|
volumes:
|
||||||
- qa-data:/data
|
- qa-data:/data
|
||||||
- qa-workspace:/tmp/workspace
|
- qa-workspace:/tmp/workspace
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
// Drives the REAL in-app AI bundle install flow (Settings > AI Features > Install
|
// Drives the REAL in-app AI bundle install flow (Settings > AI Features > Install
|
||||||
// All) against a running Docker container, exactly as a user would click through
|
// All) against a running Docker container, exactly as a user would click through
|
||||||
// it. This is the primary AI-install path per docs/prompts/engineering/QA_PROMPT.md
|
// it. This is the primary AI-install path per docs/prompts/engineering/QA_PROMPT.md
|
||||||
// Phase 2 -- the curl-based /api/v1/admin/features/<bundle>/install route is a
|
// Phase 2: the curl-based /api/v1/admin/features/<bundle>/install route is a
|
||||||
// verification/fallback path only, never the primary install.
|
// verification/fallback path only, never the primary install.
|
||||||
//
|
//
|
||||||
// This script only KICKS OFF the install and captures pre/mid-install evidence; the
|
// This script only KICKS OFF the install and captures pre/mid-install evidence; the
|
||||||
@@ -47,11 +47,11 @@ async function main() {
|
|||||||
console.log(" already authenticated (no login form shown)");
|
console.log(" already authenticated (no login form shown)");
|
||||||
}
|
}
|
||||||
|
|
||||||
// Bail loudly if we land on the forced password-change screen -- the caller
|
// Bail loudly if we land on the forced password-change screen; the caller
|
||||||
// should set SKIP_MUST_CHANGE_PASSWORD=true for automated runs.
|
// should set SKIP_MUST_CHANGE_PASSWORD=true for automated runs.
|
||||||
if (page.url().includes("/change-password")) {
|
if (page.url().includes("/change-password")) {
|
||||||
throw new Error(
|
throw new Error(
|
||||||
"Landed on /change-password -- set SKIP_MUST_CHANGE_PASSWORD=true on the container for automated QA runs",
|
"Landed on /change-password; set SKIP_MUST_CHANGE_PASSWORD=true on the container for automated QA runs",
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1251,7 +1251,7 @@ describe("bridge - initDispatcher", () => {
|
|||||||
// pipe that teardown can surface as an EPIPE/ERR_STREAM_DESTROYED on the
|
// pipe that teardown can surface as an EPIPE/ERR_STREAM_DESTROYED on the
|
||||||
// stdin stream (observed in the container shutdown log as a spurious
|
// stdin stream (observed in the container shutdown log as a spurious
|
||||||
// "[bridge] Dispatcher crash #1"). The "close" handler alone is guarded,
|
// "[bridge] Dispatcher crash #1"). The "close" handler alone is guarded,
|
||||||
// but the stdin error handler must be too -- otherwise shutdownDispatcher()
|
// but the stdin error handler must be too; otherwise shutdownDispatcher()
|
||||||
// on every AI bundle install accrues false crashes toward the disable cap.
|
// on every AI bundle install accrues false crashes toward the disable cap.
|
||||||
const mock = createMockProcess();
|
const mock = createMockProcess();
|
||||||
vi.mocked(spawn).mockReturnValue(mock.process);
|
vi.mocked(spawn).mockReturnValue(mock.process);
|
||||||
|
|||||||
@@ -366,7 +366,7 @@ describe("applyCorrections pipeline (CLAHE + normalise + gamma)", () => {
|
|||||||
const claheDisabledBuf = await claheDisabled.toBuffer();
|
const claheDisabledBuf = await claheDisabled.toBuffer();
|
||||||
|
|
||||||
// Skipping CLAHE via the size cap must produce byte-identical output to
|
// Skipping CLAHE via the size cap must produce byte-identical output to
|
||||||
// skipping it via the explicit toggle -- proof the cap actually took effect.
|
// skipping it via the explicit toggle: proof the cap actually took effect.
|
||||||
expect(Buffer.compare(overCapBuf, claheDisabledBuf)).toBe(0);
|
expect(Buffer.compare(overCapBuf, claheDisabledBuf)).toBe(0);
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -380,7 +380,7 @@ describe("applyCorrections pipeline (CLAHE + normalise + gamma)", () => {
|
|||||||
denoise: 0,
|
denoise: 0,
|
||||||
};
|
};
|
||||||
// Use the real buffer's actual 200x150 dimensions (30,000 px, well under
|
// Use the real buffer's actual 200x150 dimensions (30,000 px, well under
|
||||||
// the 16M cap) -- CLAHE's tile size is derived from imageSize, and Sharp
|
// the 16M cap). CLAHE's tile size is derived from imageSize, and Sharp
|
||||||
// rejects a tile window larger than the real underlying image, so a fake
|
// rejects a tile window larger than the real underlying image, so a fake
|
||||||
// imageSize far bigger than the actual small test buffer isn't valid here
|
// imageSize far bigger than the actual small test buffer isn't valid here
|
||||||
// (that's exactly what the "above the cap" test above uses instead,
|
// (that's exactly what the "above the cap" test above uses instead,
|
||||||
@@ -408,7 +408,7 @@ describe("applyCorrections pipeline (CLAHE + normalise + gamma)", () => {
|
|||||||
);
|
);
|
||||||
const claheDisabledBuf = await claheDisabled.toBuffer();
|
const claheDisabledBuf = await claheDisabled.toBuffer();
|
||||||
|
|
||||||
// Under the cap, CLAHE should still run -- output must differ from the
|
// Under the cap, CLAHE should still run: output must differ from the
|
||||||
// contrast-disabled baseline.
|
// contrast-disabled baseline.
|
||||||
expect(Buffer.compare(underCapBuf, claheDisabledBuf)).not.toBe(0);
|
expect(Buffer.compare(underCapBuf, claheDisabledBuf)).not.toBe(0);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -202,7 +202,7 @@ describe("useFeaturesStore (expanded)", () => {
|
|||||||
|
|
||||||
await useFeaturesStore.getState().installBundle("done-bundle");
|
await useFeaturesStore.getState().installBundle("done-bundle");
|
||||||
|
|
||||||
// The client always POSTs now -- the server owns the queue/dedup decision.
|
// The client always POSTs now; the server owns the queue/dedup decision.
|
||||||
expect(apiPostMock).toHaveBeenCalledWith("/v1/admin/features/done-bundle/install", {});
|
expect(apiPostMock).toHaveBeenCalledWith("/v1/admin/features/done-bundle/install", {});
|
||||||
// A 409 already-installed clears installing + error and refreshes silently.
|
// A 409 already-installed clears installing + error and refreshes silently.
|
||||||
const state = useFeaturesStore.getState();
|
const state = useFeaturesStore.getState();
|
||||||
|
|||||||
@@ -448,6 +448,17 @@ describe("useFeaturesStore", () => {
|
|||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// The poll for the queued bundle reads server state from GET /v1/features.
|
||||||
|
let bundleBStatus: FeatureBundleState["status"] = "queued";
|
||||||
|
apiGetMock.mockImplementation(() =>
|
||||||
|
Promise.resolve({
|
||||||
|
bundles: [
|
||||||
|
makeBundleState({ id: "bundle-a", status: "installing" }),
|
||||||
|
makeBundleState({ id: "bundle-b", status: bundleBStatus }),
|
||||||
|
],
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
await useFeaturesStore.getState().installAll();
|
await useFeaturesStore.getState().installAll();
|
||||||
|
|
||||||
await vi.waitFor(() => {
|
await vi.waitFor(() => {
|
||||||
@@ -456,21 +467,32 @@ describe("useFeaturesStore", () => {
|
|||||||
});
|
});
|
||||||
expect(useFeaturesStore.getState().installing["bundle-b"]).toBeUndefined();
|
expect(useFeaturesStore.getState().installing["bundle-b"]).toBeUndefined();
|
||||||
|
|
||||||
// Finish both; the second transitions queued -> installing on its first
|
// A queued bundle must NOT hold an SSE connection open (it polls
|
||||||
// progress frame, then completes.
|
// instead): only the actively-installing bundle has an EventSource.
|
||||||
const esB = FakeEventSource.instances.find((es) => es.url.includes("bundle-b"));
|
expect(FakeEventSource.instances).toHaveLength(1);
|
||||||
esB?.onmessage?.({ data: JSON.stringify({ phase: "installing", percent: 10, stage: "Go" }) });
|
expect(FakeEventSource.instances[0].url).toContain("bundle-a");
|
||||||
await vi.waitFor(() => {
|
|
||||||
expect(useFeaturesStore.getState().queued).not.toContain("bundle-b");
|
|
||||||
});
|
|
||||||
|
|
||||||
for (const es of FakeEventSource.instances) {
|
// The server starts bundle-b; its poll moves it queued -> installing.
|
||||||
es.onmessage?.({ data: JSON.stringify({ phase: "complete" }) });
|
bundleBStatus = "installing";
|
||||||
}
|
await vi.waitFor(
|
||||||
await vi.waitFor(() => {
|
() => {
|
||||||
|
expect(useFeaturesStore.getState().queued).not.toContain("bundle-b");
|
||||||
|
expect(useFeaturesStore.getState().installing["bundle-b"]).toBeDefined();
|
||||||
|
},
|
||||||
|
{ timeout: 8000 },
|
||||||
|
);
|
||||||
|
|
||||||
|
// Finish both: bundle-a via its SSE stream, bundle-b via its poll
|
||||||
|
// observing the terminal server status.
|
||||||
|
bundleBStatus = "installed";
|
||||||
|
FakeEventSource.instances[0].onmessage?.({ data: JSON.stringify({ phase: "complete" }) });
|
||||||
|
await vi.waitFor(
|
||||||
|
() => {
|
||||||
expect(useFeaturesStore.getState().installAllActive).toBe(false);
|
expect(useFeaturesStore.getState().installAllActive).toBe(false);
|
||||||
});
|
},
|
||||||
}, 15000);
|
{ timeout: 8000 },
|
||||||
|
);
|
||||||
|
}, 25000);
|
||||||
|
|
||||||
it("retries a bundle once if it fails during Install All, then stops", async () => {
|
it("retries a bundle once if it fails during Install All, then stops", async () => {
|
||||||
const bundles = [makeBundleState({ id: "flaky", status: "not_installed" })];
|
const bundles = [makeBundleState({ id: "flaky", status: "not_installed" })];
|
||||||
|
|||||||
Reference in New Issue
Block a user