Commit Graph
13 Commits
Author SHA1 Message Date
SnapOtterandGitHub 7d37f6e6f5 fix(pdf): flag scanned PDFs in pdf-to-text and serve text as UTF-8 (#603)
When a PDF has no text layer (scanned or image-only), pdf-to-text now
returns a 422 that points at the OCR tool instead of a silent empty file,
and text downloads carry charset=utf-8 so UTF-8 Arabic renders correctly
when the .txt is viewed inline.

Fixes #589
2026-07-21 16:26:43 +08:00
SnapOtterandGitHub 991c981529 fix: make OCR portable and reliable across AMD64 and ARM64 (#519)
* fix: make OCR portable and reliable

* fix: harden OCR installation portability

* fix: pin OCR partials across downloads

* fix: make OCR execution reliably asynchronous

* fix: harden OCR portability and docs routes

* fix: preserve decoder and docs safeguards
2026-07-15 03:34:24 +08:00
SnapOtterandGitHub dadf766899 fix(migrator): correct and harden the 1.x to 2.0 SQLite import (#434)
* feat(api): parse DATA_DIR from env for 1.x import auto-detection

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* test(migrator): build 1.17.2 fixtures by replaying legacy migrations

Discovered the legacy migrations seed a Default team (0005) and builtin roles
(0007), so the replayed fixture carries them. Seed uses a distinct custom team.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* fix(migrator): self-adjusting column copy, jobs.status map, drop sessions, advisory lock

The importer now inserts only the intersection of source and live target columns,
so the three analytics_* columns 2.x dropped no longer break the first users INSERT
(and future dropped columns are handled generically). jobs.status is mapped onto the
2.x enum (error->failed). Sessions are no longer migrated. A pg_advisory_xact_lock
serializes concurrent replicas. Includes login-after-migrate and library assertions.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* test(migrator): CI drift guard fails when a required column is unfillable from 1.17.2

Introspects every NOT-NULL-no-default column of each migrated table in the current
schema and asserts the engine can fill it from a real 1.17.2 source. Turns a future
breaking schema change into a PR-time failure instead of a production import break.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* feat(migrator): orchestrator with detection, boot states, marker, blob count

sqlite-import.ts owns source resolution (explicit path, 'off' sentinel, DATA_DIR
probe), the four boot states (import/leftover/locked/none), the persisted
sqlite_import marker, and a read-only library-blob count. runBootImport wires them
together and catches TargetNonEmptyError as a benign multi-replica skip.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* feat(api): route boot through the 1.x import orchestrator; hide marker from non-admins

index.ts now calls runBootImport (which owns detection + the four boot states)
instead of the inline SQLITE_MIGRATE_PATH block. The sqlite_import marker is added
to SENSITIVE_KEYS (but not REDACTED_KEYS) so admins see the counts for the banner
while non-admins don't see the key at all.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* feat(migrator): add analyzeSqlite + dry-run/verify CLI

analyzeSqlite is a read-only pre-flight (no live Postgres): per-table row counts,
library-blob presence, and out-of-enum job statuses. The migrate:sqlite CLI now
lives in the orchestrator and supports --dry-run/--verify (prints the analysis and
exits without writing) alongside the existing import and --force.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* docs: add 1.x to 2.0 upgrade guide; fix volume-name casing

New apps/docs upgrade guide covering auto-detect, the SQLITE_MIGRATE_PATH override +
off opt-out, the dry-run, what carries over, locked-state recovery, and non-destructive
rollback. Leads with 'back up the WHOLE /data volume, not just snapotter.db' because
1.x WAL mode leaves data in snapotter.db-wal (surfaced by the real-image upgrade test).
Standardizes README/DOCKERHUB compose volume names on the canonical SnapOtter-data
casing so they match the repo compose and don't orphan an upgrader's volume.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* feat(web): admin 1.x migration banner + 21-locale strings

A one-time admin banner reads the sqlite_import marker from /v1/settings and shows
the import result (user + saved-file counts) on success, or a warning when a 1.x
database was found but not imported. Dismissal persists to a sqlite_import.dismissedAt
settings key. shouldShowMigrationBanner/parseMigrationMarker sit in feedback.ts with
the other shouldShow helpers; strings added to en.ts and all 20 other locales.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* style(landing): biome-format Hero.astro trustBadges array

Pre-existing formatting drift on main (its Lint check was skipped on the merge that
introduced it); this PR's full Lint run surfaced it. Formatting-only, applied via
the repo's own biome formatter to unblock the required Lint check.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w
2026-07-04 15:15:39 +00:00
SnapOtterandGitHub e0dbf2a5c3 fix: set a writable HOME for the app user so PaddleOCR works in non-root deployments (#430)
The container dropped privileges to the non-root snapotter user via gosu
(external) and s6-setuidgid (embedded), both of which preserve the
environment without setting HOME. The app therefore kept root's HOME=/root,
which is not writable by snapotter, and PaddleOCR died with
PermissionError: '/root/.paddlex/temp' -- breaking the ocr tool at default
quality in every non-root deployment. Prior GPU QA ran the app as root, which
masked it.

Fix: export HOME=/data/.home (persistent, writable, hidden) at every
privilege-drop point:
- entrypoint.sh external gosu path and non-root tini path (the latter uses
  $DD/.home so a DATA_DIR override stays consistent).
- the s6 snapotter/run service (scoped there, not globally before /init, so
  postgres/redis do not inherit a snapotter-owned HOME).
The root preflight creates /data/.home and the existing chown sweep owns it as
the PUID/PGID-remapped snapotter; the dir is added to both ensure_writable
probes so an unwritable HOME fails fast with the storage-permission guidance
instead of crashing late. The Dockerfile passwd home moves from /app
(read-only) to /data/.home as the getpwuid fallback when HOME is unset.

Because bridge.ts forwards HOME to the Python sidecar, this also repairs the
expanduser("~") caches in inpaint/outpaint/restore/noise_removal/remove_bg,
not just PaddleOCR.

Also fixes a test-harness inconsistency: tool-default-settings passport-photo
countryCode "us" -> "US" (the route exact-matches uppercase PASSPORT_SPECS
codes; the UI already sends "US", so users were never affected).

Claude-Session: https://claude.ai/code/session_01XGB4pGvTvb7sUX4JN745U7
2026-07-04 09:40:04 +00:00
SnapOtterandGitHub d8cf979d4b fix: resolve 18 QA-discovered bugs across tools, previews, and the AI pipeline (#242)
Exhaustive QA sweep of all 157 tools. Fixes: CSP blob media, csv-excel ExcelJS interop, ocr-pdf segfault, chart-maker upload, non-PDF doc preview, RAW decode, merge-tool multi-file path, html-to-image chromium, ogv/wma/amr/ac3 preview fallbacks, meme/gif/stabilize codecs, nav+home a11y. Plus orphan-format and test-debt cleanup, the AI bundle build script, and a reusable Playwright QA harness under tests/qa/.
2026-06-15 22:26:24 +08:00
SnapOtter e7bef34918 style: apply Biome formatting to enterprise files 2026-06-14 14:53:44 +08:00
SnapOtter f6d5479334 test: add enterprise feature mock utilities 2026-06-13 16:22:01 +08:00
SnapOtter fc7c1f850e feat(tools): 2.0 phase 5 wave 4 - office, ebooks, data, archives (14 tools) (#224) 2026-06-13 10:19:11 +08:00
SnapOtter 638288e196 feat(tools): 2.0 phase 5 wave 3b - audio depth (14 tools) (#222) 2026-06-13 10:19:06 +08:00
SnapOtter 5f98b48593 feat(tools): 2.0 phase 5 wave 3a - video depth (22 tools) (#221) 2026-06-13 10:19:00 +08:00
SnapOtter 2f39e38162 feat(tools): 2.0 phase 5 wave 2 - pdf depth (21 tools) (#220) 2026-06-13 10:18:55 +08:00
SnapOtter ae1337901d feat(tools)!: SnapOtter 2.0 phase 4 wave 1: 45 core tools across all modalities (#219) 2026-06-13 10:18:49 +08:00
SnapOtterandGitHub 4ec39c556f test: testing overhaul -- CI e2e gates, parallel suites, generated matrices, mutation testing (#215)
Closes the "e2e never runs in CI" hole. Adds per-PR e2e smoke gate,
nightly full-suite workflows, parallel vitest forks (per-fork DBs),
Playwright parallel/serial/visual projects against production builds,
metadata-generated test suites (drift guards, hostile inputs, format
matrix, pairwise settings, property-based fuzz), Stryker mutation
testing, Schemathesis API fuzz, coverage ratchet, and fixes for three
session-poisoning bugs that caused 200+ serial-bucket failures.

Bug fix included: favicon/split/bulk-rename could hang clients forever
when ZIP streaming failed after reply.hijack().
2026-06-10 22:01:13 +08:00