SnapOtter

2,001 Commits 10 Branches 89 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
SnapOtterandGitHub	b4375e558d	fix: harden install queue/dispatcher lifecycle and repair review-sweep regressions (#395 ) Fixes 15 defects found by a max-effort multi-agent review of the last 6 merged PRs (#388, #390, #391, #392, #393, #394), all adversarially verified before fixing. Install queue + dispatcher (the serious cluster): - features.ts: finalize the installer child exactly once. A failed spawn fires both "error" and "close", and the second event released the file lock and active slot that pump() had just handed to the next queued bundle, letting two pip processes write the same venv concurrently. Outcome recording now happens before pump() so the next bundle's first progress frame cannot race the previous install's bookkeeping. - feature-status.ts: keep failed-install errors in a per-bundle map instead of the single progress slot. With the queue auto-starting the next install, the slot was overwritten within seconds and a failed install vanished without ever surfacing to GET /features. - bridge.ts: scope child lifecycle per process (stopped-children set + request generation tags) instead of an instance-wide shuttingDown flag that the next spawn reset. A stale SIGTERMed child's late close event could record a phantom crash (5 of which permanently disable the dispatcher), null out the freshly spawned child, and reject the new child's pending requests. The request-timeout kill path still counts as a real crash. - install_feature.py: the pre-write disk re-check measured ai_dir's filesystem even when budgeting the cross-filesystem copy that lands on the venv's disk; now each budget is checked against the filesystem the bytes actually land on, so ENOSPC cannot strike mid-write and leave site-packages half overwritten. Behavior regressions: - embed-subtitles: preserve pre-existing subtitle tracks (0:s?) and MKV attachments (0:t?) that the -map 0:v:0/0:a? rewrite silently dropped; data streams stay unmapped on purpose (the actual MPEG remux fix). The new subtitle maps first so the language tag hits the right stream. - usage-survey-overlay: fail closed when the settings fetch fails; the fail-open path rendered the blocking survey against an unhealthy API and soft-locked admins, the lock-out class #392 fixed. - features-store: queued bundles poll instead of each holding an SSE connection (Install All could pin 7 EventSources and exhaust the browser's 6-per-origin HTTP/1.1 limit, hanging the whole app); listenToProgress closes any prior stream and stops any poll before subscribing; installAll skips bundles already installing or queued. Contracts, tests, i18n: - openapi.yaml: add "queued" to the features status enum and document downloadBytes/installedBytes (Schemathesis conformance). - feature-lifecycle e2e: queue transcription (~0.5 GB) instead of ocr (~6 GB) and give the test a budget that covers both install drains (the stacked waits exceeded the old 900s timeout). - docker-compose.qa.yml: parameterize the host port (QA_APP_PORT) so QA_PROJECT_NAME concurrent stacks can actually bind. - compare + watermark-image: restore per-input error attribution ("Invalid first/second image", "Invalid watermark image") lost in the shared-handler migration. - ai-features-section: the "{size} on disk" suffix now goes through i18n; key added to all 21 locales. - watermark-image + content-aware-resize: migrate to the shared inputHandlerFor("image") chain like compare/vectorize/compose, fixing drift in the inline copies (no SVG sanitize, no RAW extension hint, no AVIF probe). Verified: typecheck across 9 workspaces, Biome clean on all changed files, 584 targeted unit tests and 249 integration tests green (including real-ffmpeg embed-subtitles runs). One unit test updated to the new poll-while-queued contract with a single-EventSource assertion. Claude-Session: https://claude.ai/code/session_017mR1HiHaf3a1BmUtrHX4j3	2026-07-03 13:47:15 +08:00
SnapOtterandGitHub	bd1838e40b	fix: repair docker validation QA tooling, dispatcher crash-accounting, and image-enhancement RAW hang (#391 ) Found and fixed during a full local Docker build validation (amd64/arm64, all four fleet targets, AI bundle installs, QA harness) and the follow-up bug sweep requested afterward. None of the affected scripts run in CI, so these had been silently broken indefinitely. - docker/feature-manifest.json: pythonVersion was a flat "3.11", but the amd64 base (Ubuntu 24.04) ships Python 3.12 while arm64 (Debian bookworm) ships 3.11. Changed to a per-arch object matching the file's existing convention. - tests/qa/api-sweep.mts and verify-ai.mts: bare "@snapotter/shared" import can't resolve since tests/ is not a pnpm workspace member, making both silently unrunnable via their own documented command on any fresh checkout. Switched to a relative import. - tests/qa/generate-ledger.mts: wrote to docs/qa/ without creating the directory first; docs/ is gitignored except COMMUNITY_GUIDE.md, so a fresh checkout threw ENOENT. - Seven QA Playwright spec files (input-preview, settings, settings-extended, multifile, output-preview, pipeline-ui, smoke) had ~115 fixture() calls using directory names that don't exist. Resolved every call programmatically against the real fixture tree. - packages/ai/src/bridge.ts: AI dispatcher restart (happens on every bundle install) was falsely counted as a crash, risking permanent dispatcher disable after enough legitimate restarts within the crash window. Added a shuttingDown flag checked at all three recordCrash() call sites. - packages/image-engine/src/operations/auto-enhance.ts: image-enhancement hung 40+ seconds on large RAW photos (confirmed on a real 20.2MP file) in Sharp's .clahe() step, whose cost scales with total pixel count regardless of tile size. Added a 16-megapixel cap above which CLAHE is skipped; verified against the real file (40+s -> 2.0s) with no regression to other RAW formats or normal-sized images. Fixing this surfaced a second, smaller bug where the saturation step's CLAHE compensation boost was keyed off the raw toggle instead of whether CLAHE actually ran. - Two QA-harness robustness gaps closed per "fix everything, even the small bugs": the passport-photo/erase-object input-preview tests now skip cleanly with a clear reason on a container without their AI bundle installed, and docker-compose.qa.yml's hardcoded project/container name (the actual root cause of a mid-validation container swap between two concurrent sessions) is now parameterized via QA_PROJECT_NAME. Full validation report is local-only per repo convention.	2026-07-02 14:21:13 +08:00
SnapOtter	1727a7a73e	test: reorganize flat test files into purpose-based subdirectories (phase 6) Group 245 flat integration tests and 25 loose unit tests into discoverable subdirectories per spec section 6: integration/tools/{image,video,audio,document,data}/ (156 files) integration/platform/ (64 files) integration/generated/ (14 files) integration/security/ (10 files) unit/security/ (8 files, new subdir) unit/api/ (7 files moved in) unit/web/ (3 files moved in) unit/shared/ (6 files moved in) unit/image-engine/ (1 file moved in) All moves via git mv (history preserved). Relative imports repaired for both depth levels (platform/generated/security = +1, tools/ = +2): static from-imports, dynamic import() calls, vi.mock() paths, import.meta.dirname joins, and __dirname joins. Vitest discovery unchanged (no test.include in config, recursive glob matches subdirs, shard-by-hash unaffected). test-server.ts and tool-route-drift.test.ts stay at integration root. fixtures/ untouched. Parity gate: 13189 passing test names before = 13189 after (0 dropped).	2026-06-20 05:07:46 +08:00

Author

SHA1

Message

Date

SnapOtterandGitHub

b4375e558d

fix: harden install queue/dispatcher lifecycle and repair review-sweep regressions (#395 )

Fixes 15 defects found by a max-effort multi-agent review of the last 6
merged PRs (#388, #390, #391, #392, #393, #394), all adversarially
verified before fixing.

Install queue + dispatcher (the serious cluster):

- features.ts: finalize the installer child exactly once. A failed spawn
  fires both "error" and "close", and the second event released the file
  lock and active slot that pump() had just handed to the next queued
  bundle, letting two pip processes write the same venv concurrently.
  Outcome recording now happens before pump() so the next bundle's first
  progress frame cannot race the previous install's bookkeeping.
- feature-status.ts: keep failed-install errors in a per-bundle map
  instead of the single progress slot. With the queue auto-starting the
  next install, the slot was overwritten within seconds and a failed
  install vanished without ever surfacing to GET /features.
- bridge.ts: scope child lifecycle per process (stopped-children set +
  request generation tags) instead of an instance-wide shuttingDown flag
  that the next spawn reset. A stale SIGTERMed child's late close event
  could record a phantom crash (5 of which permanently disable the
  dispatcher), null out the freshly spawned child, and reject the new
  child's pending requests. The request-timeout kill path still counts
  as a real crash.
- install_feature.py: the pre-write disk re-check measured ai_dir's
  filesystem even when budgeting the cross-filesystem copy that lands on
  the venv's disk; now each budget is checked against the filesystem the
  bytes actually land on, so ENOSPC cannot strike mid-write and leave
  site-packages half overwritten.

Behavior regressions:

- embed-subtitles: preserve pre-existing subtitle tracks (0:s?) and MKV
  attachments (0:t?) that the -map 0:v:0/0:a? rewrite silently dropped;
  data streams stay unmapped on purpose (the actual MPEG remux fix). The
  new subtitle maps first so the language tag hits the right stream.
- usage-survey-overlay: fail closed when the settings fetch fails; the
  fail-open path rendered the blocking survey against an unhealthy API
  and soft-locked admins, the lock-out class #392 fixed.
- features-store: queued bundles poll instead of each holding an SSE
  connection (Install All could pin 7 EventSources and exhaust the
  browser's 6-per-origin HTTP/1.1 limit, hanging the whole app);
  listenToProgress closes any prior stream and stops any poll before
  subscribing; installAll skips bundles already installing or queued.

Contracts, tests, i18n:

- openapi.yaml: add "queued" to the features status enum and document
  downloadBytes/installedBytes (Schemathesis conformance).
- feature-lifecycle e2e: queue transcription (~0.5 GB) instead of ocr
  (~6 GB) and give the test a budget that covers both install drains
  (the stacked waits exceeded the old 900s timeout).
- docker-compose.qa.yml: parameterize the host port (QA_APP_PORT) so
  QA_PROJECT_NAME concurrent stacks can actually bind.
- compare + watermark-image: restore per-input error attribution
  ("Invalid first/second image", "Invalid watermark image") lost in the
  shared-handler migration.
- ai-features-section: the "{size} on disk" suffix now goes through
  i18n; key added to all 21 locales.
- watermark-image + content-aware-resize: migrate to the shared
  inputHandlerFor("image") chain like compare/vectorize/compose, fixing
  drift in the inline copies (no SVG sanitize, no RAW extension hint,
  no AVIF probe).

Verified: typecheck across 9 workspaces, Biome clean on all changed
files, 584 targeted unit tests and 249 integration tests green
(including real-ffmpeg embed-subtitles runs). One unit test updated to
the new poll-while-queued contract with a single-EventSource assertion.

Claude-Session: https://claude.ai/code/session_017mR1HiHaf3a1BmUtrHX4j3

2026-07-03 13:47:15 +08:00

SnapOtterandGitHub

bd1838e40b

fix: repair docker validation QA tooling, dispatcher crash-accounting, and image-enhancement RAW hang (#391 )

Found and fixed during a full local Docker build validation (amd64/arm64, all
four fleet targets, AI bundle installs, QA harness) and the follow-up bug
sweep requested afterward. None of the affected scripts run in CI, so these
had been silently broken indefinitely.

- docker/feature-manifest.json: pythonVersion was a flat "3.11", but the
  amd64 base (Ubuntu 24.04) ships Python 3.12 while arm64 (Debian bookworm)
  ships 3.11. Changed to a per-arch object matching the file's existing
  convention.
- tests/qa/api-sweep.mts and verify-ai.mts: bare "@snapotter/shared" import
  can't resolve since tests/ is not a pnpm workspace member, making both
  silently unrunnable via their own documented command on any fresh
  checkout. Switched to a relative import.
- tests/qa/generate-ledger.mts: wrote to docs/qa/ without creating the
  directory first; docs/ is gitignored except COMMUNITY_GUIDE.md, so a fresh
  checkout threw ENOENT.
- Seven QA Playwright spec files (input-preview, settings,
  settings-extended, multifile, output-preview, pipeline-ui, smoke) had
  ~115 fixture() calls using directory names that don't exist. Resolved
  every call programmatically against the real fixture tree.
- packages/ai/src/bridge.ts: AI dispatcher restart (happens on every bundle
  install) was falsely counted as a crash, risking permanent dispatcher
  disable after enough legitimate restarts within the crash window. Added a
  shuttingDown flag checked at all three recordCrash() call sites.
- packages/image-engine/src/operations/auto-enhance.ts: image-enhancement
  hung 40+ seconds on large RAW photos (confirmed on a real 20.2MP file) in
  Sharp's .clahe() step, whose cost scales with total pixel count regardless
  of tile size. Added a 16-megapixel cap above which CLAHE is skipped;
  verified against the real file (40+s -> 2.0s) with no regression to other
  RAW formats or normal-sized images. Fixing this surfaced a second,
  smaller bug where the saturation step's CLAHE compensation boost was
  keyed off the raw toggle instead of whether CLAHE actually ran.
- Two QA-harness robustness gaps closed per "fix everything, even the small
  bugs": the passport-photo/erase-object input-preview tests now skip
  cleanly with a clear reason on a container without their AI bundle
  installed, and docker-compose.qa.yml's hardcoded project/container name
  (the actual root cause of a mid-validation container swap between two
  concurrent sessions) is now parameterized via QA_PROJECT_NAME.

Full validation report is local-only per repo convention.

2026-07-02 14:21:13 +08:00

SnapOtter

1727a7a73e

test: reorganize flat test files into purpose-based subdirectories (phase 6)

Group 245 flat integration tests and 25 loose unit tests into
discoverable subdirectories per spec section 6:

  integration/tools/{image,video,audio,document,data}/  (156 files)
  integration/platform/                                  (64 files)
  integration/generated/                                 (14 files)
  integration/security/                                  (10 files)

  unit/security/     (8 files, new subdir)
  unit/api/          (7 files moved in)
  unit/web/          (3 files moved in)
  unit/shared/       (6 files moved in)
  unit/image-engine/ (1 file moved in)

All moves via git mv (history preserved). Relative imports repaired
for both depth levels (platform/generated/security = +1, tools/ = +2):
static from-imports, dynamic import() calls, vi.mock() paths,
import.meta.dirname joins, and __dirname joins.

Vitest discovery unchanged (no test.include in config, recursive glob
matches subdirs, shard-by-hash unaffected). test-server.ts and
tool-route-drift.test.ts stay at integration root. fixtures/ untouched.

Parity gate: 13189 passing test names before = 13189 after (0 dropped).

2026-06-20 05:07:46 +08:00