mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
935861bced95cd778ea77d5a3741ff1da1a2ffed
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 SnapOtterandGitHub
|
935861bced |
fix(release): resolve the release by id, and make the vulnerability gate cover HIGH (#661)
Two release-pipeline defects found while pre-flighting 2.2.0, plus the image hardening that the second one exposed. The release job would have died immediately after pushing the v2.2.0 tag. draftRelease was turned on in #649 and never executed, and GitHub's /releases/tags/{tag} endpoint does not return draft releases, so all nine tag lookups in release.yml would have 404'd against the draft semantic-release had just created. Verified against this repo with a throwaway draft: the tag endpoint 404s while gh release view reads it and /releases/{id} returns the same REST shape. Every site now resolves the numeric id first, so existing jq expressions are untouched. The unfixed-vulnerability gate was measuring almost nothing. The blocking Trivy steps run ignore-unfixed, and trivy-unfixed-gate.mjs was meant to cover the remainder but defaults to CRITICAL with neither call site passing --severity. An unfixed HIGH was gated by nothing, and the arm64 image carried 79 of them while the summary read clean. Rather than document 79 findings, the image lost what it did not need: libde265 1.1.1 and libheif 1.23.1 are now built from source (the old libheif pin was itself affected by CVE-2026-3950, and Debian's libde265 1.0.11 was the decoder every .heic upload actually reached), and xvfb, wget and openssh-client are purged. 15 CVEs left the image outright and the HIGH gap fell to 65, each now carrying a rationale verified against the running container. curl gets its own section: bookworm-backports has a fixed 8.14.1, so claiming no fix was available would have been false. It is recorded as a declined fix. Verified on both architectures: gate exits 0, the source-built libde265 is the one libheif links, and HEIC, RAW, ImageMagick, Sharp AVIF and headless chromium all still work after the purge. |
||
|
SnapOtterandGitHub
|
d10d0f544f |
fix: release QA hardening across processing, media, security, and CI gates (#649)
A release-readiness QA pass over the whole product. The commits split into defects a user would hit and gates that were reporting green while measuring nothing. ## Fixes that change behaviour Rate limiting was bypassable on every install: TRUST_PROXY defaulted to true, so request.ip came from a client-set header and a forged X-Forwarded-For got past the login limiter. The default is now a private-network trust list. A transient Postgres outage stranded in-flight jobs, leaving finished output on disk with no row pointing at it. A reconciler now resolves those rows and adopts the bytes rather than dropping the work. A Redis connection that moved to a new address wedged every read-blocked consumer, so completions stopped signalling while health still answered 200. Socket timeouts plus subscriber pings recover it. Installing more than one AI bundle left the shared venv multi-versioned and silently broke three tools. The installer now reconciles distributions to one version each. Converting an image to JXL at quality 1 through 4 returned a 500, because libjxl 0.7 rejects the distance those values compute. The quality is floored at what the encoder honours. A missing ffmpeg was also reported to the user as a corrupt upload; it now says the engine is unavailable. RAW uploads reached an unpatched LibRaw on arm64, so it is built from source at 0.22.2, and the release scan was split so it can fail on an unfixed critical instead of hiding it behind ignore-unfixed. ## Gates that could not fail Two mutation lanes ran zero mutants because Stryker crawled the gitignored docs build; coverage discarded its whole report on any failing test; the lint gate skipped root tests, scripts, and two workspaces; and several generated matrices counted a host missing ffmpeg as a passing tool. Each now measures what it claims. Full evidence and the outstanding release items are tracked locally and are not part of this branch. |
||
|
SnapOtterandGitHub
|
4ba7503f15 |
fix(docker): harden Postgres readiness checks in compose and startup (#595)
Pin the compose Postgres healthchecks to POSTGRES_DB (pg_isready was defaulting to the username, silently reporting healthy while spamming FATAL logs when USER and DB differ), and make docker/wait-for-postgres.mjs log the target host and error code instead of a silent retry loop. Adds a change-me note next to the default password in README and Docker Hub. Refs #592 |
||
|
SnapOtterandGitHub
|
60d01ab2dd |
fix: release-acceptance QA follow-ups (upload crash, scipy ABI conflict, rate limit, OCR fallback) (#458)
* fix(api): prevent a crash when an over-limit upload stream has no consumer yet busboy's "limit" handler destroyed the file stream with an error but never attached its own error listener, relying entirely on whatever consumes part.file downstream to do so. On a fast enough connection (or a fully buffered body, e.g. Fastify inject()), busboy can process enough bytes to hit the size limit before the route handler's receiveUpload() call has attached its own stream listener, leaving the resulting "error" event with zero listeners -- which crashes the whole process by default in Node. Surfaced by tonight's FULL_MATRIX+FUZZ integration run (880 uncaught exceptions, all the same root cause). Reproduces deterministically in isolation; unrelated to this release's actual code delta (file untouched since PR #413, well before the baseline QA pass). Fix: attach a baseline no-op error listener the moment the stream is created, guaranteeing at least one listener always exists. EventEmitter delivers "error" to every registered listener, so the real consumer's own error handling is unaffected. * fix(ai-bundles): rebuild upscale-enhance and photo-restoration to reconcile scipy ABI upscale-enhance and photo-restoration both depend on codeformer-pip, whose transitive closure (basicsr -> realesrgan -> gfpgan) pulls in an unpinned scipy. Both bundles were last built ~June 18-19, before PR #437 added the manifest's `constraints` array (numpy==1.26.4, scipy==1.12.0, etc.) to pin exactly this kind of dependency during bundle builds. Only the ocr bundle was rebuilt after that fix landed. install_feature.py has no pip install step -- it's a raw tarfile extraction with no cross-bundle conflict resolution, so installing OCR alongside either stale bundle left three incompatible scipy versions' files mixed in the same site-packages directory (a compiled _rotation.*.so from one release next to Python files expecting a different release's API), breaking the `upscale` tool and OCR's higher-quality tiers with an ImportError. Rebuilt both bundles for amd64-gpu and arm64-cpu from the current manifest, verified scipy/scikit-learn/scikit-image/pandas all resolve to the pinned versions in the tarballs themselves, then verified end-to-end on real hardware (Mac arm64 CPU and ubuntu_gpu .248 RTX 4070): installing all affected bundles together now yields exactly one version of each constrained package, `upscale` produces correct output, and OCR's balanced/best tiers correctly use PaddleOCR-GPU instead of erroring out. Published the rebuilt tarballs to the public deepsafe/feature-bundles HuggingFace repo and updated this manifest's sha256/sizes to match. Also adds verify-bundle-compatibility.sh: verify-bundle.sh checks each bundle in isolation (a fresh venv per bundle), which is exactly why this shipped twice -- nothing ever checked that bundles built at different times agree once layered into the one shared venv real installs use. The new script installs every bundle for an arch into one venv and asserts each constrained package has exactly one, correct version. Known follow-up (not fixed here, needs separate discussion): uninstalling a bundle only removes its downloaded model weights, never the site-packages it added, so existing installations that already hit this bug have no clean self-service fix via uninstall+reinstall -- they need a full AI-venv wipe. * fix(docker): bake a real rate limit default for the all-in-one one-liner The documented single-container `docker run` install had RATE_LIMIT_PER_MIN=0 (effectively unlimited, ~50k/min) baked in, since only docker-compose.yml carried a hardened override. A self-hoster following the one-liner path got no meaningful throttling anywhere, including auth-adjacent routes with no dedicated per-route limit. Bakes a generous-but-real 1000/min default into the Dockerfile, raises both compose files' fallback to match so the two documented install paths converge on the same posture, and updates the Zod schema default plus docs that quoted the old value. * fix(api): boot log undercounted tool routes by the conversion-preset total The "Tool routes: N active" line logged before registerConversionPresets(app) ran, so it only ever reported the base 158 tools, 83 short of the real 241-tool total. Presets have to register after the base loop (they delegate to each base tool's own processV2), so the fix moves the log line to after that call and has registerConversionPresets return its count instead of reordering the dependency. * fix(ai): forward {info}/{warning} stderr JSON instead of dropping it The dispatcher stderr parser only recognized {ready} and {progress,stage} shaped JSON lines; anything else that parsed as valid JSON (like ocr.py's GPU-to-tesseract downgrade notice, an {"info": ...} line) matched neither branch and fell through silently, never reaching docker logs. Adds explicit {info}/{warning} handling that forwards to console.log/console.warn, same as the existing [prefix]-tagged non-JSON path. * fix(api): fall back to a lower OCR tier when PaddleOCR itself is unusable ocr.ts already retries lower quality tiers on a crashed dispatcher, but the condition only matched crash-style messages (segfault, exited unexpectedly). ocr.py's own ImportError/exception handlers already produce messages telling the caller to use a lower tier (e.g. on the scipy ABI conflict class of bug), but nothing ever acted on them, so a broken PaddleOCR hard-failed with 422 instead of degrading to Tesseract like ocr-pdf effectively does. Broadens the retry condition to also catch PaddleOCR-engine-unusable messages. Note: ocr-pdf's tesseract-only behavior turned out to be an unrelated, pre-existing, deliberate design choice (PaddleOCR segfaults on rasterized PDF pages on arm64), not a graceful-fallback mechanism to copy -- the two tools weren't actually solving the same problem, so this fixes ocr.ts's own gap rather than trying to mirror ocr-pdf. |
||
|
SnapOtterandGitHub
|
f3342a1e57 |
fix: harden Docker image and async job responses
Harden Docker runtime packaging, preserve async job response semantics, fix Redis subscriber startup connections, clear lint warnings, and harden enterprise S3 object body handling. |