Commit Graph
7 Commits
Author SHA1 Message Date
SnapOtterandGitHub d10d0f544f fix: release QA hardening across processing, media, security, and CI gates (#649)
A release-readiness QA pass over the whole product. The commits split into
defects a user would hit and gates that were reporting green while measuring
nothing.

## Fixes that change behaviour

Rate limiting was bypassable on every install: TRUST_PROXY defaulted to true, so
request.ip came from a client-set header and a forged X-Forwarded-For got past
the login limiter. The default is now a private-network trust list.

A transient Postgres outage stranded in-flight jobs, leaving finished output on
disk with no row pointing at it. A reconciler now resolves those rows and adopts
the bytes rather than dropping the work.

A Redis connection that moved to a new address wedged every read-blocked
consumer, so completions stopped signalling while health still answered 200.
Socket timeouts plus subscriber pings recover it.

Installing more than one AI bundle left the shared venv multi-versioned and
silently broke three tools. The installer now reconciles distributions to one
version each.

Converting an image to JXL at quality 1 through 4 returned a 500, because
libjxl 0.7 rejects the distance those values compute. The quality is floored at
what the encoder honours. A missing ffmpeg was also reported to the user as a
corrupt upload; it now says the engine is unavailable.

RAW uploads reached an unpatched LibRaw on arm64, so it is built from source at
0.22.2, and the release scan was split so it can fail on an unfixed critical
instead of hiding it behind ignore-unfixed.

## Gates that could not fail

Two mutation lanes ran zero mutants because Stryker crawled the gitignored docs
build; coverage discarded its whole report on any failing test; the lint gate
skipped root tests, scripts, and two workspaces; and several generated matrices
counted a host missing ffmpeg as a passing tool. Each now measures what it
claims.

Full evidence and the outstanding release items are tracked locally and are not
part of this branch.
2026-07-27 15:37:30 +08:00
SnapOtterandGitHub ae6a4c8b7c fix: error-only Sentry telemetry, storm-proof capture, and crash fixes (#476)
Removes Sentry tracing entirely (BullMQ idle polling burned 4.8M transactions in 2 days at the baked 0.1 rate), decouples PostHog sampling, and replaces the type-only error scrub with a vetted-field sanitizer plus SafeError/ToolInputError contracts. One classified capture path with per-signature throttles and a per-process ceiling makes storms impossible (NODE-1E was 4,541 events from one 30s loop). Browser errors move to a dedicated web Sentry project with their own source maps. Adds the SNAPOTTER_TELEMETRY runtime kill switch and silences test fleets.

Crash fixes: remote 204/304 SSRF process kill (NODE-20), conversion-preset boot crash loop (NODE-21), Redis version preflight + unhandled subscribe rejection (NODE-1T), Sign PDF on plain-http origins (NODE-1K/1M), wavesurfer/pdf.js teardown rejections (NODE-1P/1N), bundle-import ZlibError to 400 (NODE-1Z), chart-maker input errors declassified (NODE-1H/1J), asset requests skip the session DB lookup (NODE-1D).
2026-07-10 21:41:49 +08:00
SnapOtterandGitHub 63a03d26f2 feat: pipeline templates, analytics opt-out, 83 conversion presets, positioning + e2e modernization
Lands five integrated branches: pipeline templates (#355), analytics opt-out (#354), 83 conversion presets bringing the catalog to 240 tools (#356), self-hosted positioning (#353), and e2e modernization (#351).

Integration fixes: aligned stale web analytics tests with the opt-out/allow-list model, closed 3 CodeQL incomplete-sanitization alerts in the i18n generator, resolved settings/index/docs/format-matrix conflicts, and corrected tool counts to 240.
2026-06-28 18:57:53 +08:00
SnapOtterandGitHub 8f4235d2c6 fix(enterprise): ship enterprise package in prod image + S3, analytics, tracing, queue fixes (#342)
* fix(enterprise): ship enterprise pkg in prod image, full license features, tracing key fallback

docker/Dockerfile: COPY packages/enterprise manifest+src into the production stage.
Without it, apps/api's workspace link to @snapotter/enterprise dangles and every
import() throws (silently caught), so all 19 enterprise features failed closed
(enterprise.active=false) regardless of a valid license.

scripts/generate-license.mjs: sync PLAN_FEATURES with packages/enterprise/src/license.ts
so a --plan enterprise license unlocks all 19 features (was 8) and team unlocks 8.

apps/api/src/tracing.ts: accept SNAPOTTER_LICENSE_KEY as a fallback to LICENSE_KEY so
distributed_tracing activates with the same key as the rest of the app.

* fix(docker): keep scripts/bake-analytics.mjs in build context

.dockerignore excluded the whole scripts/ dir (PR #82, V1 hardening), but
docker/Dockerfile later added 'COPY scripts/bake-analytics.mjs' for the analytics
bake step. A clean production image build therefore fails with
'scripts/bake-analytics.mjs: not found'. The published image build is gated off in
CI so this latent break went unnoticed. Exclude scripts/* but re-include the one
file the Dockerfile needs.

* fix: S3 upload stream, analytics bake reaches API, dedupe retention field, reconcile orphan jobs

storage-s3.ts: wrap the upload AsyncIterable in Readable.from() so @aws-sdk/lib-storage
accepts it. STORAGE_MODE=s3 file uploads failed with 'Body Data is unsupported format'
for every tool because a bare async generator is not a Readable.

docker/Dockerfile: COPY the builder-baked analytics baked.ts into the API runtime stage.
The API re-copied the committed (off) baked.ts from the build context, so the
SNAPOTTER_ANALYTICS build arg had no effect on the API -- and since the SPA reads
/api/v1/config/analytics, analytics was off everywhere regardless of the arg.

settings-dialog.tsx: remove the duplicate tempFileMaxAgeHours control under Data
Retention; it bound the same setting key as the File Management control with a different
default, so editing either silently overwrote the other.

apps/api/src/index.ts: reconcile orphaned job rows (empty tool_id, never enqueued to
BullMQ) at boot so they don't sit in processing/queued forever and inflate the per-user
concurrent-job count and the upgrade-check in-flight gate.

* fix(web): style the SSO login buttons (they referenced undefined theme tokens)

The OIDC/SAML 'Sign in with <provider>' buttons used bg-secondary /
text-secondary-foreground, which the web theme never defines (it has primary,
background, foreground, muted, border, card, primary-subtle). Those classes resolved
to nothing, so the buttons rendered as bare unstyled text on the login page.

Restyle: the optional (non-enforced) buttons become white-card outline buttons with a
key icon and an orange hover tint, secondary to the primary Login button; the
SSO-enforced buttons become solid primary with the icon.

* fix: gate S3 behind license, custom-role enterprise perms, wire retention UI, cleanup

S3 is a licensed feature, but shipping packages/enterprise in every image removed the
implicit gate, so STORAGE_MODE=s3 worked without a license. Enforce
isFeatureEnabled('s3_storage') at boot and fail fast if unlicensed.

Custom roles can now be granted security:manage / compliance:manage / webhooks:manage
(roles.ts ALL_PERMISSIONS + the Roles UI) so admins can build least-privilege
compliance/security roles instead of only the built-in admin role.

retentionSweep now reads the jobsRetentionDays / auditRetentionDays DB settings the
System Settings UI writes (env vars become the fallback default), mirroring how the
temp-file sweep reads tempFileMaxAgeHours. Previously those two UI controls were no-ops.

Cleanup: drop the never-set snapotter_storage_bytes gauge and the unused
MAX_WORKSPACE_SIZE_GB env var; emit tool_client_error to PostHog from the web
ErrorBoundary (client crashes were not reaching analytics); add the Python
OpenTelemetry packages so the innermost sidecar.<script> span exports; fix the stale
'only local storage' line in the docs; delete two e2e-analytics specs that tested the
removed consent UI.

* fix(env): restore MAX_WORKSPACE_SIZE_GB default

security-auth-hardening.test.ts asserts env.MAX_WORKSPACE_SIZE_GB defaults to 10, so
the var is an intentional (tested) default, not dead code. Removing it in the cleanup
commit broke that unit test. Keep the declaration.
2026-06-24 17:27:59 +08:00
SnapOtterandGitHub 5d36ac06d8 feat(analytics): build-time bake + telemetry depth (#336)
Bake PostHog + Sentry into the published Docker image (SNAPOTTER_ANALYTICS
build arg, codegen script). Delete entire consent system. Move event emission
to BullMQ worker. Add cross-tier identity stitching, Sentry performance
tracing on both tiers, frontend funnel events. Fix stateful regex bug.

86 files changed, 1593 insertions(+), 3747 deletions(-)
2026-06-24 11:05:39 +08:00
SnapOtter 649ad5db9e test: massive test coverage expansion (+1,437 tests, 22 new files)
Expand test coverage across all layers via 14 parallel agents:

Unit tests (3,378 total, +534):
- First-ever AI sidecar tests (157 tests covering bridge lifecycle, all 12 tool modules)
- API route infrastructure (auth, pipeline, batch, settings, teams, roles, audit, api-keys, files, docs)
- Lib coverage improvements (audit 7%->95%, worker-pool 33%->100%)
- Web store/lib gap fills (features-store, tool-registry)

Integration tests (4,403 total, +903):
- Expanded 19 tool test files with parameter variations, format edge cases, boundary values
- Cross-format matrix: 290 tests covering 14 tools x 17 formats
- Adversarial/edge cases: 63 tests for extreme inputs, concurrent requests, corrupted files

E2E-Docker (125 new tests):
- Expanded 8 spec files + 1 new file covering all 49 tools
- Added HEIC/format handling, auth failures, download verification

GUI E2E (expanded 28 spec files):
- Navigation, responsive layout, keyboard shortcuts
- All 51 tool UIs with settings, processing, display modes
- Batch/pipeline workflows, settings/RBAC, visual regression
- Resilience, accessibility (ARIA, contrast, focus), performance budgets
2026-05-09 09:02:29 +08:00
SnapOtter fc8b549d78 fix: gate captureException on user consent and fix HEIC PII scrubbing
captureException now checks isRequestOptedIn before forwarding errors
to Sentry, closing a gap where server errors leaked to an external
service even when no user had consented. The PII scrubbing regex is
also fixed: he[ic]f? failed to match .heic due to word-boundary
behavior and is replaced with hei[cf]? which correctly covers .heic,
.heif, and .hei.

Adds 88 new analytics tests across unit, integration, and e2e layers
proving PostHog/Sentry are never invoked when analytics is disabled or
users have not consented, plus full 7-day reminder lifecycle coverage.
2026-04-29 23:47:19 +08:00