Five fixes to the PostHog event stream, from an audit of what we actually collect versus what's flowing in. Each one is test-first.
## What changed
**Silenced the `$opt_in` noise.** `initAnalytics` called `opt_in_capturing()` on every page load to clear a stale opt-out flag, and posthog-js emits an `$opt_in` event on every call. That was 10k+ events a month (up to 55 per user) carrying no signal: analytics is on by default with an admin opt-out, so there is no per-user consent to record. Both call sites now pass `captureEventName: false`.
**Split the onboarding survey out of `feedback_submitted`.** The onboarding usage survey rode the same event as real feedback, so about 93% of "feedback" was actually onboarding profiling. It now emits `onboarding_survey_submitted`, so feedback metrics mean feedback again.
**Set `pipeline_executed.file_count`.** It was declared in the properties interface but never populated. A pure `pipelineExecutedProps` helper now derives it (batch size for a batch run, else 1) and is shared by the success and failure paths, which also drops a duplicated payload.
**Tracked OIDC login failures.** All six OIDC callback failure branches bumped the Prometheus counter and wrote an audit log but never emitted `auth_login_failed`. A `recordOidcFailure` helper mirrors the password path.
**Added `TELEMETRY.md`.** A contributor-facing event dictionary: every event, its properties, where it fires, and the privacy invariants, with the allowlists as source of truth. A drift test fails if any `ANALYTICS_EVENTS` value goes undocumented.
I left the published telemetry guide (`apps/docs/guide/telemetry.md`) alone. It is high-level and still accurate, and editing it would pull in the 21-locale stale-gate for no gain.
## Verification
- Unit (63 tests): `analytics-events`, `telemetry-doc-drift`, `api/analytics`, `web/analytics`, `worker.behavior`
- Integration (41 tests): `oidc-auth`, `feedback`
- Full typecheck across all 9 workspaces
- Biome clean on the changed files
All green locally.
Add a once-per-boot instance_started event (arch, os, deploy_mode,
gpu_present) so the fleet architecture mix is measurable. It reuses the
existing per-instance instance_id and is exempt from the volume sample
rate, since a census that fires once per boot must not be thinned.
Restore useful capture depth now that the sponsored plan removes the
quota pressure behind the earlier hardening:
- PostHog sample rate 0.1 to 1.0 (full analytics when enabled); the
property allowlist still blocks file data.
- Sentry per-instance ceiling 20 to 500/hr, breadcrumb trail restored
(sanitized: urls/paths redacted, data payloads dropped), full stack
paths kept; local vars, request bodies, and PII still dropped. Both
api and web.
Honor ANALYTICS_ENABLED=false as an opt-out alias: it was documented on
the Docker Hub README but never wired in 2.x, so anyone who set it was
still tracked.
All capture stays behind the analytics opt-out gate.
Removes Sentry tracing entirely (BullMQ idle polling burned 4.8M transactions in 2 days at the baked 0.1 rate), decouples PostHog sampling, and replaces the type-only error scrub with a vetted-field sanitizer plus SafeError/ToolInputError contracts. One classified capture path with per-signature throttles and a per-process ceiling makes storms impossible (NODE-1E was 4,541 events from one 30s loop). Browser errors move to a dedicated web Sentry project with their own source maps. Adds the SNAPOTTER_TELEMETRY runtime kill switch and silences test fleets.
Crash fixes: remote 204/304 SSRF process kill (NODE-20), conversion-preset boot crash loop (NODE-21), Redis version preflight + unhandled subscribe rejection (NODE-1T), Sign PDF on plain-http origins (NODE-1K/1M), wavesurfer/pdf.js teardown rejections (NODE-1P/1N), bundle-import ZlibError to 400 (NODE-1Z), chart-maker input errors declassified (NODE-1H/1J), asset requests skip the session DB lookup (NODE-1D).
* feat(analytics): upload web source maps to Sentry + tie release to build
Web crash reports were unusable: the bundle ships minified with no source
maps uploaded, and every build reported as the frozen APP_VERSION, so a
Sentry error showed an unreadable stack under a single release.
- Add @sentry/vite-plugin: emit hidden source maps and upload them by debug
id when SENTRY_AUTH_TOKEN is present (published Docker build only), then
delete the maps so they never ship. No-op for dev and the source archive.
- Set the Sentry release from SENTRY_RELEASE / VITE_SENTRY_RELEASE (the Docker
build passes the release version), falling back to APP_VERSION.
- Relax beforeSend so app bundle frames keep a host-stripped path (Sentry needs
it to match the uploaded map) while the instance hostname, error message, and
PII stay stripped. Filesystem paths still collapse to the basename.
- Wire the Dockerfile (sentry_auth_token build secret + SENTRY_RELEASE arg/env)
and the release docker job.
* fix(analytics): point source map upload at the snapotter org (project node)
Lands five integrated branches: pipeline templates (#355), analytics opt-out (#354), 83 conversion presets bringing the catalog to 240 tools (#356), self-hosted positioning (#353), and e2e modernization (#351).
Integration fixes: aligned stale web analytics tests with the opt-out/allow-list model, closed 3 CodeQL incomplete-sanitization alerts in the i18n generator, resolved settings/index/docs/format-matrix conflicts, and corrected tool counts to 240.
- Fix resize 20% failure rate: add Zod refine requiring at least one
dimension, enforce integer/max constraints, clamp percentage scaling
to minimum 1px, and guard against missing metadata in withoutEnlargement
- Fix PostHog init race condition: move consent check before async import
so frontend events (search, pageview) are no longer silently dropped
- Fix identify() passing nested $set/$set_once wrappers instead of flat
properties, so version person property now appears on PostHog profiles
- Add error_code and error_message to failed tool_used analytics events
for debugging tool failures from PostHog
The production CSP had connect-src/script-src/font-src set to 'self' only,
silently blocking all analytics and error reporting in production while
working fine in dev (where CSP is not applied).
CSP fixes:
- Add PostHog ingest + assets origins to connect-src and script-src
- Add Sentry ingest origin to connect-src
- Add Scalar fonts origin to font-src for API docs pages
- Extract CSP construction into testable buildCsp() function
Silent failure hardening:
- Settings/features stores now set loadError flag and allow retry on
subsequent fetch() calls instead of permanently caching failed state
- Analytics init no longer sets initialized=true before the try block,
allowing retry on failure
- Settings dialog Tools section disables save button when settings
failed to load, preventing accidental config wipe
- Branding logo storage moved from process.cwd() to FILES_STORAGE_PATH
so logos persist across Docker container recreation
Test coverage:
- 16 CSP directive tests covering all external service domains
- Store retry-on-error behavior tests for settings and features stores
- Analytics init retry-after-failure test
PostHog SDK was initialized on app mount based only on the server-level
config flag, ignoring user consent. This caused network requests to
us-assets.i.posthog.com (config.js, web-vitals.js, dead-clicks-autocapture.js)
even when the user had not opted in or had explicitly declined telemetry.
- Replace static imports of posthog-js and @sentry/react with dynamic
import() so the SDK bundles are not downloaded until consent is granted
- Gate initAnalytics on analyticsConsent.analyticsEnabled === true,
not just server config.enabled
- Add consent re-check after each await import() to handle revocation
during the async load
- Add shutdownAnalytics() that calls opt_out_capturing() + reset()
for mid-session consent revocation
- setAnalyticsConsent(false) now triggers full SDK shutdown automatically
- Rewrite analytics test suite with 44 tests covering init gating,
shutdown lifecycle, consent toggle, race conditions, and Sentry callbacks
Closes#98
Add ~500 new E2E tests and ~300 new integration tests covering:
- 24 new GUI E2E specs: navigation, responsive layout, keyboard shortcuts,
tool UI for all 35 non-AI tools, batch/pipeline workflows, settings/RBAC,
visual regression, accessibility, and performance budgets
- 3 new E2E-Docker specs: batch workflows, advanced pipelines, cross-format
- 1 new adversarial integration test: memory pressure, corrupted files,
unicode filenames, extreme dimensions, pipeline/batch edge cases
- 29 expanded integration test files: HEIC/HEIF input, large files, parameter
boundaries, batch processing, format edge cases across all tools
- Cross-format matrix expanded: 641 tests covering every tool x 18 formats
- AI bridge unit tests expanded: lifecycle, tool modules, error propagation
- Unit test gaps filled: analytics, tool-registry, web stores
Also fixes:
- vitest.config.ts: exclude e2e-docs and e2e-landing from Vitest runner
- AI E2E specs: add sidecar health check to skip gracefully when Python
AI backend is not running instead of timing out
