mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
fix(telemetry): data-quality pass (opt-in noise, onboarding split, file_count, OIDC) (#614)
Five fixes to the PostHog event stream, from an audit of what we actually collect versus what's flowing in. Each one is test-first. ## What changed **Silenced the `$opt_in` noise.** `initAnalytics` called `opt_in_capturing()` on every page load to clear a stale opt-out flag, and posthog-js emits an `$opt_in` event on every call. That was 10k+ events a month (up to 55 per user) carrying no signal: analytics is on by default with an admin opt-out, so there is no per-user consent to record. Both call sites now pass `captureEventName: false`. **Split the onboarding survey out of `feedback_submitted`.** The onboarding usage survey rode the same event as real feedback, so about 93% of "feedback" was actually onboarding profiling. It now emits `onboarding_survey_submitted`, so feedback metrics mean feedback again. **Set `pipeline_executed.file_count`.** It was declared in the properties interface but never populated. A pure `pipelineExecutedProps` helper now derives it (batch size for a batch run, else 1) and is shared by the success and failure paths, which also drops a duplicated payload. **Tracked OIDC login failures.** All six OIDC callback failure branches bumped the Prometheus counter and wrote an audit log but never emitted `auth_login_failed`. A `recordOidcFailure` helper mirrors the password path. **Added `TELEMETRY.md`.** A contributor-facing event dictionary: every event, its properties, where it fires, and the privacy invariants, with the allowlists as source of truth. A drift test fails if any `ANALYTICS_EVENTS` value goes undocumented. I left the published telemetry guide (`apps/docs/guide/telemetry.md`) alone. It is high-level and still accurate, and editing it would pull in the 21-locale stale-gate for no gain. ## Verification - Unit (63 tests): `analytics-events`, `telemetry-doc-drift`, `api/analytics`, `web/analytics`, `worker.behavior` - Integration (41 tests): `oidc-auth`, `feedback` - Full typecheck across all 9 workspaces - Biome clean on the changed files All green locally.
This commit is contained in:
@@ -4,14 +4,16 @@ import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vites
|
||||
const mockInit = vi.fn(() => ({
|
||||
capture: mockCapture,
|
||||
startSessionRecording: vi.fn(),
|
||||
opt_in_capturing: vi.fn(),
|
||||
opt_in_capturing: mockOptInCapturing,
|
||||
opt_out_capturing: vi.fn(),
|
||||
has_opted_out_capturing: vi.fn(() => false),
|
||||
reset: vi.fn(),
|
||||
register: vi.fn(),
|
||||
get_distinct_id: vi.fn(() => "test-distinct-id"),
|
||||
persistence: { disabled: false },
|
||||
}));
|
||||
const mockCapture = vi.fn();
|
||||
const mockOptInCapturing = vi.fn();
|
||||
|
||||
vi.mock("posthog-js", () => ({
|
||||
__esModule: true,
|
||||
@@ -68,6 +70,7 @@ let mod: AnalyticsModule;
|
||||
beforeEach(async () => {
|
||||
mockInit.mockClear();
|
||||
mockCapture.mockClear();
|
||||
mockOptInCapturing.mockClear();
|
||||
mockSentryInit.mockClear();
|
||||
vi.resetModules();
|
||||
mod = await import("../../../apps/web/src/lib/analytics");
|
||||
@@ -179,6 +182,23 @@ describe("analytics lib (baked model)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("opt-in capturing", () => {
|
||||
// opt_in_capturing() clears a stale persisted opt-out flag, but posthog-js
|
||||
// emits a noisy $opt_in event on every call by default. We fire it once per
|
||||
// page load, so it must suppress that event (captureEventName: false).
|
||||
it("suppresses the $opt_in event when clearing a stale opt-out on init", async () => {
|
||||
await mod.initAnalytics(enabledConfig);
|
||||
expect(mockOptInCapturing).toHaveBeenCalledWith({ captureEventName: false });
|
||||
});
|
||||
|
||||
it("suppresses the $opt_in event when resuming capture via optIn()", async () => {
|
||||
await mod.initAnalytics(enabledConfig);
|
||||
mockOptInCapturing.mockClear();
|
||||
mod.optIn();
|
||||
expect(mockOptInCapturing).toHaveBeenCalledWith({ captureEventName: false });
|
||||
});
|
||||
});
|
||||
|
||||
describe("Sentry beforeSend callback", () => {
|
||||
async function getBeforeSend() {
|
||||
mockSentryInit.mockClear();
|
||||
|
||||
Reference in New Issue
Block a user