Files
SnapOtter/tests/integration/platform/sqlite-import-orchestrator.test.ts
T
SnapOtterandGitHub dadf766899 fix(migrator): correct and harden the 1.x to 2.0 SQLite import (#434)
* feat(api): parse DATA_DIR from env for 1.x import auto-detection

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* test(migrator): build 1.17.2 fixtures by replaying legacy migrations

Discovered the legacy migrations seed a Default team (0005) and builtin roles
(0007), so the replayed fixture carries them. Seed uses a distinct custom team.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* fix(migrator): self-adjusting column copy, jobs.status map, drop sessions, advisory lock

The importer now inserts only the intersection of source and live target columns,
so the three analytics_* columns 2.x dropped no longer break the first users INSERT
(and future dropped columns are handled generically). jobs.status is mapped onto the
2.x enum (error->failed). Sessions are no longer migrated. A pg_advisory_xact_lock
serializes concurrent replicas. Includes login-after-migrate and library assertions.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* test(migrator): CI drift guard fails when a required column is unfillable from 1.17.2

Introspects every NOT-NULL-no-default column of each migrated table in the current
schema and asserts the engine can fill it from a real 1.17.2 source. Turns a future
breaking schema change into a PR-time failure instead of a production import break.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* feat(migrator): orchestrator with detection, boot states, marker, blob count

sqlite-import.ts owns source resolution (explicit path, 'off' sentinel, DATA_DIR
probe), the four boot states (import/leftover/locked/none), the persisted
sqlite_import marker, and a read-only library-blob count. runBootImport wires them
together and catches TargetNonEmptyError as a benign multi-replica skip.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* feat(api): route boot through the 1.x import orchestrator; hide marker from non-admins

index.ts now calls runBootImport (which owns detection + the four boot states)
instead of the inline SQLITE_MIGRATE_PATH block. The sqlite_import marker is added
to SENSITIVE_KEYS (but not REDACTED_KEYS) so admins see the counts for the banner
while non-admins don't see the key at all.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* feat(migrator): add analyzeSqlite + dry-run/verify CLI

analyzeSqlite is a read-only pre-flight (no live Postgres): per-table row counts,
library-blob presence, and out-of-enum job statuses. The migrate:sqlite CLI now
lives in the orchestrator and supports --dry-run/--verify (prints the analysis and
exits without writing) alongside the existing import and --force.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* docs: add 1.x to 2.0 upgrade guide; fix volume-name casing

New apps/docs upgrade guide covering auto-detect, the SQLITE_MIGRATE_PATH override +
off opt-out, the dry-run, what carries over, locked-state recovery, and non-destructive
rollback. Leads with 'back up the WHOLE /data volume, not just snapotter.db' because
1.x WAL mode leaves data in snapotter.db-wal (surfaced by the real-image upgrade test).
Standardizes README/DOCKERHUB compose volume names on the canonical SnapOtter-data
casing so they match the repo compose and don't orphan an upgrader's volume.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* feat(web): admin 1.x migration banner + 21-locale strings

A one-time admin banner reads the sqlite_import marker from /v1/settings and shows
the import result (user + saved-file counts) on success, or a warning when a 1.x
database was found but not imported. Dismissal persists to a sqlite_import.dismissedAt
settings key. shouldShowMigrationBanner/parseMigrationMarker sit in feedback.ts with
the other shouldShow helpers; strings added to en.ts and all 20 other locales.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w

* style(landing): biome-format Hero.astro trustBadges array

Pre-existing formatting drift on main (its Lint check was skipped on the merge that
introduced it); this PR's full Lint run surfaced it. Formatting-only, applied via
the repo's own biome formatter to unblock the required Lint check.

Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w
2026-07-04 15:15:39 +00:00

82 lines
3.3 KiB
TypeScript

import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import {
analyzeSqlite,
countLibraryBlobs,
evaluateBootState,
resolveSource,
} from "../../../apps/api/src/db/sqlite-import.js";
import { buildLegacySqlite, seedRealistic1xData } from "../../helpers/legacy-sqlite-fixture.js";
describe("resolveSource", () => {
it("returns null for the 'off' sentinel", () => {
expect(resolveSource({ SQLITE_MIGRATE_PATH: "off", DATA_DIR: "/data" })).toBeNull();
expect(resolveSource({ SQLITE_MIGRATE_PATH: "OFF", DATA_DIR: "/data" })).toBeNull();
});
it("prefers an explicit path", () => {
expect(resolveSource({ SQLITE_MIGRATE_PATH: "/x/foo.db", DATA_DIR: "/data" })).toBe(
"/x/foo.db",
);
});
it("probes DATA_DIR/snapotter.db when unset", () => {
const dir = mkdtempSync(join(tmpdir(), "src-"));
writeFileSync(join(dir, "snapotter.db"), "x");
expect(resolveSource({ SQLITE_MIGRATE_PATH: "", DATA_DIR: dir })).toBe(
join(dir, "snapotter.db"),
);
});
it("returns null when nothing is present", () => {
const dir = mkdtempSync(join(tmpdir(), "src-"));
expect(resolveSource({ SQLITE_MIGRATE_PATH: "", DATA_DIR: dir })).toBeNull();
});
});
describe("evaluateBootState", () => {
it("import when users empty, source present, no marker", () => {
expect(evaluateBootState({ usersCount: 0, source: "/a.db", marker: null })).toBe("import");
});
it("leftover when a completed marker exists", () => {
expect(
evaluateBootState({ usersCount: 5, source: "/a.db", marker: { status: "completed" } }),
).toBe("leftover");
});
it("locked when source present, users non-empty, no marker", () => {
expect(evaluateBootState({ usersCount: 1, source: "/a.db", marker: null })).toBe("locked");
});
it("none when no source", () => {
expect(evaluateBootState({ usersCount: 0, source: null, marker: null })).toBe("none");
});
});
describe("countLibraryBlobs", () => {
it("counts present and missing library blobs", async () => {
const dir = mkdtempSync(join(tmpdir(), "blob-"));
const dbPath = join(dir, "legacy.db");
buildLegacySqlite(dbPath);
await seedRealistic1xData(dbPath); // seeds user_files uf-1 -> stored_name abc123.png
const filesDir = join(dir, "files");
mkdirSync(filesDir, { recursive: true });
writeFileSync(join(filesDir, "abc123.png"), "img");
expect(await countLibraryBlobs(dbPath, filesDir)).toEqual({ present: 1, missing: 0 });
});
});
describe("analyzeSqlite", () => {
it("reports counts, blobs, and out-of-enum statuses without a live Postgres", async () => {
const dir = mkdtempSync(join(tmpdir(), "analyze-"));
const dbPath = join(dir, "legacy.db");
buildLegacySqlite(dbPath);
await seedRealistic1xData(dbPath); // 1 user, 2 jobs (one "error"), 1 user_file
const filesDir = join(dir, "files");
mkdirSync(filesDir, { recursive: true });
writeFileSync(join(filesDir, "abc123.png"), "img"); // the uf-1 blob
const report = await analyzeSqlite(dbPath, filesDir);
expect(report.tables.users).toBe(1);
expect(report.tables.jobs).toBe(2);
expect(report.blobs).toEqual({ present: 1, missing: 0 });
expect(report.badStatuses).toContain("error"); // will be mapped to "failed" on import
});
});