mirror of
https://github.com/snapotter-hq/SnapOtter.git
synced 2026-08-03 07:46:42 +02:00
* feat(api): parse DATA_DIR from env for 1.x import auto-detection Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w * test(migrator): build 1.17.2 fixtures by replaying legacy migrations Discovered the legacy migrations seed a Default team (0005) and builtin roles (0007), so the replayed fixture carries them. Seed uses a distinct custom team. Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w * fix(migrator): self-adjusting column copy, jobs.status map, drop sessions, advisory lock The importer now inserts only the intersection of source and live target columns, so the three analytics_* columns 2.x dropped no longer break the first users INSERT (and future dropped columns are handled generically). jobs.status is mapped onto the 2.x enum (error->failed). Sessions are no longer migrated. A pg_advisory_xact_lock serializes concurrent replicas. Includes login-after-migrate and library assertions. Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w * test(migrator): CI drift guard fails when a required column is unfillable from 1.17.2 Introspects every NOT-NULL-no-default column of each migrated table in the current schema and asserts the engine can fill it from a real 1.17.2 source. Turns a future breaking schema change into a PR-time failure instead of a production import break. Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w * feat(migrator): orchestrator with detection, boot states, marker, blob count sqlite-import.ts owns source resolution (explicit path, 'off' sentinel, DATA_DIR probe), the four boot states (import/leftover/locked/none), the persisted sqlite_import marker, and a read-only library-blob count. runBootImport wires them together and catches TargetNonEmptyError as a benign multi-replica skip. Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w * feat(api): route boot through the 1.x import orchestrator; hide marker from non-admins index.ts now calls runBootImport (which owns detection + the four boot states) instead of the inline SQLITE_MIGRATE_PATH block. The sqlite_import marker is added to SENSITIVE_KEYS (but not REDACTED_KEYS) so admins see the counts for the banner while non-admins don't see the key at all. Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w * feat(migrator): add analyzeSqlite + dry-run/verify CLI analyzeSqlite is a read-only pre-flight (no live Postgres): per-table row counts, library-blob presence, and out-of-enum job statuses. The migrate:sqlite CLI now lives in the orchestrator and supports --dry-run/--verify (prints the analysis and exits without writing) alongside the existing import and --force. Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w * docs: add 1.x to 2.0 upgrade guide; fix volume-name casing New apps/docs upgrade guide covering auto-detect, the SQLITE_MIGRATE_PATH override + off opt-out, the dry-run, what carries over, locked-state recovery, and non-destructive rollback. Leads with 'back up the WHOLE /data volume, not just snapotter.db' because 1.x WAL mode leaves data in snapotter.db-wal (surfaced by the real-image upgrade test). Standardizes README/DOCKERHUB compose volume names on the canonical SnapOtter-data casing so they match the repo compose and don't orphan an upgrader's volume. Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w * feat(web): admin 1.x migration banner + 21-locale strings A one-time admin banner reads the sqlite_import marker from /v1/settings and shows the import result (user + saved-file counts) on success, or a warning when a 1.x database was found but not imported. Dismissal persists to a sqlite_import.dismissedAt settings key. shouldShowMigrationBanner/parseMigrationMarker sit in feedback.ts with the other shouldShow helpers; strings added to en.ts and all 20 other locales. Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w * style(landing): biome-format Hero.astro trustBadges array Pre-existing formatting drift on main (its Lint check was skipped on the merge that introduced it); this PR's full Lint run surfaced it. Formatting-only, applied via the repo's own biome formatter to unblock the required Lint check. Claude-Session: https://claude.ai/code/session_01721WHAUGxnVk22qEeTub7w
109 lines
5.6 KiB
Markdown
109 lines
5.6 KiB
Markdown
# Upgrading from 1.x to 2.0
|
|
|
|
SnapOtter 1.x stored everything in a single SQLite file and ran as one container. SnapOtter 2.0 uses PostgreSQL and Redis. This guide walks through moving a 1.x install to 2.0 without losing data.
|
|
|
|
The short version: reuse your existing `/data` volume, and 2.0 imports your 1.x database automatically on first boot. Your users, saved files, settings, API keys, and pipelines come across. The old database is never modified, so you can always roll back.
|
|
|
|
## Before you start: back up the whole `/data` volume
|
|
|
|
Do this first, every time. Back up the **entire** `/data` volume, not just the `snapotter.db` file.
|
|
|
|
Here is why it matters. 1.x runs SQLite in WAL mode, so a stopped 1.x container routinely leaves most of its committed data in `snapotter.db-wal` beside an almost-empty `snapotter.db`. Copying only `snapotter.db` captures an empty database and silently loses everything. The volume carries `snapotter.db`, `snapotter.db-wal`, `snapotter.db-shm`, and your `files/` directory together, and they must travel as a set.
|
|
|
|
```bash
|
|
# Adjust the volume name to match yours (see "Check your volume name" below).
|
|
docker run --rm -v SnapOtter-data:/data -v "$PWD":/backup \
|
|
alpine tar czf /backup/snapotter-1x-data.tgz -C /data .
|
|
```
|
|
|
|
## Upgrade to 1.17.2 first
|
|
|
|
Upgrade your 1.x install to the latest 1.x release (1.17.2) before moving to 2.0. That lets 1.x run its own final schema migrations, so 2.0 imports from a known, complete schema. Upgrading from an older 1.x straight to 2.0 is not supported.
|
|
|
|
## Check your volume name
|
|
|
|
The importer only sees your data if the 2.0 stack mounts the same volume your 1.x install used. Docker volume names are case sensitive, and older README snippets used a lowercase `snapotter-data` while the Compose files use `SnapOtter-data`. Confirm which one you have:
|
|
|
|
```bash
|
|
docker volume ls | grep -i snapotter
|
|
```
|
|
|
|
Use that exact name in your 2.0 configuration.
|
|
|
|
## Path A: single container (quickest)
|
|
|
|
If you run SnapOtter with a single `docker run`, keep doing that. 2.0 boots an embedded PostgreSQL and Redis inside the container when you do not set `DATABASE_URL` or `REDIS_URL`, and it auto-detects and imports `/data/snapotter.db` on first boot.
|
|
|
|
```bash
|
|
docker run -d --name snapotter -p 1349:1349 \
|
|
-v SnapOtter-data:/data \
|
|
snapotter/snapotter:latest
|
|
```
|
|
|
|
Watch the logs for a line like:
|
|
|
|
```
|
|
Imported 1.x SQLite database: {"tables":{"users":2,"teams":1,...},"blobs":{"present":1,"missing":0}}
|
|
```
|
|
|
|
That is it. Log in with your existing credentials.
|
|
|
|
## Path B: Compose (recommended for production)
|
|
|
|
The 2.0 Compose stack runs three services (app, Postgres, Redis). Reuse your 1.x `/data` volume for the app service. The app auto-detects `/data/snapotter.db` and imports it into Postgres on first boot.
|
|
|
|
```yaml
|
|
services:
|
|
SnapOtter:
|
|
image: snapotter/snapotter:latest
|
|
volumes:
|
|
- SnapOtter-data:/data # your existing 1.x volume
|
|
- SnapOtter-workspace:/tmp/workspace
|
|
environment:
|
|
- DATABASE_URL=postgres://snapotter:snapotter@postgres:5432/snapotter
|
|
- REDIS_URL=redis://:snapotter@redis:6379
|
|
# ...
|
|
```
|
|
|
|
If you would rather point at the old database explicitly, set `SQLITE_MIGRATE_PATH=/data/snapotter.db`. An explicit path always wins over auto-detect.
|
|
|
|
## Preview the import first (optional)
|
|
|
|
To see exactly what would be imported without writing anything, run a dry run against your database file:
|
|
|
|
```bash
|
|
pnpm --filter @snapotter/api migrate:sqlite -- /path/to/snapotter.db --dry-run
|
|
```
|
|
|
|
It prints the row counts per table, how many saved-library files it found on disk, and any job statuses it will normalize. It needs no running Postgres.
|
|
|
|
## What carries over, and what does not
|
|
|
|
Carried over:
|
|
|
|
- Users, and the ability to log in. Password hashes are unchanged, so the same username and password work.
|
|
- Teams, settings (including your instance identity), roles, API keys (they keep working), and saved pipelines.
|
|
- Job history records.
|
|
- Your saved-file library, both the records and the actual files, because `/data/files` is preserved on the volume.
|
|
|
|
Not carried over:
|
|
|
|
- Login sessions. Everyone signs in once after the upgrade. Credentials are unchanged, so it is a single re-login, nothing more.
|
|
- The input and output files of old processing jobs. Those lived in a temporary workspace and are gone by design. The job history records remain.
|
|
- Per-user analytics-consent flags from 1.x, which have no 2.0 equivalent (2.0 analytics is an instance-level setting).
|
|
|
|
## Turning the import off
|
|
|
|
If you deliberately want a fresh database even though a `snapotter.db` is present on the volume, set `SQLITE_MIGRATE_PATH=off`.
|
|
|
|
## If you already have data in the 2.0 instance
|
|
|
|
The importer only runs into an empty database. If you started 2.0 fresh (creating data), then later mounted an old `snapotter.db`, 2.0 will detect it but will not import, because merging two datasets can collide on IDs. You will see a warning in the logs. To import the 1.x data you need an empty instance:
|
|
|
|
- If the 2.0 instance only holds the default admin (you have not really used it), stop the stack, remove the Postgres volume (`SnapOtter-pgdata`), and boot again with the old `/data` present. It will import cleanly. This wipes only the throwaway Postgres data, not your 1.x database.
|
|
- If the 2.0 instance holds real data you want to keep, the two datasets cannot be auto-merged. Export what you need and import the 1.x data into a separate fresh deployment.
|
|
|
|
## Rolling back
|
|
|
|
The upgrade never modifies or deletes your 1.x `snapotter.db`. If you need to go back to 1.x, redeploy the 1.x image against the same volume. Anything you created in 2.0 after the upgrade lives in Postgres and would not be in the 1.x database, so roll back promptly if you are going to.
|