From a7bb790615724af5a1c80d84db15db9bf675b7ae Mon Sep 17 00:00:00 2001 From: Snow Lee Date: Wed, 8 Jul 2026 07:13:00 -0700 Subject: [PATCH] feat: multi-project sync hub, bdrive login/init onboarding, .bdrive rename MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The web server (bdrive web) becomes a full sync hub, and client devices get one-command onboarding — without ever seeing storage info or holding cloud credentials: - bdrive web -c config.json: server configurable from a JSON file (remote/addr/upload/upload_ttl/projects_db); explicit flags win. - Hub mode: pointing bdrive web at a storage root hosts many projects, each under // (remote.Prefixed). Projects live in a file-backed registry (projects.json — loaded at open, rewritten atomically per change) with create-or-join-by-name semantics. - Per-project APIs: /api/projects (list/create/get) and /api/p//{tree,file,render,download,upload/*,store/*}. The web UI grows a project list with per-project browsing and hash deep links. - Browser uploads and a store proxy for syncing devices: presigned direct-to-storage PUTs when the backend can sign (S3 presign, GCS V4 signed URLs; expiring, credential-free), relayed through the server otherwise. Journals are never presigned — only immutable blobs. Blobs-before-journal and one-writer-per-journal invariants hold. - https:// remote backend: a device syncs one hub project through /api/p//store/* — mnt/sync/daemon/log all work unchanged. - bdrive login : verify a hub and remember it as the device default (settings.json). bdrive init: create-or-join a project named after the folder (--name/--project override), write .bdrive, seed a starter .bdriveignore, mount, and start the daemon — one command per project. - Hard-break rename: .beardrive->.bdrive, .beardriveignore->.bdriveignore, ~/.beardrive->~/.bdrive, BEARDRIVE_HOME->BDRIVE_HOME, temp/conflict prefixes; old names are no longer read. - Tests: presigning, project registry persistence, store API validation and gating, project isolation over live HTTP, browser upload flows, and two-device convergence through a hub (incl. read-only pull-only mode). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01R7Q9ZKSZRTdvrSJkYLUmYs --- CLAUDE.md | 22 +- README.md | 137 +++++++++--- cmd/bdrive/cmds.go | 8 +- cmd/bdrive/helpers.go | 6 +- cmd/bdrive/init.go | 255 +++++++++++++++++++++ cmd/bdrive/main.go | 2 + cmd/bdrive/mount.go | 192 ++++++++-------- cmd/bdrive/web.go | 153 ++++++++++--- internal/config/config.go | 10 +- internal/config/project.go | 14 +- internal/config/settings.go | 54 +++++ internal/daemon/daemon.go | 4 +- internal/journal/journal.go | 27 ++- internal/remote/gcs.go | 19 ++ internal/remote/http.go | 216 ++++++++++++++++++ internal/remote/local.go | 4 +- internal/remote/prefixed.go | 69 ++++++ internal/remote/remote.go | 24 +- internal/remote/s3.go | 22 ++ internal/remote/sign_test.go | 77 +++++++ internal/store/store.go | 2 +- internal/syncer/filter_sync_test.go | 14 +- internal/syncer/http_remote_test.go | 110 ++++++++++ internal/syncer/ignore.go | 8 +- internal/syncer/syncer.go | 12 +- internal/syncer/syncer_test.go | 2 +- internal/webapp/dir.go | 6 +- internal/webapp/dir_test.go | 4 +- internal/webapp/projects.go | 154 +++++++++++++ internal/webapp/server.go | 328 ++++++++++++++++++++++----- internal/webapp/server_test.go | 10 +- internal/webapp/static/app.js | 215 ++++++++++++++++-- internal/webapp/static/index.html | 4 +- internal/webapp/static/style.css | 27 ++- internal/webapp/store.go | 174 +++++++++++++++ internal/webapp/store_test.go | 318 +++++++++++++++++++++++++++ internal/webapp/upload.go | 323 +++++++++++++++++++++++++++ internal/webapp/upload_test.go | 329 ++++++++++++++++++++++++++++ plugin/commands/mount.md | 8 +- plugin/scripts/beardrive-sync.sh | 4 +- plugin/skills/beardrive/SKILL.md | 43 ++-- 41 files changed, 3097 insertions(+), 313 deletions(-) create mode 100644 cmd/bdrive/init.go create mode 100644 internal/config/settings.go create mode 100644 internal/remote/http.go create mode 100644 internal/remote/prefixed.go create mode 100644 internal/remote/sign_test.go create mode 100644 internal/syncer/http_remote_test.go create mode 100644 internal/webapp/projects.go create mode 100644 internal/webapp/store.go create mode 100644 internal/webapp/store_test.go create mode 100644 internal/webapp/upload.go create mode 100644 internal/webapp/upload_test.go diff --git a/CLAUDE.md b/CLAUDE.md index 6a7885a..53472e0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,9 +4,9 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## What this is -**BearDrive** is the product name; **`bdrive`** is its CLI binary (file conventions use the full name: `.beardrive`, `.beardriveignore`, `~/.beardrive`, `BEARDRIVE_HOME`). BearDrive is a Go CLI that mounts any folder as a synced volume: contents sync across devices through cloud object storage (S3, GCS, S3-compatible, or a plain directory), with per-file change history and offline support. No server — devices converge through append-only journals in a dumb object store. +**BearDrive** is the product name; **`bdrive`** is its CLI binary (file conventions use the full name: `.bdrive`, `.bdriveignore`, `~/.bdrive`, `BDRIVE_HOME`). BearDrive is a Go CLI that mounts any folder as a synced volume: contents sync across devices through cloud object storage (S3, GCS, S3-compatible, or a plain directory), with per-file change history and offline support. No server required — devices converge through append-only journals in a dumb object store; an optional `bdrive web` server can front the store as a sync hub for storage-blind client devices. -The repo ships one binary: `cmd/bdrive` — the CLI, the sync daemon, and the read-only web viewer (`bdrive web`). +The repo ships one binary: `cmd/bdrive` — the CLI, the sync daemon, and the web server (`bdrive web`: viewer, uploads, multi-project sync hub). ## Commands @@ -20,14 +20,14 @@ go build -o bdrive ./cmd/bdrive # build the binary (gitignored There is no Makefile, linter config, or CI config in-repo. Releases run `goreleaser release` on a tagged commit (see `.goreleaser.yaml`); the version is injected via `-ldflags "-X main.version=..."` into `cmd/bdrive/main.go`. -When testing the CLI manually, set `BEARDRIVE_HOME=/some/tmp/dir` to relocate all beardrive state (device identity, mount registry, volume stores) away from the real `~/.beardrive`. +When testing the CLI manually, set `BDRIVE_HOME=/some/tmp/dir` to relocate all beardrive state (device identity, mount registry, volume stores) away from the real `~/.bdrive`. ## Architecture Data flows in two hops; the local volume store is the pivot: ``` -working folder ←scan/materialize→ volume store (~/.beardrive/volumes/) ←push/pull→ object store +working folder ←scan/materialize→ volume store (~/.bdrive/volumes/) ←push/pull→ object store (real files) blobs/ + journal/ + state + sync s3:// gs:// file:// ``` @@ -35,13 +35,13 @@ Package roles (`internal/`): - **`journal`** — the core data model. Every change is an `Op` (`put`/`delete`) in a per-device append-only JSONL log. `Less` defines the total order `(lamport, time, device, seq)`; `Replay` folds all ops into the volume state, last-writer-wins per path. Everything else is machinery around this. - **`store`** — a volume's local on-disk state: content-addressed blob store (`blobs//`), per-device journal copies, the per-mount materialization cache (`state-.json`, size+mtime fingerprints for cheap change detection), sync state (lamport clock + push cursor), and the exclusive flock that serializes cycles. -- **`remote`** — the `Backend` interface (Put/Get/List/Exists) with `file://`, `s3://`, `gs://` implementations. Remote layout: `blobs/` + `journal/.jsonl` under the URL prefix. -- **`syncer`** — the heart: `Session.Cycle()` runs one pass: scan → commit local ops → pull peer journals → preserve conflict copies → materialize merged state → push blobs + own journal. Read the package doc comment in `syncer.go` first. `ignore.go` holds the path filter (`.beardriveignore` rules + the `.beardrive` include list), applied symmetrically in scan and materialize; a newly filtered path is dropped from the cache *without* a delete op so opting out locally never deletes remotely. +- **`remote`** — the `Backend` interface (Put/Get/List/Exists) with `file://`, `s3://`, `gs://`, and `https://` implementations (`https://` syncs through a `bdrive web` server's `/api/store` API — the client device holds no storage credentials). `PutSigner` is the optional presign capability (S3/GCS). Remote layout: `blobs/` + `journal/.jsonl` under the URL prefix. +- **`syncer`** — the heart: `Session.Cycle()` runs one pass: scan → commit local ops → pull peer journals → preserve conflict copies → materialize merged state → push blobs + own journal. Read the package doc comment in `syncer.go` first. `ignore.go` holds the path filter (`.bdriveignore` rules + the `.bdrive` include list), applied symmetrically in scan and materialize; a newly filtered path is dropped from the cache *without* a delete op so opting out locally never deletes remotely. - **`daemon`** — per-mount background loop (detached process, pidfile `daemon-.pid` and log `daemon-.log` in the volume dir). Scans every `--scan-interval` (3s), talks to the remote every `--remote-interval` (10s) or immediately after local edits. Re-reads `mounts.json` each tick to pick up `bdrive remote set` / `umnt --forget` without restart. -- **`config`** — global state under `$BEARDRIVE_HOME` (default `~/.beardrive`): device identity (`device.json`), mount registry (`mounts.json`), `MountID()` (sha256 of the folder path — one volume can be mounted at several folders, and everything folder-specific is keyed by it). Also the per-folder `.beardrive` project file (`project.go`): volume/remote/include settings that live in the mounted folder itself, win over the registry (`EffectiveMount`), and are never synced. -- **`webapp`** — the `bdrive web` server: a `Source` interface with two implementations — `DirSource` (serves a local folder straight from disk; the default when no remote is given) and `RemoteSource` (reads journals straight from the remote, no local store, folds them into a file tree with per-file provenance). Renders markdown (goldmark + Obsidian `[[wikilinks]]`), streams/downloads content. Frontend is dependency-free vanilla JS embedded via `go:embed static`. +- **`config`** — global state under `$BDRIVE_HOME` (default `~/.bdrive`): device identity (`device.json`), mount registry (`mounts.json`), `MountID()` (sha256 of the folder path — one volume can be mounted at several folders, and everything folder-specific is keyed by it). Also the per-folder `.bdrive` project file (`project.go`): volume/remote/include settings that live in the mounted folder itself, win over the registry (`EffectiveMount`), and are never synced. +- **`webapp`** — the `bdrive web` server, in two modes. Single-volume: `Source` is a `DirSource` (plain folder from disk) or `RemoteSource` (folds journals into a file tree with per-file provenance). Hub: `Root` + `Projects` host many projects on one storage root, each under `//` via `remote.Prefixed`; `ProjectDB` (`projects.go`) is a file-backed registry (JSON, loaded at open, rewritten atomically per change) with create-or-join-by-name semantics. Renders markdown (goldmark + Obsidian `[[wikilinks]]`). With `--upload` it accepts writes: browser uploads (`upload.go` — direct-to-storage via presigned URLs when the backend implements `remote.PutSigner`, relayed otherwise; ops journaled under the server's own device) and the per-project `/api/p//store/*` proxy (`store.go`) that whole devices sync through — the `https://` remote backend (`remote/http.go`) is its client; journals are never presigned, only immutable blobs. Frontend is dependency-free vanilla JS embedded via `go:embed static`; it learns everything from `/api/config` (+ `/api/projects` in hub mode) and never sees storage info or credentials. -`cmd/bdrive/` is a thin cobra CLI over these packages (`mnt`, `umnt`, `sync`, `status`, `log`, `remote`, `web`, `whoami`, `daemon`, `version`). +`cmd/bdrive/` is a thin cobra CLI over these packages (`login`, `init`, `mnt`, `umnt`, `sync`, `status`, `log`, `remote`, `web`, `whoami`, `daemon`, `version`). `bdrive login ` verifies a hub and saves it as the device default (`settings.json`); `bdrive init` is the one-command project onboarding: creates-or-joins a project on the logged-in hub, writes `.bdrive` (remote `https://host/p/`), seeds a starter `.bdriveignore`, and mounts via the shared `runMount`. `bdrive web -c config.json` configures the server from a file, explicit flags winning. ## Invariants — do not break these @@ -50,7 +50,7 @@ Package roles (`internal/`): - **Scan happens before pull** in `Cycle`, so local edits are journaled (and content captured) before remote state can overwrite the working folder. - **Replay must stay deterministic.** Any change to `journal.Less` or `Replay` changes what every device converges to. - **Materialize never clobbers dirty files**: a file whose size/mtime differs from the state cache changed mid-cycle and is left for the next scan. -- **All state files are written atomically** (temp file + rename, see `store.WriteFileAtomic`). Temp files are prefixed `.beardrive-tmp-` and ignored by the scanner. +- **All state files are written atomically** (temp file + rename, see `store.WriteFileAtomic`). Temp files are prefixed `.bdrive-tmp-` and ignored by the scanner. - **`Cycle` runs under the volume flock** — the daemon and one-shot CLI commands (`bdrive sync`) coexist through it. - Errors during pull/push degrade to `Result.Offline` rather than failing the cycle; unreadable/vanished files during scan are skipped and retried next cycle. Follow this "never break sync, retry next cycle" posture. @@ -60,7 +60,7 @@ The real coverage is the integration tests in `internal/syncer/syncer_test.go`: ## Claude Code plugin -`plugin/` is a Claude Code plugin (skill + `/beardrive:mount` + `/beardrive:status` commands + turn-boundary sync hooks), published via the marketplace manifest at `.claude-plugin/marketplace.json` (`/plugin marketplace add runbear-io/beardrive`). The canonical skill lives at `plugin/skills/beardrive/SKILL.md`; `.claude/skills/beardrive` is a symlink to it. The hook script `plugin/scripts/beardrive-sync.sh` must stay a fast no-op for folders without a `.beardrive` file — it runs on every turn in every project. +`plugin/` is a Claude Code plugin (skill + `/beardrive:mount` + `/beardrive:status` commands + turn-boundary sync hooks), published via the marketplace manifest at `.claude-plugin/marketplace.json` (`/plugin marketplace add runbear-io/beardrive`). The canonical skill lives at `plugin/skills/beardrive/SKILL.md`; `.claude/skills/beardrive` is a symlink to it. The hook script `plugin/scripts/beardrive-sync.sh` must stay a fast no-op for folders without a `.bdrive` file — it runs on every turn in every project. ## Docs to keep in sync diff --git a/README.md b/README.md index 7ad8552..5fd6cfd 100644 --- a/README.md +++ b/README.md @@ -45,9 +45,9 @@ $ bdrive mnt ./workspace --remote s3://my-bucket/workspace reachable again. - **Conflict-safe** — concurrent edits resolve deterministically (last-writer-wins), and the losing version is preserved as a - `name.beardrive-conflict--