From 2d29ff8aaccfe7b5b9328face0670bb6d64e3195 Mon Sep 17 00:00:00 2001 From: Snow Lee Date: Tue, 7 Jul 2026 14:05:36 -0700 Subject: [PATCH] feat: web viewer, project-local config, selective sync, Claude Code plugin - sfs-web (cmd/sfs-web, internal/webapp): read-only Obsidian-style web UI serving a local folder (default) or an sfs remote; markdown rendering with [[wikilinks]], task lists and tables, file downloads with ETags, per-file provenance from the journals; added to goreleaser builds - .sfs project file (internal/config): per-folder volume/remote/include settings that travel with the folder, win over the global registry, and never sync; daemon picks up edits live - .sfsignore + include lists (internal/syncer): gitignore-style selective sync with ! re-includes, applied symmetrically in scan and materialize; newly ignored files stop syncing without being deleted anywhere - Claude Code plugin (plugin/, .claude-plugin/): sfs skill, /sfs:mount and /sfs:status commands, turn-boundary sync hooks (blocking pull on prompt, async push on stop); installable via the repo's marketplace manifest - CLAUDE.md and .claude project settings for Claude Code development Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01HHEUaYfFHhmDvqLYw74Ehz --- .claude-plugin/marketplace.json | 14 ++ .claude/settings.json | 14 ++ .claude/skills/sfs | 1 + .goreleaser.yaml | 13 + CLAUDE.md | 67 +++++ README.md | 89 ++++++- cmd/sfs-web/main.go | 117 +++++++++ cmd/sfs/cmds.go | 21 +- cmd/sfs/helpers.go | 13 +- cmd/sfs/mount.go | 47 +++- go.mod | 1 + go.sum | 2 + internal/config/project.go | 71 ++++++ internal/daemon/daemon.go | 21 +- internal/journal/journal.go | 4 +- internal/syncer/filter_sync_test.go | 78 ++++++ internal/syncer/ignore.go | 154 ++++++++++++ internal/syncer/ignore_test.go | 94 +++++++ internal/syncer/syncer.go | 45 +++- internal/webapp/dir.go | 66 +++++ internal/webapp/dir_test.go | 67 +++++ internal/webapp/markdown.go | 42 ++++ internal/webapp/server.go | 315 ++++++++++++++++++++++++ internal/webapp/server_test.go | 213 ++++++++++++++++ internal/webapp/static/app.js | 188 ++++++++++++++ internal/webapp/static/index.html | 30 +++ internal/webapp/static/style.css | 149 +++++++++++ plugin/.claude-plugin/plugin.json | 11 + plugin/commands/mount.md | 28 +++ plugin/commands/status.md | 13 + plugin/hooks/hooks.json | 27 ++ plugin/scripts/sfs-sync.sh | 10 + {.claude => plugin}/skills/sfs/SKILL.md | 47 +++- 33 files changed, 2016 insertions(+), 56 deletions(-) create mode 100644 .claude-plugin/marketplace.json create mode 100644 .claude/settings.json create mode 120000 .claude/skills/sfs create mode 100644 CLAUDE.md create mode 100644 cmd/sfs-web/main.go create mode 100644 internal/config/project.go create mode 100644 internal/syncer/filter_sync_test.go create mode 100644 internal/syncer/ignore.go create mode 100644 internal/syncer/ignore_test.go create mode 100644 internal/webapp/dir.go create mode 100644 internal/webapp/dir_test.go create mode 100644 internal/webapp/markdown.go create mode 100644 internal/webapp/server.go create mode 100644 internal/webapp/server_test.go create mode 100644 internal/webapp/static/app.js create mode 100644 internal/webapp/static/index.html create mode 100644 internal/webapp/static/style.css create mode 100644 plugin/.claude-plugin/plugin.json create mode 100644 plugin/commands/mount.md create mode 100644 plugin/commands/status.md create mode 100644 plugin/hooks/hooks.json create mode 100755 plugin/scripts/sfs-sync.sh rename {.claude => plugin}/skills/sfs/SKILL.md (85%) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json new file mode 100644 index 0000000..667d7e9 --- /dev/null +++ b/.claude-plugin/marketplace.json @@ -0,0 +1,14 @@ +{ + "name": "sfs", + "owner": { "name": "runbear", "email": "snow@runbear.io" }, + "description": "sfs — a synced file system for AI agents", + "plugins": [ + { + "name": "sfs", + "source": "./plugin", + "description": "Mount folders that stay in sync across devices through S3/GCS/any object store. Installs the sfs skill, /sfs:mount and /sfs:status commands, and turn-boundary sync hooks in one step.", + "category": "workflow", + "tags": ["sync", "files", "workspace", "s3", "gcs", "agents"] + } + ] +} diff --git a/.claude/settings.json b/.claude/settings.json new file mode 100644 index 0000000..889e3b9 --- /dev/null +++ b/.claude/settings.json @@ -0,0 +1,14 @@ +{ + "permissions": { + "allow": [ + "Bash(go build:*)", + "Bash(go test:*)", + "Bash(go vet:*)", + "Bash(go run ./cmd/sfs:*)", + "Bash(gofmt:*)", + "Bash(go mod tidy)", + "Bash(go doc:*)", + "Bash(go env:*)" + ] + } +} diff --git a/.claude/skills/sfs b/.claude/skills/sfs new file mode 120000 index 0000000..436f55b --- /dev/null +++ b/.claude/skills/sfs @@ -0,0 +1 @@ +../../plugin/skills/sfs \ No newline at end of file diff --git a/.goreleaser.yaml b/.goreleaser.yaml index 0d39b0e..f121408 100644 --- a/.goreleaser.yaml +++ b/.goreleaser.yaml @@ -19,6 +19,19 @@ builds: - arm64 ldflags: - -s -w -X main.version={{.Version}} + - id: sfs-web + main: ./cmd/sfs-web + binary: sfs-web + env: + - CGO_ENABLED=0 + goos: + - darwin + - linux + goarch: + - amd64 + - arm64 + ldflags: + - -s -w archives: - formats: [tar.gz] diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..c2bc63e --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,67 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this is + +`sfs` is a Go CLI that mounts any folder as a synced volume: contents sync across devices through cloud object storage (S3, GCS, S3-compatible, or a plain directory), with per-file change history and offline support. No server — devices converge through append-only journals in a dumb object store. + +The repo ships two binaries from one Go module: `cmd/sfs` (the CLI + sync daemon) and `cmd/sfs-web` (a read-only web viewer for a remote). + +## Commands + +```sh +go build ./... # build everything +go test ./... # run all tests +go test ./internal/syncer -run TestConflict -v # run a single test +go vet ./... # vet +go build -o sfs ./cmd/sfs # build the binary (gitignored at repo root) +``` + +There is no Makefile, linter config, or CI config in-repo. Releases run `goreleaser release` on a tagged commit (see `.goreleaser.yaml`); the version is injected via `-ldflags "-X main.version=..."` into `cmd/sfs/main.go`. + +When testing the CLI manually, set `SFS_HOME=/some/tmp/dir` to relocate all sfs state (device identity, mount registry, volume stores) away from the real `~/.sfs`. + +## Architecture + +Data flows in two hops; the local volume store is the pivot: + +``` +working folder ←scan/materialize→ volume store (~/.sfs/volumes/) ←push/pull→ object store + (real files) blobs/ + journal/ + state + sync s3:// gs:// file:// +``` + +Package roles (`internal/`): + +- **`journal`** — the core data model. Every change is an `Op` (`put`/`delete`) in a per-device append-only JSONL log. `Less` defines the total order `(lamport, time, device, seq)`; `Replay` folds all ops into the volume state, last-writer-wins per path. Everything else is machinery around this. +- **`store`** — a volume's local on-disk state: content-addressed blob store (`blobs//`), per-device journal copies, the per-mount materialization cache (`state-.json`, size+mtime fingerprints for cheap change detection), sync state (lamport clock + push cursor), and the exclusive flock that serializes cycles. +- **`remote`** — the `Backend` interface (Put/Get/List/Exists) with `file://`, `s3://`, `gs://` implementations. Remote layout: `blobs/` + `journal/.jsonl` under the URL prefix. +- **`syncer`** — the heart: `Session.Cycle()` runs one pass: scan → commit local ops → pull peer journals → preserve conflict copies → materialize merged state → push blobs + own journal. Read the package doc comment in `syncer.go` first. `ignore.go` holds the path filter (`.sfsignore` rules + the `.sfs` include list), applied symmetrically in scan and materialize; a newly filtered path is dropped from the cache *without* a delete op so opting out locally never deletes remotely. +- **`daemon`** — per-mount background loop (detached process, pidfile `daemon-.pid` and log `daemon-.log` in the volume dir). Scans every `--scan-interval` (3s), talks to the remote every `--remote-interval` (10s) or immediately after local edits. Re-reads `mounts.json` each tick to pick up `sfs remote set` / `umnt --forget` without restart. +- **`config`** — global state under `$SFS_HOME` (default `~/.sfs`): device identity (`device.json`), mount registry (`mounts.json`), `MountID()` (sha256 of the folder path — one volume can be mounted at several folders, and everything folder-specific is keyed by it). Also the per-folder `.sfs` project file (`project.go`): volume/remote/include settings that live in the mounted folder itself, win over the registry (`EffectiveMount`), and are never synced. +- **`webapp`** — the `sfs-web` server: a `Source` interface with two implementations — `DirSource` (serves a local folder straight from disk; the default when no remote is given) and `RemoteSource` (reads journals straight from the remote, no local store, folds them into a file tree with per-file provenance). Renders markdown (goldmark + Obsidian `[[wikilinks]]`), streams/downloads content. Frontend is dependency-free vanilla JS embedded via `go:embed static`. + +`cmd/sfs/` is a thin cobra CLI over these packages (`mnt`, `umnt`, `sync`, `status`, `log`, `remote`, `whoami`, `daemon`, `version`); `cmd/sfs-web/` wraps `webapp` with flags. + +## Invariants — do not break these + +- **Each device writes only its own journal.** This is the whole concurrency story: no locking service is needed because no object ever has two writers. Never write to another device's journal file or remote key. +- **Blobs are pushed before the journal** (`syncer.push`), so a peer never sees an op whose content is missing. Preserve this ordering. +- **Scan happens before pull** in `Cycle`, so local edits are journaled (and content captured) before remote state can overwrite the working folder. +- **Replay must stay deterministic.** Any change to `journal.Less` or `Replay` changes what every device converges to. +- **Materialize never clobbers dirty files**: a file whose size/mtime differs from the state cache changed mid-cycle and is left for the next scan. +- **All state files are written atomically** (temp file + rename, see `store.WriteFileAtomic`). Temp files are prefixed `.sfs-tmp-` and ignored by the scanner. +- **`Cycle` runs under the volume flock** — the daemon and one-shot CLI commands (`sfs sync`) coexist through it. +- Errors during pull/push degrade to `Result.Offline` rather than failing the cycle; unreadable/vanished files during scan are skipped and retried next cycle. Follow this "never break sync, retry next cycle" posture. + +## Testing conventions + +The real coverage is the integration tests in `internal/syncer/syncer_test.go`: each test builds multiple simulated devices (`newDevice`) syncing through a shared `file://` remote (`sharedRemote`), then drives explicit `cycle()` calls to test convergence, offline operation, and concurrent-edit conflicts. Extend these when touching sync behavior — a new sync feature without a multi-device test is untested where it matters. + +## Claude Code plugin + +`plugin/` is a Claude Code plugin (skill + `/sfs:mount` + `/sfs:status` commands + turn-boundary sync hooks), published via the marketplace manifest at `.claude-plugin/marketplace.json` (`/plugin marketplace add runbear-io/sfs`). The canonical skill lives at `plugin/skills/sfs/SKILL.md`; `.claude/skills/sfs` is a symlink to it. The hook script `plugin/scripts/sfs-sync.sh` must stay a fast no-op for folders without a `.sfs` file — it runs on every turn in every project. + +## Docs to keep in sync + +- `README.md` and `plugin/skills/sfs/SKILL.md` both document CLI behavior, flags, output formats, and the on-disk layout. When changing CLI commands, flags, output, or layout, update both — the skill is what makes Claude Code sfs-aware for end users and must match the actual binary. diff --git a/README.md b/README.md index 9d30552..492c81a 100644 --- a/README.md +++ b/README.md @@ -45,6 +45,9 @@ $ sfs mnt ./workspace --remote s3://my-bucket/workspace - **Conflict-safe** — concurrent edits resolve deterministically (last-writer-wins), and the losing version is preserved as a `name.sfs-conflict--