mirror of
https://github.com/runbear-io/beardrive.git
synced 2026-08-25 08:08:08 +02:00
feat: web viewer, project-local config, selective sync, Claude Code plugin
- sfs-web (cmd/sfs-web, internal/webapp): read-only Obsidian-style web UI serving a local folder (default) or an sfs remote; markdown rendering with [[wikilinks]], task lists and tables, file downloads with ETags, per-file provenance from the journals; added to goreleaser builds - .sfs project file (internal/config): per-folder volume/remote/include settings that travel with the folder, win over the global registry, and never sync; daemon picks up edits live - .sfsignore + include lists (internal/syncer): gitignore-style selective sync with ! re-includes, applied symmetrically in scan and materialize; newly ignored files stop syncing without being deleted anywhere - Claude Code plugin (plugin/, .claude-plugin/): sfs skill, /sfs:mount and /sfs:status commands, turn-boundary sync hooks (blocking pull on prompt, async push on stop); installable via the repo's marketplace manifest - CLAUDE.md and .claude project settings for Claude Code development Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HHEUaYfFHhmDvqLYw74Ehz
This commit is contained in:
Snow Lee
co-authored by
Claude Fable 5
Claude Fable 5
parent
d22fe71d1c
commit
2d29ff8aac
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"name": "sfs",
|
||||
"owner": { "name": "runbear", "email": "snow@runbear.io" },
|
||||
"description": "sfs — a synced file system for AI agents",
|
||||
"plugins": [
|
||||
{
|
||||
"name": "sfs",
|
||||
"source": "./plugin",
|
||||
"description": "Mount folders that stay in sync across devices through S3/GCS/any object store. Installs the sfs skill, /sfs:mount and /sfs:status commands, and turn-boundary sync hooks in one step.",
|
||||
"category": "workflow",
|
||||
"tags": ["sync", "files", "workspace", "s3", "gcs", "agents"]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"Bash(go build:*)",
|
||||
"Bash(go test:*)",
|
||||
"Bash(go vet:*)",
|
||||
"Bash(go run ./cmd/sfs:*)",
|
||||
"Bash(gofmt:*)",
|
||||
"Bash(go mod tidy)",
|
||||
"Bash(go doc:*)",
|
||||
"Bash(go env:*)"
|
||||
]
|
||||
}
|
||||
}
|
||||
Symlink
+1
@@ -0,0 +1 @@
|
||||
../../plugin/skills/sfs
|
||||
@@ -19,6 +19,19 @@ builds:
|
||||
- arm64
|
||||
ldflags:
|
||||
- -s -w -X main.version={{.Version}}
|
||||
- id: sfs-web
|
||||
main: ./cmd/sfs-web
|
||||
binary: sfs-web
|
||||
env:
|
||||
- CGO_ENABLED=0
|
||||
goos:
|
||||
- darwin
|
||||
- linux
|
||||
goarch:
|
||||
- amd64
|
||||
- arm64
|
||||
ldflags:
|
||||
- -s -w
|
||||
|
||||
archives:
|
||||
- formats: [tar.gz]
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## What this is
|
||||
|
||||
`sfs` is a Go CLI that mounts any folder as a synced volume: contents sync across devices through cloud object storage (S3, GCS, S3-compatible, or a plain directory), with per-file change history and offline support. No server — devices converge through append-only journals in a dumb object store.
|
||||
|
||||
The repo ships two binaries from one Go module: `cmd/sfs` (the CLI + sync daemon) and `cmd/sfs-web` (a read-only web viewer for a remote).
|
||||
|
||||
## Commands
|
||||
|
||||
```sh
|
||||
go build ./... # build everything
|
||||
go test ./... # run all tests
|
||||
go test ./internal/syncer -run TestConflict -v # run a single test
|
||||
go vet ./... # vet
|
||||
go build -o sfs ./cmd/sfs # build the binary (gitignored at repo root)
|
||||
```
|
||||
|
||||
There is no Makefile, linter config, or CI config in-repo. Releases run `goreleaser release` on a tagged commit (see `.goreleaser.yaml`); the version is injected via `-ldflags "-X main.version=..."` into `cmd/sfs/main.go`.
|
||||
|
||||
When testing the CLI manually, set `SFS_HOME=/some/tmp/dir` to relocate all sfs state (device identity, mount registry, volume stores) away from the real `~/.sfs`.
|
||||
|
||||
## Architecture
|
||||
|
||||
Data flows in two hops; the local volume store is the pivot:
|
||||
|
||||
```
|
||||
working folder ←scan/materialize→ volume store (~/.sfs/volumes/<vol>) ←push/pull→ object store
|
||||
(real files) blobs/ + journal/ + state + sync s3:// gs:// file://
|
||||
```
|
||||
|
||||
Package roles (`internal/`):
|
||||
|
||||
- **`journal`** — the core data model. Every change is an `Op` (`put`/`delete`) in a per-device append-only JSONL log. `Less` defines the total order `(lamport, time, device, seq)`; `Replay` folds all ops into the volume state, last-writer-wins per path. Everything else is machinery around this.
|
||||
- **`store`** — a volume's local on-disk state: content-addressed blob store (`blobs/<aa>/<sha256>`), per-device journal copies, the per-mount materialization cache (`state-<mountID>.json`, size+mtime fingerprints for cheap change detection), sync state (lamport clock + push cursor), and the exclusive flock that serializes cycles.
|
||||
- **`remote`** — the `Backend` interface (Put/Get/List/Exists) with `file://`, `s3://`, `gs://` implementations. Remote layout: `blobs/<sha256>` + `journal/<device>.jsonl` under the URL prefix.
|
||||
- **`syncer`** — the heart: `Session.Cycle()` runs one pass: scan → commit local ops → pull peer journals → preserve conflict copies → materialize merged state → push blobs + own journal. Read the package doc comment in `syncer.go` first. `ignore.go` holds the path filter (`.sfsignore` rules + the `.sfs` include list), applied symmetrically in scan and materialize; a newly filtered path is dropped from the cache *without* a delete op so opting out locally never deletes remotely.
|
||||
- **`daemon`** — per-mount background loop (detached process, pidfile `daemon-<mountID>.pid` and log `daemon-<mountID>.log` in the volume dir). Scans every `--scan-interval` (3s), talks to the remote every `--remote-interval` (10s) or immediately after local edits. Re-reads `mounts.json` each tick to pick up `sfs remote set` / `umnt --forget` without restart.
|
||||
- **`config`** — global state under `$SFS_HOME` (default `~/.sfs`): device identity (`device.json`), mount registry (`mounts.json`), `MountID()` (sha256 of the folder path — one volume can be mounted at several folders, and everything folder-specific is keyed by it). Also the per-folder `.sfs` project file (`project.go`): volume/remote/include settings that live in the mounted folder itself, win over the registry (`EffectiveMount`), and are never synced.
|
||||
- **`webapp`** — the `sfs-web` server: a `Source` interface with two implementations — `DirSource` (serves a local folder straight from disk; the default when no remote is given) and `RemoteSource` (reads journals straight from the remote, no local store, folds them into a file tree with per-file provenance). Renders markdown (goldmark + Obsidian `[[wikilinks]]`), streams/downloads content. Frontend is dependency-free vanilla JS embedded via `go:embed static`.
|
||||
|
||||
`cmd/sfs/` is a thin cobra CLI over these packages (`mnt`, `umnt`, `sync`, `status`, `log`, `remote`, `whoami`, `daemon`, `version`); `cmd/sfs-web/` wraps `webapp` with flags.
|
||||
|
||||
## Invariants — do not break these
|
||||
|
||||
- **Each device writes only its own journal.** This is the whole concurrency story: no locking service is needed because no object ever has two writers. Never write to another device's journal file or remote key.
|
||||
- **Blobs are pushed before the journal** (`syncer.push`), so a peer never sees an op whose content is missing. Preserve this ordering.
|
||||
- **Scan happens before pull** in `Cycle`, so local edits are journaled (and content captured) before remote state can overwrite the working folder.
|
||||
- **Replay must stay deterministic.** Any change to `journal.Less` or `Replay` changes what every device converges to.
|
||||
- **Materialize never clobbers dirty files**: a file whose size/mtime differs from the state cache changed mid-cycle and is left for the next scan.
|
||||
- **All state files are written atomically** (temp file + rename, see `store.WriteFileAtomic`). Temp files are prefixed `.sfs-tmp-` and ignored by the scanner.
|
||||
- **`Cycle` runs under the volume flock** — the daemon and one-shot CLI commands (`sfs sync`) coexist through it.
|
||||
- Errors during pull/push degrade to `Result.Offline` rather than failing the cycle; unreadable/vanished files during scan are skipped and retried next cycle. Follow this "never break sync, retry next cycle" posture.
|
||||
|
||||
## Testing conventions
|
||||
|
||||
The real coverage is the integration tests in `internal/syncer/syncer_test.go`: each test builds multiple simulated devices (`newDevice`) syncing through a shared `file://` remote (`sharedRemote`), then drives explicit `cycle()` calls to test convergence, offline operation, and concurrent-edit conflicts. Extend these when touching sync behavior — a new sync feature without a multi-device test is untested where it matters.
|
||||
|
||||
## Claude Code plugin
|
||||
|
||||
`plugin/` is a Claude Code plugin (skill + `/sfs:mount` + `/sfs:status` commands + turn-boundary sync hooks), published via the marketplace manifest at `.claude-plugin/marketplace.json` (`/plugin marketplace add runbear-io/sfs`). The canonical skill lives at `plugin/skills/sfs/SKILL.md`; `.claude/skills/sfs` is a symlink to it. The hook script `plugin/scripts/sfs-sync.sh` must stay a fast no-op for folders without a `.sfs` file — it runs on every turn in every project.
|
||||
|
||||
## Docs to keep in sync
|
||||
|
||||
- `README.md` and `plugin/skills/sfs/SKILL.md` both document CLI behavior, flags, output formats, and the on-disk layout. When changing CLI commands, flags, output, or layout, update both — the skill is what makes Claude Code sfs-aware for end users and must match the actual binary.
|
||||
@@ -45,6 +45,9 @@ $ sfs mnt ./workspace --remote s3://my-bucket/workspace
|
||||
- **Conflict-safe** — concurrent edits resolve deterministically
|
||||
(last-writer-wins), and the losing version is preserved as a
|
||||
`name.sfs-conflict-<device>-<time>` file. Nothing is silently dropped.
|
||||
- **Selective sync** — a gitignore-style `.sfsignore` opts files out, and an
|
||||
optional `include` list in the folder's `.sfs` settings narrows sync to
|
||||
chosen paths.
|
||||
- **macOS & Linux.**
|
||||
|
||||
## Install
|
||||
@@ -106,16 +109,81 @@ sfs uses each provider's standard credential chain — nothing sfs-specific:
|
||||
| `sfs remote [folder]` / `sfs remote set <folder> <url>` | Show / set the cloud remote |
|
||||
| `sfs whoami` | Device identity used in change tracking |
|
||||
|
||||
## Claude Code skill
|
||||
## Project files
|
||||
|
||||
This repo ships an [`sfs` agent skill](.claude/skills/sfs/SKILL.md) so
|
||||
Claude Code is sfs-aware out of the box when working in a clone — covering
|
||||
mount/unmount/sync, picking a backend and credentials, and inspecting
|
||||
status/logs/identity.
|
||||
Each mounted folder carries its own settings, so configuration travels with
|
||||
the project:
|
||||
|
||||
Claude Code auto-discovers skills in `.claude/skills/`; no configuration
|
||||
needed. To make it available globally, symlink the directory into
|
||||
`~/.claude/skills/`.
|
||||
- **`.sfs`** — the folder's settings (JSON): `volume`, `remote`, and an
|
||||
optional `include` list. Written by `sfs mnt`, safe to hand-edit (a running
|
||||
daemon picks changes up automatically). Never synced — remotes are
|
||||
device-specific. Copy a folder containing `.sfs` to another machine and
|
||||
plain `sfs mnt <folder>` reuses its volume and remote.
|
||||
- **`.sfsignore`** — gitignore-style opt-out list at the mount root. Syncs
|
||||
like a normal file, so every device shares the same rules. Supports `#`
|
||||
comments, `*`, `**`, `?`, trailing `/` for directories, leading `/` (or any
|
||||
`/`) for root-anchoring, and `!` to re-include.
|
||||
|
||||
```jsonc
|
||||
// .sfs
|
||||
{ "volume": "notes", "remote": "s3://my-bucket/notes", "include": ["docs/", "*.md"] }
|
||||
```
|
||||
|
||||
Opting out is non-destructive: when a pattern starts matching an
|
||||
already-synced file, the file stops syncing but is deleted nowhere.
|
||||
|
||||
## Web viewer
|
||||
|
||||
`sfs-web` serves a read-only website for a folder or an sfs remote —
|
||||
browse folders and files, read markdown rendered Obsidian-style (including
|
||||
`[[wikilinks]]`, task lists, and tables), and download any file.
|
||||
|
||||
```sh
|
||||
sfs-web # serve the current directory
|
||||
sfs-web ./notes # serve a folder from disk
|
||||
sfs-web s3://my-bucket/workspace # serve an sfs remote
|
||||
```
|
||||
|
||||
With no remote given it serves the folder straight from the local file
|
||||
system — on an sfs mount the daemon keeps those files fresh, so this is
|
||||
the simplest way to run it in production (and needs no cloud credentials
|
||||
on the serving machine). Pointing it at a remote instead reads the object
|
||||
store directly — no mount, daemon, or local sfs state — and each file
|
||||
shows who changed it last, from which device, and when: the same
|
||||
provenance as `sfs log`.
|
||||
|
||||
Flags: `--addr` (default `:4173`), `--volume` (display name), `--refresh`
|
||||
(listing cache, default `10s`), `--dir` / `--remote` (explicit forms of
|
||||
the positional argument).
|
||||
|
||||
Install alongside sfs, or from source:
|
||||
|
||||
```sh
|
||||
go install github.com/runbear-io/sfs/cmd/sfs-web@latest
|
||||
```
|
||||
|
||||
## Claude Code plugin
|
||||
|
||||
Install sfs support in Claude Code with two commands:
|
||||
|
||||
```
|
||||
/plugin marketplace add runbear-io/sfs
|
||||
/plugin install sfs@sfs
|
||||
```
|
||||
|
||||
The plugin sets up everything at once:
|
||||
|
||||
- **`/sfs:mount [folder] [remote]`** — one command that installs sfs if
|
||||
needed, mounts the folder (daemon + `.sfs` config), and verifies the sync.
|
||||
`/sfs:status` diagnoses problems.
|
||||
- **Turn-boundary sync hooks**, registered automatically: a blocking pull
|
||||
when you send a message (Claude always reads fresh files) and an async
|
||||
push when the turn ends. The hook no-ops instantly in folders that aren't
|
||||
sfs mounts, so it's safe globally.
|
||||
- **The `sfs` skill** ([plugin/skills/sfs](plugin/skills/sfs/SKILL.md)),
|
||||
covering mount/unmount/sync, backends and credentials, selective sync, and
|
||||
troubleshooting. Working in a clone of this repo picks the same skill up
|
||||
automatically via `.claude/skills/`.
|
||||
|
||||
## How it works
|
||||
|
||||
@@ -147,14 +215,15 @@ working folder ←materialize/scan→ local volume store ←push/pull→ obj
|
||||
### What sfs does not sync
|
||||
|
||||
`.git` directories (per-file LWW would corrupt repositories), `.DS_Store`,
|
||||
and its own temp files. Empty directories are not tracked (like git).
|
||||
the `.sfs` settings file, its own temp files, and anything excluded by
|
||||
`.sfsignore` or omitted from an `include` list. Empty directories are not
|
||||
tracked (like git).
|
||||
|
||||
## Roadmap
|
||||
|
||||
- `sfs restore <path>@<time>` — restore any file from history (all content
|
||||
is already retained)
|
||||
- FUSE/NFS mount mode for lazy-loading huge volumes
|
||||
- `.sfsignore` patterns
|
||||
- Journal compaction & blob GC policies
|
||||
- Per-path access scopes for multi-agent setups
|
||||
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
// sfs-web serves a read-only website for a folder or an sfs remote: browse
|
||||
// folders and files, read rendered markdown (Obsidian-style, including
|
||||
// [[wikilinks]]), and download any file.
|
||||
//
|
||||
// Two sources:
|
||||
//
|
||||
// - a local folder, served straight from disk (the default — on an sfs
|
||||
// mount the daemon keeps it fresh, so this is the simplest deployment);
|
||||
// - an sfs remote, read directly from the object store with per-file
|
||||
// provenance from the journals — no mount, daemon, or local state needed.
|
||||
//
|
||||
// Examples:
|
||||
//
|
||||
// sfs-web # serve the current directory
|
||||
// sfs-web ./notes # serve a folder
|
||||
// sfs-web s3://bucket/prefix # serve an sfs remote
|
||||
// sfs-web --remote gs://bucket/prefix --addr :8080
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/runbear-io/sfs/internal/remote"
|
||||
"github.com/runbear-io/sfs/internal/webapp"
|
||||
)
|
||||
|
||||
func main() {
|
||||
remoteURL := flag.String("remote", "", "sfs remote to serve (s3://bucket/prefix, gs://bucket/prefix, file:///path)")
|
||||
dir := flag.String("dir", "", "local folder to serve (default: current directory)")
|
||||
addr := flag.String("addr", ":4173", "address to listen on")
|
||||
volume := flag.String("volume", "", "volume display name (default: folder or remote basename)")
|
||||
refresh := flag.Duration("refresh", 10*time.Second, "how long to cache the file listing")
|
||||
flag.Parse()
|
||||
|
||||
// Positional argument: a URL selects remote mode, anything else is a
|
||||
// folder. With nothing specified at all, serve the current directory.
|
||||
if *remoteURL == "" && *dir == "" && flag.NArg() > 0 {
|
||||
if arg := flag.Arg(0); strings.Contains(arg, "://") {
|
||||
*remoteURL = arg
|
||||
} else {
|
||||
*dir = arg
|
||||
}
|
||||
}
|
||||
if *remoteURL != "" && *dir != "" {
|
||||
fmt.Fprintln(os.Stderr, "usage: sfs-web [folder | remote-url] [--addr :4173] (--remote and --dir are mutually exclusive)")
|
||||
os.Exit(2)
|
||||
}
|
||||
if *remoteURL == "" && *dir == "" {
|
||||
*dir = "."
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
var src webapp.Source
|
||||
var display, name string
|
||||
if *dir != "" {
|
||||
abs, err := filepath.Abs(*dir)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if fi, err := os.Stat(abs); err != nil || !fi.IsDir() {
|
||||
fmt.Fprintf(os.Stderr, "error: %s is not a directory\n", abs)
|
||||
os.Exit(1)
|
||||
}
|
||||
src = &webapp.DirSource{Root: abs}
|
||||
display, name = abs, filepath.Base(abs)
|
||||
} else {
|
||||
be, err := remote.Open(ctx, *remoteURL)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer be.Close()
|
||||
src = &webapp.RemoteSource{Backend: be}
|
||||
display, name = *remoteURL, volumeName(*remoteURL)
|
||||
}
|
||||
if *volume != "" {
|
||||
name = *volume
|
||||
}
|
||||
srv := &webapp.Server{
|
||||
Source: src,
|
||||
Remote: display,
|
||||
Volume: name,
|
||||
Refresh: *refresh,
|
||||
}
|
||||
|
||||
shown := *addr
|
||||
if strings.HasPrefix(shown, ":") {
|
||||
shown = "localhost" + shown
|
||||
}
|
||||
fmt.Printf("sfs-web serving %s\n volume: %s\n url: http://%s\n", display, name, shown)
|
||||
if err := http.ListenAndServe(*addr, srv.Handler()); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func volumeName(remoteURL string) string {
|
||||
if u, err := url.Parse(remoteURL); err == nil {
|
||||
if base := path.Base(strings.Trim(u.Path, "/")); base != "" && base != "." {
|
||||
return base
|
||||
}
|
||||
if u.Host != "" {
|
||||
return u.Host
|
||||
}
|
||||
}
|
||||
return "sfs"
|
||||
}
|
||||
+16
-5
@@ -75,6 +75,9 @@ func statusCmd() *cobra.Command {
|
||||
fmt.Println()
|
||||
}
|
||||
first = false
|
||||
if eff, _, found, err := config.EffectiveMount(folder); err == nil && found {
|
||||
mi = eff // .sfs project file wins over the registry
|
||||
}
|
||||
fmt.Printf("%s\n", folder)
|
||||
fmt.Printf(" volume: %s\n", mi.Volume)
|
||||
if mi.Remote != "" {
|
||||
@@ -203,19 +206,27 @@ func remoteCmd() *cobra.Command {
|
||||
if err != nil || (u.Scheme != "s3" && u.Scheme != "gs" && u.Scheme != "file") {
|
||||
return fmt.Errorf("invalid remote %q (want s3://bucket/prefix, gs://bucket/prefix, or file:///path)", raw)
|
||||
}
|
||||
mi, err := mustMount(folder)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
mi.Remote = raw
|
||||
mounts, err := config.LoadMounts()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
mi, ok := mounts[folder]
|
||||
if !ok {
|
||||
return fmt.Errorf("%s is not an sfs mount (run `sfs mnt %s` first)", folder, folder)
|
||||
}
|
||||
mi.Remote = raw
|
||||
mounts[folder] = mi
|
||||
if err := config.SaveMounts(mounts); err != nil {
|
||||
return err
|
||||
}
|
||||
proj, _, err := config.LoadProject(folder)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
proj.Volume, proj.Remote = mi.Volume, raw
|
||||
if err := config.SaveProject(folder, proj); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("remote of %s set to %s\n", folder, raw)
|
||||
fmt.Println("run `sfs sync` to sync now (a running daemon picks it up automatically)")
|
||||
return nil
|
||||
|
||||
+9
-4
@@ -20,15 +20,20 @@ func absFolder(args []string) (string, error) {
|
||||
return filepath.Abs(arg)
|
||||
}
|
||||
|
||||
// mustMount resolves a folder's settings: the .sfs project file wins over
|
||||
// the global registry, so a folder that carries its own .sfs works even
|
||||
// before it is registered on this device.
|
||||
func mustMount(folder string) (config.MountInfo, error) {
|
||||
mounts, err := config.LoadMounts()
|
||||
mi, _, found, err := config.EffectiveMount(folder)
|
||||
if err != nil {
|
||||
return config.MountInfo{}, err
|
||||
return mi, err
|
||||
}
|
||||
mi, ok := mounts[folder]
|
||||
if !ok {
|
||||
if !found {
|
||||
return mi, fmt.Errorf("%s is not an sfs mount (run `sfs mnt %s` first)", folder, folder)
|
||||
}
|
||||
if mi.Volume == "" {
|
||||
mi.Volume = filepath.Base(folder)
|
||||
}
|
||||
return mi, nil
|
||||
}
|
||||
|
||||
|
||||
+34
-13
@@ -41,29 +41,48 @@ daemon keeps the folder in sync until "sfs umnt".`,
|
||||
return err
|
||||
}
|
||||
|
||||
// Settings resolution: flags win, then the folder's .sfs file,
|
||||
// then the global registry. The result is written back to both,
|
||||
// so the project file travels with the folder and the registry
|
||||
// knows which mounts are active on this device.
|
||||
mounts, err := config.LoadMounts()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
mi, exists := mounts[folder]
|
||||
if exists {
|
||||
if volume != "" && volume != mi.Volume {
|
||||
return fmt.Errorf("%s is already mounted as volume %q", folder, mi.Volume)
|
||||
}
|
||||
} else {
|
||||
v := volume
|
||||
if v == "" {
|
||||
v = filepath.Base(folder)
|
||||
}
|
||||
mi = config.MountInfo{Volume: v}
|
||||
reg, registered := mounts[folder]
|
||||
proj, _, err := config.LoadProject(folder)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if remoteURL != "" {
|
||||
mi.Remote = remoteURL
|
||||
effVolume := proj.Volume
|
||||
if effVolume == "" && registered {
|
||||
effVolume = reg.Volume
|
||||
}
|
||||
if volume != "" && effVolume != "" && volume != effVolume {
|
||||
return fmt.Errorf("%s is already mounted as volume %q", folder, effVolume)
|
||||
}
|
||||
if volume != "" {
|
||||
effVolume = volume
|
||||
}
|
||||
if effVolume == "" {
|
||||
effVolume = filepath.Base(folder)
|
||||
}
|
||||
effRemote := remoteURL
|
||||
if effRemote == "" {
|
||||
effRemote = proj.Remote
|
||||
}
|
||||
if effRemote == "" && registered {
|
||||
effRemote = reg.Remote
|
||||
}
|
||||
mi := config.MountInfo{Volume: effVolume, Remote: effRemote}
|
||||
mounts[folder] = mi
|
||||
if err := config.SaveMounts(mounts); err != nil {
|
||||
return err
|
||||
}
|
||||
proj.Volume, proj.Remote = effVolume, effRemote
|
||||
if err := config.SaveProject(folder, proj); err != nil {
|
||||
return err
|
||||
}
|
||||
vdir, err := config.VolumeDir(mi.Volume)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -96,6 +115,8 @@ daemon keeps the folder in sync until "sfs umnt".`,
|
||||
fmt.Printf(" remote: (none — local only; set one with `sfs remote set %s <url>`)\n", folder)
|
||||
}
|
||||
fmt.Printf(" device: %s (%s) as %s\n", dev.Name, dev.ID, dev.Author)
|
||||
fmt.Printf(" config: %s (volume/remote/include; add a .sfsignore next to it to exclude paths)\n",
|
||||
filepath.Join(folder, config.ProjectFile))
|
||||
printCycle(res)
|
||||
|
||||
if foreground {
|
||||
|
||||
@@ -9,6 +9,7 @@ require (
|
||||
github.com/aws/aws-sdk-go-v2/service/s3 v1.103.3
|
||||
github.com/aws/smithy-go v1.27.2
|
||||
github.com/spf13/cobra v1.10.2
|
||||
github.com/yuin/goldmark v1.8.2
|
||||
google.golang.org/api v0.284.0
|
||||
)
|
||||
|
||||
|
||||
@@ -117,6 +117,8 @@ github.com/spiffe/go-spiffe/v2 v2.6.0 h1:l+DolpxNWYgruGQVV0xsfeya3CsC7m8iBzDnMps
|
||||
github.com/spiffe/go-spiffe/v2 v2.6.0/go.mod h1:gm2SeUoMZEtpnzPNs2Csc0D/gX33k1xIx7lEzqblHEs=
|
||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||
github.com/yuin/goldmark v1.8.2 h1:kEGpgqJXdgbkhcOgBxkC0X0PmoPG1ZyoZ117rDVp4zE=
|
||||
github.com/yuin/goldmark v1.8.2/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
||||
go.opentelemetry.io/contrib/detectors/gcp v1.42.0 h1:kpt2PEJuOuqYkPcktfJqWWDjTEd/FNgrxcniL7kQrXQ=
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
// ProjectFile is the name of the per-folder settings file at the mount root.
|
||||
// It travels with the project (copy the folder, `sfs mnt .`, and the same
|
||||
// volume/remote apply) but is never synced — remotes and credentials setups
|
||||
// are often device-specific, and syncing it would let one device silently
|
||||
// repoint another.
|
||||
const ProjectFile = ".sfs"
|
||||
|
||||
// Project holds the settings stored in <folder>/.sfs.
|
||||
type Project struct {
|
||||
Volume string `json:"volume,omitempty"`
|
||||
Remote string `json:"remote,omitempty"`
|
||||
// Include optionally narrows what syncs: when non-empty, only paths
|
||||
// matching one of these patterns (gitignore-style, same syntax as
|
||||
// .sfsignore) are scanned and materialized.
|
||||
Include []string `json:"include,omitempty"`
|
||||
}
|
||||
|
||||
// LoadProject reads <folder>/.sfs; ok is false if the file does not exist.
|
||||
func LoadProject(folder string) (Project, bool, error) {
|
||||
var p Project
|
||||
data, err := os.ReadFile(filepath.Join(folder, ProjectFile))
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return p, false, nil
|
||||
}
|
||||
return p, false, err
|
||||
}
|
||||
if err := json.Unmarshal(data, &p); err != nil {
|
||||
return p, false, fmt.Errorf("parse %s: %w", ProjectFile, err)
|
||||
}
|
||||
return p, true, nil
|
||||
}
|
||||
|
||||
// SaveProject writes <folder>/.sfs.
|
||||
func SaveProject(folder string, p Project) error {
|
||||
return writeJSON(filepath.Join(folder, ProjectFile), p)
|
||||
}
|
||||
|
||||
// EffectiveMount resolves a folder's mount settings: the project file wins
|
||||
// over the global registry, so hand-edits to .sfs (or a folder copied with
|
||||
// its .sfs) take effect without re-registering. Found reports whether the
|
||||
// folder is known at all (registered or carrying a project file).
|
||||
func EffectiveMount(folder string) (mi MountInfo, proj Project, found bool, err error) {
|
||||
mounts, err := LoadMounts()
|
||||
if err != nil {
|
||||
return mi, proj, false, err
|
||||
}
|
||||
mi, registered := mounts[folder]
|
||||
proj, hasProj, err := LoadProject(folder)
|
||||
if err != nil {
|
||||
return mi, proj, false, err
|
||||
}
|
||||
if hasProj {
|
||||
if proj.Volume != "" {
|
||||
mi.Volume = proj.Volume
|
||||
}
|
||||
if proj.Remote != "" {
|
||||
mi.Remote = proj.Remote
|
||||
}
|
||||
}
|
||||
return mi, proj, registered || hasProj, nil
|
||||
}
|
||||
@@ -106,6 +106,22 @@ func Stop(volDir, mountID string) (bool, error) {
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// overlayProject applies the folder's .sfs settings on top of the registry
|
||||
// entry; the project file wins so hand-edits take effect on the next tick.
|
||||
func overlayProject(folder string, mi config.MountInfo) config.MountInfo {
|
||||
proj, ok, err := config.LoadProject(folder)
|
||||
if err != nil || !ok {
|
||||
return mi
|
||||
}
|
||||
if proj.Volume != "" {
|
||||
mi.Volume = proj.Volume
|
||||
}
|
||||
if proj.Remote != "" {
|
||||
mi.Remote = proj.Remote
|
||||
}
|
||||
return mi
|
||||
}
|
||||
|
||||
// Run is the daemon main loop, executed in the foreground of the (usually
|
||||
// detached) `sfs daemon run` process.
|
||||
func Run(folder string, scanInterval, remoteInterval time.Duration) error {
|
||||
@@ -120,6 +136,7 @@ func Run(folder string, scanInterval, remoteInterval time.Duration) error {
|
||||
if !ok {
|
||||
return fmt.Errorf("%s is not an sfs mount", folder)
|
||||
}
|
||||
mi = overlayProject(folder, mi)
|
||||
volDir, err := config.VolumeDir(mi.Volume)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -150,13 +167,15 @@ func Run(folder string, scanInterval, remoteInterval time.Duration) error {
|
||||
var lastRemote time.Time
|
||||
|
||||
for {
|
||||
// Pick up `sfs remote set` / `sfs umnt --forget` without restarting.
|
||||
// Pick up `sfs remote set`, .sfs edits, and `sfs umnt --forget`
|
||||
// without restarting.
|
||||
if m, err := config.LoadMounts(); err == nil {
|
||||
cur, ok := m[folder]
|
||||
if !ok {
|
||||
log.Printf("mount unregistered; exiting")
|
||||
return nil
|
||||
}
|
||||
cur = overlayProject(folder, cur)
|
||||
if cur.Remote != mi.Remote {
|
||||
log.Printf("remote changed: %q -> %q", mi.Remote, cur.Remote)
|
||||
if be != nil {
|
||||
|
||||
@@ -31,8 +31,8 @@ type Op struct {
|
||||
Device string `json:"device"`
|
||||
DeviceName string `json:"device_name,omitempty"`
|
||||
Author string `json:"author,omitempty"`
|
||||
Kind string `json:"kind"` // "put" or "delete"
|
||||
Path string `json:"path"` // slash-separated, relative to volume root
|
||||
Kind string `json:"kind"` // "put" or "delete"
|
||||
Path string `json:"path"` // slash-separated, relative to volume root
|
||||
Blob string `json:"blob,omitempty"` // sha256 hex of content (put only)
|
||||
Size int64 `json:"size,omitempty"`
|
||||
Mode uint32 `json:"mode,omitempty"` // permission bits
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
package syncer
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Multi-device behavior of .sfsignore and the .sfs include list.
|
||||
|
||||
func TestIgnoredFilesDoNotSync(t *testing.T) {
|
||||
be := sharedRemote(t)
|
||||
a := newDevice(t, "deva", be)
|
||||
b := newDevice(t, "devb", be)
|
||||
|
||||
write(t, a.Folder, IgnoreFile, "*.secret\n")
|
||||
write(t, a.Folder, "notes.md", "hello")
|
||||
write(t, a.Folder, "key.secret", "hunter2")
|
||||
cycle(t, a)
|
||||
cycle(t, b)
|
||||
|
||||
if got := read(t, b.Folder, "notes.md"); got != "hello" {
|
||||
t.Fatalf("notes.md = %q", got)
|
||||
}
|
||||
if got := read(t, b.Folder, IgnoreFile); got != "*.secret\n" {
|
||||
t.Fatalf(".sfsignore should sync like a normal file, got %q", got)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(b.Folder, "key.secret")); !os.IsNotExist(err) {
|
||||
t.Fatal("ignored file must not reach other devices")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewlyIgnoredFileIsNotDeletedRemotely(t *testing.T) {
|
||||
be := sharedRemote(t)
|
||||
a := newDevice(t, "deva", be)
|
||||
b := newDevice(t, "devb", be)
|
||||
|
||||
write(t, a.Folder, "debug.log", "lines")
|
||||
cycle(t, a)
|
||||
cycle(t, b)
|
||||
if read(t, b.Folder, "debug.log") != "lines" {
|
||||
t.Fatal("setup: file should have synced")
|
||||
}
|
||||
|
||||
// A opts out afterwards; the file must stop syncing without a delete
|
||||
// op, so it stays on disk on every device.
|
||||
write(t, a.Folder, IgnoreFile, "*.log\n")
|
||||
res := cycle(t, a)
|
||||
if res.LocalOps != 1 { // only the .sfsignore put, no delete for debug.log
|
||||
t.Fatalf("LocalOps = %d, want 1 (the .sfsignore itself)", res.LocalOps)
|
||||
}
|
||||
cycle(t, b)
|
||||
cycle(t, b) // second cycle: filter from the pulled .sfsignore is active
|
||||
if read(t, a.Folder, "debug.log") != "lines" || read(t, b.Folder, "debug.log") != "lines" {
|
||||
t.Fatal("newly ignored file must remain on disk everywhere")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIncludeListLimitsSync(t *testing.T) {
|
||||
be := sharedRemote(t)
|
||||
a := newDevice(t, "deva", be)
|
||||
b := newDevice(t, "devb", be)
|
||||
|
||||
write(t, a.Folder, ".sfs", `{"include": ["docs/"]}`)
|
||||
write(t, a.Folder, "docs/guide.md", "included")
|
||||
write(t, a.Folder, "src/main.go", "excluded")
|
||||
cycle(t, a)
|
||||
cycle(t, b)
|
||||
|
||||
if got := read(t, b.Folder, "docs/guide.md"); got != "included" {
|
||||
t.Fatalf("docs/guide.md = %q", got)
|
||||
}
|
||||
for _, absent := range []string{"src/main.go", ".sfs"} {
|
||||
if _, err := os.Stat(filepath.Join(b.Folder, absent)); !os.IsNotExist(err) {
|
||||
t.Fatalf("%s must not sync", absent)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,154 @@
|
||||
package syncer
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// IgnoreFile is the per-folder opt-out list at the mount root. It uses a
|
||||
// gitignore-style syntax and, unlike the .sfs settings file, syncs like any
|
||||
// other file so every device shares the same rules.
|
||||
const IgnoreFile = ".sfsignore"
|
||||
|
||||
// Filter decides which paths sync. A path syncs when it is not ignored and,
|
||||
// if an include list is set, matches at least one include pattern.
|
||||
//
|
||||
// Pattern syntax (a practical gitignore subset): one pattern per line,
|
||||
// blank lines and #-comments skipped, `!` re-includes, a trailing `/`
|
||||
// matches directories only, a `/` anywhere else anchors the pattern to the
|
||||
// mount root (otherwise it matches at any depth), `*` matches within a path
|
||||
// segment, `**` across segments, `?` a single character.
|
||||
type Filter struct {
|
||||
ignore []pattern
|
||||
include []pattern
|
||||
negated bool // any `!` rules → directory pruning is unsafe
|
||||
}
|
||||
|
||||
type pattern struct {
|
||||
re *regexp.Regexp
|
||||
negate bool
|
||||
}
|
||||
|
||||
// loadFilter builds the filter for a folder from its .sfsignore (if any)
|
||||
// plus the include list from the .sfs settings file.
|
||||
func loadFilter(folder string, include []string) (*Filter, error) {
|
||||
f := &Filter{}
|
||||
for _, line := range include {
|
||||
if p, ok := compile(line); ok {
|
||||
f.include = append(f.include, p)
|
||||
}
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(folder, IgnoreFile))
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return f, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
for _, line := range strings.Split(string(data), "\n") {
|
||||
p, ok := compile(line)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
f.ignore = append(f.ignore, p)
|
||||
if p.negate {
|
||||
f.negated = true
|
||||
}
|
||||
}
|
||||
return f, nil
|
||||
}
|
||||
|
||||
// compile turns one pattern line into a regexp over slash-separated paths.
|
||||
// The regexp also matches everything under a matched directory. Returns
|
||||
// ok=false for blanks, comments, and invalid patterns.
|
||||
func compile(line string) (pattern, bool) {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || strings.HasPrefix(line, "#") {
|
||||
return pattern{}, false
|
||||
}
|
||||
var p pattern
|
||||
if strings.HasPrefix(line, "!") {
|
||||
p.negate = true
|
||||
line = strings.TrimSpace(line[1:])
|
||||
}
|
||||
anchored := strings.HasPrefix(line, "/")
|
||||
dirOnly := strings.HasSuffix(line, "/")
|
||||
line = strings.Trim(line, "/")
|
||||
if line == "" {
|
||||
return pattern{}, false
|
||||
}
|
||||
anchored = anchored || strings.Contains(line, "/")
|
||||
|
||||
var b strings.Builder
|
||||
if anchored {
|
||||
b.WriteString("^")
|
||||
} else {
|
||||
b.WriteString("(^|.*/)")
|
||||
}
|
||||
for i := 0; i < len(line); i++ {
|
||||
switch line[i] {
|
||||
case '*':
|
||||
if i+1 < len(line) && line[i+1] == '*' {
|
||||
b.WriteString(".*")
|
||||
i++
|
||||
} else {
|
||||
b.WriteString("[^/]*")
|
||||
}
|
||||
case '?':
|
||||
b.WriteString("[^/]")
|
||||
default:
|
||||
b.WriteString(regexp.QuoteMeta(line[i : i+1]))
|
||||
}
|
||||
}
|
||||
if dirOnly {
|
||||
b.WriteString("/.*$") // must match something *inside* the directory
|
||||
} else {
|
||||
b.WriteString("(/.*)?$")
|
||||
}
|
||||
re, err := regexp.Compile(b.String())
|
||||
if err != nil {
|
||||
return pattern{}, false
|
||||
}
|
||||
p.re = re
|
||||
return p, true
|
||||
}
|
||||
|
||||
// Skip reports whether a file path should not sync.
|
||||
func (f *Filter) Skip(rel string) bool {
|
||||
if f.ignoredFile(rel) {
|
||||
return true
|
||||
}
|
||||
if len(f.include) == 0 {
|
||||
return false
|
||||
}
|
||||
for _, p := range f.include {
|
||||
if p.re.MatchString(rel) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// PruneDir reports whether a whole directory can be skipped during the
|
||||
// scan walk. Pruning is conservative: never with `!` rules (a child could
|
||||
// be re-included) or an include list (a deep child could match).
|
||||
func (f *Filter) PruneDir(rel string) bool {
|
||||
if f.negated || len(f.include) > 0 {
|
||||
return false
|
||||
}
|
||||
return f.ignoredFile(rel + "/")
|
||||
}
|
||||
|
||||
// ignoredFile applies the ignore rules in order; the last match wins, so
|
||||
// `!` patterns can re-include what an earlier pattern excluded.
|
||||
func (f *Filter) ignoredFile(rel string) bool {
|
||||
ignored := false
|
||||
for _, p := range f.ignore {
|
||||
if p.re.MatchString(rel) {
|
||||
ignored = !p.negate
|
||||
}
|
||||
}
|
||||
return ignored
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
package syncer
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func filterFrom(t *testing.T, ignore string, include []string) *Filter {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
if ignore != "" {
|
||||
if err := os.WriteFile(filepath.Join(dir, IgnoreFile), []byte(ignore), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
f, err := loadFilter(dir, include)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
func TestIgnorePatterns(t *testing.T) {
|
||||
cases := []struct {
|
||||
ignore string
|
||||
path string
|
||||
want bool
|
||||
}{
|
||||
{"*.log", "a.log", true},
|
||||
{"*.log", "sub/dir/a.log", true},
|
||||
{"*.log", "a.log.txt", false},
|
||||
{"# comment\n\n*.tmp", "x.tmp", true},
|
||||
{"build/", "build/out.bin", true},
|
||||
{"build/", "build", false}, // dir-only pattern must not match a file
|
||||
{"build/", "sub/build/x", true},
|
||||
{"/docs", "docs/a.md", true},
|
||||
{"/docs", "sub/docs/a.md", false}, // anchored
|
||||
{"docs/*.md", "docs/a.md", true},
|
||||
{"docs/*.md", "docs/sub/a.md", false}, // * stays within a segment
|
||||
{"docs/**/a.md", "docs/x/y/a.md", true},
|
||||
{"secret?.txt", "secret1.txt", true},
|
||||
{"secret?.txt", "secret10.txt", false},
|
||||
{"*.log\n!keep.log", "keep.log", false},
|
||||
{"*.log\n!keep.log", "other.log", true},
|
||||
}
|
||||
for _, c := range cases {
|
||||
f := filterFrom(t, c.ignore, nil)
|
||||
if got := f.Skip(c.path); got != c.want {
|
||||
t.Errorf("ignore %q: Skip(%q) = %v, want %v", c.ignore, c.path, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIncludePatterns(t *testing.T) {
|
||||
f := filterFrom(t, "", []string{"docs/", "*.md"})
|
||||
for path, want := range map[string]bool{
|
||||
"docs/deep/x.bin": false, // under an included dir
|
||||
"README.md": false, // matches *.md anywhere
|
||||
"notes/plan.md": false,
|
||||
"src/main.go": true, // matches nothing → skipped
|
||||
} {
|
||||
if got := f.Skip(path); got != want {
|
||||
t.Errorf("Skip(%q) = %v, want %v", path, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIgnoreBeatsInclude(t *testing.T) {
|
||||
f := filterFrom(t, "docs/private.md", []string{"docs/"})
|
||||
if !f.Skip("docs/private.md") {
|
||||
t.Error("ignored file inside an included dir should be skipped")
|
||||
}
|
||||
if f.Skip("docs/public.md") {
|
||||
t.Error("non-ignored file inside an included dir should sync")
|
||||
}
|
||||
}
|
||||
|
||||
func TestPruneDir(t *testing.T) {
|
||||
f := filterFrom(t, "node_modules/", nil)
|
||||
if !f.PruneDir("node_modules") || !f.PruneDir("sub/node_modules") {
|
||||
t.Error("plain dir ignore should prune the walk")
|
||||
}
|
||||
if f.PruneDir("src") {
|
||||
t.Error("unmatched dir must not be pruned")
|
||||
}
|
||||
// negations and includes make pruning unsafe
|
||||
if filterFrom(t, "node_modules/\n!node_modules/keep.js", nil).PruneDir("node_modules") {
|
||||
t.Error("must not prune when ! rules exist")
|
||||
}
|
||||
if filterFrom(t, "node_modules/", []string{"docs/"}).PruneDir("node_modules") {
|
||||
t.Error("must not prune when an include list exists")
|
||||
}
|
||||
}
|
||||
+34
-11
@@ -60,7 +60,9 @@ func (r *Result) Activity() bool {
|
||||
return r.LocalOps > 0 || r.PulledOps > 0 || r.Conflicts > 0 || r.Materialized > 0
|
||||
}
|
||||
|
||||
var ignoreNames = map[string]bool{".DS_Store": true}
|
||||
// config.ProjectFile (.sfs) never syncs: remotes are device-specific and
|
||||
// syncing it would let one device silently repoint another.
|
||||
var ignoreNames = map[string]bool{".DS_Store": true, config.ProjectFile: true}
|
||||
var ignoreDirs = map[string]bool{".git": true, ".sfs": true}
|
||||
|
||||
func ignoredFile(name string) bool {
|
||||
@@ -88,9 +90,17 @@ func (s *Session) Cycle(ctx context.Context) (*Result, error) {
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read own journal: %w", err)
|
||||
}
|
||||
proj, _, err := config.LoadProject(s.Folder)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
filter, err := loadFilter(s.Folder, proj.Include)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("load %s: %w", IgnoreFile, err)
|
||||
}
|
||||
|
||||
// 1. Scan the working folder and journal any local changes.
|
||||
localOps, err := s.scan(cache, &st, int64(len(myOps)))
|
||||
localOps, err := s.scan(cache, &st, int64(len(myOps)), filter)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("scan: %w", err)
|
||||
}
|
||||
@@ -139,7 +149,7 @@ func (s *Session) Cycle(ctx context.Context) (*Result, error) {
|
||||
return nil, fmt.Errorf("read journals: %w", err)
|
||||
}
|
||||
target := journal.Replay(all)
|
||||
n, err := s.materialize(target, cache)
|
||||
n, err := s.materialize(target, cache, filter)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("materialize: %w", err)
|
||||
}
|
||||
@@ -165,8 +175,11 @@ func (s *Session) Cycle(ctx context.Context) (*Result, error) {
|
||||
}
|
||||
|
||||
// scan diffs the working folder against the state cache and returns ops for
|
||||
// every local change, storing new content in the blob store.
|
||||
func (s *Session) scan(cache map[string]store.CachedFile, st *store.SyncState, seqBase int64) ([]journal.Op, error) {
|
||||
// every local change, storing new content in the blob store. Filtered paths
|
||||
// are neither journaled nor deleted: a path that becomes ignored is dropped
|
||||
// from the cache without a delete op, so opting out locally never removes
|
||||
// the file from other devices.
|
||||
func (s *Session) scan(cache map[string]store.CachedFile, st *store.SyncState, seqBase int64, filter *Filter) ([]journal.Op, error) {
|
||||
seen := make(map[string]bool, len(cache))
|
||||
var ops []journal.Op
|
||||
nextOp := func(kind, rel string) journal.Op {
|
||||
@@ -189,12 +202,12 @@ func (s *Session) scan(cache map[string]store.CachedFile, st *store.SyncState, s
|
||||
}
|
||||
rel = filepath.ToSlash(rel)
|
||||
if d.IsDir() {
|
||||
if ignoreDirs[d.Name()] {
|
||||
if ignoreDirs[d.Name()] || filter.PruneDir(rel) {
|
||||
return fs.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !d.Type().IsRegular() || ignoredFile(d.Name()) {
|
||||
if !d.Type().IsRegular() || ignoredFile(d.Name()) || filter.Skip(rel) {
|
||||
return nil
|
||||
}
|
||||
info, err := d.Info()
|
||||
@@ -229,10 +242,15 @@ func (s *Session) scan(cache map[string]store.CachedFile, st *store.SyncState, s
|
||||
}
|
||||
|
||||
for rel := range cache {
|
||||
if !seen[rel] {
|
||||
ops = append(ops, nextOp(journal.KindDelete, rel))
|
||||
delete(cache, rel)
|
||||
if seen[rel] {
|
||||
continue
|
||||
}
|
||||
if filter.Skip(rel) {
|
||||
delete(cache, rel) // newly filtered, not deleted: stop tracking silently
|
||||
continue
|
||||
}
|
||||
ops = append(ops, nextOp(journal.KindDelete, rel))
|
||||
delete(cache, rel)
|
||||
}
|
||||
return ops, nil
|
||||
}
|
||||
@@ -390,9 +408,14 @@ func sanitize(s string) string {
|
||||
|
||||
// materialize applies the merged state to the working folder, never
|
||||
// clobbering files that changed since the scan earlier in this cycle.
|
||||
func (s *Session) materialize(target map[string]journal.FileState, cache map[string]store.CachedFile) (int, error) {
|
||||
// Filtered paths are not written: other devices' files that match the local
|
||||
// ignore/include rules simply don't appear here.
|
||||
func (s *Session) materialize(target map[string]journal.FileState, cache map[string]store.CachedFile, filter *Filter) (int, error) {
|
||||
changed := 0
|
||||
for rel, want := range target {
|
||||
if filter.Skip(rel) {
|
||||
continue
|
||||
}
|
||||
c, ok := cache[rel]
|
||||
if ok && c.Blob == want.Blob && c.Mode == want.Mode {
|
||||
continue
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
package webapp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// DirSource serves a plain local folder straight from disk — no sfs remote
|
||||
// or volume needed. Meant for debugging the webapp (and as a quick local
|
||||
// markdown browser): the tree reflects the folder live, provenance is just
|
||||
// file mtimes, and content streams from the filesystem.
|
||||
type DirSource struct {
|
||||
Root string
|
||||
}
|
||||
|
||||
var skipNames = map[string]bool{".DS_Store": true, ".sfs": true}
|
||||
var skipDirs = map[string]bool{".git": true, ".sfs": true}
|
||||
|
||||
func (d *DirSource) Files(_ context.Context) (map[string]FileInfo, error) {
|
||||
files := make(map[string]FileInfo)
|
||||
err := filepath.WalkDir(d.Root, func(p string, e fs.DirEntry, walkErr error) error {
|
||||
if walkErr != nil {
|
||||
return nil // skip unreadable entries
|
||||
}
|
||||
rel, err := filepath.Rel(d.Root, p)
|
||||
if err != nil || rel == "." {
|
||||
return nil
|
||||
}
|
||||
if e.IsDir() {
|
||||
if skipDirs[e.Name()] {
|
||||
return fs.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !e.Type().IsRegular() || skipNames[e.Name()] || strings.HasPrefix(e.Name(), ".sfs-tmp-") {
|
||||
return nil
|
||||
}
|
||||
info, err := e.Info()
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
files[filepath.ToSlash(rel)] = FileInfo{
|
||||
// Synthetic content identity for the ETag; changes when the
|
||||
// file does, which is all revalidation needs.
|
||||
Blob: fmt.Sprintf("dir-%d-%d", info.ModTime().UnixNano(), info.Size()),
|
||||
Size: info.Size(),
|
||||
Time: info.ModTime().UTC(),
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
// Open streams a file from disk. Paths are only ever snapshot map keys
|
||||
// (produced by Files above), so they cannot escape Root.
|
||||
func (d *DirSource) Open(_ context.Context, path string, _ FileInfo) (io.ReadCloser, error) {
|
||||
return os.Open(filepath.Join(d.Root, filepath.FromSlash(path)))
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
package webapp
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func dirServer(t *testing.T, files map[string]string) http.Handler {
|
||||
t.Helper()
|
||||
root := t.TempDir()
|
||||
for rel, content := range files {
|
||||
abs := filepath.Join(root, filepath.FromSlash(rel))
|
||||
if err := os.MkdirAll(filepath.Dir(abs), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(abs, []byte(content), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
s := &Server{Source: &DirSource{Root: root}, Remote: root, Volume: "local", Refresh: 0}
|
||||
return s.Handler()
|
||||
}
|
||||
|
||||
func TestDirSourceServesFolder(t *testing.T) {
|
||||
h := dirServer(t, map[string]string{
|
||||
"README.md": "# Local",
|
||||
"notes/plan.md": "content",
|
||||
".sfs": `{"volume":"x"}`, // settings file must be hidden
|
||||
".git/config": "noise", // .git must be skipped
|
||||
})
|
||||
|
||||
var root Node
|
||||
if err := json.Unmarshal(get(t, h, "/api/tree").Body.Bytes(), &root); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
names := []string{}
|
||||
for _, n := range root.Children {
|
||||
names = append(names, n.Name)
|
||||
}
|
||||
if len(root.Children) != 2 || root.Children[0].Name != "notes" || root.Children[1].Name != "README.md" {
|
||||
t.Fatalf("tree children = %v, want [notes README.md]", names)
|
||||
}
|
||||
|
||||
rec := get(t, h, "/api/file?path=notes/plan.md")
|
||||
if rec.Code != 200 || rec.Body.String() != "content" {
|
||||
t.Fatalf("file: %d %q", rec.Code, rec.Body)
|
||||
}
|
||||
if rec.Header().Get("ETag") == "" {
|
||||
t.Fatal("dir source should still produce ETags")
|
||||
}
|
||||
|
||||
rec = get(t, h, "/api/render?path=README.md")
|
||||
if rec.Code != 200 || !strings.Contains(rec.Body.String(), "Local") {
|
||||
t.Fatalf("render: %d %s", rec.Code, rec.Body)
|
||||
}
|
||||
|
||||
if rec := get(t, h, "/api/file?path=.git/config"); rec.Code != 404 {
|
||||
t.Fatalf(".git content must be hidden, got %d", rec.Code)
|
||||
}
|
||||
if rec := get(t, h, "/api/file?path=../escape"); rec.Code != 404 {
|
||||
t.Fatalf("path traversal must 404, got %d", rec.Code)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
package webapp
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"net/url"
|
||||
"regexp"
|
||||
|
||||
"github.com/yuin/goldmark"
|
||||
"github.com/yuin/goldmark/extension"
|
||||
"github.com/yuin/goldmark/parser"
|
||||
)
|
||||
|
||||
var md = goldmark.New(
|
||||
goldmark.WithExtensions(extension.GFM),
|
||||
goldmark.WithParserOptions(parser.WithAutoHeadingID()),
|
||||
)
|
||||
|
||||
// wikiRe matches Obsidian-style [[target]] and [[target|label]] links.
|
||||
var wikiRe = regexp.MustCompile(`\[\[([^\]|]+)(?:\|([^\]]+))?\]\]`)
|
||||
|
||||
// expandWikilinks rewrites [[target]] to a markdown link with a wiki: URL;
|
||||
// the frontend resolves the target against the file tree by basename.
|
||||
func expandWikilinks(src []byte) []byte {
|
||||
return wikiRe.ReplaceAllFunc(src, func(m []byte) []byte {
|
||||
g := wikiRe.FindSubmatch(m)
|
||||
target, label := g[1], g[2]
|
||||
if len(label) == 0 {
|
||||
label = target
|
||||
}
|
||||
return []byte("[" + string(label) + "](wiki:" + url.PathEscape(string(target)) + ")")
|
||||
})
|
||||
}
|
||||
|
||||
// RenderMarkdown converts markdown to HTML (GFM + wikilinks). Raw HTML in
|
||||
// the source is escaped by goldmark's safe default.
|
||||
func RenderMarkdown(src []byte) (string, error) {
|
||||
var buf bytes.Buffer
|
||||
if err := md.Convert(expandWikilinks(src), &buf); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return buf.String(), nil
|
||||
}
|
||||
@@ -0,0 +1,315 @@
|
||||
// Package webapp serves a read-only web view of an sfs remote: the volume's
|
||||
// file tree reconstructed from the journals, rendered markdown, and file
|
||||
// downloads. It talks straight to the object store — no local volume state,
|
||||
// mount, or daemon is needed.
|
||||
package webapp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"embed"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/fs"
|
||||
"maps"
|
||||
"mime"
|
||||
"net/http"
|
||||
"path"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/runbear-io/sfs/internal/journal"
|
||||
"github.com/runbear-io/sfs/internal/remote"
|
||||
)
|
||||
|
||||
//go:embed static
|
||||
var staticFiles embed.FS
|
||||
|
||||
// Source supplies the file set and content the server renders. Two
|
||||
// implementations: RemoteSource (an sfs remote, the normal mode) and
|
||||
// DirSource (a plain local folder, for debugging without any remote).
|
||||
type Source interface {
|
||||
Files(ctx context.Context) (map[string]FileInfo, error)
|
||||
Open(ctx context.Context, path string, fi FileInfo) (io.ReadCloser, error)
|
||||
}
|
||||
|
||||
// Server renders one source as a website. File listings are cached for
|
||||
// Refresh between fetches; if the source becomes unreachable, the last good
|
||||
// snapshot keeps being served.
|
||||
type Server struct {
|
||||
Source Source
|
||||
Remote string // display only
|
||||
Volume string // display only
|
||||
Refresh time.Duration
|
||||
|
||||
mu sync.Mutex
|
||||
snap *snapshot
|
||||
at time.Time
|
||||
}
|
||||
|
||||
// FileInfo is the resolved state of one path: content identity (Blob doubles
|
||||
// as the ETag), plus provenance where the source knows it.
|
||||
type FileInfo struct {
|
||||
Blob string
|
||||
Size int64
|
||||
Time time.Time
|
||||
Author string
|
||||
Device string
|
||||
}
|
||||
|
||||
type snapshot struct {
|
||||
files map[string]FileInfo
|
||||
}
|
||||
|
||||
func (s *Server) snapshot(ctx context.Context) (*snapshot, error) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if s.snap != nil && time.Since(s.at) < s.Refresh {
|
||||
return s.snap, nil
|
||||
}
|
||||
files, err := s.Source.Files(ctx)
|
||||
if err != nil {
|
||||
if s.snap != nil {
|
||||
return s.snap, nil // serve stale rather than fail
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
s.snap, s.at = &snapshot{files: files}, time.Now()
|
||||
return s.snap, nil
|
||||
}
|
||||
|
||||
// RemoteSource reads an sfs remote: it fetches every journal and folds the
|
||||
// ops into the current volume state (same total order as journal.Replay,
|
||||
// but keeping author/device/time of the winning op per path).
|
||||
type RemoteSource struct {
|
||||
Backend remote.Backend
|
||||
}
|
||||
|
||||
func (r *RemoteSource) Files(ctx context.Context) (map[string]FileInfo, error) {
|
||||
objs, err := r.Backend.List(ctx, "journal/")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list journals: %w", err)
|
||||
}
|
||||
var all []journal.Op
|
||||
for _, o := range objs {
|
||||
if !strings.HasSuffix(o.Key, ".jsonl") {
|
||||
continue
|
||||
}
|
||||
rc, err := r.Backend.Get(ctx, o.Key)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("fetch %s: %w", o.Key, err)
|
||||
}
|
||||
data, err := io.ReadAll(rc)
|
||||
rc.Close()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ops, err := journal.Parse(data)
|
||||
if err != nil {
|
||||
continue // corrupt journal; ignore rather than break the view
|
||||
}
|
||||
all = append(all, ops...)
|
||||
}
|
||||
journal.Sort(all)
|
||||
files := make(map[string]FileInfo)
|
||||
for _, op := range all {
|
||||
switch op.Kind {
|
||||
case journal.KindPut:
|
||||
files[op.Path] = FileInfo{
|
||||
Blob: op.Blob, Size: op.Size, Time: op.Time,
|
||||
Author: op.Author, Device: op.DeviceName,
|
||||
}
|
||||
case journal.KindDelete:
|
||||
delete(files, op.Path)
|
||||
}
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func (r *RemoteSource) Open(ctx context.Context, _ string, fi FileInfo) (io.ReadCloser, error) {
|
||||
return r.Backend.Get(ctx, "blobs/"+fi.Blob)
|
||||
}
|
||||
|
||||
// Handler returns the HTTP handler: /api/* plus the embedded frontend.
|
||||
func (s *Server) Handler() http.Handler {
|
||||
static, err := fs.Sub(staticFiles, "static")
|
||||
if err != nil {
|
||||
panic(err) // embedded FS; cannot fail at runtime
|
||||
}
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("GET /api/volume", s.handleVolume)
|
||||
mux.HandleFunc("GET /api/tree", s.handleTree)
|
||||
mux.HandleFunc("GET /api/file", s.handleFile)
|
||||
mux.HandleFunc("GET /api/download", s.handleDownload)
|
||||
mux.HandleFunc("GET /api/render", s.handleRender)
|
||||
mux.Handle("GET /", http.FileServerFS(static))
|
||||
return mux
|
||||
}
|
||||
|
||||
func (s *Server) handleVolume(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, map[string]string{"volume": s.Volume, "remote": s.Remote})
|
||||
}
|
||||
|
||||
// Node is one entry of the file tree returned by /api/tree.
|
||||
type Node struct {
|
||||
Name string `json:"name"`
|
||||
Path string `json:"path"`
|
||||
Dir bool `json:"dir"`
|
||||
Size int64 `json:"size,omitempty"`
|
||||
Time time.Time `json:"time,omitzero"`
|
||||
Author string `json:"author,omitempty"`
|
||||
Device string `json:"device,omitempty"`
|
||||
Children []*Node `json:"children,omitempty"`
|
||||
}
|
||||
|
||||
func (s *Server) handleTree(w http.ResponseWriter, r *http.Request) {
|
||||
snap, err := s.snapshot(r.Context())
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusBadGateway)
|
||||
return
|
||||
}
|
||||
writeJSON(w, buildTree(snap.files))
|
||||
}
|
||||
|
||||
func buildTree(files map[string]FileInfo) *Node {
|
||||
root := &Node{Name: "/", Dir: true}
|
||||
dirs := map[string]*Node{"": root}
|
||||
for _, p := range slices.Sorted(maps.Keys(files)) {
|
||||
fi := files[p]
|
||||
parent := root
|
||||
segs := strings.Split(p, "/")
|
||||
for i := 0; i < len(segs)-1; i++ {
|
||||
dp := strings.Join(segs[:i+1], "/")
|
||||
n, ok := dirs[dp]
|
||||
if !ok {
|
||||
n = &Node{Name: segs[i], Path: dp, Dir: true}
|
||||
dirs[dp] = n
|
||||
parent.Children = append(parent.Children, n)
|
||||
}
|
||||
parent = n
|
||||
}
|
||||
parent.Children = append(parent.Children, &Node{
|
||||
Name: segs[len(segs)-1], Path: p,
|
||||
Size: fi.Size, Time: fi.Time, Author: fi.Author, Device: fi.Device,
|
||||
})
|
||||
}
|
||||
sortTree(root)
|
||||
return root
|
||||
}
|
||||
|
||||
func sortTree(n *Node) {
|
||||
sort.SliceStable(n.Children, func(i, j int) bool {
|
||||
a, b := n.Children[i], n.Children[j]
|
||||
if a.Dir != b.Dir {
|
||||
return a.Dir // folders first, like Obsidian
|
||||
}
|
||||
return strings.ToLower(a.Name) < strings.ToLower(b.Name)
|
||||
})
|
||||
for _, c := range n.Children {
|
||||
if c.Dir {
|
||||
sortTree(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// lookup resolves ?path= against the current snapshot.
|
||||
func (s *Server) lookup(r *http.Request) (string, FileInfo, int, error) {
|
||||
p := r.URL.Query().Get("path")
|
||||
if p == "" {
|
||||
return "", FileInfo{}, http.StatusBadRequest, fmt.Errorf("missing ?path=")
|
||||
}
|
||||
snap, err := s.snapshot(r.Context())
|
||||
if err != nil {
|
||||
return "", FileInfo{}, http.StatusBadGateway, err
|
||||
}
|
||||
fi, ok := snap.files[p]
|
||||
if !ok {
|
||||
return "", FileInfo{}, http.StatusNotFound, fmt.Errorf("no such file: %s", p)
|
||||
}
|
||||
return p, fi, 0, nil
|
||||
}
|
||||
|
||||
func (s *Server) serveBlob(w http.ResponseWriter, r *http.Request, attach bool) {
|
||||
p, fi, code, err := s.lookup(r)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), code)
|
||||
return
|
||||
}
|
||||
etag := `"` + fi.Blob + `"`
|
||||
if r.Header.Get("If-None-Match") == etag {
|
||||
w.WriteHeader(http.StatusNotModified)
|
||||
return
|
||||
}
|
||||
rc, err := s.Source.Open(r.Context(), p, fi)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("fetch content: %v", err), http.StatusBadGateway)
|
||||
return
|
||||
}
|
||||
defer rc.Close()
|
||||
w.Header().Set("ETag", etag)
|
||||
w.Header().Set("Content-Type", contentType(p))
|
||||
w.Header().Set("Content-Length", fmt.Sprint(fi.Size))
|
||||
if attach {
|
||||
w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%q", path.Base(p)))
|
||||
}
|
||||
io.Copy(w, rc)
|
||||
}
|
||||
|
||||
func (s *Server) handleFile(w http.ResponseWriter, r *http.Request) {
|
||||
s.serveBlob(w, r, false)
|
||||
}
|
||||
|
||||
func (s *Server) handleDownload(w http.ResponseWriter, r *http.Request) {
|
||||
s.serveBlob(w, r, true)
|
||||
}
|
||||
|
||||
func (s *Server) handleRender(w http.ResponseWriter, r *http.Request) {
|
||||
p, fi, code, err := s.lookup(r)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), code)
|
||||
return
|
||||
}
|
||||
rc, err := s.Source.Open(r.Context(), p, fi)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("fetch content: %v", err), http.StatusBadGateway)
|
||||
return
|
||||
}
|
||||
src, err := io.ReadAll(rc)
|
||||
rc.Close()
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusBadGateway)
|
||||
return
|
||||
}
|
||||
html, err := RenderMarkdown(src)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("render: %v", err), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
writeJSON(w, map[string]any{
|
||||
"path": p, "html": html,
|
||||
"size": fi.Size, "time": fi.Time, "author": fi.Author, "device": fi.Device,
|
||||
})
|
||||
}
|
||||
|
||||
func contentType(p string) string {
|
||||
switch strings.ToLower(path.Ext(p)) {
|
||||
case ".md", ".markdown":
|
||||
return "text/markdown; charset=utf-8"
|
||||
case ".txt", ".log", ".go", ".py", ".js", ".ts", ".sh", ".yaml", ".yml", ".toml", ".csv":
|
||||
return "text/plain; charset=utf-8"
|
||||
case ".json":
|
||||
return "application/json"
|
||||
}
|
||||
if t := mime.TypeByExtension(path.Ext(p)); t != "" {
|
||||
return t
|
||||
}
|
||||
return "application/octet-stream"
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, v any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(v)
|
||||
}
|
||||
@@ -0,0 +1,213 @@
|
||||
package webapp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/runbear-io/sfs/internal/journal"
|
||||
"github.com/runbear-io/sfs/internal/remote"
|
||||
)
|
||||
|
||||
// fakeRemote builds an sfs remote layout (journal/<dev>.jsonl + blobs/<sha>)
|
||||
// in a temp dir and returns a Server over it.
|
||||
type fakeRemote struct {
|
||||
t *testing.T
|
||||
dir string
|
||||
seq map[string]int64
|
||||
lam int64
|
||||
}
|
||||
|
||||
func newFakeRemote(t *testing.T) *fakeRemote {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
for _, d := range []string{"journal", "blobs"} {
|
||||
if err := os.MkdirAll(filepath.Join(dir, d), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
return &fakeRemote{t: t, dir: dir, seq: map[string]int64{}}
|
||||
}
|
||||
|
||||
func (f *fakeRemote) put(dev, path, content string) {
|
||||
f.t.Helper()
|
||||
sum := sha256.Sum256([]byte(content))
|
||||
blob := hex.EncodeToString(sum[:])
|
||||
if err := os.WriteFile(filepath.Join(f.dir, "blobs", blob), []byte(content), 0o644); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
f.append(dev, journal.Op{
|
||||
Kind: journal.KindPut, Path: path,
|
||||
Blob: blob, Size: int64(len(content)), Mode: 0o644,
|
||||
})
|
||||
}
|
||||
|
||||
func (f *fakeRemote) del(dev, path string) {
|
||||
f.t.Helper()
|
||||
f.append(dev, journal.Op{Kind: journal.KindDelete, Path: path})
|
||||
}
|
||||
|
||||
func (f *fakeRemote) append(dev string, op journal.Op) {
|
||||
f.t.Helper()
|
||||
f.lam++
|
||||
f.seq[dev]++
|
||||
op.Seq, op.Lamport = f.seq[dev], f.lam
|
||||
op.Time = time.Now().UTC()
|
||||
op.Device, op.DeviceName, op.Author = dev, dev, dev+"@test"
|
||||
if err := journal.Append(filepath.Join(f.dir, "journal", dev+".jsonl"), []journal.Op{op}); err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func (f *fakeRemote) server() *Server {
|
||||
f.t.Helper()
|
||||
be, err := remote.Open(context.Background(), "file://"+f.dir)
|
||||
if err != nil {
|
||||
f.t.Fatal(err)
|
||||
}
|
||||
f.t.Cleanup(func() { be.Close() })
|
||||
return &Server{Source: &RemoteSource{Backend: be}, Remote: "file://" + f.dir, Volume: "testvol", Refresh: 0}
|
||||
}
|
||||
|
||||
func get(t *testing.T, h http.Handler, url string) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest("GET", url, nil))
|
||||
return rec
|
||||
}
|
||||
|
||||
func TestTreeAndFile(t *testing.T) {
|
||||
f := newFakeRemote(t)
|
||||
f.put("deva", "readme.md", "# Hello")
|
||||
f.put("deva", "notes/plan.md", "- step one")
|
||||
f.put("devb", "notes/img.png", "not-really-a-png")
|
||||
h := f.server().Handler()
|
||||
|
||||
rec := get(t, h, "/api/tree")
|
||||
if rec.Code != 200 {
|
||||
t.Fatalf("tree: %d %s", rec.Code, rec.Body)
|
||||
}
|
||||
var root Node
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &root); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(root.Children) != 2 {
|
||||
t.Fatalf("root children = %d, want 2 (notes/, readme.md)", len(root.Children))
|
||||
}
|
||||
if !root.Children[0].Dir || root.Children[0].Name != "notes" {
|
||||
t.Fatalf("first child = %+v, want dir notes (folders first)", root.Children[0])
|
||||
}
|
||||
if got := len(root.Children[0].Children); got != 2 {
|
||||
t.Fatalf("notes/ children = %d, want 2", got)
|
||||
}
|
||||
|
||||
rec = get(t, h, "/api/file?path=readme.md")
|
||||
if rec.Code != 200 || rec.Body.String() != "# Hello" {
|
||||
t.Fatalf("file: %d %q", rec.Code, rec.Body)
|
||||
}
|
||||
if ct := rec.Header().Get("Content-Type"); !strings.HasPrefix(ct, "text/markdown") {
|
||||
t.Fatalf("content-type = %q", ct)
|
||||
}
|
||||
etag := rec.Header().Get("ETag")
|
||||
if etag == "" {
|
||||
t.Fatal("no ETag")
|
||||
}
|
||||
req := httptest.NewRequest("GET", "/api/file?path=readme.md", nil)
|
||||
req.Header.Set("If-None-Match", etag)
|
||||
rec = httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusNotModified {
|
||||
t.Fatalf("etag revalidate: %d, want 304", rec.Code)
|
||||
}
|
||||
|
||||
if rec := get(t, h, "/api/file?path=nope.md"); rec.Code != 404 {
|
||||
t.Fatalf("missing file: %d, want 404", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteHidesFile(t *testing.T) {
|
||||
f := newFakeRemote(t)
|
||||
f.put("deva", "a.md", "a")
|
||||
f.put("deva", "b.md", "b")
|
||||
f.del("devb", "a.md")
|
||||
h := f.server().Handler()
|
||||
|
||||
var root Node
|
||||
if err := json.Unmarshal(get(t, h, "/api/tree").Body.Bytes(), &root); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(root.Children) != 1 || root.Children[0].Name != "b.md" {
|
||||
t.Fatalf("tree after delete = %+v, want only b.md", root.Children)
|
||||
}
|
||||
if rec := get(t, h, "/api/file?path=a.md"); rec.Code != 404 {
|
||||
t.Fatalf("deleted file: %d, want 404", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderMarkdown(t *testing.T) {
|
||||
f := newFakeRemote(t)
|
||||
f.put("deva", "doc.md", "# Title\n\nsee [[plan]] and [[plan|the plan]]\n\n<script>x</script>")
|
||||
h := f.server().Handler()
|
||||
|
||||
rec := get(t, h, "/api/render?path=doc.md")
|
||||
if rec.Code != 200 {
|
||||
t.Fatalf("render: %d %s", rec.Code, rec.Body)
|
||||
}
|
||||
var doc struct {
|
||||
HTML string `json:"html"`
|
||||
Author string `json:"author"`
|
||||
}
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &doc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, want := range []string{"<h1", "Title", `href="wiki:plan"`, ">the plan</a>"} {
|
||||
if !strings.Contains(doc.HTML, want) {
|
||||
t.Errorf("html missing %q:\n%s", want, doc.HTML)
|
||||
}
|
||||
}
|
||||
if strings.Contains(doc.HTML, "<script>") {
|
||||
t.Errorf("raw HTML not escaped:\n%s", doc.HTML)
|
||||
}
|
||||
if doc.Author != "deva@test" {
|
||||
t.Errorf("author = %q", doc.Author)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDownload(t *testing.T) {
|
||||
f := newFakeRemote(t)
|
||||
f.put("deva", "notes/plan.md", "content")
|
||||
h := f.server().Handler()
|
||||
|
||||
rec := get(t, h, "/api/download?path=notes/plan.md")
|
||||
if rec.Code != 200 || rec.Body.String() != "content" {
|
||||
t.Fatalf("download: %d %q", rec.Code, rec.Body)
|
||||
}
|
||||
if cd := rec.Header().Get("Content-Disposition"); cd != `attachment; filename="plan.md"` {
|
||||
t.Fatalf("content-disposition = %q", cd)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFrontendServed(t *testing.T) {
|
||||
f := newFakeRemote(t)
|
||||
h := f.server().Handler()
|
||||
rec := get(t, h, "/")
|
||||
if rec.Code != 200 || !strings.Contains(rec.Body.String(), "<title>sfs</title>") {
|
||||
t.Fatalf("index: %d", rec.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExpandWikilinks(t *testing.T) {
|
||||
got := string(expandWikilinks([]byte("a [[x y]] b [[u|v]] c [[no")))
|
||||
want := "a [x y](wiki:x%20y) b [v](wiki:u) c [[no"
|
||||
if got != want {
|
||||
t.Errorf("got %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,188 @@
|
||||
/* sfs web viewer: file tree + obsidian-like markdown pane. No dependencies. */
|
||||
"use strict";
|
||||
|
||||
const $ = (id) => document.getElementById(id);
|
||||
let flatFiles = []; // [{path, name}] for wikilink resolution
|
||||
let currentPath = null;
|
||||
let collapsed = new Set(); // dir paths the user collapsed
|
||||
|
||||
const MD_EXT = /\.(md|markdown)$/i;
|
||||
const IMG_EXT = /\.(png|jpe?g|gif|svg|webp|ico|bmp|avif)$/i;
|
||||
const TEXT_EXT = /\.(txt|log|json|ya?ml|toml|csv|go|py|js|ts|jsx|tsx|sh|bash|zsh|rb|rs|c|h|cpp|java|kt|swift|sql|html|css|xml|ini|conf|env|mod|sum|jsonl)$/i;
|
||||
|
||||
const fileURL = (p) => "api/file?path=" + encodeURIComponent(p);
|
||||
|
||||
async function getJSON(url) {
|
||||
const r = await fetch(url);
|
||||
if (!r.ok) throw new Error(await r.text());
|
||||
return r.json();
|
||||
}
|
||||
|
||||
/* ---- boot ---- */
|
||||
async function boot() {
|
||||
try {
|
||||
const v = await getJSON("api/volume");
|
||||
$("vault-name").textContent = v.volume || "sfs";
|
||||
$("vault-remote").textContent = v.remote || "";
|
||||
document.title = (v.volume || "sfs") + " — sfs";
|
||||
} catch { /* non-fatal */ }
|
||||
await refreshTree();
|
||||
const p = decodeURIComponent(location.hash.slice(1));
|
||||
if (p) openFile(p);
|
||||
setInterval(refreshTree, 15000); // pick up synced changes
|
||||
}
|
||||
|
||||
/* ---- tree ---- */
|
||||
async function refreshTree() {
|
||||
let root;
|
||||
try {
|
||||
root = await getJSON("api/tree");
|
||||
} catch { return; } // keep the last good tree
|
||||
flatFiles = [];
|
||||
const nav = $("tree");
|
||||
nav.innerHTML = "";
|
||||
nav.appendChild(renderChildren(root.children || []));
|
||||
markActive();
|
||||
}
|
||||
|
||||
function renderChildren(children) {
|
||||
const ul = document.createElement("ul");
|
||||
for (const n of children) ul.appendChild(renderNode(n));
|
||||
return ul;
|
||||
}
|
||||
|
||||
function renderNode(n) {
|
||||
const li = document.createElement("li");
|
||||
li.className = n.dir ? "dir" : "file";
|
||||
const row = document.createElement("div");
|
||||
row.className = "row";
|
||||
row.dataset.path = n.path;
|
||||
const chev = document.createElement("span");
|
||||
chev.className = "chev";
|
||||
chev.textContent = "▾"; // ▾
|
||||
const label = document.createElement("span");
|
||||
label.textContent = n.name;
|
||||
row.append(chev, label);
|
||||
li.appendChild(row);
|
||||
if (n.dir) {
|
||||
li.appendChild(renderChildren(n.children || []));
|
||||
if (collapsed.has(n.path)) li.classList.add("collapsed");
|
||||
row.onclick = () => {
|
||||
li.classList.toggle("collapsed");
|
||||
li.classList.contains("collapsed") ? collapsed.add(n.path) : collapsed.delete(n.path);
|
||||
};
|
||||
} else {
|
||||
flatFiles.push({ path: n.path, name: n.name });
|
||||
row.onclick = () => openFile(n.path);
|
||||
}
|
||||
return li;
|
||||
}
|
||||
|
||||
function markActive() {
|
||||
for (const el of document.querySelectorAll("#tree .row.active")) el.classList.remove("active");
|
||||
if (!currentPath) return;
|
||||
const row = document.querySelector(`#tree .row[data-path="${CSS.escape(currentPath)}"]`);
|
||||
if (row) row.classList.add("active");
|
||||
}
|
||||
|
||||
/* ---- file pane ---- */
|
||||
async function openFile(p) {
|
||||
currentPath = p;
|
||||
location.hash = encodeURIComponent(p);
|
||||
markActive();
|
||||
$("crumb").textContent = p.split("/").join(" / ");
|
||||
const dl = $("download");
|
||||
dl.href = "api/download?path=" + encodeURIComponent(p);
|
||||
dl.hidden = false;
|
||||
const content = $("content");
|
||||
content.innerHTML = "";
|
||||
$("meta").textContent = "";
|
||||
try {
|
||||
if (MD_EXT.test(p)) {
|
||||
const doc = await getJSON("api/render?path=" + encodeURIComponent(p));
|
||||
content.innerHTML = doc.html;
|
||||
fixLinks(content, p);
|
||||
showMeta(doc);
|
||||
} else if (IMG_EXT.test(p)) {
|
||||
const img = document.createElement("img");
|
||||
img.src = fileURL(p);
|
||||
img.alt = p;
|
||||
content.appendChild(img);
|
||||
} else if (TEXT_EXT.test(p)) {
|
||||
const r = await fetch(fileURL(p));
|
||||
if (!r.ok) throw new Error(await r.text());
|
||||
const pre = document.createElement("pre");
|
||||
pre.className = "plain";
|
||||
pre.textContent = await r.text();
|
||||
content.appendChild(pre);
|
||||
} else {
|
||||
content.innerHTML =
|
||||
`<div class="filecard"><div class="name"></div>` +
|
||||
`<p>No preview for this file type.</p>` +
|
||||
`<a class="btn" download href="${dl.href}">Download</a></div>`;
|
||||
content.querySelector(".name").textContent = p.split("/").pop();
|
||||
}
|
||||
} catch (err) {
|
||||
content.innerHTML = `<div class="empty"></div>`;
|
||||
content.querySelector(".empty").textContent = "Could not load file: " + err.message;
|
||||
}
|
||||
}
|
||||
|
||||
function showMeta(doc) {
|
||||
const parts = [];
|
||||
if (doc.author) parts.push(doc.author + (doc.device ? " on " + doc.device : ""));
|
||||
if (doc.time) parts.push(new Date(doc.time).toLocaleString());
|
||||
$("meta").textContent = parts.join(" · ");
|
||||
}
|
||||
|
||||
/* Rewrite rendered-markdown links: wiki: targets resolve by basename, and
|
||||
relative links/images resolve against the current file's folder. */
|
||||
function fixLinks(scope, p) {
|
||||
const dir = p.includes("/") ? p.slice(0, p.lastIndexOf("/")) : "";
|
||||
for (const img of scope.querySelectorAll("img")) {
|
||||
const src = img.getAttribute("src") || "";
|
||||
if (!/^([a-z]+:|\/)/i.test(src)) img.src = fileURL(join(dir, src));
|
||||
}
|
||||
for (const a of scope.querySelectorAll("a")) {
|
||||
const href = a.getAttribute("href") || "";
|
||||
if (href.startsWith("wiki:")) {
|
||||
const target = decodeURIComponent(href.slice(5));
|
||||
a.onclick = (e) => { e.preventDefault(); openWikilink(target); };
|
||||
} else if (!/^([a-z]+:|\/|#)/i.test(href)) {
|
||||
const target = join(dir, decodeURIComponent(href));
|
||||
a.onclick = (e) => { e.preventDefault(); openFile(target); };
|
||||
} else if (/^https?:/i.test(href)) {
|
||||
a.target = "_blank";
|
||||
a.rel = "noopener";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function openWikilink(target) {
|
||||
const want = target.toLowerCase();
|
||||
const hit =
|
||||
flatFiles.find((f) => f.path.toLowerCase() === want || f.path.toLowerCase() === want + ".md") ||
|
||||
flatFiles.find((f) => {
|
||||
const n = f.name.toLowerCase();
|
||||
return n === want || n === want + ".md";
|
||||
});
|
||||
if (hit) openFile(hit.path);
|
||||
}
|
||||
|
||||
function join(dir, rel) {
|
||||
const parts = (dir ? dir.split("/") : []).concat(rel.split("/"));
|
||||
const out = [];
|
||||
for (const s of parts) {
|
||||
if (s === "" || s === ".") continue;
|
||||
if (s === "..") out.pop();
|
||||
else out.push(s);
|
||||
}
|
||||
return out.join("/");
|
||||
}
|
||||
|
||||
window.addEventListener("hashchange", () => {
|
||||
const p = decodeURIComponent(location.hash.slice(1));
|
||||
if (p && p !== currentPath) openFile(p);
|
||||
});
|
||||
|
||||
boot();
|
||||
@@ -0,0 +1,30 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>sfs</title>
|
||||
<link rel="stylesheet" href="style.css">
|
||||
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 100 100'><text y='.9em' font-size='90'>📁</text></svg>">
|
||||
</head>
|
||||
<body>
|
||||
<aside id="sidebar">
|
||||
<header id="vault">
|
||||
<span id="vault-name">…</span>
|
||||
<span id="vault-remote"></span>
|
||||
</header>
|
||||
<nav id="tree" aria-label="Files"></nav>
|
||||
</aside>
|
||||
<main id="main">
|
||||
<header id="topbar">
|
||||
<span id="crumb"></span>
|
||||
<span id="meta"></span>
|
||||
<a id="download" class="btn" hidden download>Download</a>
|
||||
</header>
|
||||
<article id="content" class="markdown">
|
||||
<div class="empty">Select a file from the sidebar</div>
|
||||
</article>
|
||||
</main>
|
||||
<script src="app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,149 @@
|
||||
/* Obsidian-inspired dark theme */
|
||||
:root {
|
||||
--bg: #1e1e1e;
|
||||
--bg-side: #262626;
|
||||
--bg-hover: #333333;
|
||||
--bg-active: #3f3550;
|
||||
--border: #363636;
|
||||
--text: #dadada;
|
||||
--text-dim: #9a9a9a;
|
||||
--text-faint: #6e6e6e;
|
||||
--accent: #a882ff;
|
||||
--accent-dim: #7c5cd6;
|
||||
--code-bg: #2a2a2a;
|
||||
}
|
||||
|
||||
* { box-sizing: border-box; }
|
||||
html, body { height: 100%; margin: 0; }
|
||||
body {
|
||||
display: flex;
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
font: 15px/1.6 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
|
||||
}
|
||||
|
||||
/* ---- sidebar ---- */
|
||||
#sidebar {
|
||||
width: 280px;
|
||||
min-width: 200px;
|
||||
background: var(--bg-side);
|
||||
border-right: 1px solid var(--border);
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
overflow: hidden;
|
||||
}
|
||||
#vault {
|
||||
padding: 14px 16px 10px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
#vault-name { display: block; font-weight: 600; font-size: 14px; }
|
||||
#vault-remote {
|
||||
display: block;
|
||||
font-size: 11px;
|
||||
color: var(--text-faint);
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
#tree { flex: 1; overflow-y: auto; padding: 8px 6px 24px; font-size: 13.5px; }
|
||||
#tree ul { list-style: none; margin: 0; padding-left: 14px; }
|
||||
#tree > ul { padding-left: 4px; }
|
||||
#tree li > .row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 5px;
|
||||
padding: 2px 8px;
|
||||
border-radius: 5px;
|
||||
cursor: pointer;
|
||||
color: var(--text-dim);
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
#tree li > .row:hover { background: var(--bg-hover); color: var(--text); }
|
||||
#tree li > .row.active { background: var(--bg-active); color: var(--accent); }
|
||||
#tree .chev { width: 12px; flex: none; font-size: 10px; color: var(--text-faint); transition: transform .12s; }
|
||||
#tree li.collapsed > ul { display: none; }
|
||||
#tree li.collapsed > .row .chev { transform: rotate(-90deg); }
|
||||
#tree .file .chev { visibility: hidden; }
|
||||
|
||||
/* ---- main pane ---- */
|
||||
#main { flex: 1; display: flex; flex-direction: column; min-width: 0; }
|
||||
#topbar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
padding: 10px 24px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
min-height: 46px;
|
||||
}
|
||||
#crumb { font-size: 13px; color: var(--text-dim); }
|
||||
#meta { flex: 1; font-size: 12px; color: var(--text-faint); text-align: right; }
|
||||
.btn {
|
||||
flex: none;
|
||||
padding: 4px 12px;
|
||||
border-radius: 6px;
|
||||
background: var(--accent-dim);
|
||||
color: #fff;
|
||||
font-size: 12.5px;
|
||||
text-decoration: none;
|
||||
}
|
||||
.btn:hover { background: var(--accent); }
|
||||
#content { flex: 1; overflow-y: auto; padding: 28px 48px 80px; }
|
||||
.empty { color: var(--text-faint); text-align: center; margin-top: 20vh; }
|
||||
|
||||
/* ---- markdown ---- */
|
||||
.markdown { max-width: 820px; }
|
||||
.markdown h1, .markdown h2, .markdown h3, .markdown h4 {
|
||||
color: #f0f0f0;
|
||||
line-height: 1.3;
|
||||
margin: 1.4em 0 .5em;
|
||||
}
|
||||
.markdown h1:first-child { margin-top: 0; }
|
||||
.markdown h1 { font-size: 1.7em; }
|
||||
.markdown h2 { font-size: 1.35em; border-bottom: 1px solid var(--border); padding-bottom: .25em; }
|
||||
.markdown a { color: var(--accent); text-decoration: none; }
|
||||
.markdown a:hover { text-decoration: underline; }
|
||||
.markdown code {
|
||||
background: var(--code-bg);
|
||||
padding: .15em .4em;
|
||||
border-radius: 4px;
|
||||
font: 12.5px/1.5 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
|
||||
}
|
||||
.markdown pre {
|
||||
background: var(--code-bg);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 14px 16px;
|
||||
overflow-x: auto;
|
||||
}
|
||||
.markdown pre code { background: none; padding: 0; }
|
||||
.markdown blockquote {
|
||||
margin: 1em 0;
|
||||
padding: .1em 1em;
|
||||
border-left: 3px solid var(--accent-dim);
|
||||
color: var(--text-dim);
|
||||
}
|
||||
.markdown table { border-collapse: collapse; margin: 1em 0; }
|
||||
.markdown th, .markdown td { border: 1px solid var(--border); padding: 6px 12px; }
|
||||
.markdown th { background: var(--bg-side); }
|
||||
.markdown img { max-width: 100%; border-radius: 6px; }
|
||||
.markdown hr { border: none; border-top: 1px solid var(--border); margin: 2em 0; }
|
||||
.markdown input[type="checkbox"] { accent-color: var(--accent); }
|
||||
|
||||
/* plain file / binary views */
|
||||
pre.plain {
|
||||
background: var(--code-bg);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 14px 16px;
|
||||
overflow-x: auto;
|
||||
font: 12.5px/1.55 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
|
||||
white-space: pre-wrap;
|
||||
}
|
||||
.filecard {
|
||||
margin-top: 15vh;
|
||||
text-align: center;
|
||||
color: var(--text-dim);
|
||||
}
|
||||
.filecard .name { font-size: 1.2em; color: var(--text); margin-bottom: .3em; }
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"name": "sfs",
|
||||
"displayName": "sfs",
|
||||
"description": "Synced file system for AI agents: mount folders that stay in sync across devices through S3/GCS/any object store, with automatic sync at turn boundaries and full change history.",
|
||||
"version": "0.1.0",
|
||||
"author": { "name": "runbear", "url": "https://github.com/runbear-io" },
|
||||
"homepage": "https://github.com/runbear-io/sfs",
|
||||
"repository": "https://github.com/runbear-io/sfs",
|
||||
"license": "MIT",
|
||||
"keywords": ["sync", "files", "workspace", "s3", "gcs", "memory", "agents"]
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
description: Mount a folder as a synced sfs volume — one command sets up the sync daemon, the .sfs project config, and (via this plugin's hooks) automatic sync at every turn boundary
|
||||
argument-hint: [folder] [remote e.g. s3://bucket/prefix]
|
||||
---
|
||||
|
||||
Mount a folder as a synced sfs volume. Arguments: `$ARGUMENTS` (optional folder, optional remote URL).
|
||||
|
||||
Follow these steps:
|
||||
|
||||
1. **Check sfs is installed**: run `command -v sfs`. If missing, offer to install it (`brew install runbear-io/tap/sfs`, or `go install github.com/runbear-io/sfs/cmd/sfs@latest`) and wait for the user's choice before installing.
|
||||
|
||||
2. **Determine the folder**: first argument if given, otherwise the current directory. If the folder already contains a `.sfs` file, the volume and remote are already configured — just run `sfs mnt <folder>` and skip step 3.
|
||||
|
||||
3. **Determine the remote**: second argument if given. If not given, ask the user which backend they want:
|
||||
- `s3://bucket/prefix` — Amazon S3 or any S3-compatible store (R2/MinIO via `AWS_ENDPOINT_URL`)
|
||||
- `gs://bucket/prefix` — Google Cloud Storage
|
||||
- `file:///abs/path` — a plain shared directory (NAS, external drive)
|
||||
- none — local-only for now (`sfs remote set` can add one later)
|
||||
|
||||
4. **Mount**: run `sfs mnt <folder> [--remote <url>]`. This registers the background sync daemon and writes the folder's settings to `<folder>/.sfs`.
|
||||
|
||||
5. **Verify**: run `sfs status <folder>` and show the result. If the remote errored, consult the sfs skill's troubleshooting table (credentials are the usual cause).
|
||||
|
||||
6. **Tell the user what's now active** (briefly):
|
||||
- the daemon syncs continuously (every few seconds);
|
||||
- this plugin's hooks also sync at every turn boundary — a blocking pull when they send a message, an async push when the turn ends — so Claude always works on fresh files;
|
||||
- `.sfsignore` in the folder root excludes files (gitignore-style); an `"include"` list in `.sfs` narrows what syncs;
|
||||
- `sfs log` shows who changed what, `sfs umnt` stops syncing.
|
||||
@@ -0,0 +1,13 @@
|
||||
---
|
||||
description: Show sfs sync status — mounts, daemon state, pending changes — and diagnose any sync problems
|
||||
argument-hint: [folder]
|
||||
---
|
||||
|
||||
Show the sfs sync status. Argument: `$ARGUMENTS` (optional folder; default all mounts).
|
||||
|
||||
1. Run `sfs status $ARGUMENTS` and show the output.
|
||||
2. If anything looks wrong, diagnose using the sfs skill:
|
||||
- `daemon: stopped` → restart with `sfs mnt <folder>`
|
||||
- `pending` stuck above 0 → run `sfs sync <folder>` and read the error; it usually points at credentials or the remote
|
||||
- changes not appearing from another device → run `sfs log <folder>` to see whether the ops arrived
|
||||
3. Summarize the state in one or two sentences.
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"hooks": {
|
||||
"UserPromptSubmit": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"${CLAUDE_PLUGIN_ROOT}/scripts/sfs-sync.sh\"",
|
||||
"timeout": 30,
|
||||
"statusMessage": "sfs: pulling latest files"
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"Stop": [
|
||||
{
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"${CLAUDE_PLUGIN_ROOT}/scripts/sfs-sync.sh\"",
|
||||
"async": true
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
Executable
+10
@@ -0,0 +1,10 @@
|
||||
#!/bin/sh
|
||||
# Sync the current project if it is an sfs mount (has a .sfs settings file).
|
||||
# Fast no-op otherwise, so this hook is safe on every turn in every project.
|
||||
#
|
||||
# Runs blocking on UserPromptSubmit (fresh files before Claude reads them)
|
||||
# and async on Stop (push edits out without delaying the turn).
|
||||
cd "${CLAUDE_PROJECT_DIR:-.}" || exit 0
|
||||
[ -f .sfs ] || exit 0
|
||||
command -v sfs >/dev/null 2>&1 || exit 0
|
||||
sfs sync . >/dev/null 2>&1 || true
|
||||
@@ -26,6 +26,36 @@ Use this skill whenever the user is working with the `sfs` CLI: mounting, unmoun
|
||||
|
||||
`<folder>` is created if missing. Omitting it on `sync`/`status`/`log` defaults to the current working directory.
|
||||
|
||||
## Project files
|
||||
|
||||
Two files at the mount root control a folder's sync behavior:
|
||||
|
||||
- **`.sfs`** — the folder's settings (JSON): `volume`, `remote`, and optional `include`. Written by `sfs mnt` / `sfs remote set`; safe to hand-edit (a running daemon picks changes up on its next tick). It is **never synced** — remotes are device-specific — and it travels with the folder: copy a folder containing `.sfs` to a new machine and plain `sfs mnt <folder>` reuses its volume and remote.
|
||||
- **`.sfsignore`** — opt-out list, gitignore-style. **Syncs like a normal file**, so all devices share the same rules. Syntax subset: `#` comments, `*` within a segment, `**` across segments, `?`, trailing `/` for directories-only, a `/` elsewhere anchors to the mount root, `!` re-includes.
|
||||
|
||||
```jsonc
|
||||
// .sfs
|
||||
{
|
||||
"volume": "agent-workspace",
|
||||
"remote": "s3://acme-sfs/agent-workspace",
|
||||
"include": ["docs/", "notes/", "*.md"] // optional: sync ONLY these
|
||||
}
|
||||
```
|
||||
|
||||
```gitignore
|
||||
# .sfsignore
|
||||
*.log
|
||||
node_modules/
|
||||
build/
|
||||
!build/keep.txt
|
||||
```
|
||||
|
||||
Selective-sync semantics — important when advising users:
|
||||
|
||||
- A path syncs when it is **not ignored** and (if `include` is non-empty) **matches an include pattern**. Ignore beats include.
|
||||
- Adding a pattern for an already-synced file makes this device **stop tracking it without deleting it anywhere** — the file stays on disk locally and on every other device. Deleting it locally after that does not propagate either.
|
||||
- Because `.sfsignore` syncs, adding a rule on one device applies it everywhere on the next cycle.
|
||||
|
||||
---
|
||||
|
||||
## 1. Mount / unmount / sync
|
||||
@@ -35,7 +65,7 @@ Use this skill whenever the user is working with the `sfs` CLI: mounting, unmoun
|
||||
1. Pick a folder. New empty or with existing files — existing files are imported on the first cycle.
|
||||
2. Decide on a remote (optional at mount time; configurable later via `sfs remote set`).
|
||||
3. Run `sfs mnt`. sfs:
|
||||
- registers the folder in `~/.sfs/mounts.json`,
|
||||
- writes the folder's settings to `<folder>/.sfs` and registers it in `~/.sfs/mounts.json`,
|
||||
- opens/creates the volume under `~/.sfs/volumes/<volume>/`,
|
||||
- runs an initial cycle (import locals; pull remote state if a remote is set),
|
||||
- starts a background daemon (unless `-f`).
|
||||
@@ -94,7 +124,7 @@ sfs umnt ./notes --forget
|
||||
|
||||
### What sfs does not sync
|
||||
|
||||
`.git` directories, `.DS_Store`, sfs's own temp files, and empty directories. Don't suggest mounting a folder where `.git` is the content the user expects synced — they want git, not sfs.
|
||||
`.git` directories, `.DS_Store`, the `.sfs` settings file, sfs's own temp files, empty directories, and anything excluded by `.sfsignore` or left out of an `include` list. Don't suggest mounting a folder where `.git` is the content the user expects synced — they want git, not sfs.
|
||||
|
||||
---
|
||||
|
||||
@@ -320,14 +350,11 @@ To change name/author, edit `~/.sfs/device.json` and restart the daemon (`sfs um
|
||||
### The per-mount daemon log
|
||||
|
||||
```sh
|
||||
# Volume contents
|
||||
# Volume contents (daemon pid + log files live here, one pair per mount)
|
||||
ls ~/.sfs/volumes/<volume>/
|
||||
|
||||
# Daemon state + log for each mount of this volume
|
||||
ls ~/.sfs/volumes/<volume>/daemons/
|
||||
|
||||
# Tail
|
||||
tail -F ~/.sfs/volumes/<volume>/daemons/*.log
|
||||
tail -F ~/.sfs/volumes/<volume>/daemon-*.log
|
||||
```
|
||||
|
||||
Useful when `pending` is stuck > 0, the daemon flips to `stopped` after a restart, or you changed credentials and want to confirm uptake.
|
||||
@@ -338,7 +365,7 @@ Useful when `pending` is stuck > 0, the daemon flips to `stopped` after a restar
|
||||
2. `daemon: stopped` → `sfs mnt <folder>` to restart it.
|
||||
3. `pending` stuck → `sfs sync <folder>` and read the cycle output. Errors here point at the remote — see the cloud-storage troubleshooting table above.
|
||||
4. Sync succeeds but the other device doesn't see changes → `sfs sync` on the other device + `sfs log` to confirm the op crossed over.
|
||||
5. Daemon keeps dying → tail `~/.sfs/volumes/<volume>/daemons/*.log` for the cause.
|
||||
5. Daemon keeps dying → tail `~/.sfs/volumes/<volume>/daemon-*.log` for the cause.
|
||||
|
||||
---
|
||||
|
||||
@@ -351,9 +378,9 @@ Useful when `pending` is stuck > 0, the daemon flips to `stopped` after a restar
|
||||
└── volumes/<volume>/
|
||||
├── blobs/ # content-addressed file content
|
||||
├── journal/ # per-device append-only op logs
|
||||
├── state.json # what's currently materialized
|
||||
├── state-<mountID>.json # what's currently materialized (per mount)
|
||||
├── sync.json # lamport clock + push cursor
|
||||
└── daemons/ # one pid+log file per mount of this volume
|
||||
└── daemon-<mountID>.pid/.log # daemon state + log, per mount
|
||||
```
|
||||
|
||||
Don't suggest editing files under `volumes/` directly — sfs owns them. `device.json` and `mounts.json` are safe to inspect; `mounts.json` is safe to hand-edit if a mount entry needs surgery, but prefer `sfs umnt --forget` then `sfs mnt`.
|
||||
Reference in New Issue
Block a user