From dcd0517e92e4606a5841dd00d98616ceeea90f43 Mon Sep 17 00:00:00 2001 From: "Snow W. Lee (Sungwon)" Date: Wed, 29 Jul 2026 17:10:51 +0900 Subject: [PATCH] =?UTF-8?q?feat(cli):=20bdrive=20scope=20--explain=20?= =?UTF-8?q?=E2=80=94=20prove=20what=20leaves=20this=20machine=20(BEA-24)?= =?UTF-8?q?=20(#70)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The local-first claim was asserted, never demonstrated: nothing anywhere told you what your laptop chose *not* to send. `bdrive scope --explain` walks the folder and prints two sorted lists — synced and not synced — with counts and a pointer at what it does not answer. The decisions come from the same walk the sync cycle uses. scan()'s WalkDir decision tree moves into walkFolder (internal/syncer/walk.go), the only copy of the rules; scan and Explain both go through it, so the output provably cannot drift from real sync behavior. Pure read: its own Filter, no Session, no volume flock, no network. Fully-excluded directories collapse to one counted line; nested mounts are annotated as syncing through their own project rather than called "not synced", which would be a lie in a trust surface. Known gap, deliberate: this answers "what leaves from now on", not "what is already on the hub" — the footer points at `bdrive forget`. Co-authored-by: Claude Opus 5 (1M context) --- README.md | 2 +- architecture/cli-sync.md | 24 ++- cmd/bdrive/scope.go | 58 ++++++- internal/syncer/explain.go | 148 +++++++++++++++++ internal/syncer/explain_test.go | 172 ++++++++++++++++++++ internal/syncer/syncer.go | 23 +-- internal/syncer/walk.go | 63 +++++++ plugin/skills/beardrive/SKILL.md | 1 + web/docs/src/content/docs/guides/scoping.md | 50 ++++++ web/docs/src/content/docs/reference/cli.md | 1 + 10 files changed, 518 insertions(+), 24 deletions(-) create mode 100644 internal/syncer/explain.go create mode 100644 internal/syncer/explain_test.go create mode 100644 internal/syncer/walk.go diff --git a/README.md b/README.md index abec2d9..e8e4ff6 100644 --- a/README.md +++ b/README.md @@ -143,7 +143,7 @@ hub's own storage, never something a syncing client points at directly: | `bdrive logout` | Sign this device out — clear the saved token/account (`--forget` also drops the remembered server) | | `bdrive init [folder]` | Create/connect a project and start syncing — the mount is always exactly the folder named. Interactive on a TTY, flags (`--name/--project/--server/--only/--yes`) for scripts; installs the agent skill, registers agent sync hooks in each platform's user config (`--no-hooks` skips the hooks), prints the project link; re-run to resume | | `bdrive stop [folder]` | Stop syncing, including agent sync hooks (files stay; `bdrive init` resumes) | -| `bdrive scope [add\|rm ]` | Show or change which subfolders sync — edits the managed block of `.bdriveignore` rules that `init --only` writes, so no one hand-writes negation syntax. The daemon picks changes up in seconds; `rm` deletes nothing, locally or on the hub | +| `bdrive scope [add\|rm ]` | Show or change which subfolders sync — edits the managed block of `.bdriveignore` rules that `init --only` writes, so no one hand-writes negation syntax. The daemon picks changes up in seconds; `rm` deletes nothing, locally or on the hub. `--explain` lists every path in the folder split into what syncs and what does not, so you can verify what leaves this machine (pure read — no daemon, no lock, no network) | | `bdrive forget ...` | Stop syncing a path *and* remove it from the hub — adds the rule to `.bdriveignore` (which syncs) and prunes in one step. Local files are never touched, here or on teammates' devices | | `bdrive url [path]` | Internal hub link for a file/folder (sign-in + membership required; `--sync` pushes first; no arg = project home). Computed locally | | `bdrive share ` | Public URL for a synced file (`--list`, `--revoke`, `--expires`) | diff --git a/architecture/cli-sync.md b/architecture/cli-sync.md index af2b5dc..1a18188 100644 --- a/architecture/cli-sync.md +++ b/architecture/cli-sync.md @@ -38,7 +38,24 @@ classDiagram +Skip(rel) bool +PruneDir(rel) bool } - note for Filter "ignore.go — .bdriveignore rules (incl. the managed `# bdrive scope` negation block written by init --only / bdrive scope) + a legacy .bdrive include list, applied symmetrically in scan and materialize; Negated() is what makes sync --prune refuse on a scoped project" + note for Filter "ignore.go — .bdriveignore rules (incl. the managed `# bdrive scope` negation block written by init --only / bdrive scope) + a legacy .bdrive include list, applied symmetrically in scan and materialize; Negated() is what makes sync --prune refuse on a scoped project. NOT the whole predicate: walkFolder adds .git/.bdrive pruning, non-regular and .DS_Store/.bdrive-tmp-* skips, and nested-mount handoff" + + class walkFolder { + +walkFolder(folder, filter, fn) + verdict: vSync vSkipFile vDescend vPruneDir vNested + } + note for walkFolder "walk.go — the ONLY copy of the sync predicate; scan and Explain both go through it, so what --explain reports cannot drift from what leaves" + + class Explain { + +Explain(folder, include) two lists + +NotSyncedFiles(entries) int + } + class Entry { + +Path string + +Files int + +Nested bool + } + note for Explain "explain.go — bdrive scope --explain. Pure read: own Filter, no Session, no flock, no network. Collapses fully-excluded dirs to one counted line; nested mounts annotated, counted as zero (they sync via their own project)" class Store { -dir volume dir @@ -76,6 +93,11 @@ classDiagram Session --> Store : volume state Session --> Backend : pull and push Session --> Filter : scan and materialize + Session --> walkFolder : scan + Explain --> walkFolder : same predicate + Explain --> Filter : own fresh instance + Explain ..> Entry : not-synced lines + walkFolder --> Filter : Skip / PruneDir / addNestedMount Session ..> Op : commits, replays Session --> Result Store o-- Op : journal files diff --git a/cmd/bdrive/scope.go b/cmd/bdrive/scope.go index 517884a..32c25db 100644 --- a/cmd/bdrive/scope.go +++ b/cmd/bdrive/scope.go @@ -4,11 +4,13 @@ import ( "fmt" "os" "path/filepath" + "strconv" "strings" "github.com/spf13/cobra" "github.com/runbear-io/beardrive/internal/config" + "github.com/runbear-io/beardrive/internal/syncer" ) // bdrive scope shows and edits which of the mount's subfolders sync. The @@ -17,6 +19,7 @@ import ( // reason to hand-write the negation syntax. The daemon re-reads the rules // every tick, so changes apply within seconds. func scopeCmd() *cobra.Command { + var explain bool c := &cobra.Command{ Use: "scope", Short: "Show or change which subfolders sync", @@ -26,10 +29,15 @@ The whole mount syncs by default. Narrowing it writes a managed block of .bdriveignore rules ("only these folders"), which syncs to the team like any other rule — so everyone sees the same scope. Run from the mount root. +--explain walks the folder and prints every path it found, split into what +syncs and what does not, so you can verify what leaves this machine instead +of taking it on trust. It is a pure read: no daemon, no lock, no network. + Removing a folder stops syncing it but deletes nothing — local files stay, and the hub keeps everything already synced. To take something off the hub too, use ` + "`bdrive forget `" + `.`, Example: ` bdrive scope # show what syncs + bdrive scope --explain # list every path: synced vs not synced bdrive scope add docs # also sync ./docs bdrive scope rm docs # stop syncing ./docs (files stay everywhere)`, Args: cobra.NoArgs, @@ -42,13 +50,61 @@ too, use ` + "`bdrive forget `" + `.`, if err != nil { return err } - return printScope(folder, proj) + if err := printScope(folder, proj); err != nil { + return err + } + if !explain { + return nil + } + synced, notSynced, err := syncer.Explain(folder, proj.Include) + if err != nil { + return err + } + printExplain(synced, notSynced) + return nil }, } + // Flags, not PersistentFlags: add/rm must not inherit --explain. + c.Flags().BoolVar(&explain, "explain", false, "list every path, split into what syncs and what does not") c.AddCommand(scopeAddCmd(), scopeRmCmd()) return c } +// printExplain renders the two lists. All the filtering rules live in +// internal/syncer — this only formats what the walk decided. +func printExplain(synced []string, notSynced []syncer.Entry) { + fmt.Printf("\nsynced (%s)\n", comma(len(synced))) + for _, p := range synced { + fmt.Println(" " + p) + } + fmt.Printf("\nnot synced (%s)\n", comma(syncer.NotSyncedFiles(notSynced))) + for _, e := range notSynced { + note := "" + switch { + case e.Nested: + note = "(own project — syncs separately)" + case e.IsDir(): + note = fmt.Sprintf("(%s files)", comma(e.Files)) + if e.Files == 1 { + note = "(1 file)" + } + } + fmt.Println(strings.TrimRight(fmt.Sprintf(" %-30s %s", e.Path, note), " ")) + } + fmt.Printf("\n%s files sync, %s do not.\n\n", comma(len(synced)), comma(syncer.NotSyncedFiles(notSynced))) + fmt.Println("Excluded paths never leave this machine. Anything that synced before its rule") + fmt.Println("existed may still be on the hub — `bdrive forget ` takes it off.") +} + +// comma groups thousands: 2486 -> "2,486". +func comma(n int) string { + s := strconv.Itoa(n) + for i := len(s) - 3; i > 0; i -= 3 { + s = s[:i] + "," + s[i:] + } + return s +} + func scopeAddCmd() *cobra.Command { return &cobra.Command{ Use: "add ...", diff --git a/internal/syncer/explain.go b/internal/syncer/explain.go new file mode 100644 index 0000000..128aebb --- /dev/null +++ b/internal/syncer/explain.go @@ -0,0 +1,148 @@ +package syncer + +import ( + "io/fs" + "path" + "path/filepath" + "sort" + "strings" +) + +// Entry is one line of the not-synced list. Path ends in "/" when a whole +// directory collapsed to a single line; Files is how many files it holds. +type Entry struct { + Path string + Files int + Nested bool // syncs through its own project — not excluded +} + +// IsDir reports whether the entry stands for a whole directory rather than a +// single file. +func (e Entry) IsDir() bool { return strings.HasSuffix(e.Path, "/") } + +// Explain reports what the sync cycle would and would not send for a folder. +// It is a pure read: no Session, no volume lock, no network, no writes — the +// answer comes from the same walk the cycle itself uses, so it cannot drift. +func Explain(folder string, include []string) (synced []string, notSynced []Entry, err error) { + // A fresh filter: addNestedMount mutates it during the walk, so this must + // never be shared with a live cycle. + filter, err := loadFilter(folder, include) + if err != nil { + return nil, nil, err + } + + var skipped []string + dirs := map[string]*Entry{} + var dirOrder []string // walk order: parents before children + keep := map[string]bool{} // dir holds something that must stay visible + count := map[string]int{} // files under a dir that do not sync + + err = walkFolder(folder, filter, func(abs, rel string, d fs.DirEntry, v verdict) error { + switch v { + case vSync: + synced = append(synced, rel) + for _, a := range ancestors(rel) { + keep[a] = true + } + case vSkipFile: + skipped = append(skipped, rel) + for _, a := range ancestors(rel) { + count[a]++ + } + case vDescend: + dirs[rel] = &Entry{Path: rel + "/"} + dirOrder = append(dirOrder, rel) + case vPruneDir: + n := countFiles(abs) + dirs[rel] = &Entry{Path: rel + "/", Files: n} + dirOrder = append(dirOrder, rel) + for _, a := range ancestors(rel) { + count[a] += n + } + case vNested: + // Not excluded — it syncs through its own project, so its files + // are not counted as "do not sync" and its parents stay visible + // rather than collapsing the annotation away. + dirs[rel] = &Entry{Path: rel + "/", Nested: true} + dirOrder = append(dirOrder, rel) + for _, a := range ancestors(rel) { + keep[a] = true + } + } + return nil + }) + if err != nil { + return nil, nil, err + } + + // Collapse: a directory with nothing to show individually prints as one + // counted line, and everything under it is dropped. Parents come first in + // walk order, so the topmost such directory wins. + collapsed := map[string]bool{} + for _, rel := range dirOrder { + if keep[rel] || underCollapsed(rel, collapsed) { + continue + } + collapsed[rel] = true + e := dirs[rel] + if !e.Nested && e.Files == 0 { + e.Files = count[rel] + } + notSynced = append(notSynced, *e) + } + for _, rel := range skipped { + if !underCollapsed(rel, collapsed) { + notSynced = append(notSynced, Entry{Path: rel}) + } + } + + sort.Strings(synced) + sort.Slice(notSynced, func(i, j int) bool { return notSynced[i].Path < notSynced[j].Path }) + return synced, notSynced, nil +} + +// NotSyncedFiles is how many files the not-synced list stands for: collapsed +// directories count their whole subtree, nested mounts count zero because +// they do sync — through their own project. +func NotSyncedFiles(notSynced []Entry) int { + n := 0 + for _, e := range notSynced { + if e.IsDir() { + n += e.Files + } else { + n++ + } + } + return n +} + +func ancestors(rel string) []string { + var out []string + for d := path.Dir(rel); d != "."; d = path.Dir(d) { + out = append(out, d) + } + return out +} + +func underCollapsed(rel string, collapsed map[string]bool) bool { + for _, a := range ancestors(rel) { + if collapsed[a] { + return true + } + } + return false +} + +// countFiles counts the files under an already-excluded directory. Readdir +// only — never Stat — because this runs over trees like .git and +// node_modules; a partial count on an unreadable subtree is fine. +func countFiles(abs string) int { + n := 0 + filepath.WalkDir(abs, func(_ string, d fs.DirEntry, err error) error { + if err == nil && !d.IsDir() { + n++ + } + return nil + }) + return n +} diff --git a/internal/syncer/explain_test.go b/internal/syncer/explain_test.go new file mode 100644 index 0000000..3f27167 --- /dev/null +++ b/internal/syncer/explain_test.go @@ -0,0 +1,172 @@ +package syncer + +import ( + "io/fs" + "os" + "path/filepath" + "reflect" + "sort" + "testing" + + "github.com/runbear-io/beardrive/internal/journal" +) + +// TestExplainMatchesScan is the whole point of `bdrive scope --explain`: the +// paths it calls "synced" must be exactly the paths the sync cycle journals, +// and exactly the paths that show up on a peer device. If these ever diverge +// the command is lying in a trust surface, so this asserts the identity from +// both ends for a whole-folder project and for a scoped one. +func TestExplainMatchesScan(t *testing.T) { + for _, tc := range []struct { + name string + include []string + setup func(t *testing.T, folder string) + }{ + { + name: "whole folder", + setup: func(t *testing.T, folder string) { + write(t, folder, IgnoreFile, "node_modules/\n*.log\n") + }, + }, + { + // PruneDir refuses to prune when an include list exists, so the + // walk descends node_modules in full — this is the case the + // collapse pass has to handle. + name: "scoped", + include: []string{"/docs/"}, + setup: func(t *testing.T, folder string) { + write(t, folder, ".bdrive/config.json", `{"include":["/docs/"]}`) + write(t, folder, IgnoreFile, "*.log\n") + }, + }, + } { + t.Run(tc.name, func(t *testing.T) { + be := sharedRemote(t) + a := newDevice(t, "deva", be) + b := newDevice(t, "devb", be) + tc.setup(t, a.Folder) + + write(t, a.Folder, "docs/guide.md", "synced") + write(t, a.Folder, "docs/deep/spec.md", "synced too") + write(t, a.Folder, "src/main.go", "maybe") + write(t, a.Folder, "scratch/notes.md", "maybe") + write(t, a.Folder, "debug.log", "never") + write(t, a.Folder, ".DS_Store", "never") + write(t, a.Folder, ".bdrive-tmp-x", "never") + write(t, a.Folder, ".git/HEAD", "never") + for i := 0; i < 12; i++ { + write(t, a.Folder, filepath.Join("node_modules/pkg", "f"+string(rune('a'+i))+".js"), "never") + } + write(t, a.Folder, "vendor/acme/.bdrive/config.json", `{"mount_id":"m-nested"}`) + write(t, a.Folder, "vendor/acme/inner.md", "own project") + if err := os.Symlink(filepath.Join(a.Folder, "docs/guide.md"), filepath.Join(a.Folder, "docs/link.md")); err != nil { + t.Fatal(err) + } + + synced, notSynced, err := Explain(a.Folder, tc.include) + if err != nil { + t.Fatal(err) + } + + // 1. explain's synced set == the paths scan journals. + cycle(t, a) + ops, err := a.Store.DeviceOps(a.Device.ID) + if err != nil { + t.Fatal(err) + } + var put []string + for _, op := range ops { + if op.Kind == journal.KindPut { + put = append(put, op.Path) + } + } + sort.Strings(put) + if !reflect.DeepEqual(synced, put) { + t.Fatalf("explain synced != journaled puts\n explain: %v\n scan: %v", synced, put) + } + + // 2. and == what actually landed on a second device. + cycle(t, b) + var got []string + filepath.WalkDir(b.Folder, func(p string, d fs.DirEntry, err error) error { + if err != nil || d.IsDir() { + return nil + } + rel, _ := filepath.Rel(b.Folder, p) + got = append(got, filepath.ToSlash(rel)) + return nil + }) + sort.Strings(got) + if !reflect.DeepEqual(synced, got) { + t.Fatalf("explain synced != files on peer\n explain: %v\n peer: %v", synced, got) + } + + // 3. the nested mount is annotated, never a plain exclusion. + nested := find(notSynced, "vendor/acme/") + if nested == nil { + t.Fatalf("nested mount missing from not-synced: %v", notSynced) + } + if !nested.Nested { + t.Fatal("nested mount must be marked Nested, not listed as excluded") + } + + // 4. node_modules collapses to one counted line. + nm := find(notSynced, "node_modules/") + if nm == nil || nm.Files != 12 { + t.Fatalf("node_modules should be one entry with 12 files, got %+v (all: %v)", nm, notSynced) + } + if len(notSynced) > 12 { + t.Fatalf("not-synced list should stay small, got %d entries: %v", len(notSynced), notSynced) + } + + // the builtin exclusions are visible — that is the reassurance + for _, want := range []string{".git/", ".DS_Store", ".bdrive-tmp-x", "docs/link.md"} { + if find(notSynced, want) == nil { + t.Fatalf("%s missing from not-synced: %v", want, notSynced) + } + } + + // 5. byte-stable across runs. + synced2, notSynced2, err := Explain(a.Folder, tc.include) + if err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(synced, synced2) || !reflect.DeepEqual(notSynced, notSynced2) { + t.Fatal("Explain output is not stable across runs") + } + }) + } +} + +// TestExplainScopedCollapsesUnsharedDirs pins the scoped-project shape: a +// directory outside the include list prints as one counted line rather than +// every file under it. +func TestExplainScopedCollapsesUnsharedDirs(t *testing.T) { + a := newDevice(t, "deva", nil) + write(t, a.Folder, ".bdrive/config.json", `{"include":["/docs/"]}`) + write(t, a.Folder, "docs/guide.md", "synced") + for i := 0; i < 5; i++ { + write(t, a.Folder, filepath.Join("private/deep", "f"+string(rune('a'+i))+".txt"), "no") + } + + _, notSynced, err := Explain(a.Folder, []string{"/docs/"}) + if err != nil { + t.Fatal(err) + } + e := find(notSynced, "private/") + if e == nil || e.Files != 5 { + t.Fatalf("private/ should collapse to one line with 5 files, got %+v (all: %v)", e, notSynced) + } + if n := NotSyncedFiles(notSynced); n < 5 { + t.Fatalf("NotSyncedFiles = %d, should count the collapsed subtree", n) + } +} + +func find(entries []Entry, path string) *Entry { + for i, e := range entries { + if e.Path == path { + return &entries[i] + } + } + return nil +} diff --git a/internal/syncer/syncer.go b/internal/syncer/syncer.go index ab246ff..26060d7 100644 --- a/internal/syncer/syncer.go +++ b/internal/syncer/syncer.go @@ -333,27 +333,8 @@ func (s *Session) scan(cache map[string]store.CachedFile, st *store.SyncState, s } } - err := filepath.WalkDir(s.Folder, func(p string, d fs.DirEntry, walkErr error) error { - if walkErr != nil { - return nil // skip unreadable entries - } - rel, err := filepath.Rel(s.Folder, p) - if err != nil || rel == "." { - return nil - } - rel = filepath.ToSlash(rel) - if d.IsDir() { - if ignoreDirs[d.Name()] || filter.PruneDir(rel) { - return fs.SkipDir - } - if config.IsMount(p) { - // A mount of its own: it syncs through its own project. - filter.addNestedMount(rel) - return fs.SkipDir - } - return nil - } - if !d.Type().IsRegular() || ignoredFile(d.Name()) || filter.Skip(rel) { + err := walkFolder(s.Folder, filter, func(p, rel string, d fs.DirEntry, v verdict) error { + if v != vSync { return nil } info, err := d.Info() diff --git a/internal/syncer/walk.go b/internal/syncer/walk.go new file mode 100644 index 0000000..f909aa0 --- /dev/null +++ b/internal/syncer/walk.go @@ -0,0 +1,63 @@ +package syncer + +import ( + "io/fs" + "path/filepath" + + "github.com/runbear-io/beardrive/internal/config" +) + +// verdict is what the sync predicate decided about one entry on disk. +type verdict int + +const ( + vDescend verdict = iota // directory, walk into it + vPruneDir // directory excluded whole (.git, .bdrive, PruneDir) + vNested // directory is a mount of its own (syncs separately) + vSkipFile // file excluded (non-regular, .DS_Store/.bdrive-tmp-*, Skip) + vSync // file syncs +) + +// walkFolder walks a mount applying the exact predicate the sync cycle uses. +// fn sees every entry with its verdict; pruning happens here, so no caller can +// descend where scan would not. This is the only copy of the rules: scan and +// Explain both go through it, so what `bdrive scope --explain` reports cannot +// drift from what actually leaves the machine. +func walkFolder(folder string, filter *Filter, fn func(abs, rel string, d fs.DirEntry, v verdict) error) error { + return filepath.WalkDir(folder, func(p string, d fs.DirEntry, walkErr error) error { + if walkErr != nil { + return nil // skip unreadable entries + } + rel, err := filepath.Rel(folder, p) + if err != nil || rel == "." { + return nil + } + rel = filepath.ToSlash(rel) + + var v verdict + switch { + case !d.IsDir(): + if !d.Type().IsRegular() || ignoredFile(d.Name()) || filter.Skip(rel) { + v = vSkipFile + } else { + v = vSync + } + case ignoreDirs[d.Name()] || filter.PruneDir(rel): + v = vPruneDir + case config.IsMount(p): + // A mount of its own: it syncs through its own project. + filter.addNestedMount(rel) + v = vNested + default: + v = vDescend + } + + if err := fn(p, rel, d, v); err != nil { + return err + } + if v == vPruneDir || v == vNested { + return fs.SkipDir + } + return nil + }) +} diff --git a/plugin/skills/beardrive/SKILL.md b/plugin/skills/beardrive/SKILL.md index 9418697..ce4619d 100644 --- a/plugin/skills/beardrive/SKILL.md +++ b/plugin/skills/beardrive/SKILL.md @@ -17,6 +17,7 @@ Use this skill whenever the user is working with the `bdrive` CLI: initializing | Run the daemon in the foreground | `bdrive init -f` | | Stop syncing | `bdrive stop []` — pauses daemon *and* agent hooks; `bdrive init` resumes (`--forget` also unregisters) | | Show/change which subfolders sync | `bdrive scope` / `bdrive scope add ` / `bdrive scope rm ` — edits the managed block of `.bdriveignore` rules that `init --only` writes (run from the mount root; never hand-write the negation syntax). The daemon applies it within seconds. `rm` stops syncing a folder but deletes nothing, locally or on the hub; removing the last entry is refused (that would flip to whole-folder sync — use `bdrive stop` instead) | +| Verify what actually leaves this machine | `bdrive scope --explain` — walks the folder and prints two sorted lists, `synced` and `not synced`, with counts; fully-excluded directories collapse to one counted line and nested mounts are marked as syncing through their own project. Pure read: safe with the daemon running and offline, takes no lock, makes no network call. Answers "what leaves from now on", NOT "what is already on the hub" — that's `bdrive forget` below. Output is stable, so `bdrive scope --explain > before.txt` + `diff` is a real audit | | One sync cycle now | `bdrive sync []` — `--note ` stamps session context; `--prune` also removes from the hub whatever `.bdriveignore` now excludes (refuses when `.bdriveignore` narrows the scope with `!` rules — see below); `--hook