feat(cli): bdrive grep — search the text inside the files a project syncs (BEA-99) (#136)

The ⌘K palette searches file names, projects and actions; nothing in the
product searched file contents. Three personas independently typed a phrase
that lives inside a synced file and got "No matches".

`bdrive grep <pattern> [folder]` searches the working folder — RE2 or -F
literal, -i, -l, -n (default 200, 0 = all), output `path:line: text`, exit 0
on match and 1 on none.

It searches exactly what the project syncs, via a new syncer.SyncedFiles that
wraps the existing walkFolder: the one copy of the sync predicate, so an
ignore rule or a narrowed scope excludes a file from search the same way it
excludes it from sync, and .bdrive/ state can never surface. Not Explain,
which countFiles every pruned dir — a grep in a repo with node_modules/ would
walk it in full for a count it discards.

A read stays a read: LoadProject, not ResolveMount (no registry self-heal, no
device enrollment), no session, no flock, and the volume store is opened for
IgnoreAccepted only when it already exists, so a search creates nothing.
Both the path and the matched line go through safeField — a matched line is a
teammate's file content, the widest version of the surface that function
exists for.

The hub-side content index stays deliberately unbuilt; the issue records its
cost. ROADMAP's "Search across the hub" line is reworded rather than removed.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Snow Lee (Sungwon)
2026-08-11 04:27:20 +09:00
committed by GitHub
co-authored by Claude Opus 5
parent 5f1ac98dae
commit 594a027c15
9 changed files with 758 additions and 4 deletions
+1
View File
@@ -234,6 +234,7 @@ hub's own storage, never something a syncing client points at directly:
| `bdrive autostart [install\|uninstall]` | Show, add, or remove the login registration that runs `bdrive resume` after a reboot — a launchd user agent on macOS, a systemd user unit on Linux. `bdrive init` installs it; `--no-autostart` skips it |
| `bdrive stop [folder]` | Stop syncing, including agent sync hooks (files stay; `bdrive init` resumes) |
| `bdrive scope [add\|rm <dirs...>]` | Show or change which subfolders sync — edits the managed block of `.bdriveignore` rules that `init --only` writes, so no one hand-writes negation syntax. The daemon picks changes up in seconds; `rm` deletes nothing, locally or on the hub. `--explain` lists every path in the folder split into what syncs and what does not, so you can verify what leaves this machine (pure read — no daemon, no lock, no network) |
| `bdrive grep <pattern> [folder]` | Search the text **inside** the files a project syncs — Go RE2 regexp, or a literal with `-F`; `-i` ignores case, `-l` prints matching paths only, `-n` caps the lines printed (default 200, `0` = all). Output is `path:line: text`. Only files the project actually syncs are searched, so a `.bdriveignore` rule or a narrowed `bdrive scope` excludes a file from search exactly as it excludes it from sync; binary files are skipped. Pure local read — no daemon, no lock, no network, works offline and never blocks a sync in progress. Exit status 0 on match, 1 on none, so it composes in scripts |
| `bdrive forget <path>...` | Stop syncing a path *and* remove it from the hub — adds the rule to `.bdriveignore` (which syncs) and prunes in one step. Local files are never touched, here or on teammates' devices |
| `bdrive url [path]` | Internal hub link for a file/folder (sign-in + membership required; `--sync` pushes first; no arg = project home). Computed locally |
| `bdrive share <file>` | Public URL for a synced file (`--list`, `--revoke`, `--expires`) |
+5 -1
View File
@@ -27,7 +27,11 @@ are deliberately scoped for outside contributors.
- **FUSE / NFS mount mode** — lazy-loading huge volumes instead of full
materialization.
- **Search across the hub** — full-text + wikilink graph over a project.
- **Search across the hub** — full-text + wikilink graph over a project, in the
browser and across projects. Searching *contents* already works locally today
(`bdrive grep`, over the files a project syncs); what is still exploring is
doing it hub-side, which needs a content index the storage layer has no home
for yet.
- **Webhooks / notifications** — "a file changed in `reports/`" pushed to
Slack or similar (hub-side).
+11 -3
View File
@@ -82,13 +82,18 @@ classDiagram
+walkFolder(folder, filter, fn)
verdict: vSync vSkipFile vDescend vPruneDir vNested
}
note for walkFolder "walk.go — the ONLY copy of the sync predicate; scan, Explain and Measure all go through it, so what --explain reports and what init warns about cannot drift from what leaves"
note for walkFolder "walk.go — the ONLY copy of the sync predicate; scan, Explain, Measure and SyncedFiles all go through it, so what --explain reports, what init warns about and what bdrive grep searches cannot drift from what leaves"
class Measure {
+Measure(folder, include) files, bytes
}
note for Measure "walk.go — sizes what a FIRST sync would upload, for the oversized-folder warning bdrive init prints (1 GiB / 20k files). Filter-aware on purpose: a 40 GB repo whose bulk is already ignored measures as the few MB that really sync, so the warning fires on the folder nobody meant to share and stays quiet on an ordinary checkout. Advice only — an unreadable subtree is skipped, never fatal"
class SyncedFiles {
+SyncedFiles(folder, include, accepted) paths
}
note for SyncedFiles "walk.go — the mount-relative paths that sync, in walk order: what bdrive grep searches, so a .bdriveignore rule or a narrowed scope excludes a file from search exactly as it excludes it from sync. Deliberately NOT Explain, which countFiles every pruned dir — a grep in a repo with node_modules/ would walk it in full for a count it discards"
class Explain {
+Explain(folder, include, accepted) two lists
+NotSyncedFiles(entries) int
@@ -143,6 +148,8 @@ classDiagram
Session --> Filter : SkipUp on scan, Skip on materialize
Session --> walkFolder : scan
Explain --> walkFolder : same predicate
SyncedFiles --> walkFolder : same predicate
SyncedFiles --> Filter : own fresh instance
Measure --> walkFolder : same predicate
Measure --> Filter : own fresh instance
Explain --> Filter : own fresh instance
@@ -169,13 +176,14 @@ classDiagram
class Commands {
init login logout
sync stop scope forget status log
sync stop scope grep forget status log
restore url share export import
web daemon hooks read-log
resume autostart
}
note for Commands "cmd/bdrive — thin cobra layer; init is the front door (one command: login + hooks + sync + link), stop pauses"
note for Commands "Every peer-authored string status / log / whoami print goes through safeField first — a teammate's file name is attacker-controlled text landing in your terminal, and an escape sequence there rewrites the line above it. login now does PKCE on the loopback callback (no compat arm) and both its client and init's refuse to follow a redirect off the hub's origin with the device token attached"
note for Commands "grep searches file CONTENTS in the working folder via syncer.SyncedFiles — LoadProject not ResolveMount (a read must not enroll the device), no session, no flock, and the volume store is opened only if it already exists, so a search creates nothing. Exit 1 on no match is a status, not an error (errNoMatch + SilenceErrors)"
note for Commands "Every peer-authored string status / log / whoami print goes through safeField first — a teammate's file name is attacker-controlled text landing in your terminal, and an escape sequence there rewrites the line above it. grep runs BOTH the path and the matched line through it — a matched line is a teammate's file content, the widest version of that surface. login now does PKCE on the loopback callback (no compat arm) and both its client and init's refuse to follow a redirect off the hub's origin with the device token attached"
class Templates {
<<internal/templates>>
+214
View File
@@ -0,0 +1,214 @@
package main
import (
"bufio"
"bytes"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"regexp"
"github.com/spf13/cobra"
"github.com/runbear-io/beardrive/internal/config"
"github.com/runbear-io/beardrive/internal/store"
"github.com/runbear-io/beardrive/internal/syncer"
)
// errNoMatch is grep's "found nothing" exit, not a failure: the convention is
// status 1 with no message, so `bdrive grep x || echo none` composes. Real
// errors (bad pattern, not a project) still print — see grepCmd.
var errNoMatch = errors.New("no match")
// binarySniff is how much of a file is read to decide it is not text. A NUL
// byte in the first 8 KB is the rule: cheap, and the same one grep uses.
const binarySniff = 8 << 10
// maxLineScan bounds one line, so a minified bundle that syncs cannot make
// grep buffer it whole. Past this the rest of that file is skipped.
const maxLineScan = 1 << 20
func grepCmd() *cobra.Command {
var (
ignoreCase bool
fixed bool
filesOnly bool
limit int
)
c := &cobra.Command{
Use: "grep <pattern> [folder]",
Short: "Search the text inside the files this project syncs",
Long: `Search file contents in a synced folder.
pattern is a Go regular expression (RE2), or a literal string with -F. Only
the files this project actually syncs are searched: a .bdriveignore rule or a
narrowed ` + "`bdrive scope`" + ` excludes a file from search exactly as it excludes it
from sync, and binary files are skipped.
It searches the real files on disk, not the hub a pure read with no daemon,
no lock, and no network, so it works offline and never blocks on a sync in
progress. Exit status is 0 when something matched and 1 when nothing did.`,
Example: ` bdrive grep 'retention.*fold' # regexp over the whole project
bdrive grep -i -l TODO # case-insensitive, paths only
bdrive grep -F 'a[b]c' docs # literal string, inside ./docs`,
Args: cobra.RangeArgs(1, 2),
RunE: func(cmd *cobra.Command, args []string) error {
err := runGrep(cmd, args[0], args[1:], ignoreCase, fixed, filesOnly, limit)
// SilenceErrors below is for errNoMatch alone, so anything else
// has to print itself — cobra no longer will.
if err != nil && !errors.Is(err, errNoMatch) {
fmt.Fprintln(cmd.ErrOrStderr(), "Error:", err)
}
return err
},
}
// Silenced so a no-match exits 1 without printing anything, the way grep
// does — errNoMatch is a status, not a failure, and cobra would otherwise
// render it as an error and a usage block.
c.SilenceErrors = true
c.SilenceUsage = true
c.Flags().BoolVarP(&ignoreCase, "ignore-case", "i", false, "match case-insensitively")
c.Flags().BoolVarP(&fixed, "fixed-strings", "F", false, "treat the pattern as a literal string, not a regexp")
c.Flags().BoolVarP(&filesOnly, "files-with-matches", "l", false, "print matching paths only, one per line")
// -n means the same here as in `bdrive log`: max rows out.
c.Flags().IntVarP(&limit, "limit", "n", 200, "max matching lines printed (0 = all)")
return c
}
func runGrep(cmd *cobra.Command, pattern string, folderArg []string, ignoreCase, fixed, filesOnly bool, limit int) error {
if fixed {
pattern = regexp.QuoteMeta(pattern)
}
if ignoreCase {
pattern = "(?i)" + pattern
}
re, err := regexp.Compile(pattern)
if err != nil {
return fmt.Errorf("bad pattern: %w", err)
}
folder, err := absFolder(folderArg)
if err != nil {
return err
}
// LoadProject, not ResolveMount: ResolveMount self-heals the registry
// path, i.e. it enrolls this device. A read-only query must not have that
// side effect — the rule logReads follows for the same reason.
proj, found, err := config.LoadProject(folder)
if err != nil {
return err
}
if !found {
return fmt.Errorf("%s is not a beardrive project (run `bdrive init` there first)", folder)
}
// The rules this device has ACCEPTED, so results match what the cycle
// actually uploads (syncer.Filter.SkipUp). Best-effort, exactly as
// `bdrive scope --explain` does it: store.Open takes no volume flock, so
// this cannot block behind a running daemon, and a store that will not
// open degrades the answer to the live rules rather than failing a read.
//
// The Stat guard is this command's own: store.Open MkdirAlls the volume
// directory, and a search must not create one for a project that has never
// synced. No store means no accepted rules, which is what "" already says.
var accepted string
if vdir, verr := config.VolumeDir(proj.ID); verr == nil && dirExists(vdir) {
if st, serr := store.Open(vdir); serr == nil {
if sync, serr := st.LoadSync(); serr == nil {
accepted = sync.IgnoreAccepted
}
}
}
paths, err := syncer.SyncedFiles(folder, proj.Include, accepted)
if err != nil {
return err
}
out := cmd.OutOrStdout()
var lines, files int
truncated := false
for _, rel := range paths {
hits := grepFile(filepath.Join(folder, rel), re, filesOnly)
if len(hits) == 0 {
continue
}
files++
for _, h := range hits {
if limit > 0 && lines >= limit {
truncated = true
break
}
if filesOnly {
fmt.Fprintln(out, safeField(rel, 160))
} else {
// Both the path and the matched text are content a teammate
// wrote and synced — the same trust level `bdrive log` treats
// journal strings with, over a wider surface. safeField, or a
// lone CR repaints the row and U+202E reverses it.
fmt.Fprintf(out, "%s:%d: %s\n", safeField(rel, 160), h.line, safeField(h.text, 400))
}
lines++
}
if truncated {
break
}
}
if lines == 0 {
return errNoMatch
}
if !filesOnly {
fmt.Fprintf(out, "%s, %s\n", plural(files, "file"), plural(lines, "matching line"))
}
if truncated {
fmt.Fprintf(out, "output limited to %d lines — use -n 0 for all\n", limit)
}
return nil
}
func dirExists(p string) bool {
fi, err := os.Stat(p)
return err == nil && fi.IsDir()
}
// hit is one matching line: its 1-based number and its text.
type hit struct {
line int
text string
}
// grepFile returns the matching lines in one file, or nothing when the file is
// binary or unreadable — an unreadable file is skipped, never fatal, the same
// posture the scan takes. With filesOnly it stops at the first match.
func grepFile(abs string, re *regexp.Regexp, filesOnly bool) []hit {
f, err := os.Open(abs)
if err != nil {
return nil
}
defer f.Close()
head := make([]byte, binarySniff)
n, err := f.Read(head)
if err != nil && err != io.EOF {
return nil
}
head = head[:n]
if bytes.IndexByte(head, 0) >= 0 {
return nil // binary
}
var hits []hit
sc := bufio.NewScanner(io.MultiReader(bytes.NewReader(head), f))
sc.Buffer(make([]byte, 0, 64<<10), maxLineScan)
for lineno := 1; sc.Scan(); lineno++ {
if !re.Match(sc.Bytes()) {
continue
}
hits = append(hits, hit{line: lineno, text: sc.Text()})
if filesOnly {
break
}
}
return hits // sc.Err() ignored: an over-long line ends this file, not the run
}
+338
View File
@@ -0,0 +1,338 @@
package main
import (
"bytes"
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/runbear-io/beardrive/internal/config"
"github.com/runbear-io/beardrive/internal/store"
)
// grepMount builds an isolated BDRIVE_HOME with one enrolled project folder.
// Nothing here contacts a hub: grep is a local read.
func grepMount(t *testing.T) string {
t.Helper()
t.Setenv("BDRIVE_HOME", t.TempDir())
folder := t.TempDir()
folder, _ = filepath.EvalSymlinks(folder)
if _, err := config.SaveProject(folder, config.Project{
Volume: "wiki",
Remote: "https://hub.example.com/p/p-12345678",
}); err != nil {
t.Fatal(err)
}
if _, _, err := config.EnrollMount(folder); err != nil {
t.Fatal(err)
}
return folder
}
func grepWrite(t *testing.T, folder, rel, body string) {
t.Helper()
abs := filepath.Join(folder, filepath.FromSlash(rel))
if err := os.MkdirAll(filepath.Dir(abs), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(abs, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
// grepRun drives the real cobra command and returns its combined output plus
// the error it exited with (errNoMatch is grep's status-1 no-match).
func grepRun(t *testing.T, args ...string) (string, error) {
t.Helper()
c := grepCmd()
var out bytes.Buffer
c.SetOut(&out)
c.SetErr(&out)
c.SetArgs(args)
err := c.Execute()
return out.String(), err
}
func TestGrepPrintsMatchingLines(t *testing.T) {
folder := grepMount(t)
grepWrite(t, folder, "wiki/runbook.md", "intro\nthe retention fold collapses day buckets\ntail\n")
grepWrite(t, folder, "specs/reads.md", "retention folding happens at boot\n")
grepWrite(t, folder, "unrelated.md", "nothing to see\n")
out, err := grepRun(t, "retention.*fold", folder)
if err != nil {
t.Fatalf("grep: %v\n%s", err, out)
}
for _, want := range []string{
"wiki/runbook.md:2: the retention fold collapses day buckets",
"specs/reads.md:1: retention folding happens at boot",
"2 files, 2 matching lines",
} {
if !strings.Contains(out, want) {
t.Errorf("missing %q in:\n%s", want, out)
}
}
if strings.Contains(out, "unrelated.md") {
t.Errorf("non-matching file printed:\n%s", out)
}
// -l prints bare paths, one per line, and no summary.
out, err = grepRun(t, "-l", "retention", folder)
if err != nil {
t.Fatalf("grep -l: %v\n%s", err, out)
}
got := strings.Fields(out)
if len(got) != 2 || got[0] != "specs/reads.md" || got[1] != "wiki/runbook.md" {
t.Errorf("-l should print two bare paths, got %q", out)
}
}
func TestGrepFlags(t *testing.T) {
folder := grepMount(t)
grepWrite(t, folder, "notes.md", "TODO later\ntodo now\nliteral a[b]c here\n")
// Default is case-sensitive; -i widens it.
out, _ := grepRun(t, "TODO", folder)
if strings.Contains(out, "todo now") {
t.Errorf("case-sensitive match leaked:\n%s", out)
}
out, err := grepRun(t, "-i", "TODO", folder)
if err != nil || !strings.Contains(out, "todo now") {
t.Errorf("-i should match both cases: %v\n%s", err, out)
}
// -F takes the pattern literally: as a regexp a[b]c matches "abc", which
// is not in the file — only the literal text is.
if _, err := grepRun(t, "a[b]c", folder); !errors.Is(err, errNoMatch) {
t.Errorf("regexp a[b]c should not match, got %v", err)
}
out, err = grepRun(t, "-F", "a[b]c", folder)
if err != nil || !strings.Contains(out, "literal a[b]c here") {
t.Errorf("-F should match literally: %v\n%s", err, out)
}
// A bad pattern is a real error, printed, not a silent no-match.
out, err = grepRun(t, "a(", folder)
if err == nil || errors.Is(err, errNoMatch) {
t.Errorf("bad pattern should error, got %v", err)
}
if !strings.Contains(out, "bad pattern") {
t.Errorf("bad pattern should say so:\n%s", out)
}
}
func TestGrepLimit(t *testing.T) {
folder := grepMount(t)
var body strings.Builder
for i := 0; i < 500; i++ {
body.WriteString("hit\n")
}
grepWrite(t, folder, "many.md", body.String())
// Default caps at 200 and says so.
out, err := grepRun(t, "hit", folder)
if err != nil {
t.Fatal(err)
}
if n := strings.Count(out, "many.md:"); n != 200 {
t.Errorf("default limit should print 200 lines, got %d", n)
}
if !strings.Contains(out, "output limited to 200 lines") {
t.Errorf("truncation should be announced:\n%s", out[len(out)-200:])
}
// -n 0 means all.
out, err = grepRun(t, "-n", "0", "hit", folder)
if err != nil {
t.Fatal(err)
}
if n := strings.Count(out, "many.md:"); n != 500 {
t.Errorf("-n 0 should print every line, got %d", n)
}
if strings.Contains(out, "output limited") {
t.Errorf("-n 0 must not announce truncation:\n%s", out)
}
}
// Results match what the project actually syncs: the same .bdriveignore rule
// that keeps a file off the hub keeps it out of search.
func TestGrepRespectsTheSyncFilter(t *testing.T) {
folder := grepMount(t)
grepWrite(t, folder, "keep.md", "needle\n")
grepWrite(t, folder, "drafts/skip.md", "needle\n")
out, err := grepRun(t, "needle", folder)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(out, "drafts/skip.md") {
t.Fatalf("drafts/skip.md should match before the rule:\n%s", out)
}
grepWrite(t, folder, ".bdriveignore", "drafts/\n")
out, err = grepRun(t, "needle", folder)
if err != nil {
t.Fatal(err)
}
if strings.Contains(out, "drafts/skip.md") {
t.Errorf("an ignored path must not be searchable:\n%s", out)
}
if !strings.Contains(out, "keep.md") {
t.Errorf("keep.md should still match:\n%s", out)
}
}
// .bdrive/ holds the project's own settings and .bdrive-tmp-* are half-written
// state files. Neither is project content and neither may ever surface.
func TestGrepNeverSearchesBdriveState(t *testing.T) {
folder := grepMount(t)
grepWrite(t, folder, "real.md", "p-12345678\n")
grepWrite(t, folder, ".bdrive-tmp-half", "p-12345678\n")
out, err := grepRun(t, "p-12345678", folder)
if err != nil {
t.Fatal(err)
}
// .bdrive/config.json contains the remote URL, so it would match.
if strings.Contains(out, ".bdrive/") || strings.Contains(out, ".bdrive-tmp-") {
t.Errorf("bdrive state leaked into results:\n%s", out)
}
if !strings.Contains(out, "real.md") {
t.Errorf("real.md should match:\n%s", out)
}
}
func TestGrepSkipsBinaryFiles(t *testing.T) {
folder := grepMount(t)
grepWrite(t, folder, "text.md", "needle here\n")
// A NUL inside the first 8 KB is the binary rule.
grepWrite(t, folder, "blob.bin", "needle here\x00 and more\n")
// A NUL past the sniff window is not: the file reads as text.
grepWrite(t, folder, "late.bin", strings.Repeat("x\n", 6000)+"needle here\n\x00")
out, err := grepRun(t, "needle", folder)
if err != nil {
t.Fatal(err)
}
if strings.Contains(out, "blob.bin") {
t.Errorf("binary file searched:\n%s", out)
}
if !strings.Contains(out, "text.md") || !strings.Contains(out, "late.bin") {
t.Errorf("text files should match:\n%s", out)
}
}
// Every matched line is content a teammate wrote and synced — a strictly wider
// version of the surface safeField exists for. A planted file must not be able
// to repaint or reverse the operator's terminal.
func TestGrepOutputCannotRewriteTheTerminal(t *testing.T) {
folder := grepMount(t)
hostile := "needle \x1b[2J\x1b[3J\x1b[H\rgnitset\x9b31m\x7f end"
grepWrite(t, folder, "planted.md", hostile+"\n")
// The path is attacker-controlled too: a teammate chooses the file name.
grepWrite(t, folder, "name\x1b[31m.md", "needle\n")
for _, args := range [][]string{{"needle", folder}, {"-l", "needle", folder}} {
out, err := grepRun(t, args...)
if err != nil {
t.Fatalf("grep %v: %v", args, err)
}
for _, bad := range []string{"\x1b", "\r", "", "", "\x9b", "\x7f"} {
if strings.Contains(out, bad) {
t.Errorf("grep %v leaked %q into the terminal:\n%q", args, bad, out)
}
}
// One match is one line: no forged rows.
for _, line := range strings.Split(strings.TrimRight(out, "\n"), "\n") {
for _, r := range line {
if r < 0x20 && r != '\t' || r == 0x7f {
t.Errorf("control rune %q survived in %q", r, line)
}
}
}
}
}
// A read-only query must not enroll this device: LoadProject, never
// ResolveMount. Outside a project it says so, exits non-zero, and the registry
// is untouched.
func TestGrepOutsideAProjectWritesNothing(t *testing.T) {
t.Setenv("BDRIVE_HOME", t.TempDir())
folder := t.TempDir()
grepWrite(t, folder, "loose.md", "needle\n")
mounts := filepath.Join(os.Getenv("BDRIVE_HOME"), "mounts.json")
before, beforeErr := os.ReadFile(mounts)
out, err := grepRun(t, "needle", folder)
if err == nil || errors.Is(err, errNoMatch) {
t.Fatalf("should fail outside a project, got %v", err)
}
if !strings.Contains(out, "not a beardrive project") {
t.Errorf("message should name the problem:\n%s", out)
}
after, afterErr := os.ReadFile(mounts)
if (beforeErr == nil) != (afterErr == nil) || !bytes.Equal(before, after) {
t.Errorf("the registry was written by a read-only query")
}
}
// Exit codes are grep's, so `bdrive grep x || …` composes: 0 on match, 1 on
// no match with nothing printed at all.
func TestGrepExitCodes(t *testing.T) {
folder := grepMount(t)
grepWrite(t, folder, "a.md", "hay\n")
if out, err := grepRun(t, "hay", folder); err != nil {
t.Errorf("match should exit 0: %v\n%s", err, out)
}
out, err := grepRun(t, "needle", folder)
if !errors.Is(err, errNoMatch) {
t.Errorf("no match should return errNoMatch, got %v", err)
}
if out != "" {
t.Errorf("no match must print nothing, got %q", out)
}
}
// grep opens the volume store read-only for one field and never takes the
// volume flock, so a daemon mid-cycle cannot make a search hang.
func TestGrepDoesNotBlockOnTheVolumeLock(t *testing.T) {
folder := grepMount(t)
grepWrite(t, folder, "a.md", "needle\n")
proj, found, err := config.LoadProject(folder)
if err != nil || !found {
t.Fatal(err)
}
vdir, err := config.VolumeDir(proj.ID)
if err != nil {
t.Fatal(err)
}
st, err := store.Open(vdir)
if err != nil {
t.Fatal(err)
}
unlock, err := st.Lock() // stand in for a cycle in progress
if err != nil {
t.Fatal(err)
}
defer unlock()
done := make(chan string, 1)
go func() {
out, _ := grepRun(t, "needle", folder)
done <- out
}()
select {
case out := <-done:
if !strings.Contains(out, "a.md:1: needle") {
t.Errorf("grep under the lock should still find it:\n%s", out)
}
case <-time.After(5 * time.Second):
t.Fatal("grep blocked on the volume lock")
}
}
+1
View File
@@ -53,6 +53,7 @@ everything keeps working offline; changes sync when the remote is reachable.`,
urlCmd(),
stopCmd(),
scopeCmd(),
grepCmd(),
forgetCmd(),
syncCmd(),
readLogCmd(),
+125
View File
@@ -0,0 +1,125 @@
package syncer
import (
"os"
"path/filepath"
"testing"
)
// SyncedFiles is what `bdrive grep` searches, so it has to be exactly the set
// the cycle uploads — and it has to get there without descending a pruned
// directory, which is the whole reason it exists instead of Explain.
func TestSyncedFilesMatchesTheSyncSet(t *testing.T) {
a := newDevice(t, "deva", nil)
write(t, a.Folder, IgnoreFile, "node_modules/\n*.log\n")
write(t, a.Folder, "docs/guide.md", "yes")
write(t, a.Folder, "docs/deep/spec.md", "yes")
write(t, a.Folder, "README.md", "yes")
write(t, a.Folder, "debug.log", "no")
write(t, a.Folder, ".DS_Store", "no")
write(t, a.Folder, ".bdrive-tmp-x", "no")
write(t, a.Folder, ".bdrive/config.json", `{}`)
write(t, a.Folder, ".git/HEAD", "no")
write(t, a.Folder, "node_modules/pkg/index.js", "no")
// A nested mount syncs through its own project, not this one.
nested := filepath.Join(a.Folder, "sub")
write(t, a.Folder, "sub/inner.md", "own project")
if err := os.MkdirAll(filepath.Join(nested, ".bdrive"), 0o755); err != nil {
t.Fatal(err)
}
write(t, a.Folder, "sub/.bdrive/config.json", `{"id":"m-other"}`)
got, err := SyncedFiles(a.Folder, nil, "")
if err != nil {
t.Fatal(err)
}
set := map[string]bool{}
for _, p := range got {
set[p] = true
}
for _, want := range []string{"docs/guide.md", "docs/deep/spec.md", "README.md", IgnoreFile} {
if !set[want] {
t.Errorf("%s should sync, got %v", want, got)
}
}
for _, never := range []string{
"debug.log", ".DS_Store", ".bdrive-tmp-x", ".bdrive/config.json",
".git/HEAD", "node_modules/pkg/index.js", "sub/inner.md",
} {
if set[never] {
t.Errorf("%s must not be listed, got %v", never, got)
}
}
// The same answer Explain gives, since both go through walkFolder — if
// these ever disagree, `bdrive grep` and `bdrive scope --explain` are
// telling the operator two different stories about one folder.
synced, _, err := Explain(a.Folder, nil, "")
if err != nil {
t.Fatal(err)
}
if len(synced) != len(got) {
t.Fatalf("SyncedFiles %v != Explain %v", got, synced)
}
for i := range synced {
if synced[i] != got[i] {
t.Fatalf("SyncedFiles %v != Explain %v", got, synced)
}
}
}
// The reason SyncedFiles is not Explain: Explain counts the files inside a
// pruned directory, so it reads every entry of node_modules/. A grep must not.
func TestSyncedFilesDoesNotDescendPrunedDirs(t *testing.T) {
a := newDevice(t, "deva", nil)
write(t, a.Folder, IgnoreFile, "heavy/\n")
write(t, a.Folder, "keep.md", "yes")
write(t, a.Folder, "heavy/a.md", "no")
write(t, a.Folder, "heavy/deep/b.md", "no")
got, err := SyncedFiles(a.Folder, nil, "")
if err != nil {
t.Fatal(err)
}
for _, p := range got {
if p != "keep.md" && p != IgnoreFile {
t.Fatalf("walked into a pruned dir: %v", got)
}
}
}
// A rule a TEAMMATE pushed must not widen what this device reports as synced
// until this device accepts it — the SkipUp asymmetry, which is why accepted
// is a parameter at all.
func TestSyncedFilesHonorsAcceptedRules(t *testing.T) {
a := newDevice(t, "deva", nil)
write(t, a.Folder, IgnoreFile, "*.log\n!keep.log\n")
write(t, a.Folder, "keep.log", "negated")
// Local rules only: the negation is this device's own file, so it applies.
got, err := SyncedFiles(a.Folder, nil, "*.log\n!keep.log\n")
if err != nil {
t.Fatal(err)
}
found := false
for _, p := range got {
if p == "keep.log" {
found = true
}
}
if !found {
t.Fatalf("accepted negation should sync keep.log, got %v", got)
}
// The negation is new and unaccepted: the upload door stays shut.
got, err = SyncedFiles(a.Folder, nil, "*.log\n")
if err != nil {
t.Fatal(err)
}
for _, p := range got {
if p == "keep.log" {
t.Fatalf("unaccepted negation must not widen the sync set, got %v", got)
}
}
}
+32
View File
@@ -71,6 +71,38 @@ func walkFolder(folder string, filter *Filter, fn func(abs, rel string, d fs.Dir
})
}
// SyncedFiles lists the mount-relative paths that sync, in walk order. It is
// the same pure read as Explain — no Session, no volume lock, no network — but
// without Explain's not-synced accounting, which calls countFiles on every
// pruned directory: a `bdrive grep` in a repo with node_modules/ would walk
// node_modules/ in full just to produce a count it throws away.
//
// accepted is the ignore text this device has accepted (store.SyncState's
// IgnoreAccepted; "" when there is none), for the reason Explain documents:
// the walk applies Filter.SkipUp, and omitting it would list a file a peer's
// `!` rule points at that the cycle will not actually send.
//
// Unreadable entries are skipped rather than failing, as everywhere else in
// this walk.
func SyncedFiles(folder string, include []string, accepted string) ([]string, error) {
// A fresh filter: addNestedMount mutates it during the walk, so this must
// never be shared with a live cycle.
filter, err := loadFilter(folder, include)
if err != nil {
return nil, err
}
filter.AcceptRules(accepted)
var out []string
err = walkFolder(folder, filter, func(_, rel string, _ fs.DirEntry, v verdict) error {
if v == vSync {
out = append(out, rel)
}
return nil
})
return out, err
}
// Measure reports what a first sync of this folder would actually upload:
// the number of files and their total bytes, after the same filter the cycle
// uses. It exists so `bdrive init` can warn about a folder nobody meant to
@@ -17,6 +17,7 @@ One binary, `bdrive` — the CLI, the sync daemon, and the web server.
| `bdrive stop [folder]` | Stop syncing — daemon and agent sync hooks both pause. Files stay on disk; `bdrive init` resumes |
| `bdrive scope [add\|rm <dirs...>]` | Show or change which subfolders sync — edits the managed block of `.bdriveignore` rules that `init --only` writes. Run from the mount root; the daemon picks changes up in seconds. `rm` stops syncing a folder but deletes nothing, locally or on the hub |
| `bdrive scope --explain` | List every path in the folder, split into what syncs and what does not, with counts — the verifiable answer to "what leaves this machine". Pure read: no daemon, no lock, no network |
| `bdrive grep <pattern> [folder]` | Search the text **inside** the files a project syncs. `pattern` is a Go RE2 regexp, or a literal string with `-F`. `-i` ignores case, `-l` prints matching paths only, `-n` caps the lines printed (default 200, `0` = all). Pure read: no daemon, no lock, no network |
| `bdrive forget <path>...` | Stop syncing a path and remove it from the hub. Adds the rule to `.bdriveignore` (which syncs) and prunes in one step. Local files are never touched, here or on teammates' devices |
| `bdrive url [path]` | Internal hub link for a file or folder — sign-in and membership required. `--sync` pushes first; no argument gives the project home. Computed locally |
| `bdrive share <file>` | Public URL for a synced file. `--list`, `--revoke`, `--expires` (the hub's Share dialog can also set an expiry on an existing link) |
@@ -123,6 +124,36 @@ run *created*, open that run in the hub's History view and use the row's
device, and the DELETED row it leaves behind restores it). From the CLI, delete
the file yourself and let the next sync carry that.
### `bdrive grep` — search what is inside the files
`bdrive grep` searches file *contents*, not names:
```sh
bdrive grep 'retention.*fold'
# wiki/runbook.md:42: the retention fold collapses day buckets
# specs/reads.md:8: retention folding happens at boot
# 2 files, 2 matching lines
bdrive grep -i -l TODO # matching paths only, case-insensitive
bdrive grep -F 'a[b]c' # literal string, not a regexp
```
**It searches exactly what the project syncs.** A `.bdriveignore` rule or a
narrowed `bdrive scope` excludes a file from search the same way it excludes it
from sync, so results never mention a file your teammates cannot see, and
`.bdrive/` state never appears. Binary files are skipped.
It reads the real files on disk, so it works offline, takes no lock, and never
blocks on a sync in progress. Exit status is 0 when something matched and 1
when nothing did — `bdrive grep -q` has no equivalent, use `-l` and check the
status.
:::note[Local, not hub-wide]
This searches *this* folder. Searching contents across a whole hub, or from the
browser, is not built yet — the ⌘K palette covers file names, projects and
actions.
:::
### `bdrive forget` and `bdrive sync --prune` — cleaning up the hub
Adding a rule to `.bdriveignore` only stops *future* uploads. Anything that