mirror of
https://github.com/runbear-io/beardrive.git
synced 2026-08-25 08:08:08 +02:00
feat(cli): bdrive verify proves this folder matches the hub (BEA-161)
"Your folder is the same everywhere" was a belief with no receipt. `bdrive status` counts pending ops and unscanned changes but never reads a byte of content, so a file whose bytes changed while its size and mtime stayed put was invisible to every check we shipped. `bdrive verify` hashes every synced file and compares it against journal.Replay(AllOps), reporting drifted / never-pushed / missing-locally / not-yet-scanned, and with --remote also missing-on-hub. Exit 0 when every category is empty, 1 otherwise, so it composes as a pre-flight check. The logic lives in internal/syncer/verify.go next to Drift and Explain — the two read-only siblings it completes — because neverSync, chunkThreshold and loadFilter are unexported there, and because a multi-device test cannot drive a func main package. cmd/bdrive/verify.go is the thin cobra shell. Two things that would have made it silently wrong: - --remote must probe BOTH blobs/<sha> and manifests/<sha>. Files over 4 MiB are pushed as chunks plus a manifest keyed by the file's own sha, so a check asking only blobs/ would call every large file missing from the hub. Size only orders the probe — it can never be a filter, because browser uploads always write blobs/<sha> at any size, pushChunked falls back to a whole blob when the manifest key is refused, and pre-delta-sync history is whole blobs regardless. - missing-locally applies filter.Skip + neverSync, the same guard materialize uses. The rules are symmetric in scan and materialize, so a path the local filter excludes is legitimately absent — without this, every project narrowed by `bdrive scope --only` would report its whole out-of-scope set as missing. Pure read throughout: no Session, no volume flock, no ops, no journal writes, no materialize, and no network without --remote. LoadProject rather than ResolveMount and remote.Open rather than openSession, so a read never enrolls the device. An unreachable hub degrades to a printed warning and the local verdict still decides. The command says its own caveat out loud: the journals it replays are this device's local copies, so it proves "this folder matches what I last pulled". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Snow Lee
co-authored by
Claude Opus 5
Claude Opus 5
parent
899a840439
commit
46cfeb7898
@@ -253,6 +253,7 @@ hub's own storage, never something a syncing client points at directly:
|
||||
| `bdrive scope [add\|rm <dirs...>]` | Show or change which subfolders sync — edits the managed block of `.bdriveignore` rules that `init --only` writes, so no one hand-writes negation syntax. The daemon picks changes up in seconds; `rm` deletes nothing, locally or on the hub. `--explain` lists every path in the folder split into what syncs and what does not, so you can verify what leaves this machine (pure read — no daemon, no lock, no network) |
|
||||
| `bdrive grep <pattern> [folder]` | Search the text **inside** the files a project syncs — Go RE2 regexp, or a literal with `-F`; `-i` ignores case, `-l` prints matching paths only, `-n` caps the lines printed (default 200, `0` = all). Output is `path:line: text`. Only files the project actually syncs are searched, so a `.bdriveignore` rule or a narrowed `bdrive scope` excludes a file from search exactly as it excludes it from sync; binary files are skipped. Pure local read — no daemon, no lock, no network, works offline and never blocks a sync in progress. Exit status 0 on match, 1 on none, so it composes in scripts |
|
||||
| `bdrive stale [folder]` | Find docs the code has outgrown: synced markdown (`.md`/`.markdown`) that links to a file written **after** the doc itself. Staleness here is not age — a doc goes stale when what it describes moves. Write times come from the **journal**, not `os.Stat`: materialize stamps a peer's file with this device's mtime, so on a freshly synced machine every mtime is identical and only the journal still knows. `-l` prints outgrown paths only, `-n` caps the docs printed (default 50, `0` = all). A reference that does not resolve to a file this project syncs — a URL, a `../` escape, a made-up path — is silently ignored. Pure local read — no daemon, no lock, no network. **Exit status is 0 whether or not anything is stale**: this is advisory, not a gate |
|
||||
| `bdrive verify [folder]` | Prove this folder matches the content the journal records: **sha256 every file the project syncs** and compare. Reports `drifted` (on disk, but not what the journal records), `never-pushed` (committed here, never reached the hub), `missing-locally` (the journal has it, this folder does not), `not-yet-scanned` (on disk, no op yet), and with `--remote` also `missing-on-hub` (one existence check per distinct blob, asking **both** `blobs/<sha>` and `manifests/<sha>` — files over 4 MiB are pushed as chunks plus a manifest, so a check that asked only `blobs/` would call every large file missing). Unlike `bdrive status` this reads content, so a file whose bytes changed while its size and mtime stayed put is caught; there is deliberately no fast path, and it is slower than `status` on a big project. Pure read — no daemon, no lock, no ops, no journal writes, and without `--remote` no network. It never repairs anything. **Exit status is 0 when every category is empty, 1 otherwise**, so it composes as a pre-flight check. An unreachable hub degrades `--remote` to a warning, never a failure. The journals it replays are this device's local copies, so it proves "this folder matches what I last pulled" |
|
||||
| `bdrive forget <path>...` | Stop syncing a path *and* remove it from the hub — adds the rule to `.bdriveignore` (which syncs) and prunes in one step. Local files are never touched, here or on teammates' devices |
|
||||
| `bdrive url [path]` | Internal hub link for a file/folder (sign-in + membership required; `--sync` pushes first; no arg = project home). Computed locally |
|
||||
| `bdrive share <file>` | Public URL for a synced file (`--list`, `--revoke`, `--expires`) |
|
||||
|
||||
@@ -115,6 +115,22 @@ classDiagram
|
||||
}
|
||||
note for Drift "drift.go — the `local:` line in bdrive status: what is on disk that the state cache has not seen, using the scan's own size+mtime compare. Pure read like its siblings, and load-bearing that it stays one: status is what someone runs when sync is stuck, so it stores no blob, mints no op, rewrites no cache — it does not even mutate the cache map it is handed, which status prints `files:` from"
|
||||
|
||||
class Verify {
|
||||
+Verify(ctx, folder, include, st, device, be) VerifyReport
|
||||
-existsEither(ctx, be, blob, size) bool
|
||||
}
|
||||
class VerifyReport {
|
||||
+Files int
|
||||
+Bytes int64
|
||||
+Elapsed Duration
|
||||
+Drifted / NeverPushed / MissingLocally paths
|
||||
+NotYetScanned / MissingOnHub paths
|
||||
+NotFetched int
|
||||
+RemoteErr error
|
||||
+Problems() int
|
||||
}
|
||||
note for Verify "verify.go — bdrive verify, the third read-only sibling and the only one that HASHES: sha256 every synced file against journal.Replay(st.AllOps()), which is why it catches the file whose bytes changed while size and mtime stayed put — precisely what Drift cannot see. Pure read like its siblings: no Session, no flock, no ops, no journal write, no materialize, and no network unless a Backend is passed. missing-locally applies filter.Skip + neverSync, the same guard materialize uses, so a scope-narrowed project does not report every out-of-scope path as missing. The --remote leg is one Exists per DISTINCT blob probing BOTH blobs/<sha> and manifests/<sha> — size only orders the probe, because a large file legitimately lives under blobs/ (browser upload, pushChunked fallback, pre-delta-sync history) — and any Exists error sets RemoteErr and stops that leg, never the local verdict"
|
||||
|
||||
class Explain {
|
||||
+Explain(folder, include, accepted) two lists
|
||||
+NotSyncedFiles(entries) int
|
||||
@@ -191,6 +207,11 @@ classDiagram
|
||||
Session --> Filter : SkipUp on scan, Skip on materialize
|
||||
Session --> walkFolder : scan
|
||||
Explain --> walkFolder : same predicate
|
||||
Verify --> SyncedFiles : hashes what syncs
|
||||
Verify --> Filter : own fresh instance, for missing-locally
|
||||
Verify --> Store : AllOps / DeviceOps / LoadSync / HasBlob
|
||||
Verify --> Backend : Exists per blob, --remote only
|
||||
Verify ..> VerifyReport : findings and exit status
|
||||
Drift --> walkFolder : same predicate
|
||||
Drift --> Filter : own fresh instance
|
||||
SyncedFiles --> walkFolder : same predicate
|
||||
@@ -223,13 +244,15 @@ classDiagram
|
||||
|
||||
class Commands {
|
||||
init login logout
|
||||
sync stop scope grep stale forget status log
|
||||
sync stop scope grep stale verify forget status log
|
||||
restore url share export import
|
||||
web daemon hooks read-log
|
||||
resume autostart
|
||||
}
|
||||
note for Commands "cmd/bdrive — thin cobra layer; init is the front door (one command: login + hooks + sync + link), stop pauses"
|
||||
note for Commands "grep searches file CONTENTS in the working folder via syncer.SyncedFiles — LoadProject not ResolveMount (a read must not enroll the device), no session, no flock, and the volume store is opened only if it already exists, so a search creates nothing. Exit 1 on no match is a status, not an error (errNoMatch + SilenceErrors). stale copies that whole posture and swaps the predicate: it extracts path-shaped references from synced markdown, keeps only the ones resolving into the SyncedFiles set, and flags a doc whose reference was written later. It dates a path from the JOURNAL, not os.Stat — materialize stamps a peer's file with this device's mtime, so mtime comparison reports nothing on a freshly cloned machine — folding st.AllOps() to the max syncer.DisplayTime per path, which drops a forged future stamp instead of dating that path to year 1. Unlike grep it exits 0 either way: advisory output, not a gate"
|
||||
note for Commands "verify is the same pure-read shell again — LoadProject not ResolveMount, stat-guarded store.Open, safeField on every path (missing-locally and missing-on-hub come out of a PEER's journal) — and it opens its Backend with remote.Open directly rather than openSession, because openSession goes through mustProject and would enroll the device. Exit 1 when any category is non-empty is a status, not an error (errVerifyProblems + SilenceErrors), so it composes as a pre-flight check; an unreachable hub prints a warning and the local verdict still decides"
|
||||
|
||||
note for Commands "Every peer-authored string status / log / whoami print goes through safeField first — a teammate's file name is attacker-controlled text landing in your terminal, and an escape sequence there rewrites the line above it. grep runs BOTH the path and the matched line through it — a matched line is a teammate's file content, the widest version of that surface. login now does PKCE on the loopback callback (no compat arm) and both its client and init's refuse to follow a redirect off the hub's origin with the device token attached"
|
||||
|
||||
class Templates {
|
||||
|
||||
@@ -55,6 +55,7 @@ everything keeps working offline; changes sync when the remote is reachable.`,
|
||||
scopeCmd(),
|
||||
grepCmd(),
|
||||
staleCmd(),
|
||||
verifyCmd(),
|
||||
forgetCmd(),
|
||||
syncCmd(),
|
||||
readLogCmd(),
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"time"
|
||||
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
"github.com/runbear-io/beardrive/internal/config"
|
||||
"github.com/runbear-io/beardrive/internal/remote"
|
||||
"github.com/runbear-io/beardrive/internal/store"
|
||||
"github.com/runbear-io/beardrive/internal/syncer"
|
||||
)
|
||||
|
||||
// errVerifyProblems is verify's "something is wrong" exit, not a failure of
|
||||
// the command: the convention is status 1 with the findings already printed,
|
||||
// so `bdrive verify || ...` composes in a script or a pre-flight check. Real
|
||||
// errors (not a project, bad remote) still print — see verifyCmd.
|
||||
var errVerifyProblems = errors.New("verify found problems")
|
||||
|
||||
// verifyMaxList caps how many paths are listed per category. The counts are
|
||||
// always exact; only the listing is bounded, so a project that drifted whole
|
||||
// does not scroll the finding off the screen.
|
||||
const verifyMaxList = 20
|
||||
|
||||
func verifyCmd() *cobra.Command {
|
||||
var checkRemote bool
|
||||
c := &cobra.Command{
|
||||
Use: "verify [folder]",
|
||||
Short: "Prove this folder matches the content the journal records",
|
||||
Long: `Hash every file this project syncs and compare it to the journal.
|
||||
|
||||
"Your folder is the same everywhere" is the claim; this is the receipt.
|
||||
` + "`bdrive status`" + ` counts pending ops and unscanned changes but never reads a
|
||||
byte of content — so a file whose bytes changed while its size and mtime
|
||||
stayed put is invisible to it. verify hashes, which is the whole point: there
|
||||
is deliberately no size/mtime fast path, and on a large project it is
|
||||
noticeably slower than status.
|
||||
|
||||
It reports five things:
|
||||
|
||||
drifted on disk, but not the content the journal records
|
||||
never-pushed committed here, never reached the hub
|
||||
missing-locally the journal has it, this folder does not
|
||||
not-yet-scanned on disk, with no op anywhere yet
|
||||
missing-on-hub synced here, absent from the hub (--remote only)
|
||||
|
||||
It is a pure read: no daemon, no lock, no ops, no journal writes, and without
|
||||
--remote no network at all — so it works offline and never blocks on a sync in
|
||||
progress. It never repairs anything; run ` + "`bdrive sync`" + ` for that.
|
||||
|
||||
Exit status is 0 when every category is empty and 1 when any is not.
|
||||
|
||||
One caveat it prints for itself: the journals it replays are this device's
|
||||
local copies, so without a pull it proves "this folder matches what I last
|
||||
pulled" — a teammate's newer op is invisible until the next cycle.`,
|
||||
Example: ` bdrive verify # hash the folder, compare to the journal
|
||||
bdrive verify --remote # also ask the hub whether it still holds the content
|
||||
bdrive verify ~/wiki # a project other than the current folder`,
|
||||
Args: cobra.MaximumNArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
err := runVerify(cmd, args, checkRemote)
|
||||
// SilenceErrors below is for errVerifyProblems alone, so anything
|
||||
// else has to print itself — cobra no longer will.
|
||||
if err != nil && !errors.Is(err, errVerifyProblems) {
|
||||
fmt.Fprintln(cmd.ErrOrStderr(), "Error:", err)
|
||||
}
|
||||
return err
|
||||
},
|
||||
}
|
||||
// Silenced so a findings exit is status 1 with the findings and nothing
|
||||
// else — errVerifyProblems is a status, not a failure, and cobra would
|
||||
// otherwise render it as an error and a usage block. Same mechanism
|
||||
// `bdrive grep` uses for its no-match exit.
|
||||
c.SilenceErrors = true
|
||||
c.SilenceUsage = true
|
||||
c.Flags().BoolVar(&checkRemote, "remote", false, "also ask the hub whether it still holds every blob (one check per blob)")
|
||||
return c
|
||||
}
|
||||
|
||||
func runVerify(cmd *cobra.Command, folderArg []string, checkRemote bool) error {
|
||||
folder, err := absFolder(folderArg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// LoadProject, not ResolveMount: ResolveMount self-heals the registry
|
||||
// path, i.e. it enrolls this device. A read-only query must not have that
|
||||
// side effect — the same rule grep and stale follow.
|
||||
proj, found, err := config.LoadProject(folder)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !found {
|
||||
return fmt.Errorf("%s is not a beardrive project (run `bdrive init` there first)", folder)
|
||||
}
|
||||
|
||||
out := cmd.OutOrStdout()
|
||||
fmt.Fprintf(out, " project: %s (%s)\n", safeField(proj.Volume, 120), proj.ID)
|
||||
|
||||
// Stat-guarded, because store.Open MkdirAlls and a read must not create a
|
||||
// volume for a project that has never synced; unlocked, because store.Open
|
||||
// takes no volume flock — a running daemon never blocks this.
|
||||
vdir, err := config.VolumeDir(proj.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !dirExists(vdir) {
|
||||
fmt.Fprintln(out, " nothing synced yet — no local history to verify against")
|
||||
return nil
|
||||
}
|
||||
st, err := store.Open(vdir)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var be remote.Backend
|
||||
if checkRemote {
|
||||
if proj.Remote == "" {
|
||||
return errors.New("--remote needs a hub: this project is local only")
|
||||
}
|
||||
// remote.Open directly, not openSession: openSession goes through
|
||||
// mustProject, which is the enrolling call this command must not make.
|
||||
// It picks up the device token itself, so there is nothing to wire.
|
||||
b, oerr := remote.Open(cmd.Context(), proj.Remote)
|
||||
if oerr != nil {
|
||||
// Unreachable hub is a warning, not a failure: the local verdict
|
||||
// is still the answer, and this is the repo's "never break on the
|
||||
// remote" posture.
|
||||
fmt.Fprintf(out, " warning: hub unreachable, local check only (%s)\n", safeField(oerr.Error(), 200))
|
||||
} else {
|
||||
be = b
|
||||
defer be.Close()
|
||||
}
|
||||
}
|
||||
|
||||
// Best-effort: without an identity the never-pushed count is simply
|
||||
// empty, which is a worse answer than the others but not a reason to fail
|
||||
// a read.
|
||||
dev, _ := config.LoadDevice()
|
||||
|
||||
rep, err := syncer.Verify(cmd.Context(), folder, proj.Include, st, dev.ID, be)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return printVerify(out, rep)
|
||||
}
|
||||
|
||||
func printVerify(out io.Writer, rep syncer.VerifyReport) error {
|
||||
fmt.Fprintf(out, " checked: %s, %s hashed in %s\n",
|
||||
plural(rep.Files, "file"), humanBytes(rep.Bytes), rep.Elapsed.Round(time.Millisecond))
|
||||
if rep.RemoteErr != nil {
|
||||
fmt.Fprintf(out, " warning: hub check incomplete, local result stands (%s)\n", safeField(rep.RemoteErr.Error(), 200))
|
||||
}
|
||||
|
||||
missingNote := "the journal has it, this folder does not"
|
||||
if rep.NotFetched > 0 {
|
||||
missingNote = fmt.Sprintf("%d not fetched yet — run `bdrive sync`", rep.NotFetched)
|
||||
}
|
||||
cats := []struct {
|
||||
name, note string
|
||||
paths []string
|
||||
}{
|
||||
{"drifted", "on disk, but not the content the journal records", rep.Drifted},
|
||||
{"never-pushed", "committed here, never reached the hub", rep.NeverPushed},
|
||||
{"missing-locally", missingNote, rep.MissingLocally},
|
||||
{"not-yet-scanned", "on disk, with no op anywhere yet", rep.NotYetScanned},
|
||||
{"missing-on-hub", "synced here, absent from the hub", rep.MissingOnHub},
|
||||
}
|
||||
for _, c := range cats {
|
||||
if len(c.paths) == 0 {
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(out, "\n %s (%d) - %s\n", c.name, len(c.paths), c.note)
|
||||
shown := c.paths
|
||||
if len(shown) > verifyMaxList {
|
||||
shown = shown[:verifyMaxList]
|
||||
}
|
||||
for _, p := range shown {
|
||||
// Every path in missing-locally and missing-on-hub comes out of a
|
||||
// PEER's journal — a string someone else chose. safeField, the
|
||||
// same treatment `bdrive log` and `bdrive grep` give journal
|
||||
// strings, or a lone CR repaints the row.
|
||||
fmt.Fprintf(out, " %s\n", safeField(p, 160))
|
||||
}
|
||||
if len(c.paths) > len(shown) {
|
||||
fmt.Fprintf(out, " ... and %d more\n", len(c.paths)-len(shown))
|
||||
}
|
||||
}
|
||||
|
||||
if n := rep.Problems(); n > 0 {
|
||||
fmt.Fprintf(out, "\n %s. Run `bdrive sync` to reconcile, or `bdrive log <path>` for history.\n", plural(n, "problem"))
|
||||
return errVerifyProblems
|
||||
}
|
||||
fmt.Fprintln(out, " OK - the folder matches the journal")
|
||||
// Said out loud, not just documented: the journals replayed here are this
|
||||
// device's local copies, so a clean verdict is about what was last pulled.
|
||||
// Without this line the command overclaims exactly the thing it exists to
|
||||
// prove.
|
||||
fmt.Fprintln(out, " (this compares against what this device last pulled — run `bdrive sync` first for the hub's latest)")
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
package syncer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"github.com/runbear-io/beardrive/internal/journal"
|
||||
"github.com/runbear-io/beardrive/internal/remote"
|
||||
"github.com/runbear-io/beardrive/internal/store"
|
||||
)
|
||||
|
||||
// VerifyReport is what Verify found. Every slice holds mount-relative paths,
|
||||
// sorted; an empty slice means that category is clean.
|
||||
//
|
||||
// A path can legitimately land in two categories at once — a file edited after
|
||||
// its last scan and never pushed is both drifted and never-pushed — so the
|
||||
// counts are independent and dedup happens only within a category.
|
||||
type VerifyReport struct {
|
||||
Files int // files hashed this run
|
||||
Bytes int64 // bytes hashed this run
|
||||
Elapsed time.Duration // how long the hashing took, so the cost is visible
|
||||
|
||||
// Drifted: on disk and in the journal state, but sha256(disk) != state.Blob.
|
||||
Drifted []string
|
||||
// NeverPushed: this device's own ops past SyncState.PushedOps — content
|
||||
// the hub has never seen.
|
||||
NeverPushed []string
|
||||
// MissingLocally: a path the journal state holds and the local filter
|
||||
// includes, that this folder does not have.
|
||||
MissingLocally []string
|
||||
// NotFetched is how many of MissingLocally are simply not downloaded yet
|
||||
// (their blob is absent from the local store). That is the benign half of
|
||||
// this category — materializeFile returns early on it and the next cycle
|
||||
// fixes it — and without the distinction a freshly cloned device reads as
|
||||
// broken.
|
||||
NotFetched int
|
||||
// NotYetScanned: a file on disk the filter syncs, with no op anywhere yet.
|
||||
NotYetScanned []string
|
||||
// MissingOnHub: content this device believes is synced that the hub does
|
||||
// not hold. Only populated with a backend.
|
||||
MissingOnHub []string
|
||||
// RemoteErr is set when the remote leg could not finish. The local half of
|
||||
// the report still stands — this is the repo's "never break on the remote"
|
||||
// posture, the same one Result.Offline takes.
|
||||
RemoteErr error
|
||||
}
|
||||
|
||||
// Problems is how many findings the report holds, across every category. It is
|
||||
// what decides `bdrive verify`'s exit status.
|
||||
func (r VerifyReport) Problems() int {
|
||||
return len(r.Drifted) + len(r.NeverPushed) + len(r.MissingLocally) +
|
||||
len(r.NotYetScanned) + len(r.MissingOnHub)
|
||||
}
|
||||
|
||||
// Verify compares the bytes in the working folder against the content the
|
||||
// journal records for them, hashing every synced file.
|
||||
//
|
||||
// It is a pure read, the same contract as Drift and Explain: no Session, no
|
||||
// volume lock, no ops, no journal write, no materialize — and with be == nil,
|
||||
// no network either. `bdrive verify` is what someone runs to check a folder
|
||||
// they already suspect; a version of it that repaired anything would destroy
|
||||
// the evidence it was asked to describe.
|
||||
//
|
||||
// Unlike Drift, which compares size+mtime against the state cache, this hashes
|
||||
// content — that is the whole point, and there is deliberately no fast path.
|
||||
// A file whose bytes changed but whose size and mtime were restored is exactly
|
||||
// what Drift cannot see.
|
||||
//
|
||||
// device is this device's ID, for the never-pushed count. be may be nil, which
|
||||
// skips the hub leg entirely.
|
||||
//
|
||||
// Known limitation, and the command prints it: the journals AllOps replays are
|
||||
// this device's LOCAL copies. Without a pull, this proves "the folder matches
|
||||
// what I last pulled" — and with a backend, "and the hub still holds all of
|
||||
// it". A teammate's newer op committed since the last cycle is invisible here.
|
||||
func Verify(ctx context.Context, folder string, include []string, st *store.Store, device string, be remote.Backend) (VerifyReport, error) {
|
||||
var rep VerifyReport
|
||||
start := time.Now()
|
||||
|
||||
sync, err := st.LoadSync()
|
||||
if err != nil {
|
||||
return rep, err
|
||||
}
|
||||
|
||||
// A fresh filter: addNestedMount mutates it during a walk, so this must
|
||||
// never be shared with a live cycle. AcceptRules for the reason Explain
|
||||
// documents — the scan applies Filter.SkipUp, and omitting it would
|
||||
// disagree with the very next cycle.
|
||||
filter, err := loadFilter(folder, include)
|
||||
if err != nil {
|
||||
return rep, err
|
||||
}
|
||||
filter.AcceptRules(sync.IgnoreAccepted)
|
||||
|
||||
ops, err := st.AllOps()
|
||||
if err != nil {
|
||||
return rep, err
|
||||
}
|
||||
// The identical object Cycle materializes from (syncer.go, the pull phase),
|
||||
// so what this compares against is what the cycle would write.
|
||||
state := journal.Replay(ops)
|
||||
|
||||
paths, err := SyncedFiles(folder, include, sync.IgnoreAccepted)
|
||||
if err != nil {
|
||||
return rep, err
|
||||
}
|
||||
|
||||
seen := make(map[string]bool, len(paths))
|
||||
for _, rel := range paths {
|
||||
seen[rel] = true
|
||||
want, known := state[rel]
|
||||
if !known {
|
||||
rep.NotYetScanned = append(rep.NotYetScanned, rel)
|
||||
continue
|
||||
}
|
||||
abs := filepath.Join(folder, filepath.FromSlash(rel))
|
||||
sum, err := hashFile(abs)
|
||||
if err != nil {
|
||||
continue // vanished or unreadable; skipped, never fatal, as everywhere in this walk
|
||||
}
|
||||
rep.Files++
|
||||
if fi, err := os.Stat(abs); err == nil {
|
||||
rep.Bytes += fi.Size()
|
||||
}
|
||||
if sum != want.Blob {
|
||||
rep.Drifted = append(rep.Drifted, rel)
|
||||
}
|
||||
}
|
||||
|
||||
for rel, want := range state {
|
||||
// The same guard materialize applies before it would write: a path the
|
||||
// local filter excludes is LEGITIMATELY absent from disk, because the
|
||||
// rules are applied symmetrically in scan and materialize. Without
|
||||
// this, every project narrowed by `bdrive scope --only` would report
|
||||
// every out-of-scope path as missing.
|
||||
if seen[rel] || filter.Skip(rel) || neverSync(rel) {
|
||||
continue
|
||||
}
|
||||
rep.MissingLocally = append(rep.MissingLocally, rel)
|
||||
if !st.HasBlob(want.Blob) {
|
||||
rep.NotFetched++
|
||||
}
|
||||
}
|
||||
|
||||
// The same arithmetic push and `bdrive status` use: our own ops past the
|
||||
// push cursor are the ones the hub has never seen.
|
||||
if myOps, err := st.DeviceOps(device); err == nil {
|
||||
from := sync.PushedOps
|
||||
if from > int64(len(myOps)) {
|
||||
from = int64(len(myOps))
|
||||
}
|
||||
if from < 0 {
|
||||
from = 0
|
||||
}
|
||||
dedup := map[string]bool{}
|
||||
for _, op := range myOps[from:] {
|
||||
if dedup[op.Path] {
|
||||
continue
|
||||
}
|
||||
dedup[op.Path] = true
|
||||
rep.NeverPushed = append(rep.NeverPushed, op.Path)
|
||||
}
|
||||
}
|
||||
|
||||
if be != nil {
|
||||
verifyRemote(ctx, be, state, &rep)
|
||||
}
|
||||
|
||||
for _, s := range [][]string{rep.Drifted, rep.NeverPushed, rep.MissingLocally, rep.NotYetScanned, rep.MissingOnHub} {
|
||||
sort.Strings(s)
|
||||
}
|
||||
rep.Elapsed = time.Since(start)
|
||||
return rep, nil
|
||||
}
|
||||
|
||||
// verifyRemote asks the hub whether it still holds every distinct blob the
|
||||
// journal state references — one existence check per blob, not per path.
|
||||
func verifyRemote(ctx context.Context, be remote.Backend, state map[string]journal.FileState, rep *VerifyReport) {
|
||||
byBlob := map[string][]string{}
|
||||
size := map[string]int64{}
|
||||
for rel, want := range state {
|
||||
if want.Blob == "" {
|
||||
continue
|
||||
}
|
||||
byBlob[want.Blob] = append(byBlob[want.Blob], rel)
|
||||
if want.Size > size[want.Blob] {
|
||||
size[want.Blob] = want.Size
|
||||
}
|
||||
}
|
||||
blobs := make([]string, 0, len(byBlob))
|
||||
for b := range byBlob {
|
||||
blobs = append(blobs, b)
|
||||
}
|
||||
sort.Strings(blobs) // stable order, so a partial run after RemoteErr is reproducible
|
||||
|
||||
for _, b := range blobs {
|
||||
ok, err := existsEither(ctx, be, b, size[b])
|
||||
if err != nil {
|
||||
// Stop the remote leg, keep the local report. Never a hard failure.
|
||||
rep.RemoteErr = err
|
||||
return
|
||||
}
|
||||
if !ok {
|
||||
rep.MissingOnHub = append(rep.MissingOnHub, byBlob[b]...)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// existsEither asks for a blob under both key shapes it can live under.
|
||||
//
|
||||
// A file over chunkThreshold is pushed as content-defined chunks plus a
|
||||
// manifest keyed by the FILE's own sha256 — manifests/<sha>, never
|
||||
// blobs/<sha> (chunks.go). A check that only asked blobs/<sha> would report
|
||||
// every large file as missing from the hub.
|
||||
//
|
||||
// The size only picks which key to try FIRST; it can never be a filter,
|
||||
// because a large file legitimately lives under blobs/ in three ways: the
|
||||
// browser upload path always writes blobs/<sha> at any size, pushChunked falls
|
||||
// back to a whole-blob Put when the manifest key is refused, and anything
|
||||
// pushed before delta sync existed is a whole blob regardless of size.
|
||||
//
|
||||
// Checking the manifest key alone is enough for a chunked file:
|
||||
// "a manifest exists ⟹ its chunks exist" is enforced hub-side at ingest, not
|
||||
// an honest-client convention, so there is no need to walk the chunk list.
|
||||
func existsEither(ctx context.Context, be remote.Backend, blob string, size int64) (bool, error) {
|
||||
keys := [2]string{"blobs/" + blob, "manifests/" + blob}
|
||||
if size > chunkThreshold {
|
||||
keys[0], keys[1] = keys[1], keys[0]
|
||||
}
|
||||
for _, k := range keys {
|
||||
ok, err := be.Exists(ctx, k)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if ok {
|
||||
return true, nil // short-circuit: the common case is one round trip
|
||||
}
|
||||
}
|
||||
return false, nil
|
||||
}
|
||||
@@ -0,0 +1,301 @@
|
||||
package syncer
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/runbear-io/beardrive/internal/remote"
|
||||
"github.com/runbear-io/beardrive/internal/store"
|
||||
)
|
||||
|
||||
// verifyLocal runs Verify against a device the way `bdrive verify` does with
|
||||
// no --remote: the device's own store and ID, no backend.
|
||||
func verifyLocal(t *testing.T, s *Session) VerifyReport {
|
||||
t.Helper()
|
||||
rep, err := Verify(context.Background(), s.Folder, nil, s.Store, s.Device.ID, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return rep
|
||||
}
|
||||
|
||||
func verifyRemoteRun(t *testing.T, s *Session) VerifyReport {
|
||||
t.Helper()
|
||||
rep, err := Verify(context.Background(), s.Folder, nil, s.Store, s.Device.ID, s.Backend)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if rep.RemoteErr != nil {
|
||||
t.Fatalf("unexpected RemoteErr: %v", rep.RemoteErr)
|
||||
}
|
||||
return rep
|
||||
}
|
||||
|
||||
func wantEmpty(t *testing.T, name string, got []string) {
|
||||
t.Helper()
|
||||
if len(got) != 0 {
|
||||
t.Fatalf("%s = %v, want empty", name, got)
|
||||
}
|
||||
}
|
||||
|
||||
func wantOnly(t *testing.T, name string, got []string, want ...string) {
|
||||
t.Helper()
|
||||
if strings.Join(got, ",") != strings.Join(want, ",") {
|
||||
t.Fatalf("%s = %v, want %v", name, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// TestVerifyCleanAfterConverge: two devices that have converged both report a
|
||||
// fully empty verdict — nothing drifted, nothing pending, nothing missing.
|
||||
func TestVerifyCleanAfterConverge(t *testing.T) {
|
||||
be := sharedRemote(t)
|
||||
a := newDevice(t, "deva", be)
|
||||
b := newDevice(t, "devb", be)
|
||||
|
||||
write(t, a.Folder, "doc.txt", "v1")
|
||||
write(t, a.Folder, "sub/nested.md", "deep")
|
||||
cycle(t, a)
|
||||
cycle(t, b)
|
||||
|
||||
for _, d := range []*Session{a, b} {
|
||||
rep := verifyLocal(t, d)
|
||||
if rep.Problems() != 0 {
|
||||
t.Fatalf("%s: Problems() = %d, want 0 (%+v)", d.Device.ID, rep.Problems(), rep)
|
||||
}
|
||||
if rep.Files != 2 {
|
||||
t.Fatalf("%s: hashed %d file(s), want 2", d.Device.ID, rep.Files)
|
||||
}
|
||||
if rep.Bytes == 0 {
|
||||
t.Fatalf("%s: Bytes should be > 0", d.Device.ID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestVerifyCatchesRestoredMtime is the criterion Drift cannot meet: bytes
|
||||
// changed, size and mtime put back. Drift compares size+mtime and sees
|
||||
// nothing; Verify hashes and catches it.
|
||||
func TestVerifyCatchesRestoredMtime(t *testing.T) {
|
||||
a := newDevice(t, "deva", sharedRemote(t))
|
||||
write(t, a.Folder, "doc.txt", "aaaa")
|
||||
cycle(t, a)
|
||||
|
||||
abs := filepath.Join(a.Folder, "doc.txt")
|
||||
fi, err := os.Stat(abs)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Same length, different bytes, original mtime restored.
|
||||
if err := os.WriteFile(abs, []byte("bbbb"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Chtimes(abs, fi.ModTime(), fi.ModTime()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
rep := verifyLocal(t, a)
|
||||
wantOnly(t, "Drifted", rep.Drifted, "doc.txt")
|
||||
|
||||
// And the whole point: the cheap check reports nothing.
|
||||
added, modified, removed, err := Drift(a.Folder, nil, "", cacheOf(t, a))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if added+modified+removed != 0 {
|
||||
t.Fatalf("Drift saw %d/%d/%d changes; it is supposed to be blind to this", added, modified, removed)
|
||||
}
|
||||
}
|
||||
|
||||
// cacheOf is the materialization cache Drift compares against, for the mount
|
||||
// the test session materializes into.
|
||||
func cacheOf(t *testing.T, s *Session) map[string]store.CachedFile {
|
||||
t.Helper()
|
||||
c, err := s.Store.LoadCache(s.mountID())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// TestVerifyMissingLocally: a path the journal holds and this folder does not.
|
||||
func TestVerifyMissingLocally(t *testing.T) {
|
||||
be := sharedRemote(t)
|
||||
a := newDevice(t, "deva", be)
|
||||
b := newDevice(t, "devb", be)
|
||||
|
||||
write(t, a.Folder, "keep.txt", "k")
|
||||
write(t, a.Folder, "gone.txt", "g")
|
||||
cycle(t, a)
|
||||
cycle(t, b)
|
||||
|
||||
// Delete behind the daemon's back — no cycle, so no delete op.
|
||||
if err := os.Remove(filepath.Join(b.Folder, "gone.txt")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rep := verifyLocal(t, b)
|
||||
wantOnly(t, "MissingLocally", rep.MissingLocally, "gone.txt")
|
||||
wantEmpty(t, "Drifted", rep.Drifted)
|
||||
if rep.NotFetched != 0 {
|
||||
t.Fatalf("NotFetched = %d, want 0 — the blob is right here", rep.NotFetched)
|
||||
}
|
||||
}
|
||||
|
||||
// TestVerifyIgnoredPathNotMissing is the --only false-alarm case: a path the
|
||||
// LOCAL filter excludes is legitimately absent from disk, because the rules
|
||||
// are applied symmetrically in scan and materialize.
|
||||
func TestVerifyIgnoredPathNotMissing(t *testing.T) {
|
||||
be := sharedRemote(t)
|
||||
a := newDevice(t, "deva", be)
|
||||
b := newDevice(t, "devb", be)
|
||||
|
||||
write(t, a.Folder, "keep.txt", "k")
|
||||
write(t, a.Folder, "secret/thing.txt", "s")
|
||||
cycle(t, a)
|
||||
cycle(t, b)
|
||||
|
||||
// B narrows its scope and drops the excluded path, the way materialize
|
||||
// would on the next cycle.
|
||||
if err := os.RemoveAll(filepath.Join(b.Folder, "secret")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rules := "secret/\n"
|
||||
write(t, b.Folder, IgnoreFile, rules)
|
||||
sync, err := b.Store.LoadSync()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sync.IgnoreAccepted = rules
|
||||
if err := b.Store.SaveSync(sync); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
rep := verifyLocal(t, b)
|
||||
wantEmpty(t, "MissingLocally", rep.MissingLocally)
|
||||
|
||||
// Negative control: without the rule the very same folder DOES report it,
|
||||
// so the assertion above is about the filter and not about the path
|
||||
// having quietly fallen out of the journal state.
|
||||
if err := os.Remove(filepath.Join(b.Folder, IgnoreFile)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sync.IgnoreAccepted = ""
|
||||
if err := b.Store.SaveSync(sync); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if rep := verifyLocal(t, b); len(rep.MissingLocally) != 1 || rep.MissingLocally[0] != "secret/thing.txt" {
|
||||
t.Fatalf("without the rule MissingLocally = %v, want [secret/thing.txt]", rep.MissingLocally)
|
||||
}
|
||||
}
|
||||
|
||||
// TestVerifyNeverPushed: an offline cycle commits ops the hub never sees.
|
||||
func TestVerifyNeverPushed(t *testing.T) {
|
||||
a := newDevice(t, "deva", nil) // nil backend: nothing can be pushed
|
||||
write(t, a.Folder, "notes/draft.md", "wip")
|
||||
cycle(t, a)
|
||||
|
||||
rep := verifyLocal(t, a)
|
||||
wantOnly(t, "NeverPushed", rep.NeverPushed, "notes/draft.md")
|
||||
wantEmpty(t, "Drifted", rep.Drifted)
|
||||
wantEmpty(t, "NotYetScanned", rep.NotYetScanned)
|
||||
}
|
||||
|
||||
// TestVerifyNotYetScanned: a file on disk the filter syncs, with no op at all.
|
||||
func TestVerifyNotYetScanned(t *testing.T) {
|
||||
a := newDevice(t, "deva", sharedRemote(t))
|
||||
write(t, a.Folder, "doc.txt", "v1")
|
||||
cycle(t, a)
|
||||
|
||||
write(t, a.Folder, "fresh.txt", "brand new") // no cycle
|
||||
rep := verifyLocal(t, a)
|
||||
wantOnly(t, "NotYetScanned", rep.NotYetScanned, "fresh.txt")
|
||||
wantEmpty(t, "Drifted", rep.Drifted)
|
||||
wantEmpty(t, "MissingLocally", rep.MissingLocally)
|
||||
}
|
||||
|
||||
// TestVerifyRemoteChunkedFile is the manifests/<sha> criterion: a file over
|
||||
// chunkThreshold is pushed as chunks plus a manifest keyed by the file's own
|
||||
// sha, never blobs/<sha>. A --remote check that asked only blobs/ would report
|
||||
// every large file as missing from the hub.
|
||||
func TestVerifyRemoteChunkedFile(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
a := newDevice(t, "deva", fileRemote(t, dir))
|
||||
big := strings.Repeat("beardrive chunked payload — 64 bytes of filler here!!\n", (chunkThreshold/53)+200_000/53)
|
||||
if len(big) <= chunkThreshold {
|
||||
t.Fatalf("test payload is %d bytes, need > %d", len(big), chunkThreshold)
|
||||
}
|
||||
write(t, a.Folder, "video/demo.bin", big)
|
||||
write(t, a.Folder, "small.txt", "tiny")
|
||||
cycle(t, a)
|
||||
|
||||
// The test is only meaningful if the big file really took the chunked
|
||||
// path: its content must be reachable ONLY under manifests/<sha>, so a
|
||||
// check that asked blobs/<sha> alone would have to report it missing.
|
||||
sum, err := hashFile(filepath.Join(a.Folder, "video/demo.bin"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "manifests", sum)); err != nil {
|
||||
t.Fatalf("the >4MiB file was not chunked, so this test proves nothing: %v", err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "blobs", sum)); err == nil {
|
||||
t.Fatal("the >4MiB file is also a whole blob, so this test proves nothing")
|
||||
}
|
||||
|
||||
rep := verifyRemoteRun(t, a)
|
||||
wantEmpty(t, "MissingOnHub", rep.MissingOnHub)
|
||||
if rep.Problems() != 0 {
|
||||
t.Fatalf("Problems() = %d, want 0 (%+v)", rep.Problems(), rep)
|
||||
}
|
||||
}
|
||||
|
||||
// TestVerifyRemoteMissingBlob: content this device believes is synced, gone
|
||||
// from the hub's storage.
|
||||
func TestVerifyRemoteMissingBlob(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
be := fileRemote(t, dir)
|
||||
a := newDevice(t, "deva", be)
|
||||
|
||||
write(t, a.Folder, "doc.txt", "v1")
|
||||
cycle(t, a)
|
||||
if rep := verifyRemoteRun(t, a); rep.Problems() != 0 {
|
||||
t.Fatalf("before removal: Problems() = %d, want 0 (%+v)", rep.Problems(), rep)
|
||||
}
|
||||
|
||||
// Blow the blob out of the remote directory.
|
||||
blobs := filepath.Join(dir, "blobs")
|
||||
entries, err := os.ReadDir(blobs)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
removed := 0
|
||||
for _, e := range entries {
|
||||
if err := os.RemoveAll(filepath.Join(blobs, e.Name())); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
removed++
|
||||
}
|
||||
if removed == 0 {
|
||||
t.Fatal("no blobs in the remote to remove")
|
||||
}
|
||||
|
||||
rep := verifyRemoteRun(t, a)
|
||||
wantOnly(t, "MissingOnHub", rep.MissingOnHub, "doc.txt")
|
||||
// The local half is still clean — the bytes are on this disk.
|
||||
wantEmpty(t, "Drifted", rep.Drifted)
|
||||
if rep.Elapsed <= 0 {
|
||||
t.Fatal("Elapsed should be measured")
|
||||
}
|
||||
}
|
||||
|
||||
// fileRemote is sharedRemote against a directory the test can reach into, so
|
||||
// it can delete a blob out from under a converged device.
|
||||
func fileRemote(t *testing.T, dir string) remote.Backend {
|
||||
t.Helper()
|
||||
be, err := remote.Open(context.Background(), "file://"+dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return be
|
||||
}
|
||||
@@ -1113,3 +1113,86 @@ func TestCLIStatusReportsUnscannedWork(t *testing.T) {
|
||||
t.Fatalf("drift did not clear after a sync:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCLIVerifyE2E drives `bdrive verify` against a real hub through the real
|
||||
// binary: a clean project passes, an edit made behind the daemon's back is
|
||||
// reported as drifted with a status-1 exit, --remote passes against the live
|
||||
// hub, and --remote against a hub that has gone away still returns the local
|
||||
// verdict rather than failing.
|
||||
func TestCLIVerifyE2E(t *testing.T) {
|
||||
e := newCLIEnv(t)
|
||||
run := e.run
|
||||
|
||||
work := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(work, "index.md"), []byte("# Index\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if out, err := run(work, "init", "--name", "verify-e2e", "--yes"); err != nil {
|
||||
t.Fatalf("init: %v\n%s", err, out)
|
||||
}
|
||||
defer run(work, "stop", work)
|
||||
if out, err := run(work, "sync"); err != nil {
|
||||
t.Fatalf("sync: %v\n%s", err, out)
|
||||
}
|
||||
// Stop the daemon, or it scans the edit below out from under the test.
|
||||
if out, err := run(work, "stop", work); err != nil {
|
||||
t.Fatalf("stop: %v\n%s", err, out)
|
||||
}
|
||||
|
||||
// --- Clean: exit 0, and it says so.
|
||||
out, err := run(work, "verify")
|
||||
if err != nil {
|
||||
t.Fatalf("verify on a clean project should exit 0: %v\n%s", err, out)
|
||||
}
|
||||
if !strings.Contains(out, "OK - the folder matches the journal") {
|
||||
t.Fatalf("clean verify did not report OK:\n%s", out)
|
||||
}
|
||||
// index.md plus the .bdriveignore init seeds — the point is that the cost
|
||||
// is reported at all, not the exact count.
|
||||
if !strings.Contains(out, "checked: 2 files, ") {
|
||||
t.Fatalf("verify did not report what it hashed:\n%s", out)
|
||||
}
|
||||
|
||||
// --- The hub still holds it: --remote is clean too.
|
||||
out, err = run(work, "verify", "--remote")
|
||||
if err != nil {
|
||||
t.Fatalf("verify --remote on a synced project should exit 0: %v\n%s", err, out)
|
||||
}
|
||||
if strings.Contains(out, "missing-on-hub") {
|
||||
t.Fatalf("verify --remote reported content missing from the hub:\n%s", out)
|
||||
}
|
||||
|
||||
// --- Edited behind the daemon's back: drifted, exit 1.
|
||||
if err := os.WriteFile(filepath.Join(work, "index.md"), []byte("# Index\n\nappended by hand\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out, err = run(work, "verify")
|
||||
if err == nil {
|
||||
t.Fatalf("verify should exit 1 when something drifted:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, "drifted (1)") || !strings.Contains(out, "index.md") {
|
||||
t.Fatalf("verify did not name the drifted file:\n%s", out)
|
||||
}
|
||||
// A status is not a stack trace: cobra's usage block must stay silenced.
|
||||
if strings.Contains(out, "Usage:") {
|
||||
t.Fatalf("verify printed a usage block for a findings exit:\n%s", out)
|
||||
}
|
||||
|
||||
// --- Hub gone: the remote leg degrades to a warning, the local verdict
|
||||
// still decides. Put the file back first so "local is clean" is the
|
||||
// thing being asserted.
|
||||
if err := os.WriteFile(filepath.Join(work, "index.md"), []byte("# Index\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
e.hub.Close()
|
||||
out, err = run(work, "verify", "--remote")
|
||||
if err != nil {
|
||||
t.Fatalf("verify --remote against an unreachable hub must not hard-fail: %v\n%s", err, out)
|
||||
}
|
||||
if !strings.Contains(out, "warning:") {
|
||||
t.Fatalf("verify --remote said nothing about the unreachable hub:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, "OK - the folder matches the journal") {
|
||||
t.Fatalf("verify --remote dropped the local verdict when the hub went away:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,7 @@ One binary, `bdrive` — the CLI, the sync daemon, and the web server.
|
||||
| `bdrive scope --explain` | List every path in the folder, split into what syncs and what does not, with counts — the verifiable answer to "what leaves this machine". Pure read: no daemon, no lock, no network |
|
||||
| `bdrive grep <pattern> [folder]` | Search the text **inside** the files a project syncs. `pattern` is a Go RE2 regexp, or a literal string with `-F`. `-i` ignores case, `-l` prints matching paths only, `-n` caps the lines printed (default 200, `0` = all). Pure read: no daemon, no lock, no network |
|
||||
| `bdrive stale [folder]` | Find synced markdown that links to a file written **after** the doc itself — staleness by what moved, not by the calendar. `-l` prints outgrown paths only, `-n` caps the docs printed (default 50, `0` = all). Pure read: no daemon, no lock, no network. Exit status is 0 whether or not anything is stale |
|
||||
| `bdrive verify [folder]` | Prove this folder matches the content the journal records — sha256 every synced file and compare. Reports `drifted`, `never-pushed`, `missing-locally`, `not-yet-scanned`, and with `--remote` also `missing-on-hub`. Pure read: no daemon, no lock, no ops, and without `--remote` no network. Exit status is 0 when every category is empty, 1 otherwise |
|
||||
| `bdrive forget <path>...` | Stop syncing a path and remove it from the hub. Adds the rule to `.bdriveignore` (which syncs) and prunes in one step. Local files are never touched, here or on teammates' devices |
|
||||
| `bdrive url [path]` | Internal hub link for a file or folder — sign-in and membership required. `--sync` pushes first; no argument gives the project home. Computed locally |
|
||||
| `bdrive share <file>` | Public URL for a synced file — links are per-file, so a folder is refused with a file inside it named instead. `--list`, `--revoke`, `--expires` (the hub's Share dialog can also set an expiry on an existing link). Refuses a file whose first 1 MiB holds credential-shaped strings — `--force` shares it anyway |
|
||||
@@ -204,6 +205,67 @@ would invert here and fail on a clean project. Read heat, a badge on the hub's
|
||||
file view, and injecting the flag into an agent's session context are not built
|
||||
yet; this ships the signal.
|
||||
|
||||
### `bdrive verify` — prove this folder matches the hub
|
||||
|
||||
"Your folder is the same everywhere" is the claim. `bdrive verify` is the
|
||||
receipt. It hashes every file the project syncs and compares each one against
|
||||
the content the journal records for it.
|
||||
|
||||
This is what `bdrive status` cannot do. `status` counts pending ops and
|
||||
unscanned changes, but it never reads a byte of content — so a file whose bytes
|
||||
changed while its size and mtime stayed put is invisible to it. `verify`
|
||||
hashes, which is the whole point: there is deliberately no size/mtime fast
|
||||
path, and on a large project it is noticeably slower than `status`. It reports
|
||||
what it hashed and how long it took for exactly that reason.
|
||||
|
||||
```sh
|
||||
bdrive verify
|
||||
# project: team-wiki (m-a3f9c1d2)
|
||||
# checked: 184 files, 41.2 MB hashed in 912ms
|
||||
# OK - the folder matches the journal
|
||||
# (this compares against what this device last pulled — run `bdrive sync` first for the hub's latest)
|
||||
|
||||
bdrive verify --remote # also ask the hub whether it still holds the content
|
||||
bdrive verify ~/wiki # a project other than the current folder
|
||||
```
|
||||
|
||||
It reports five things:
|
||||
|
||||
| Category | Meaning |
|
||||
| --- | --- |
|
||||
| `drifted` | on disk, but not the content the journal records |
|
||||
| `never-pushed` | committed here, never reached the hub |
|
||||
| `missing-locally` | the journal has it, this folder does not |
|
||||
| `not-yet-scanned` | on disk, with no op anywhere yet |
|
||||
| `missing-on-hub` | synced here, absent from the hub — `--remote` only |
|
||||
|
||||
A path the local `.bdriveignore` excludes is **not** reported as missing. The
|
||||
rules are applied symmetrically in scan and materialize, so its absence from
|
||||
disk is correct — a project narrowed with `bdrive scope --only` stays quiet.
|
||||
And when `missing-locally` is really just "not downloaded yet", the output says
|
||||
so and points at `bdrive sync`.
|
||||
|
||||
`--remote` adds one existence check per **distinct blob**, and asks for both
|
||||
`blobs/<sha>` and `manifests/<sha>`. Files over 4 MiB move as content-defined
|
||||
chunks plus a manifest keyed by the file's own sha256, so a check that asked
|
||||
only `blobs/` would call every large file missing from the hub. A manifest
|
||||
existing implies its chunks do — the hub enforces that at ingest — so there is
|
||||
no chunk-by-chunk walk. If the hub is unreachable, the remote half degrades to
|
||||
a printed warning and the local verdict still stands.
|
||||
|
||||
**`verify` never repairs anything.** It is a pure read: no daemon, no lock, no
|
||||
ops, no journal writes. Run `bdrive sync` to reconcile what it found, or
|
||||
`bdrive log <path>` for a file's history.
|
||||
|
||||
**Exit status is 0 when every category is empty and 1 when any is not,** so it
|
||||
composes as a pre-flight check in a script or a CI job.
|
||||
|
||||
One caveat the command prints for itself: the journals it replays are this
|
||||
device's **local copies**. Without a pull it proves "this folder matches what I
|
||||
last pulled" — and with `--remote`, "and the hub still holds all of it". A
|
||||
teammate's newer op is invisible until the next cycle, so sync first if you
|
||||
want the hub's latest.
|
||||
|
||||
### `bdrive forget` and `bdrive sync --prune` — cleaning up the hub
|
||||
|
||||
Adding a rule to `.bdriveignore` only stops *future* uploads. Anything that
|
||||
|
||||
Reference in New Issue
Block a user