mirror of
https://github.com/runbear-io/beardrive.git
synced 2026-08-25 08:08:08 +02:00
post_sync: run a local command when teammates' changes land (#163)
Inbound sync was invisible to the machine it landed on — a local index, cache or notifier had to poll. A `post_sync` command in the folder's own .bdrive/config.json now runs once per cycle that applied peer changes, with the batch as JSON on stdin. The batch rides out on a new Result.Inbound rather than the inbound spool: DrainInbound is destructive and `bdrive sync --hook` is its only consumer, so a second drainer would silently empty the agent's "teammates changed X" context. Both are kept, and both comments now say why. Cycle becomes a thin wrapper over cycleLocked so the hook is spawned after the volume flock drops — a property of the code shape, not a rule each of the seven call sites has to remember. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
0dd474baab
commit
edfe46c0aa
@@ -288,9 +288,34 @@ the project:
|
||||
```jsonc
|
||||
// .bdrive/config.json
|
||||
{ "id": "m-5a10b713", "volume": "notes",
|
||||
"remote": "https://drive.example.com/p/7f3a2c91-4d5e-4b8a-9c17-2ad0f6b3e9c4" }
|
||||
"remote": "https://drive.example.com/p/7f3a2c91-4d5e-4b8a-9c17-2ad0f6b3e9c4",
|
||||
"post_sync": "qmd update && qmd embed" }
|
||||
```
|
||||
|
||||
### `post_sync` — run something when teammates' changes land
|
||||
|
||||
Optional. A shell command run **on this device** after a cycle applies changes
|
||||
from the hub, so a local search index, cache or notifier can stop polling. The
|
||||
applied batch arrives as JSON on stdin:
|
||||
|
||||
```json
|
||||
{ "project": "m-5a10b713", "folder": "/Users/you/notes",
|
||||
"changed": [ { "path": "wiki/onboarding.md", "op": "write" },
|
||||
{ "path": "notes/retired.md", "op": "delete" } ] }
|
||||
```
|
||||
|
||||
Once per cycle that applied at least one path (an initial sync of 400 files is
|
||||
one invocation), inbound only — a cycle that just commits and pushes your own
|
||||
edits fires nothing — and never blocking: the command is spawned detached, and
|
||||
a hook that hangs, exits non-zero or does not exist is logged and forgotten.
|
||||
`bdrive restore` also counts as inbound, since it writes an older version back
|
||||
into the folder.
|
||||
|
||||
It lives in `.bdrive/config.json` and only there. That directory never syncs,
|
||||
so no hub and no teammate can put a command on your machine — but note that
|
||||
`.bdrive/` does travel with a folder you copy by hand, so a folder copied from
|
||||
a teammate brings their `post_sync` with it.
|
||||
|
||||
Opting out is non-destructive: when a pattern starts matching an
|
||||
already-synced file, the file stops syncing but is deleted nowhere — which
|
||||
also means the hub keeps the copy it already has. `bdrive forget <path>` (or
|
||||
|
||||
@@ -23,6 +23,9 @@ classDiagram
|
||||
+Prune bool
|
||||
+OnProgress func
|
||||
+Cycle(ctx) Result
|
||||
-cycleLocked(ctx) Result
|
||||
-firePostSync(res)
|
||||
-logInbound(rel, deleted)
|
||||
+Restore(ctx, path, sha) error
|
||||
-pushChunked(ctx, blob) bytes
|
||||
-fetchChunked(ctx, op, basis) error
|
||||
@@ -33,16 +36,20 @@ classDiagram
|
||||
note for Session "internal/syncer — scan → commit local ops → pull peer journals → adopt on join → re-assert withdrawn ops → preserve conflicts → refresh rules → prune → materialize → push blobs then own journal"
|
||||
note for Session "pull returns TWO lists: newly seen ops, and `gone` — ops a peer deleted from a journal this device had already applied. A peer cannot un-say what we already hold: stillHold re-signs each still-held put into OUR journal (reassertNote). Pull resumes at a byte offset by prefix-matching the local journal copy, so a peer's growing journal is read once"
|
||||
note for Session "Bounds and hardening applied throughout: sizeBound/pullBound cap a fetch against the op's declared size, maxPeerJournals caps how many peers one cycle reads, absorbLamport/tickLamport refuse an absurd peer clock and saturate instead of wrapping, safeMode masks a materialized file to 0777 &^ 0022 (no setuid/setgid, no group/other write), and safeDevice + os.SameFile keep a peer's device id from naming this device's own journal file"
|
||||
note for Session "Cycle is now a thin wrapper: cycleLocked does today's work under the volume flock, then firePostSync spawns the folder's post_sync command AFTER the lock drops, so a hook that runs a bdrive command starts working instead of blocking on the flock the cycle still holds. Splitting the body out is what makes that a property of the code rather than a rule every call site remembers"
|
||||
note for Session "logInbound records each materialized peer path BOTH ways — onto Result.Inbound for the post_sync hook (same process as the cycle) and onto the store's inbound spool for bdrive sync --hook (a later process). DrainInbound is destructive and single-consumer: the hook must never drain it"
|
||||
note for Session "Prune (bdrive forget / sync --prune, never the daemon) journals a delete for every replayed path the SHARED ignore rules exclude — the include scope is per-device and must never prune it"
|
||||
|
||||
class Result {
|
||||
+LocalOps +PulledOps
|
||||
+Conflicts +Adopted +Pruned +Materialized
|
||||
+Inbound []store.InboundEvent
|
||||
+Pushed +Offline +OfflineErr
|
||||
+ReadOnly +NoAccess +AccessErr
|
||||
+Reason() string
|
||||
}
|
||||
note for Result "Adopted counts the paths a JOINING device gave to the project (Cycle step 1b). A first cycle is a join, not an edit: a local file at a path the project already holds is demoted to lamport 0 (adoptNote), so the project's version wins on every device and no conflict copy is made — the local content stays journaled and pushed, which is what bdrive restore reads"
|
||||
note for Result "Inbound names the paths this cycle applied on a peer's behalf — what firePostSync hands the post_sync command as JSON on stdin. Empty on a scan-and-push cycle, which is why a local-edit-only cycle fires nothing"
|
||||
note for Result "Offline / ReadOnly / NoAccess are three different answers: unreachable (retry all), push refused (pull-only), pull refused (pause, touch nothing)"
|
||||
note for Result "Reason() is accessReason(AccessErr): the hub's own sentence for a refusal, minus the wrapper chain, dropped unless it passes journal.SafeText. 'read-only' summarizes the STATUS CODE — the sentence is the only thing that tells a device-registration 403 from a project the user really is a reader on"
|
||||
|
||||
@@ -128,7 +135,7 @@ classDiagram
|
||||
+Lock() flock
|
||||
}
|
||||
note for Store "internal/store — ~/.bdrive/volumes/mount-id: content-addressed blobs, per-device journal copies, state cache, paused marker (free funcs Paused/SetPaused, no flock)"
|
||||
note for Store "inbound.jsonl is the read spool's twin, running the other way: materialize appends every path it wrote or removed for a peer, and `sync --hook` drains it into the turn's context (re-read before editing). A spool and not a Result field because the daemon usually materializes the change seconds before the turn starts, so the hook's own cycle sees nothing. Capped, best-effort, never fails a cycle"
|
||||
note for Store "inbound.jsonl is the read spool's twin, running the other way: materialize appends every path it wrote or removed for a peer, and `sync --hook` drains it into the turn's context (re-read before editing). A spool and not a Result field because the daemon usually materializes the change seconds before the turn starts, so the hook's own cycle sees nothing. Capped, best-effort, never fails a cycle. Result.Inbound now carries the SAME events for the post_sync hook and is not a duplicate to delete: that consumer fires from the cycle itself, and a second drainer would silently empty the agent hook's context"
|
||||
|
||||
class Op {
|
||||
+Seq +Lamport +Time +Device
|
||||
@@ -234,8 +241,10 @@ classDiagram
|
||||
+ID stable mount id
|
||||
+Volume +Remote
|
||||
+Include legacy, read-only
|
||||
+PostSync local hook command
|
||||
}
|
||||
note for Project ".bdrive/config.json — travels with the folder (git clone, copy); presence alone is NOT consent to sync"
|
||||
note for Project "PostSync is a shell command this device runs after a cycle applies peers' changes. It lives here and ONLY here: .bdrive is in ReservedDirs and never syncs, so no hub and no teammate can put a command on someone else's machine. The daemon re-reads this file every tick, so an edit takes effect without a restart"
|
||||
|
||||
class MountRegistry {
|
||||
mounts.json
|
||||
|
||||
@@ -186,6 +186,14 @@ type Project struct {
|
||||
// matching one of these patterns (gitignore-style, same syntax as
|
||||
// .bdriveignore) are scanned and materialized.
|
||||
Include []string `json:"include,omitempty"`
|
||||
// PostSync is a shell command run on THIS device after a cycle applies a
|
||||
// teammate's changes, with the applied batch as JSON on stdin — the event
|
||||
// a local index, cache or notifier can hang off instead of polling.
|
||||
//
|
||||
// It lives here, and only here, on purpose: .bdrive is in ReservedDirs and
|
||||
// never syncs, so no hub response and no peer's journal can put a command
|
||||
// on someone else's machine.
|
||||
PostSync string `json:"post_sync,omitempty"`
|
||||
}
|
||||
|
||||
// mountIDRe is the shape of a mount identity. The id is read verbatim from a
|
||||
|
||||
@@ -1,10 +1,40 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestPostSyncRoundTrip: the hook command survives save/load, and stays out of
|
||||
// the file entirely when unset.
|
||||
func TestPostSyncRoundTrip(t *testing.T) {
|
||||
folder := t.TempDir()
|
||||
if _, err := SaveProject(folder, Project{ID: "m-1234abcd", Volume: "wiki", PostSync: "qmd update && qmd embed"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, ok, err := LoadProject(folder)
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("LoadProject: %v (ok=%v)", err, ok)
|
||||
}
|
||||
if got.PostSync != "qmd update && qmd embed" {
|
||||
t.Fatalf("post_sync = %q, want the saved command", got.PostSync)
|
||||
}
|
||||
|
||||
if _, err := SaveProject(folder, Project{ID: "m-1234abcd", Volume: "wiki"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, err := os.ReadFile(filepath.Join(folder, ProjectDir, "config.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(raw), "post_sync") {
|
||||
t.Fatalf("unset post_sync should be omitted, got %s", raw)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeInclude(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
in []string
|
||||
|
||||
@@ -14,6 +14,12 @@ import (
|
||||
// rather than a field on Result because the daemon usually materializes the
|
||||
// peer's change seconds before the turn starts, so the hook's own cycle
|
||||
// reports nothing: the record has to outlive the cycle that made it.
|
||||
//
|
||||
// syncer.Result.Inbound carries the same events and is NOT a duplicate to be
|
||||
// deleted. The post_sync hook fires from the cycle that materialized, so it
|
||||
// wants the batch in-process; DrainInbound is destructive and single-consumer,
|
||||
// so a second drainer would silently empty the agent hook's context about half
|
||||
// the time. Different lifetimes, different consumers — keep both.
|
||||
|
||||
// InboundEvent is one path a cycle wrote or removed on a peer's behalf
|
||||
// (mount-relative).
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
package syncer
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"os/exec"
|
||||
|
||||
"github.com/runbear-io/beardrive/internal/config"
|
||||
)
|
||||
|
||||
// postSyncPayload is what the hook command reads on stdin.
|
||||
type postSyncPayload struct {
|
||||
Project string `json:"project"`
|
||||
Folder string `json:"folder"`
|
||||
Changed []postSyncChanged `json:"changed"`
|
||||
}
|
||||
|
||||
type postSyncChanged struct {
|
||||
Path string `json:"path"`
|
||||
Op string `json:"op"` // "write" | "delete"
|
||||
}
|
||||
|
||||
// firePostSync runs the folder's post_sync command once, for a cycle that
|
||||
// materialized at least one path on a peer's behalf. It is called from Cycle
|
||||
// AFTER the volume flock is released, so a hook that runs a bdrive command
|
||||
// completes instead of deadlocking.
|
||||
//
|
||||
// Nothing it does can fail the cycle: a missing command, a non-zero exit, or a
|
||||
// hook that never returns is logged (daemon.log for the daemon, stderr for a
|
||||
// one-shot CLI cycle) and forgotten.
|
||||
func (s *Session) firePostSync(res *Result) {
|
||||
if len(res.Inbound) == 0 {
|
||||
return // inbound only: a scan-and-push cycle fires nothing
|
||||
}
|
||||
proj, ok, err := config.LoadProject(s.Folder)
|
||||
if err != nil || !ok || proj.PostSync == "" {
|
||||
return // off unless configured
|
||||
}
|
||||
|
||||
payload := postSyncPayload{Project: s.mountID(), Folder: s.Folder}
|
||||
for _, e := range res.Inbound {
|
||||
op := "write"
|
||||
if e.Deleted {
|
||||
op = "delete"
|
||||
}
|
||||
payload.Changed = append(payload.Changed, postSyncChanged{Path: e.Path, Op: op})
|
||||
}
|
||||
body, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
log.Printf("post_sync: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// stdin is an unlinked temp FILE, not a bytes.Reader: os/exec feeds a
|
||||
// non-*os.File stdin through a pipe served by a goroutine in the PARENT,
|
||||
// and a one-shot `bdrive sync` exits before the child has read it —
|
||||
// delivering truncated JSON. A file descriptor is handed straight to the
|
||||
// child, so the parent may exit immediately.
|
||||
f, err := os.CreateTemp("", "bdrive-postsync-")
|
||||
if err != nil {
|
||||
log.Printf("post_sync: %v", err)
|
||||
return
|
||||
}
|
||||
os.Remove(f.Name()) // the fd is the only handle left
|
||||
defer f.Close()
|
||||
if _, err := f.Write(body); err != nil {
|
||||
log.Printf("post_sync: %v", err)
|
||||
return
|
||||
}
|
||||
if _, err := f.Seek(0, io.SeekStart); err != nil {
|
||||
log.Printf("post_sync: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
// ponytail: no single-flight guard and no timeout — a hook that hangs
|
||||
// while inbound keeps arriving accumulates children. The cycle is
|
||||
// unaffected either way; add coalescing here if a real hook turns out to
|
||||
// be slow.
|
||||
cmd := exec.Command("sh", "-c", proj.PostSync)
|
||||
cmd.Dir, cmd.Stdin = s.Folder, f
|
||||
if err := cmd.Start(); err != nil {
|
||||
log.Printf("post_sync: %v", err)
|
||||
return
|
||||
}
|
||||
go func() {
|
||||
if err := cmd.Wait(); err != nil {
|
||||
log.Printf("post_sync exited: %v", err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
package syncer
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/runbear-io/beardrive/internal/config"
|
||||
)
|
||||
|
||||
// postSync writes a folder's .bdrive/config.json with a post_sync command.
|
||||
func postSync(t *testing.T, folder, cmd string) {
|
||||
t.Helper()
|
||||
if _, err := config.SaveProject(folder, config.Project{PostSync: cmd}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// waitForFile polls until path exists, because the hook is spawned detached:
|
||||
// a bare assert right after cycle() races the child.
|
||||
func waitForFile(t *testing.T, path string, d time.Duration) []byte {
|
||||
t.Helper()
|
||||
deadline := time.Now().Add(d)
|
||||
for {
|
||||
if b, err := os.ReadFile(path); err == nil && len(b) > 0 {
|
||||
return b
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatalf("post_sync marker %s never appeared", path)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
|
||||
// neverAppears is the negative of waitForFile: give the spawn a fair chance,
|
||||
// then assert nothing ran.
|
||||
func neverAppears(t *testing.T, path string) {
|
||||
t.Helper()
|
||||
time.Sleep(300 * time.Millisecond)
|
||||
if _, err := os.Stat(path); err == nil {
|
||||
t.Fatalf("post_sync ran but should not have (%s exists)", path)
|
||||
}
|
||||
}
|
||||
|
||||
func batch(t *testing.T, raw []byte) postSyncPayload {
|
||||
t.Helper()
|
||||
var p postSyncPayload
|
||||
if err := json.Unmarshal(raw, &p); err != nil {
|
||||
t.Fatalf("post_sync stdin is not the JSON batch: %v (%q)", err, raw)
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
// TestResultInboundNamesPeerPaths is step 1 on its own: the cycle carries the
|
||||
// applied paths out on Result, and a local-edit-only cycle carries none.
|
||||
func TestResultInboundNamesPeerPaths(t *testing.T) {
|
||||
rem := sharedRemote(t)
|
||||
a, b := newDevice(t, "deva", rem), newDevice(t, "devb", rem)
|
||||
|
||||
write(t, a.Folder, "wiki/onboarding.md", "hello")
|
||||
write(t, a.Folder, "notes/retired.md", "bye")
|
||||
cycle(t, a)
|
||||
|
||||
res := cycle(t, b)
|
||||
if len(res.Inbound) != 2 {
|
||||
t.Fatalf("Result.Inbound = %v, want 2 writes", res.Inbound)
|
||||
}
|
||||
for _, e := range res.Inbound {
|
||||
if e.Deleted {
|
||||
t.Fatalf("%s reported as a delete, want a write", e.Path)
|
||||
}
|
||||
}
|
||||
|
||||
if err := os.Remove(filepath.Join(a.Folder, "notes", "retired.md")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cycle(t, a)
|
||||
res = cycle(t, b)
|
||||
if len(res.Inbound) != 1 || !res.Inbound[0].Deleted || res.Inbound[0].Path != "notes/retired.md" {
|
||||
t.Fatalf("Result.Inbound = %v, want one delete of notes/retired.md", res.Inbound)
|
||||
}
|
||||
|
||||
// A cycle that only scans and pushes local work is not inbound.
|
||||
write(t, b.Folder, "mine.md", "local only")
|
||||
res = cycle(t, b)
|
||||
if len(res.Inbound) != 0 {
|
||||
t.Fatalf("local-edit cycle reported Inbound = %v, want none", res.Inbound)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPostSyncFiresOncePerInboundBatch is the "400 files, one invocation" AC.
|
||||
func TestPostSyncFiresOncePerInboundBatch(t *testing.T) {
|
||||
rem := sharedRemote(t)
|
||||
a, b := newDevice(t, "deva", rem), newDevice(t, "devb", rem)
|
||||
|
||||
marker := filepath.Join(t.TempDir(), "marker.json")
|
||||
postSync(t, b.Folder, "cat >> "+marker)
|
||||
|
||||
write(t, a.Folder, "one.md", "1")
|
||||
write(t, a.Folder, "two.md", "2")
|
||||
write(t, a.Folder, "wiki/three.md", "3")
|
||||
cycle(t, a)
|
||||
|
||||
cycle(t, b)
|
||||
got := batch(t, waitForFile(t, marker, 5*time.Second))
|
||||
if len(got.Changed) != 3 {
|
||||
t.Fatalf("batch = %+v, want 3 paths in ONE invocation", got.Changed)
|
||||
}
|
||||
for _, c := range got.Changed {
|
||||
if c.Op != "write" {
|
||||
t.Fatalf("%s op = %q, want write", c.Path, c.Op)
|
||||
}
|
||||
}
|
||||
if got.Folder != b.Folder {
|
||||
t.Fatalf("batch folder = %q, want %q", got.Folder, b.Folder)
|
||||
}
|
||||
if got.Project == "" {
|
||||
t.Fatal("batch project is empty")
|
||||
}
|
||||
// >> appends, so a second invocation would show up as a longer file.
|
||||
raw, err := os.ReadFile(marker)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := json.Unmarshal(raw, new(postSyncPayload)); err != nil {
|
||||
t.Fatalf("marker holds more than one batch: %q", raw)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPostSyncSilentWithoutConfig: no post_sync key, no new behavior.
|
||||
func TestPostSyncSilentWithoutConfig(t *testing.T) {
|
||||
rem := sharedRemote(t)
|
||||
a, b := newDevice(t, "deva", rem), newDevice(t, "devb", rem)
|
||||
|
||||
postSync(t, b.Folder, "") // a project config, deliberately with no command
|
||||
|
||||
// Only a spawned command could create this.
|
||||
marker := filepath.Join(t.TempDir(), "marker")
|
||||
|
||||
write(t, a.Folder, "one.md", "1")
|
||||
cycle(t, a)
|
||||
res := cycle(t, b)
|
||||
if res.Materialized == 0 {
|
||||
t.Fatal("nothing materialized; test proves nothing")
|
||||
}
|
||||
if len(res.Inbound) != 1 {
|
||||
t.Fatalf("Result.Inbound = %v, want the one applied path", res.Inbound)
|
||||
}
|
||||
neverAppears(t, marker)
|
||||
}
|
||||
|
||||
// TestPostSyncSkipsLocalOnlyCycle: scan + push is not an inbound event.
|
||||
func TestPostSyncSkipsLocalOnlyCycle(t *testing.T) {
|
||||
rem := sharedRemote(t)
|
||||
b := newDevice(t, "devb", rem)
|
||||
|
||||
marker := filepath.Join(t.TempDir(), "marker")
|
||||
postSync(t, b.Folder, "cat > "+marker)
|
||||
|
||||
write(t, b.Folder, "mine.md", "local only")
|
||||
res := cycle(t, b)
|
||||
if res.LocalOps == 0 {
|
||||
t.Fatal("expected a local op; test proves nothing")
|
||||
}
|
||||
neverAppears(t, marker)
|
||||
}
|
||||
|
||||
// TestPostSyncReportsDeletes: each entry distinguishes write from delete.
|
||||
func TestPostSyncReportsDeletes(t *testing.T) {
|
||||
rem := sharedRemote(t)
|
||||
a, b := newDevice(t, "deva", rem), newDevice(t, "devb", rem)
|
||||
|
||||
write(t, a.Folder, "gone.md", "here")
|
||||
cycle(t, a)
|
||||
cycle(t, b) // b now holds it
|
||||
|
||||
marker := filepath.Join(t.TempDir(), "marker.json")
|
||||
postSync(t, b.Folder, "cat > "+marker)
|
||||
|
||||
if err := os.Remove(filepath.Join(a.Folder, "gone.md")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cycle(t, a)
|
||||
cycle(t, b)
|
||||
|
||||
got := batch(t, waitForFile(t, marker, 5*time.Second))
|
||||
if len(got.Changed) != 1 || got.Changed[0].Path != "gone.md" || got.Changed[0].Op != "delete" {
|
||||
t.Fatalf("batch = %+v, want one delete of gone.md", got.Changed)
|
||||
}
|
||||
}
|
||||
|
||||
// TestPostSyncFailureNeverBreaksCycle: a hook that exits non-zero or does not
|
||||
// exist leaves the cycle and the next one alone.
|
||||
func TestPostSyncFailureNeverBreaksCycle(t *testing.T) {
|
||||
for _, cmd := range []string{"exit 3", "definitely-not-a-real-binary-xyz"} {
|
||||
rem := sharedRemote(t)
|
||||
a, b := newDevice(t, "deva", rem), newDevice(t, "devb", rem)
|
||||
postSync(t, b.Folder, cmd)
|
||||
|
||||
write(t, a.Folder, "one.md", "1")
|
||||
cycle(t, a)
|
||||
res, err := b.Cycle(t.Context())
|
||||
if err != nil {
|
||||
t.Fatalf("%q: Cycle returned %v", cmd, err)
|
||||
}
|
||||
if res.Materialized != 1 || len(res.Inbound) != 1 {
|
||||
t.Fatalf("%q: Result affected by the hook: %+v", cmd, res)
|
||||
}
|
||||
|
||||
// The next cycle still converges.
|
||||
write(t, a.Folder, "two.md", "2")
|
||||
cycle(t, a)
|
||||
cycle(t, b)
|
||||
if read(t, b.Folder, "two.md") != "2" {
|
||||
t.Fatalf("%q: sync stopped converging after a failing hook", cmd)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestPostSyncLeavesInboundSpool is the guard on the trap: firing the hook
|
||||
// must not consume what `bdrive sync --hook` reports to the agent.
|
||||
func TestPostSyncLeavesInboundSpool(t *testing.T) {
|
||||
rem := sharedRemote(t)
|
||||
a, b := newDevice(t, "deva", rem), newDevice(t, "devb", rem)
|
||||
|
||||
marker := filepath.Join(t.TempDir(), "marker.json")
|
||||
postSync(t, b.Folder, "cat > "+marker)
|
||||
|
||||
write(t, a.Folder, "wiki/page.md", "hi")
|
||||
cycle(t, a)
|
||||
cycle(t, b)
|
||||
waitForFile(t, marker, 5*time.Second)
|
||||
|
||||
evs, err := b.Store.DrainInbound()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(evs) != 1 || evs[0].Path != "wiki/page.md" {
|
||||
t.Fatalf("inbound spool = %v, want wiki/page.md still queued for the agent hook", evs)
|
||||
}
|
||||
}
|
||||
@@ -92,6 +92,21 @@ type Session struct {
|
||||
// be invoked concurrently from upload workers, so it must be safe to call
|
||||
// from multiple goroutines.
|
||||
OnProgress func(Progress)
|
||||
|
||||
// inbound accumulates this cycle's materialized peer paths, for
|
||||
// Result.Inbound. Reset at the top of every cycle — a Session is reused
|
||||
// across cycles by the daemon and by the tests.
|
||||
inbound []store.InboundEvent
|
||||
}
|
||||
|
||||
// logInbound records one materialized peer path both ways: on this cycle's
|
||||
// Result (for the post_sync hook, which fires from the cycle that did the
|
||||
// work) and on the store's spool (for `bdrive sync --hook`, which runs in a
|
||||
// later process). Both consumers want the same event; neither may consume the
|
||||
// other's copy.
|
||||
func (s *Session) logInbound(rel string, deleted bool) {
|
||||
s.inbound = append(s.inbound, store.InboundEvent{Path: rel, Deleted: deleted, Time: time.Now().UTC()})
|
||||
_ = s.Store.LogInbound(rel, deleted) // best-effort: never fails a cycle
|
||||
}
|
||||
|
||||
func (s *Session) mountID() string {
|
||||
@@ -124,6 +139,12 @@ type Result struct {
|
||||
Pruned int // paths removed from the hub by --prune (kept on disk)
|
||||
Materialized int // files written/removed in the working folder
|
||||
Pushed bool // own journal/blobs uploaded
|
||||
// Inbound names the paths this cycle materialized on a peer's behalf —
|
||||
// the same events the inbound spool queues, carried out of the cycle that
|
||||
// made them so the post_sync hook can hand them to a local command. It
|
||||
// does NOT replace the spool: see the comment in internal/store/inbound.go
|
||||
// for why the two coexist.
|
||||
Inbound []store.InboundEvent
|
||||
// Offline reports that the remote leg of this cycle had a problem worth
|
||||
// telling the user about — usually "unreachable", but also "the hub served
|
||||
// bytes that are not their content address", which is the only signal that
|
||||
@@ -215,14 +236,28 @@ func tickLamport(cur int64) int64 {
|
||||
return cur + 1
|
||||
}
|
||||
|
||||
// Cycle runs one full scan/sync/materialize pass under the volume lock.
|
||||
// Cycle runs one full scan/sync/materialize pass under the volume lock, then
|
||||
// fires the folder's post_sync hook — after the lock is released, so a hook
|
||||
// that itself runs a bdrive command starts working immediately instead of
|
||||
// blocking on the flock the cycle that spawned it is still holding (a long
|
||||
// push can hold it for a while). Splitting the body out is what makes that
|
||||
// ordering a property of the code: no call site has to remember it.
|
||||
func (s *Session) Cycle(ctx context.Context) (*Result, error) {
|
||||
res, err := s.cycleLocked(ctx)
|
||||
if err == nil && res != nil {
|
||||
s.firePostSync(res)
|
||||
}
|
||||
return res, err
|
||||
}
|
||||
|
||||
func (s *Session) cycleLocked(ctx context.Context) (*Result, error) {
|
||||
unlock, err := s.Store.Lock()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("lock volume: %w", err)
|
||||
}
|
||||
defer unlock()
|
||||
|
||||
s.inbound = nil
|
||||
res := &Result{}
|
||||
cache, err := s.Store.LoadCache(s.mountID())
|
||||
if err != nil {
|
||||
@@ -523,6 +558,7 @@ func (s *Session) Cycle(ctx context.Context) (*Result, error) {
|
||||
return nil, fmt.Errorf("materialize: %w", err)
|
||||
}
|
||||
res.Materialized += n
|
||||
res.Inbound = s.inbound
|
||||
|
||||
// 5. Push our blobs and journal.
|
||||
if s.Backend != nil && !blocked && int64(len(myOps)) > st.PushedOps {
|
||||
@@ -1300,7 +1336,7 @@ func (s *Session) materialize(target map[string]journal.FileState, cache map[str
|
||||
// a spool failure must never fail a cycle. Only the branch that
|
||||
// actually unlinked a file logs — the paths below it were already
|
||||
// gone locally, so nothing changed under the agent.
|
||||
_ = s.Store.LogInbound(rel, true)
|
||||
s.logInbound(rel, true)
|
||||
}
|
||||
delete(cache, rel)
|
||||
changed++
|
||||
@@ -1351,7 +1387,7 @@ func (s *Session) materializeFile(rel string, want journal.FileState, cache map[
|
||||
// Known trade: the first cycle on a fresh mount materializes the whole
|
||||
// project, so that one turn's list is "everything" — bounded by the
|
||||
// hook's render cap, and cheaper than special-casing an empty cache.
|
||||
_ = s.Store.LogInbound(rel, false)
|
||||
s.logInbound(rel, false)
|
||||
return true, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -36,6 +36,7 @@ type cliEnv struct {
|
||||
hub *httptest.Server
|
||||
browser *http.Client
|
||||
home string // the isolated HOME; hooks live under here now
|
||||
bin string // the real binary, for tests that need a bdrive to run
|
||||
}
|
||||
|
||||
func newCLIEnv(t *testing.T) cliEnv {
|
||||
@@ -107,7 +108,7 @@ func newCLIEnvBin(t *testing.T, hub *httptest.Server, bin string) cliEnv {
|
||||
out, _ := os.ReadFile(logFile)
|
||||
t.Fatalf("login --device: %v\n%s", err, out)
|
||||
}
|
||||
return cliEnv{run: run, hub: hub, browser: browser, home: home}
|
||||
return cliEnv{run: run, hub: hub, browser: browser, home: home, bin: bin}
|
||||
}
|
||||
|
||||
func TestCLIOnboardingE2E(t *testing.T) {
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
package webapp
|
||||
|
||||
// post_sync against the real binary and a real volume flock: a hook that runs
|
||||
// a bdrive subcommand — the case the whole "fire outside the lock" rule exists
|
||||
// for — receives the batch on stdin and runs to completion.
|
||||
//
|
||||
// It does NOT prove the ordering, and the plan's claim that it would is wrong:
|
||||
// the child is detached and nothing waits on it, so a hook spawned INSIDE the
|
||||
// flock stalls on LOCK_EX until the parent's cycle ends rather than deadlocking
|
||||
// (verified by moving the call and watching this test still pass). Firing after
|
||||
// the unlock is still the right shape — a hook must not queue behind a long
|
||||
// push — but it is a code-shape property, guarded by Cycle being a wrapper,
|
||||
// not by an assertion here.
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/runbear-io/beardrive/internal/config"
|
||||
)
|
||||
|
||||
func TestCLIPostSyncRunningBdriveDoesNotDeadlock(t *testing.T) {
|
||||
hub := startTestHub(t)
|
||||
first := newCLIEnvOn(t, hub)
|
||||
|
||||
owner := filepath.Join(t.TempDir(), "team")
|
||||
if err := os.MkdirAll(owner, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(owner, "seed.md"), []byte("# seed\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if out, err := first.run(owner, "init", "--name", "postsync-e2e", "--yes"); err != nil {
|
||||
t.Fatalf("init owner: %v\n%s", err, out)
|
||||
}
|
||||
defer first.run(owner, "stop", owner)
|
||||
|
||||
second := newCLIEnvOn(t, hub)
|
||||
joiner := filepath.Join(t.TempDir(), "joined")
|
||||
if err := os.MkdirAll(joiner, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
id := projectIDByName(t, first.browser, hub.URL, "postsync-e2e")
|
||||
if out, err := second.run(joiner, "init", "--project", id, "--yes"); err != nil {
|
||||
t.Fatalf("init joiner: %v\n%s", err, out)
|
||||
}
|
||||
defer second.run(joiner, "stop", joiner)
|
||||
|
||||
// The hook runs a bdrive command that takes the same volume flock the
|
||||
// cycle spawning it just held, then leaves a marker holding the batch.
|
||||
marker := filepath.Join(t.TempDir(), "marker.json")
|
||||
batchFile := filepath.Join(t.TempDir(), "batch.json")
|
||||
cfgPath := filepath.Join(joiner, ".bdrive", "config.json")
|
||||
proj, ok, err := config.LoadProject(joiner)
|
||||
if err != nil || !ok {
|
||||
t.Fatalf("load joiner project: %v (ok=%v)", err, ok)
|
||||
}
|
||||
proj.PostSync = "cat > " + batchFile + " && " + second.bin + " sync " + joiner + " > /dev/null 2>&1 && mv " + batchFile + " " + marker
|
||||
body, err := json.Marshal(proj)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(cfgPath, body, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// A teammate's change, then the joiner picks it up.
|
||||
if err := os.WriteFile(filepath.Join(owner, "from-owner.md"), []byte("hello\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if out, err := first.run(owner, "sync"); err != nil {
|
||||
t.Fatalf("owner sync: %v\n%s", err, out)
|
||||
}
|
||||
if out, err := second.run(joiner, "sync"); err != nil {
|
||||
t.Fatalf("joiner sync: %v\n%s", err, out)
|
||||
}
|
||||
|
||||
// Whichever cycle applied it — this sync or the joiner's daemon — the hook
|
||||
// must have run its bdrive command to completion.
|
||||
deadline := time.Now().Add(60 * time.Second)
|
||||
for {
|
||||
raw, err := os.ReadFile(marker)
|
||||
if err == nil {
|
||||
var got struct {
|
||||
Changed []struct{ Path, Op string } `json:"changed"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &got); err != nil {
|
||||
t.Fatalf("post_sync stdin was not the JSON batch: %v (%q)", err, raw)
|
||||
}
|
||||
var found bool
|
||||
for _, c := range got.Changed {
|
||||
if c.Path == "from-owner.md" && c.Op == "write" {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatalf("batch %+v does not name the teammate's file", got.Changed)
|
||||
}
|
||||
return
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("post_sync running `bdrive sync` never completed — spawned inside the volume flock?")
|
||||
}
|
||||
time.Sleep(200 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
@@ -30,6 +30,42 @@ It is **never synced** and holds **no credentials**; the session token stays in
|
||||
Because everything is keyed by the mount id, the folder can be renamed or moved
|
||||
freely. Copy it to another machine and `bdrive init` resumes the same project.
|
||||
|
||||
### `post_sync`
|
||||
|
||||
An optional shell command run **on this device** after a sync applies changes
|
||||
from the hub — the event a local search index, cache or notifier can hang off
|
||||
instead of polling.
|
||||
|
||||
```jsonc
|
||||
// .bdrive/config.json
|
||||
{ "id": "m-5a10b713", "volume": "notes",
|
||||
"remote": "https://drive.example.com/p/7f3a2c91-…",
|
||||
"post_sync": "qmd update && qmd embed" }
|
||||
```
|
||||
|
||||
The applied batch arrives as JSON on stdin, with the command's working
|
||||
directory set to the folder:
|
||||
|
||||
```json
|
||||
{ "project": "m-5a10b713", "folder": "/Users/you/notes",
|
||||
"changed": [ { "path": "wiki/onboarding.md", "op": "write" },
|
||||
{ "path": "notes/retired.md", "op": "delete" } ] }
|
||||
```
|
||||
|
||||
- **Once per cycle** that applied at least one path — an initial sync of 400
|
||||
files is one invocation, not 400.
|
||||
- **Inbound only.** A cycle that only commits and pushes your own edits fires
|
||||
nothing. (`bdrive restore` does count: it writes an older version back into
|
||||
the folder.)
|
||||
- **Never blocks sync.** The command is spawned detached; one that hangs, exits
|
||||
non-zero or does not exist is logged to `daemon.log` and forgotten.
|
||||
- **Off unless set.** No `post_sync` key, no new behavior.
|
||||
|
||||
This is local configuration and nothing else can set it: `.bdrive/` never syncs,
|
||||
so no hub response and no teammate's change can put a command on your machine.
|
||||
The one path in is your own hands — `.bdrive/` travels with a folder, so a
|
||||
folder you copy from a teammate brings their `post_sync` along with it.
|
||||
|
||||
## `.bdriveignore`
|
||||
|
||||
A gitignore-style opt-out list at the mount root. It syncs like a normal file,
|
||||
|
||||
Reference in New Issue
Block a user