Files
beardrive/architecture
0dd474baab Delta sync: large files move as content-defined chunks (#161)
* feat(sync): delta sync — large files move as content-defined chunks

Files over 4 MiB push as chunks/<sha256> pieces plus a manifests/<sha256>
chunk list keyed by the whole file's hash, so Op.Blob alone locates it and
the journal format is byte-identical. A 1-byte edit to a 20 MiB file now
transfers ~2 MB instead of ~21 MB, both directions; chunk boundaries come
from a rolling hash (restic/chunker), so front insertions stay cheap.

The hub reassembles whole blobs on demand (spool, verify, backfill, serve),
which is the entire backward-compatibility story: old clients ask for
blobs/<sha> and never learn anything changed. Proven by e2e tests that build
the real pre-change binary from the pinned merge-base commit.

The push skip-proof is one Exists per chunk — three cheaper proxies (local
basis, manifest existence, stored manifest content) each proved false or
forgeable across four CTO review rounds and are recorded in the code
comment. Hub-side, manifests are write-once and must name only chunks the
store holds; reassembly is bounded at 256 MiB against amplified manifests.

Also: per-file sync ceiling 32 -> 100 MiB; import refuses archives whose
journals reference content they do not hold (--allow-incomplete overrides).

Deploy hubs before clients: old hubs refuse chunk keys (push degrades to
offline-retry), and old clients cap reads at 32 MiB so 32-100 MiB files
report "blob corrupt on remote" until the client upgrades.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6nqxi5a9qcENmJvrgBJF7

* fix(test): fetch the pinned pre-delta commit on shallow CI clones

buildOldBinary archives the pinned merge-base sha, which a fetch-depth-1
actions/checkout does not have — all three old-binary e2e tests failed in
CI with exit 128 while passing on any full local clone. On archive failure,
fetch just that commit (--depth=1, one object; actions/checkout persists
credentials so the in-job fetch works) and retry. Verified against a real
GitHub shallow clone: archive fails, the single-sha fetch succeeds, archive
then yields the pre-delta tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R6nqxi5a9qcENmJvrgBJF7

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 07:18:37 -07:00
..

Architecture diagrams

Mermaid diagrams of the current implementation, kept next to the code so PRs can update them alongside the change.

Convention: when a PR changes the structure drawn here (new/removed types, new seams, changed relationships), update the affected diagram in the same PR and add an "Architecture changes" section to the PR description that, per changed diagram:

  1. names exactly which types/relationships changed and how (one sentence);
  2. shows a Before and an After mermaid block — each an excerpt of only the affected classes and their immediate relationships, never the full diagram (Before comes from the diagram at the merge base).

The committed diagram file stays the full current state; the before/after excerpts exist only in the PR description so reviewers see the structural delta at a glance. A pre-PR hook (.claude/hooks/check-arch-diagrams.sh) reminds Claude Code sessions when server code changed but no diagram did.

Together these cover every application package in the repo — every code change lands inside exactly one detail diagram's scope (plus the overview when the package map or cross-piece wiring changes):

  • overview.md — system diagram: every package and surface on one page, and how they connect
  • cli-sync.md — class diagram of the CLI and sync engine (cmd/bdrive + internal/{syncer,store,journal,config,daemon,agenthooks,autostart})
  • webapp-server.md — class diagram of the bdrive serve server (internal/webapp + its internal/remote seam)
  • webapp-frontend.md — module diagram of the hub's React SPA (internal/webapp/frontend/src)

Not covered on purpose: web/docs (content site, no application code) and cloud/ (private nested repo — its architecture lives there).