feat(hub): see what each agent session read, not just what it changed (BEA-98) (#135)

History showed what an agent run CHANGED. What it read lived in a daily
aggregate with no session dimension, so the two could not be joined and
nobody could answer "when my agent answered, what did it look at — and was
it the fresh version or archive/retired-spec.md?".

The join is one string carried through four places: hook -> spool -> hub ->
run card. A run card now marks each change the run also read, lists the
files it read and never touched, and says on screen why a read can be
missing.

The three landmines the issue asks be named here:

1. Op.Note is USER-SETTABLE (`bdrive sync --note`), so joining reads to
   writes on the note string would let any member with write access forge a
   note that collides with a teammate's run card and hang their reads off
   it. Fixed by adding journal.Op.Session — set only by `bdrive sync
   --hook`, never by --note — and joining on that. The note stays settable
   and stays untrusted; the join simply never reads it. Op.Session is
   additive JSONL and, like Mtime, is never an input to Less or Replay, so
   replay determinism is untouched and older ops carry "".

   The read half has the same hole one step further on: POST /reads takes
   the session id from the CLIENT, so a member could report reads under a
   teammate's session and paint files onto their card. Every session row is
   therefore pinned to the ownsDevice-validated device, and the query
   requires ?session= AND ?device= together — a forged row can only be found
   under the forger's own device, which MayActAs guarantees is never
   somebody else's.

2. BUCKET CARDINALITY. Putting the session in the read_stats key would take
   a 2k-file project from ~2k to ~100k rows/day, into a table ReadLedger
   loads whole at boot and full-scans on every heat request, hub-wide — so
   it would slow the Dashboard for projects that never ran an agent. This
   is the escape hatch the spec itself names, taken up front: session rows
   live in their own read_sessions repo, outside ReadLedger.byKey. No
   read_stats PK migration, no change to the resident-row count, ?by=device
   byte-identical. They get their own retention (session_retention_days,
   default 30) which DELETES rather than folds — no heat total was ever
   derived from them.

3. READS ARE RECORDED ONLY FOR PATHS IN THE CURRENT REPLAY, so a session
   that read a file it then deleted shows a change with no read. That is by
   design, and the run card says so in its footer rather than leaving it to
   read as a bug.

Privacy ruling, written into internal/webapp/reads.go before anything
serves it: a session id appears only in History responses on the op that
carries it, and as a ?session= filter INPUT. It is never enumerated — no
listing endpoint, no session column in /heat output, nothing new in
?by=device.

Also: PendingReads now dedupes on (path, session), not path alone. Two
agent sessions on one device between syncs used to collapse into one event
carrying whichever session flushed last — one session's reads silently
credited to another.

Tests: journal round-trip + Less-ignores-Session; the forge test (`sync
--note "claude-code session <someone-else's>"` leaves Session empty); a
multi-device syncer test carrying the session through convergence; spool
per-session dedup; hub round-trip, cross-device forge, query contract and
non-enumeration; db_conformance on file, sqlite AND postgres; runs.ts
grouping incl. legacy fallback; a Playwright spec on the seeded run card.
This commit is contained in:
Snow Lee (Sungwon)
2026-08-11 04:18:53 +09:00
committed by GitHub
parent 831d5cda31
commit 5f1ac98dae
40 changed files with 1498 additions and 218 deletions
+5 -2
View File
@@ -19,6 +19,7 @@ classDiagram
+Account config.Settings
+Backend remote.Backend
+Note string
+SessionID string
+Prune bool
+OnProgress func
+Cycle(ctx) Result
@@ -107,7 +108,8 @@ classDiagram
+LoadCache / SaveCache mountID
+LoadSync / SaveSync
+SaveNote / LoadNote
+PendingReads read spool
+LogRead(rel, session) read spool
+PendingReads dedup on path+session
+Lock() flock
}
note for Store "internal/store — ~/.bdrive/volumes/mount-id: content-addressed blobs, per-device journal copies, state cache, paused marker (free funcs Paused/SetPaused, no flock)"
@@ -117,9 +119,10 @@ classDiagram
+Author +User +UserName
+Kind put or delete
+Path +Blob +Size +Mode +Note
+Session agent session, hook-set
+Mtime when the file was written
}
note for Op "internal/journal — Less orders by (lamport, time, device, seq); Replay folds to LWW-per-path state; each device writes only its own journal. Mtime is display-only (bdrive log shows it, falling back to Time) and never feeds Less or Replay"
note for Op "internal/journal — Less orders by (lamport, time, device, seq); Replay folds to LWW-per-path state; each device writes only its own journal. Mtime is display-only (bdrive log shows it, falling back to Time) and never feeds Less or Replay. Session holds the same standing: set only by `bdrive sync --hook` (never by --note, which any member can spell), display/join-only, and the key History run cards group on — a note is forgeable, a session id is not"
note for Op "Op now owns its own JSON: a Path that is not valid UTF-8 rides as a base64 `path_raw` sidecar and is restored only when the lossy form still matches, so one line can never name two different files on two readers. Less falls through to Kind/Path/Blob/Size/Mode, making the order TOTAL — two ops can no longer tie and replay differently per device. Parse skips an undecodable line and drops an unknown Kind instead of failing the whole journal"
class Backend {
+20 -3
View File
@@ -295,15 +295,23 @@ classDiagram
class ReadLedger {
-repo ReadRepo
-retention
-byKey, dirty, seen
-sessions SessionReadRepo
-retention, sessionRetention
-byKey, dirty, seen, pendingSess
+Record(...)
+RecordSession(project, session, device, path)
+Heat(project, prefix, days)
+SessionPaths(project, session, device)
+WithSessions(repo, days)
+ShareOpens(project)
}
class ReadStat {
+Project +Path +Day +Kind +Actor +Count +Last
}
class SessionRead {
+Project +Session +Device +Path +Last
}
note for SessionRead "One row per (session, device, path) — the per-session detail a History run card joins its writes to, on the un-forgeable Op.Session and never on the note. Deliberately OUTSIDE ReadLedger.byKey: that map is loaded whole at boot and full-scanned by Heat on every request, hub-wide, so session cardinality in it would slow the Dashboard for projects that never ran an agent. Device is always the ownsDevice-validated id, never a client field, so a report naming someone else's session can only ever be found under the forger's own device. Its own, much shorter retention (session_retention_days, default 30) DELETES rather than folds — no heat total was ever derived from it"
class HeatEntry {
+Human +Agent +Share +Readers +LastRead
}
@@ -415,6 +423,7 @@ classDiagram
RemoteSource ..> sourcedOp : attribution comes from the journal key
RemoteSource *-- cachedJournal : parsed ops, keyed on size+mtime
ReadLedger ..> ReadStat
ReadLedger ..> SessionRead
ReadLedger ..> HeatEntry
ReadLedger ..> ShareOpen
ShareDB ..> ShareOpen : shares list joins the open count per path
@@ -446,6 +455,7 @@ classDiagram
+Shares() ShareRepo
+Devices() DeviceRepo
+Reads() ReadRepo
+SessionReads() SessionReadRepo
+Close()
}
@@ -500,7 +510,7 @@ classDiagram
storable / storableMap
checkAccount checkToken checkProject
checkOrg checkInvite checkShare
checkDevice checkReadStat
checkDevice checkReadStat checkSessionRead
}
note for storable "Called at the top of every repo write in BOTH backends. A NUL byte or invalid UTF-8 in a name is accepted by JSON and rejected by Postgres, so the file backend used to persist rows the SQL backend would refuse — the same hub, migrated, would silently lose them. Refusing at one gate makes the two backends agree on what is storable"
class ReadRepo {
@@ -508,6 +518,11 @@ classDiagram
+Load() +PutBatch +DeleteBatch
}
note for ReadRepo "batch-oriented: one flush = one write"
class SessionReadRepo {
<<interface>>
+PutBatch +ListBySession +PruneBefore
}
note for SessionReadRepo "read_sessions / sessions.json — never Load()ed whole; queried by (project, session, device) and pruned by date, which is what keeps the boot load and Heat's scan the size they are today"
class fileMetaStore {
JSON files, atomic rewrite per change
@@ -530,6 +545,7 @@ classDiagram
MetaStore *-- ShareRepo
MetaStore *-- DeviceRepo
MetaStore *-- ReadRepo
MetaStore *-- SessionReadRepo
class BuiltinAuth
class ProjectDB
@@ -544,6 +560,7 @@ classDiagram
ShareDB o-- ShareRepo
DeviceRegistry o-- DeviceRepo
ReadLedger o-- ReadRepo
ReadLedger o-- SessionReadRepo
BuiltinAuth *-- versionGate
ProjectDB *-- versionGate
+14
View File
@@ -44,6 +44,14 @@ type hookLink struct {
// mounts.
func hookSessionID(cmd *cobra.Command) string {
data, _ := io.ReadAll(io.LimitReader(cmd.InOrStdin(), 1<<20))
return eventSessionID(data)
}
// eventSessionID is that parse over an already-read payload — `bdrive
// read-log` consumes the same stdin for its own reasons and tags every read
// it spools with the same id, which is what lets the hub join a run's reads
// to its writes.
func eventSessionID(data []byte) string {
var event struct {
SessionID string `json:"session_id"`
}
@@ -64,6 +72,12 @@ func runHookSync(cmd *cobra.Command, target, sessionID, label string) (string, b
if err := sess.Store.SaveNote(note, hookNoteTTL); err == nil {
sess.Note = note
}
// The hook is the ONLY writer of Op.Session — `bdrive sync --note`
// cannot reach it, which is what makes a run card's identity
// un-forgeable. Unlike the note it is not persisted with a TTL: a
// later daemon scan should not credit its own changes to a session
// that has moved on.
sess.SessionID = sessionID
}
// The pull. Offline is fine — the link formula below is still valid
+8 -4
View File
@@ -41,18 +41,22 @@ to run it by hand.`,
return nil
}
data, _ := io.ReadAll(io.LimitReader(cmd.InOrStdin(), 1<<20))
// Parsed once, like syncCmd hoists it: stdin is already drained
// here, and logReads runs once per mount.
session := eventSessionID(data)
// The session's directory is rarely the mount root, so reads are
// attributed to whichever mount actually contains them.
for _, target := range syncTargets(folder) {
logReads(target, data)
logReads(target, data, session)
}
return nil
},
}
}
// logReads spools the reads from one hook event that fall inside one mount.
func logReads(folder string, data []byte) {
// logReads spools the reads from one hook event that fall inside one mount,
// tagged with the agent session they happened in (see store.ReadEvent).
func logReads(folder string, data []byte, session string) {
// LoadProject, not ResolveMount: a hook must never enroll this
// device (registry self-heal) — and syncBlocked keeps a paused
// or never-inited project's spool from even being created.
@@ -89,7 +93,7 @@ func logReads(folder string, data []byte) {
if filter.Skip(rel) {
continue // not part of the project (ignore/include rules)
}
st.LogRead(rel) // best-effort; the hook must never fail the turn
st.LogRead(rel, session) // best-effort; the hook must never fail the turn
}
}
+6
View File
@@ -135,6 +135,12 @@ func TestReadLogCommand(t *testing.T) {
if len(evs) != 1 || evs[0].Path != "wiki/a.md" {
t.Fatalf("spool = %+v, want just the in-project read, mount-relative", evs)
}
// The same session id `bdrive sync --hook` stamps onto the writes, off
// the same stdin payload — it is what lets the hub join this read to the
// run card that turn produced.
if evs[0].Session != "abc" {
t.Fatalf("spooled read Session = %q, want the event's session_id", evs[0].Session)
}
}
// read-log fires on every agent tool call in every folder, so it must be
+117
View File
@@ -0,0 +1,117 @@
package main
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
"github.com/runbear-io/beardrive/internal/config"
"github.com/runbear-io/beardrive/internal/journal"
)
// journalOps reads the ops this device wrote for a project.
func journalOps(t *testing.T, projectID, deviceID string) []journal.Op {
t.Helper()
vdir, err := config.VolumeDir(projectID)
if err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(filepath.Join(vdir, "journal", deviceID+".jsonl"))
if err != nil {
t.Fatal(err)
}
ops, err := journal.Parse(data)
if err != nil {
t.Fatal(err)
}
return ops
}
// stampFixture is a mount with one file, ready to commit.
func stampFixture(t *testing.T) (folder string, proj config.Project) {
t.Helper()
t.Setenv("BDRIVE_HOME", t.TempDir())
folder = t.TempDir()
folder, _ = filepath.EvalSymlinks(folder)
var err error
proj, err = config.SaveProject(folder, config.Project{
Volume: "wiki",
Remote: "https://hub.example.com/p/p-12345678", // unreachable: the cycle degrades offline, the scan still commits
})
if err != nil {
t.Fatal(err)
}
if _, _, err := config.EnrollMount(folder); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(folder, "a.md"), []byte("hello\n"), 0o644); err != nil {
t.Fatal(err)
}
return folder, proj
}
func thisDevice(t *testing.T) string {
t.Helper()
dev, err := config.LoadDevice()
if err != nil {
t.Fatal(err)
}
return dev.ID
}
// The hook sets BOTH the note and the session id — the note is the label a
// reader sees, the session is the key a run card joins its reads on.
func TestHookStampsSession(t *testing.T) {
folder, proj := stampFixture(t)
c := syncCmd()
c.SetOut(&bytes.Buffer{})
c.SetIn(strings.NewReader(`{"session_id":"sess-42"}`))
c.SetArgs([]string{folder, "--hook", "claude-code"})
if err := c.Execute(); err != nil {
t.Fatalf("hook mode must never fail: %v", err)
}
ops := journalOps(t, proj.ID, thisDevice(t))
if len(ops) == 0 {
t.Fatal("the hook run committed nothing")
}
for _, op := range ops {
if op.Session != "sess-42" {
t.Errorf("op %q Session = %q, want sess-42", op.Path, op.Session)
}
if op.Note != "claude-code session sess-42" {
t.Errorf("op %q Note = %q", op.Path, op.Note)
}
}
}
// Landmine 1, tested explicitly: Op.Note is user-settable, so `bdrive sync
// --note` can spell out any other member's session card verbatim. It must
// still produce an EMPTY Op.Session, so nothing it writes can attach to that
// member's run — the join reads the session, never the note.
func TestSyncNoteCannotForgeASession(t *testing.T) {
folder, proj := stampFixture(t)
c := syncCmd()
c.SetOut(&bytes.Buffer{})
c.SetArgs([]string{folder, "--note", "claude-code session sess-42"})
if err := c.Execute(); err != nil {
t.Fatalf("sync: %v", err)
}
ops := journalOps(t, proj.ID, thisDevice(t))
if len(ops) == 0 {
t.Fatal("the sync committed nothing")
}
for _, op := range ops {
if op.Session != "" {
t.Errorf("--note forged a session id on %q: %q", op.Path, op.Session)
}
if op.Note != "claude-code session sess-42" {
t.Errorf("op %q Note = %q, want the note to still be settable", op.Path, op.Note)
}
}
}
+14 -2
View File
@@ -70,6 +70,12 @@ type webConfig struct {
Reads *struct {
Enabled *bool `json:"enabled,omitempty"` // default true
RetentionDays int `json:"retention_days,omitempty"` // default 400; older days fold into all-time
// SessionRetentionDays bounds the per-session read detail behind
// History's run cards (which files an agent session read). Default
// 30, and deliberately much shorter than RetentionDays: this is
// event-shaped rather than aggregate, and rows past it are deleted,
// which changes no heat total.
SessionRetentionDays int `json:"session_retention_days,omitempty"`
} `json:"reads,omitempty"`
}
@@ -380,23 +386,29 @@ credentials); otherwise it is relayed through this server.`,
return fmt.Errorf("open share registry: %w", err)
}
srv.Shares = shares
readsOn, retention := true, 0
readsOn, retention, sessRetention := true, 0, 0
if cfg.Reads != nil {
if cfg.Reads.Enabled != nil {
readsOn = *cfg.Reads.Enabled
}
retention = cfg.Reads.RetentionDays
sessRetention = cfg.Reads.SessionRetentionDays
}
if readsOn {
var reads *webapp.ReadLedger
var sessions webapp.SessionReadRepo
if meta != nil {
reads, err = webapp.NewReadLedger(meta.Reads(), retention)
sessions = meta.SessionReads()
} else {
reads, err = webapp.OpenReadLedger(filepath.Join(filepath.Dir(projectsDB), "reads.json"), retention)
dir := filepath.Dir(projectsDB)
reads, err = webapp.OpenReadLedger(filepath.Join(dir, "reads.json"), retention)
sessions = webapp.OpenSessionReadRepo(filepath.Join(dir, "sessions.json"))
}
if err != nil {
return fmt.Errorf("open read ledger: %w", err)
}
reads.WithSessions(sessions, sessRetention)
defer reads.Close()
srv.Reads = reads
}
+12
View File
@@ -43,6 +43,18 @@ type Op struct {
Size int64 `json:"size,omitempty"`
Mode uint32 `json:"mode,omitempty"` // permission bits
Note string `json:"note,omitempty"` // e.g. "conflict copy of <path>"
// Session is the agent session this op was committed during, set ONLY by
// the agent sync hook (`bdrive sync --hook`). Display/join only — never an
// input to Less or Replay, exactly like Mtime below, so replay stays
// deterministic and ops written before this field existed simply carry "".
//
// It exists because Note is user-settable (`bdrive sync --note`): joining
// a run's reads to its writes on the note string would let any member with
// write access forge a note that collides with a teammate's session and
// hang their reads off it. This field is the un-forgeable half of that
// pair, so the join reads it and never the note.
Session string `json:"session,omitempty"`
// Mtime is when the file was last written, as opposed to Time, which is
// when the op was committed. Display only — never an input to Less or
// Replay, since it comes from the filesystem and can be anything.
+58
View File
@@ -0,0 +1,58 @@
package journal
import (
"strings"
"testing"
)
// Op.Session holds the same standing as Mtime: additive on the wire, and
// invisible to the ordering. A journal written by this code still parses in
// the old shape, and an op written before the field existed reads back as "".
func TestSessionIsAdditive(t *testing.T) {
with := op(1, "a", 1, KindPut, "x.txt", "blob1")
with.Session = "8f21e4"
without := op(2, "a", 2, KindPut, "y.txt", "blob2")
data, err := Marshal([]Op{with, without})
if err != nil {
t.Fatal(err)
}
lines := strings.Split(strings.TrimSpace(string(data)), "\n")
if !strings.Contains(lines[0], `"session":"8f21e4"`) {
t.Fatalf("op lost its session: %s", lines[0])
}
if strings.Contains(lines[1], "session") {
t.Fatalf("op without a session should emit no session key: %s", lines[1])
}
got, err := Parse(data)
if err != nil {
t.Fatal(err)
}
if got[0].Session != "8f21e4" {
t.Fatalf("Session = %q, want 8f21e4", got[0].Session)
}
if got[1].Session != "" {
t.Fatalf("Session should be empty, got %q", got[1].Session)
}
// A line written by an older device carries no session key at all.
legacy, err := Parse([]byte(`{"seq":1,"lamport":1,"device":"a","kind":"put","path":"z.txt"}`))
if err != nil {
t.Fatal(err)
}
if legacy[0].Session != "" {
t.Fatalf("legacy op invented a session: %q", legacy[0].Session)
}
}
// Replay determinism is the invariant this field must not touch: two ops
// differing ONLY in Session compare equal under Less in both directions, so
// no peer's ordering can depend on it.
func TestSessionDoesNotOrder(t *testing.T) {
a := op(1, "dev", 1, KindPut, "x.txt", "blob1")
b := a
b.Session = "8f21e4"
if Less(a, b) || Less(b, a) {
t.Fatalf("Session must not be an input to Less: Less(a,b)=%v Less(b,a)=%v", Less(a, b), Less(b, a))
}
}
+7 -2
View File
@@ -65,8 +65,13 @@ type Backend interface {
// ReadEvent is one agent file read reported to the hub for its read heatmap.
type ReadEvent struct {
Path string `json:"path"`
Time time.Time `json:"time,omitzero"`
Path string `json:"path"`
// Session is the agent session the read happened in, so the hub can join
// a run's reads to the writes journal.Op.Session carries. A client string
// — the hub pins each recorded row to the device it validated, never to
// anything in this body (see handleReadReport).
Session string `json:"session,omitempty"`
Time time.Time `json:"time,omitzero"`
}
// ReadReporter is the optional read-telemetry capability, in the PutSigner
+26 -16
View File
@@ -16,8 +16,12 @@ import (
// ReadEvent is one observed read of a synced file (mount-relative path).
type ReadEvent struct {
Path string `json:"path"`
Time time.Time `json:"time"`
Path string `json:"path"`
// Session is the agent session the read happened in, from the same hook
// payload the sync hook stamps journal.Op.Session from. Empty for reads
// with no session (a platform that reports none, or an older client).
Session string `json:"session,omitempty"`
Time time.Time `json:"time"`
}
// readSpoolMax caps the spool: past it new events are dropped rather than
@@ -32,11 +36,11 @@ func (s *Store) readFlushPath() string { return filepath.Join(s.dir, "reads-flus
// LogRead appends one read event to the spool. Single-line O_APPEND writes
// keep concurrent hook invocations from interleaving.
func (s *Store) LogRead(rel string) error {
func (s *Store) LogRead(rel, session string) error {
if fi, err := os.Stat(s.readSpoolPath()); err == nil && fi.Size() > readSpoolMax {
return nil // spool full: drop, never grow unbounded
}
line, err := json.Marshal(ReadEvent{Path: rel, Time: time.Now().UTC()})
line, err := json.Marshal(ReadEvent{Path: rel, Session: session, Time: time.Now().UTC()})
if err != nil {
return err
}
@@ -50,10 +54,14 @@ func (s *Store) LogRead(rel string) error {
return err
}
// PendingReads returns the queued batch awaiting report, deduplicated by path
// (latest time wins). The spool is rotated aside first, so events logged
// after this call land in a fresh spool; the batch survives until
// ClearPendingReads — a failed report is simply retried next cycle.
// PendingReads returns the queued batch awaiting report, deduplicated by
// (path, session) — latest time wins. Not by path alone: two agent sessions
// on one device between syncs both reading wiki/a.md are two reads by two
// sessions, and collapsing them would report one, carrying whichever session
// happened to flush last — one session's reads silently credited to another.
// The spool is rotated aside first, so events logged after this call land in
// a fresh spool; the batch survives until ClearPendingReads — a failed report
// is simply retried next cycle.
func (s *Store) PendingReads() ([]ReadEvent, error) {
if _, err := os.Stat(s.readFlushPath()); os.IsNotExist(err) {
if err := os.Rename(s.readSpoolPath(), s.readFlushPath()); err != nil {
@@ -70,8 +78,9 @@ func (s *Store) PendingReads() ([]ReadEvent, error) {
}
return nil, err
}
latest := map[string]time.Time{}
var order []string
type readKey struct{ path, session string }
latest := map[readKey]time.Time{}
var order []readKey
for _, line := range bytes.Split(data, []byte("\n")) {
if len(bytes.TrimSpace(line)) == 0 {
continue
@@ -80,19 +89,20 @@ func (s *Store) PendingReads() ([]ReadEvent, error) {
if json.Unmarshal(line, &e) != nil || e.Path == "" {
continue // torn or corrupt line; drop it
}
if _, ok := latest[e.Path]; !ok {
order = append(order, e.Path)
k := readKey{e.Path, e.Session}
if _, ok := latest[k]; !ok {
order = append(order, k)
}
if e.Time.After(latest[e.Path]) {
latest[e.Path] = e.Time
if e.Time.After(latest[k]) {
latest[k] = e.Time
}
}
if len(order) > readReportMax {
order = order[len(order)-readReportMax:]
}
out := make([]ReadEvent, 0, len(order))
for _, p := range order {
out = append(out, ReadEvent{Path: p, Time: latest[p]})
for _, k := range order {
out = append(out, ReadEvent{Path: k.path, Session: k.session, Time: latest[k]})
}
return out, nil
}
+42 -6
View File
@@ -26,11 +26,11 @@ func TestReadSpool(t *testing.T) {
// Repeat reads of one path dedupe to its latest event.
for i := 0; i < 3; i++ {
if err := s.LogRead("wiki/a.md"); err != nil {
if err := s.LogRead("wiki/a.md", ""); err != nil {
t.Fatal(err)
}
}
if err := s.LogRead("b.md"); err != nil {
if err := s.LogRead("b.md", ""); err != nil {
t.Fatal(err)
}
evs, err := s.PendingReads()
@@ -46,7 +46,7 @@ func TestReadSpool(t *testing.T) {
// The batch survives until cleared — a failed report just retries — and
// reads logged meanwhile land in a fresh spool behind it.
if err := s.LogRead("c.md"); err != nil {
if err := s.LogRead("c.md", ""); err != nil {
t.Fatal(err)
}
again, err := s.PendingReads()
@@ -74,14 +74,14 @@ func TestReadSpool(t *testing.T) {
func TestReadSpoolSurvivesCorruptLines(t *testing.T) {
s := openTestStore(t)
s.LogRead("good.md")
s.LogRead("good.md", "")
f, err := os.OpenFile(s.readSpoolPath(), os.O_WRONLY|os.O_APPEND, 0o644)
if err != nil {
t.Fatal(err)
}
f.WriteString(`{"path": "torn`) // a torn write
f.Close()
s.LogRead("also-good.md")
s.LogRead("also-good.md", "")
evs, err := s.PendingReads()
if err != nil {
t.Fatal(err)
@@ -97,7 +97,7 @@ func TestReadSpoolCap(t *testing.T) {
s := openTestStore(t)
long := strings.Repeat("d", 1024)
for i := 0; i < 1100; i++ { // ~1.1 MB of events
if err := s.LogRead(long + "/" + string(rune('a'+i%26)) + ".md"); err != nil {
if err := s.LogRead(long+"/"+string(rune('a'+i%26))+".md", ""); err != nil {
t.Fatal(err)
}
}
@@ -109,3 +109,39 @@ func TestReadSpoolCap(t *testing.T) {
t.Fatalf("spool grew past its cap: %d bytes", fi.Size())
}
}
// Two agent sessions on one device between syncs, both reading one path, are
// two reads by two sessions — the spool must not collapse them into one
// event carrying whichever session flushed last, which would credit one
// session's reads to another on the History run card.
func TestReadSpoolDedupesPerSession(t *testing.T) {
s := openTestStore(t)
for _, e := range []struct{ path, session string }{
{"wiki/a.md", "sess-1"},
{"wiki/a.md", "sess-2"},
{"wiki/a.md", "sess-1"}, // a repeat within one session still collapses
{"wiki/b.md", "sess-1"},
{"wiki/c.md", ""}, // no session (an older client / a platform that reports none)
} {
if err := s.LogRead(e.path, e.session); err != nil {
t.Fatal(err)
}
}
evs, err := s.PendingReads()
if err != nil {
t.Fatal(err)
}
got := map[string]bool{}
for _, e := range evs {
got[e.Path+"|"+e.Session] = true
}
want := []string{"wiki/a.md|sess-1", "wiki/a.md|sess-2", "wiki/b.md|sess-1", "wiki/c.md|"}
if len(evs) != len(want) {
t.Fatalf("batch = %+v, want %d entries", evs, len(want))
}
for _, w := range want {
if !got[w] {
t.Errorf("batch is missing %q: %+v", w, evs)
}
}
}
+2 -2
View File
@@ -48,7 +48,7 @@ func secdefModes(t *testing.T, dir string) map[string]os.FileMode {
// beside it in the same 0755 volume directory at mode 0644.
func TestSec_Store_ReadSpoolIsNotWorldReadable(t *testing.T) {
s, _ := secpkgStore(t)
if err := s.LogRead("secret-project/acquisition-plan.md"); err != nil {
if err := s.LogRead("secret-project/acquisition-plan.md", ""); err != nil {
t.Fatal(err)
}
found := false
@@ -88,7 +88,7 @@ func TestSec_Store_ReadSpoolSurvivesAHostilePathAsData(t *testing.T) {
"tab\there.md",
}
for _, p := range hostile {
if err := s.LogRead(p); err != nil {
if err := s.LogRead(p, ""); err != nil {
t.Fatalf("LogRead(%q): %v", p, err)
}
}
+69 -5
View File
@@ -52,9 +52,9 @@ func TestAgentReadReporting(t *testing.T) {
write(t, a.Folder, "wiki/a.md", "content")
// The agent read a.md twice and b.md once before this cycle.
a.Store.LogRead("wiki/a.md")
a.Store.LogRead("wiki/a.md")
a.Store.LogRead("b.md")
a.Store.LogRead("wiki/a.md", "")
a.Store.LogRead("wiki/a.md", "")
a.Store.LogRead("b.md", "")
res := cycle(t, a)
if !res.Pushed {
t.Fatal("cycle should have pushed")
@@ -74,7 +74,7 @@ func TestAgentReadReporting(t *testing.T) {
// Hub down: the cycle still succeeds and the batch stays queued.
hub.setFail(true)
a.Store.LogRead("wiki/a.md")
a.Store.LogRead("wiki/a.md", "")
if res := cycle(t, a); res.Offline {
t.Fatal("a failed read report must not mark the cycle offline")
}
@@ -93,9 +93,73 @@ func TestAgentReadReporting(t *testing.T) {
// queued reads: the cycle runs, the spool just keeps waiting.
b := newDevice(t, "devb", sharedRemote(t))
write(t, b.Folder, "x.md", "x")
b.Store.LogRead("x.md")
b.Store.LogRead("x.md", "")
cycle(t, b)
if evs, err := b.Store.PendingReads(); err != nil || len(evs) != 1 {
t.Fatalf("spool on a hubless device = %v, %v; want the read still queued", evs, err)
}
}
// TestSessionCarriesThroughTwoDevices is the multi-device shape of the join:
// a device syncing under an agent session stamps that session onto every op
// it commits AND onto every read it reports, its peer converges on ops that
// carry the id, and a device with no session leaves both empty — so a run
// card can never claim another device's work.
func TestSessionCarriesThroughTwoDevices(t *testing.T) {
shared := sharedRemote(t)
hubA := &readReportingRemote{Backend: shared}
hubB := &readReportingRemote{Backend: shared}
a := newDevice(t, "deva", hubA)
b := newDevice(t, "devb", hubB)
// Device A works inside an agent session: it reads two files and writes one.
a.SessionID = "8f21e4"
write(t, a.Folder, "wiki/a.md", "written by the run")
a.Store.LogRead("wiki/a.md", "8f21e4")
a.Store.LogRead("wiki/reference.md", "8f21e4")
cycle(t, a)
if reports := hubA.all(); len(reports) != 1 || len(reports[0]) != 2 {
t.Fatalf("reports = %+v, want one batch of 2", reports)
} else {
for _, e := range reports[0] {
if e.Session != "8f21e4" {
t.Fatalf("reported read %+v lost its session", e)
}
}
}
// Device B, no session at all: its own op carries none, and the read it
// reports carries none — nothing of B's can land on A's card.
write(t, b.Folder, "wiki/b.md", "written by a human")
b.Store.LogRead("wiki/b.md", "")
cycle(t, b)
if reports := hubB.all(); len(reports) != 1 || reports[0][0].Session != "" {
t.Fatalf("sessionless device reported %+v, want an empty session", reports)
}
// Both peers converge, and each op keeps the session of the device that
// wrote it — replay does not touch the field.
cycle(t, a)
cycle(t, b)
for _, d := range []*Session{a, b} {
ops, err := d.Store.AllOps()
if err != nil {
t.Fatal(err)
}
seen := map[string]string{}
for _, op := range ops {
seen[op.Path] = op.Session
}
if seen["wiki/a.md"] != "8f21e4" {
t.Errorf("%s sees wiki/a.md session %q, want 8f21e4", d.Device.ID, seen["wiki/a.md"])
}
if seen["wiki/b.md"] != "" {
t.Errorf("%s sees wiki/b.md session %q, want empty", d.Device.ID, seen["wiki/b.md"])
}
}
// Convergence itself: both folders hold both files.
if got, want := snapshotDir(t, a.Folder), snapshotDir(t, b.Folder); len(got) != len(want) {
t.Fatalf("folders diverged: %v vs %v", got, want)
}
}
+10 -2
View File
@@ -66,6 +66,14 @@ type Session struct {
// `bdrive sync --note` leave context that the daemon's later scans also
// stamp. Conflict-copy ops keep their own explanatory note.
Note string
// SessionID is the agent session every op this cycle commits is stamped
// with (journal.Op.Session). Set only by `bdrive sync --hook`, and
// deliberately NOT persisted the way Note is (store.SaveNote): the note
// is context that outlives the hook turn, the session id is an identity
// that must not be attached to changes the daemon commits on its own
// later. So a daemon scan after the hook turn carries the note and no
// session — the asymmetry is intended.
SessionID string
// Prune makes this cycle reconcile the hub against the shared ignore
// rules: every path the remote still holds that .bdriveignore (or a
// builtin never-sync rule) now excludes is journaled as a delete, so it
@@ -435,7 +443,7 @@ func (s *Session) Cycle(ctx context.Context) (*Result, error) {
if evs, err := s.Store.PendingReads(); err == nil && len(evs) > 0 {
reads := make([]remote.ReadEvent, len(evs))
for i, e := range evs {
reads[i] = remote.ReadEvent{Path: e.Path, Time: e.Time}
reads[i] = remote.ReadEvent{Path: e.Path, Session: e.Session, Time: e.Time}
}
if rr.ReportReads(ctx, reads) == nil {
s.Store.ClearPendingReads()
@@ -482,7 +490,7 @@ func (s *Session) scan(cache map[string]store.CachedFile, st *store.SyncState, s
Seq: seqBase, Lamport: st.Lamport, Time: time.Now().UTC(),
Device: s.Device.ID, DeviceName: s.Device.Name, Author: s.Device.Author,
User: s.Account.Email, UserName: s.Account.Name,
Kind: kind, Path: rel, Note: note,
Kind: kind, Path: rel, Note: note, Session: s.SessionID,
}
}
+19
View File
@@ -3,6 +3,7 @@ package webapp
import (
"fmt"
"strings"
"time"
"unicode/utf8"
)
@@ -25,6 +26,7 @@ type MetaStore interface {
Shares() ShareRepo
Devices() DeviceRepo
Reads() ReadRepo
SessionReads() SessionReadRepo
Close() error
}
@@ -78,6 +80,19 @@ type ReadRepo interface {
DeleteBatch(keys []ReadStatKey) error
}
// SessionReadRepo persists which paths one agent session read (see
// SessionRead). Deliberately its OWN repo rather than a session column on
// read_stats: ReadLedger loads every read_stats row into one map at boot and
// ReadLedger.Heat linearly scans that whole map on every heat request,
// hub-wide — so multiplying its row count by session cardinality would slow
// the Dashboard for projects that never ran an agent. These rows never enter
// that map; they are queried by primary key and pruned by date.
type SessionReadRepo interface {
PutBatch(reads []SessionRead) error // upsert by (project, session, device, path)
ListBySession(project, session, device string) ([]SessionRead, error)
PruneBefore(t time.Time) error
}
// ---- cheap change detection ---------------------------------------------
// Versioned is the optional "has anything moved?" check on a repository: a
@@ -206,3 +221,7 @@ func checkDevice(d DeviceInfo) error {
func checkReadStat(s ReadStat) error {
return storable(s.Project, s.Path, s.Day, s.Kind, s.Actor)
}
func checkSessionRead(s SessionRead) error {
return storable(s.Project, s.Session, s.Path, s.Device)
}
+45 -1
View File
@@ -65,7 +65,7 @@ func metaBackends(t *testing.T) []metaBackend {
// device_rows rows behind for the following test to inherit.
db.Exec(`DROP TABLE IF EXISTS accounts, tokens, auth_policy, projects, project_perms,
orgs, org_members, invites, shares, devices, device_rows, read_stats,
meta_version, schema_meta`)
read_sessions, meta_version, schema_meta`)
},
open: func(t *testing.T) MetaStore {
s, err := OpenSQLStore("pgx", dsn)
@@ -221,9 +221,22 @@ func TestMetaStoreConformance(t *testing.T) {
reads.Record(p1.ID, "handbook.md", ReadKindHuman, "dev@x.io")
reads.Record(p1.ID, "handbook.md", ReadKindHuman, "boss@x.io")
reads.Record(p1.ID, "wiki/deep.md", ReadKindAgent, "d1")
// Per-session read detail rides its own repo, so it needs its own
// pass on every backend: two sessions on one device must stay two
// sets of rows, and one of them must prune away by date.
reads.WithSessions(st.SessionReads(), 0)
reads.RecordSession(p1.ID, "sess-a", "d1", "handbook.md")
reads.RecordSession(p1.ID, "sess-a", "d1", "wiki/deep.md")
reads.RecordSession(p1.ID, "sess-b", "d1", "handbook.md")
if err := reads.Close(); err != nil {
t.Fatal(err)
}
if err := st.SessionReads().PutBatch([]SessionRead{{
Project: p1.ID, Session: "sess-old", Device: "d1", Path: "handbook.md",
Last: time.Now().UTC().Add(-90 * 24 * time.Hour),
}}); err != nil {
t.Fatal(err)
}
if err := st.Close(); err != nil {
t.Fatal(err)
@@ -323,6 +336,37 @@ func TestMetaStoreConformance(t *testing.T) {
if sub := reads2.Heat(p1.ID, "wiki", time.Time{}); len(sub) != 1 {
t.Fatalf("prefix heat = %+v, want only wiki/deep.md", sub)
}
sessions := st2.SessionReads()
got, err := sessions.ListBySession(p1.ID, "sess-a", "d1")
if err != nil {
t.Fatal(err)
}
if len(got) != 2 || got[0].Path != "handbook.md" || got[1].Path != "wiki/deep.md" {
t.Fatalf("session-a rows lost across reload: %+v", got)
}
if other, _ := sessions.ListBySession(p1.ID, "sess-b", "d1"); len(other) != 1 {
t.Fatalf("session-b rows = %+v, want its own single row", other)
}
// Wrong device, same session id: the query is keyed on both, which
// is what keeps a forged report off somebody else's run card.
if none, _ := sessions.ListBySession(p1.ID, "sess-a", "d2"); len(none) != 0 {
t.Fatalf("session rows leaked across devices: %+v", none)
}
if err := sessions.PruneBefore(time.Now().UTC().AddDate(0, 0, -30)); err != nil {
t.Fatal(err)
}
if old, _ := sessions.ListBySession(p1.ID, "sess-old", "d1"); len(old) != 0 {
t.Fatalf("prune left expired session rows: %+v", old)
}
if kept, _ := sessions.ListBySession(p1.ID, "sess-a", "d1"); len(kept) != 2 {
t.Fatalf("prune took recent session rows too: %+v", kept)
}
// The aggregate the run cards do NOT come from is untouched by any
// of that — session rows never enter the bucket map.
if e := reads2.Heat(p1.ID, "", time.Time{})["handbook.md"]; e.Human != 2 {
t.Fatalf("bucket heat changed with session rows: %+v", e)
}
})
}
}
+113 -7
View File
@@ -90,6 +90,7 @@ type fileMetaStore struct {
shares *fileShareRepo
devices *fileDeviceRepo
reads *fileReadRepo
sessions *fileSessionReadRepo
}
// OpenFileStore builds the file backend over dir, using the historical
@@ -102,16 +103,18 @@ func OpenFileStore(dir string) (MetaStore, error) {
shares: newFileShareRepo(filepath.Join(dir, "shares.json")),
devices: newFileDeviceRepo(filepath.Join(dir, "devices.json")),
reads: newFileReadRepo(filepath.Join(dir, "reads.json")),
sessions: newFileSessionReadRepo(filepath.Join(dir, "sessions.json")),
}, nil
}
func (s *fileMetaStore) Accounts() AccountRepo { return s.accounts }
func (s *fileMetaStore) Projects() ProjectRepo { return s.projects }
func (s *fileMetaStore) Orgs() OrgRepo { return s.orgs }
func (s *fileMetaStore) Shares() ShareRepo { return s.shares }
func (s *fileMetaStore) Devices() DeviceRepo { return s.devices }
func (s *fileMetaStore) Reads() ReadRepo { return s.reads }
func (s *fileMetaStore) Close() error { return nil }
func (s *fileMetaStore) Accounts() AccountRepo { return s.accounts }
func (s *fileMetaStore) Projects() ProjectRepo { return s.projects }
func (s *fileMetaStore) Orgs() OrgRepo { return s.orgs }
func (s *fileMetaStore) Shares() ShareRepo { return s.shares }
func (s *fileMetaStore) Devices() DeviceRepo { return s.devices }
func (s *fileMetaStore) Reads() ReadRepo { return s.reads }
func (s *fileMetaStore) SessionReads() SessionReadRepo { return s.sessions }
func (s *fileMetaStore) Close() error { return nil }
// ---- accounts (auth.json: users + tokens + policy) ----
@@ -768,3 +771,106 @@ func (r *fileReadRepo) DeleteBatch(keys []ReadStatKey) error {
}
return r.write()
}
// ---- session reads (sessions.json) ----
type fileSessionReadRepo struct {
path string
mu sync.Mutex
byKey map[sessionReadKey]SessionRead
}
func newFileSessionReadRepo(path string) *fileSessionReadRepo {
return &fileSessionReadRepo{path: path, byKey: map[sessionReadKey]SessionRead{}}
}
// reload re-reads before every write, for fileReadRepo.reload's reason: a
// stale rewrite erases rows another hub process recorded since boot. Callers
// hold mu.
func (r *fileSessionReadRepo) reload() error {
var f struct {
Sessions []SessionRead `json:"sessions"`
}
if _, err := readJSONFile(r.path, &f); err != nil {
return err
}
r.byKey = map[sessionReadKey]SessionRead{}
for _, sr := range f.Sessions {
r.byKey[sr.key()] = sr
}
return nil
}
func (r *fileSessionReadRepo) write() error {
var f struct {
Sessions []SessionRead `json:"sessions"`
}
f.Sessions = make([]SessionRead, 0, len(r.byKey))
for _, sr := range r.byKey {
f.Sessions = append(f.Sessions, sr)
}
sort.Slice(f.Sessions, func(i, j int) bool {
a, b := f.Sessions[i], f.Sessions[j]
if a.Session != b.Session {
return a.Session < b.Session
}
return a.Path < b.Path
})
data, err := json.Marshal(f) // telemetry: compact beats pretty
if err != nil {
return err
}
return writeFileAtomic(r.path, append(data, '\n'))
}
func (r *fileSessionReadRepo) PutBatch(reads []SessionRead) error {
for _, sr := range reads {
if err := checkSessionRead(sr); err != nil {
return err
}
}
r.mu.Lock()
defer r.mu.Unlock()
if err := r.reload(); err != nil {
return err
}
for _, sr := range reads {
r.byKey[sr.key()] = sr
}
return r.write()
}
func (r *fileSessionReadRepo) ListBySession(project, session, device string) ([]SessionRead, error) {
r.mu.Lock()
defer r.mu.Unlock()
if err := r.reload(); err != nil {
return nil, err
}
var out []SessionRead
for k, sr := range r.byKey {
if k.Project == project && k.Session == session && k.Device == device {
out = append(out, sr)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].Path < out[j].Path })
return out, nil
}
func (r *fileSessionReadRepo) PruneBefore(t time.Time) error {
r.mu.Lock()
defer r.mu.Unlock()
if err := r.reload(); err != nil {
return err
}
n := 0
for k, sr := range r.byKey {
if sr.Last.Before(t) {
delete(r.byKey, k)
n++
}
}
if n == 0 {
return nil
}
return r.write()
}
+75 -7
View File
@@ -36,6 +36,7 @@ type sqlMetaStore struct {
shares *sqlShareRepo
devices *sqlDeviceRepo
reads *sqlReadRepo
sessions *sqlSessionReadRepo
}
// OpenSQLStore opens (and migrates) a SQL metadata store. driver is "sqlite"
@@ -70,16 +71,18 @@ func OpenSQLStore(driver, dsn string) (MetaStore, error) {
s.shares = &sqlShareRepo{s: s, w: regWriter{s, regShares}}
s.devices = &sqlDeviceRepo{s: s, w: regWriter{s, regDevices}}
s.reads = &sqlReadRepo{s: s, w: regWriter{s, regReads}}
s.sessions = &sqlSessionReadRepo{s: s}
return s, nil
}
func (s *sqlMetaStore) Accounts() AccountRepo { return s.accounts }
func (s *sqlMetaStore) Projects() ProjectRepo { return s.projects }
func (s *sqlMetaStore) Orgs() OrgRepo { return s.orgs }
func (s *sqlMetaStore) Shares() ShareRepo { return s.shares }
func (s *sqlMetaStore) Devices() DeviceRepo { return s.devices }
func (s *sqlMetaStore) Reads() ReadRepo { return s.reads }
func (s *sqlMetaStore) Close() error { return s.db.Close() }
func (s *sqlMetaStore) Accounts() AccountRepo { return s.accounts }
func (s *sqlMetaStore) Projects() ProjectRepo { return s.projects }
func (s *sqlMetaStore) Orgs() OrgRepo { return s.orgs }
func (s *sqlMetaStore) Shares() ShareRepo { return s.shares }
func (s *sqlMetaStore) Devices() DeviceRepo { return s.devices }
func (s *sqlMetaStore) Reads() ReadRepo { return s.reads }
func (s *sqlMetaStore) SessionReads() SessionReadRepo { return s.sessions }
func (s *sqlMetaStore) Close() error { return s.db.Close() }
// q rebinds ?-placeholders to $1,$2,… for Postgres; SQLite keeps ?.
func (s *sqlMetaStore) q(query string) string {
@@ -244,6 +247,16 @@ func (s *sqlMetaStore) migrate() error {
kind TEXT NOT NULL, actor TEXT NOT NULL,
count INTEGER NOT NULL DEFAULT 0, last TEXT NOT NULL DEFAULT '',
PRIMARY KEY (project, path, day, kind, actor))`,
// Which paths one agent session read. Its own table, NOT a column on
// read_stats: read_stats is loaded whole into ReadLedger's map at boot
// and linearly scanned on every heat request, so session cardinality
// there would cost every project on the hub. Queried by primary key
// prefix, pruned by date — never loaded whole.
`CREATE TABLE IF NOT EXISTS read_sessions (
project TEXT NOT NULL, session TEXT NOT NULL, device TEXT NOT NULL,
path TEXT NOT NULL, last TEXT NOT NULL DEFAULT '',
PRIMARY KEY (project, session, device, path))`,
`CREATE INDEX IF NOT EXISTS read_sessions_last ON read_sessions (last)`,
`CREATE TABLE IF NOT EXISTS project_perms (
project TEXT NOT NULL, email TEXT NOT NULL, level TEXT NOT NULL,
PRIMARY KEY (project, email))`,
@@ -903,3 +916,58 @@ func (r *sqlReadRepo) DeleteBatch(keys []ReadStatKey) error {
return nil
})
}
// ---- session reads ----
// No regWriter: these rows are telemetry detail, never read through a
// registry's refresh path, so there is no version counter to bump.
type sqlSessionReadRepo struct {
s *sqlMetaStore
}
func (r *sqlSessionReadRepo) PutBatch(reads []SessionRead) error {
for _, sr := range reads {
if err := checkSessionRead(sr); err != nil {
return err
}
}
tx, err := r.s.db.Begin()
if err != nil {
return err
}
defer tx.Rollback()
for _, sr := range reads {
if _, err := tx.Exec(r.s.q(`INSERT INTO read_sessions (project,session,device,path,last)
VALUES (?,?,?,?,?)
ON CONFLICT(project,session,device,path) DO UPDATE SET last=excluded.last`),
sr.Project, sr.Session, sr.Device, sr.Path, tenc(sr.Last)); err != nil {
return err
}
}
return tx.Commit()
}
func (r *sqlSessionReadRepo) ListBySession(project, session, device string) ([]SessionRead, error) {
rows, err := r.s.db.Query(r.s.q(`SELECT project, session, device, path, last FROM read_sessions
WHERE project = ? AND session = ? AND device = ? ORDER BY path`), project, session, device)
if err != nil {
return nil, err
}
defer rows.Close()
var out []SessionRead
for rows.Next() {
var sr SessionRead
var last string
if err := rows.Scan(&sr.Project, &sr.Session, &sr.Device, &sr.Path, &last); err != nil {
return nil, err
}
sr.Last = tdec(last)
out = append(out, sr)
}
return out, rows.Err()
}
func (r *sqlSessionReadRepo) PruneBefore(t time.Time) error {
_, err := r.s.db.Exec(r.s.q(`DELETE FROM read_sessions WHERE last < ?`), tenc(t))
return err
}
+19 -3
View File
@@ -28,7 +28,9 @@ import (
)
const (
e2eAddr = "0.0.0.0:8993"
e2eAddr = "0.0.0.0:8993"
// e2eSession is the agent session the seeded run card belongs to.
e2eSession = "8f21e4"
e2eAdmin = "e2e@example.com"
e2eMember = "member@example.com"
e2eSolo = "solo@example.com"
@@ -103,6 +105,16 @@ func TestE2EServe(t *testing.T) {
if err != nil {
t.Fatal(err)
}
// Per-session read detail, so the seeded run card has both halves of the
// story: what the run changed AND what it read (BEA-98).
srv.Reads.WithSessions(OpenSessionReadRepo(filepath.Join(state, "sessions.json")), 0)
for _, path := range []string{
"notes/readme.md", // read AND rewritten by the run
"index.md", // read, never changed
"archive/retired-spec.md", // read, never changed — the hot+stale one
} {
srv.Reads.RecordSession(p.ID, e2eSession, "seed", path)
}
srv.Devices, _ = OpenDeviceRegistry(filepath.Join(state, "devices.json"))
srv.Devices.Observe(DeviceInfo{ID: "seed", Name: "seed-agent", OS: "linux/amd64"})
@@ -247,8 +259,12 @@ func seedE2E(t *testing.T, state, prefix, projectID string) {
// of their path, so neither row offers a restore (BEA-57).
put("notes/readme.md", "# Notes\n\nRewritten during the agent run.\n", 90*time.Minute)
put("runbook.md", "# Runbook\n\nCreated during the agent run.\n", 90*time.Minute)
ops[len(ops)-1].Note = "claude-code session 8f21e4"
ops[len(ops)-2].Note = "claude-code session 8f21e4"
ops[len(ops)-1].Note = "claude-code session " + e2eSession
ops[len(ops)-2].Note = "claude-code session " + e2eSession
// The un-forgeable half of the run identity: the note is what a reader
// sees, this is what the card groups and joins its reads on.
ops[len(ops)-1].Session = e2eSession
ops[len(ops)-2].Session = e2eSession
// A second version of the same binary, so the history diff has a
// predecessor to refuse to diff (the "binary — no diff" path).
put("assets/logo.png", png+"\x00trailing", 3*time.Hour)
+2 -1
View File
@@ -515,7 +515,8 @@ test("history groups one agent run into a single card", async ({ page }) => {
const run = page.locator(".hrun");
await expect(run).toHaveCount(1);
await expect(run.locator(".hrun-note")).toHaveText("claude-code session 8f21e4");
await expect(run.locator(".hrun-meta")).toContainText("2 files");
// Both halves of the run, since the seed gives it session reads (BEA-98).
await expect(run.locator(".hrun-meta")).toContainText("changed 2");
await expect(run.locator(".hrun-meta")).toContainText("seed-agent");
// Both of the run's changes live inside the card...
await expect(run.locator(".hentry")).toHaveCount(2);
@@ -0,0 +1,45 @@
import { test, expect } from "@playwright/test";
import { login, wikiId } from "./helpers";
/* One agent run, both halves (BEA-98). History used to show only what a run
CHANGED; the reads lived in a daily aggregate with no session dimension and
could not be joined to it. The seeded run reads three files and rewrites
one of them. */
test("a run card shows what the session read as well as what it changed", async ({ page }) => {
await login(page);
const pid = await wikiId(page);
await page.goto(`/${pid}/history`);
const card = page.locator(".hrun").first();
await expect(card).toBeVisible();
// The header counts both halves now.
await expect(card.locator(".hrun-meta")).toContainText("read 3");
await expect(card.locator(".hrun-meta")).toContainText("changed 2");
// The file the run read AND rewrote carries the read marker on its own row.
const rewritten = card.locator(".hentry", { hasText: "notes/readme.md" });
await expect(rewritten.locator(".hread")).toHaveText("read");
// The file it created was never read, so that row has no marker.
await expect(card.locator(".hentry", { hasText: "runbook.md" }).locator(".hread")).toHaveCount(0);
// What it read and did not touch is its own list.
const readOnly = card.locator(".hrun-read");
await expect(readOnly).toHaveCount(2);
await expect(readOnly.first()).toContainText("archive/retired-spec.md");
await expect(readOnly.last()).toContainText("index.md");
// Landmine 3 is on screen, not folded into a comment: a file the run read
// and then deleted shows a write with no read, and the card says why.
await expect(card.locator(".hrun-foot")).toHaveText(
"Reads shown only for files the project still has.",
);
});
test("a read-only row opens the file it names", async ({ page }) => {
await login(page);
const pid = await wikiId(page);
await page.goto(`/${pid}/history`);
await page.locator(".hrun-read", { hasText: "index.md" }).click();
await expect(page).toHaveURL(new RegExp(`/${pid}/index.md`));
});
@@ -183,6 +183,10 @@ export interface HistoryEntry {
author?: string;
device: DeviceInfo;
note?: string;
// The agent session this change was committed during (hook-set, and unlike
// the note not settable by hand). It groups a run card and is the key the
// card's reads are fetched with.
session?: string;
}
// POST .../shares (handleShareCreate, shares.go)
@@ -58,6 +58,7 @@ export function HistoryRow({
remove,
restoreSha,
inRun,
read,
}: {
entry: HistoryEntry;
// Its own prop, not something nested in `diff`: the version controls below
@@ -80,6 +81,9 @@ export function HistoryRow({
// Inside a run card, where "this run created the file" is a statement we
// can actually make.
inRun?: boolean;
// The run that wrote this row also READ this path. Only ever set inside a
// run card, where a session id makes the join possible at all.
read?: boolean;
}) {
const [noteOpen, setNoteOpen] = useState(false);
const [diffOpen, setDiffOpen] = useState(false);
@@ -125,6 +129,13 @@ export function HistoryRow({
>
<div className="hline">
<span className="hkind">{KIND_LABEL[kind] || kind}</span>
{/* The run read this file before it wrote it the whole point of the
card, so it sits on the row rather than in a separate list. */}
{read && (
<span className="hread" title="This run read this file before changing it">
read
</span>
)}
<span className="hpath">{e.path}</span>
<span className="htime">{when}</span>
</div>
@@ -1,5 +1,5 @@
import { useEffect, useRef, useState } from "react";
import { useInfiniteQuery } from "@tanstack/react-query";
import { useInfiniteQuery, useQuery } from "@tanstack/react-query";
import { getJSON } from "../api/http";
import type { HistoryEntry } from "../api/types";
import { HistoryRow, NoteText, type RemoveAction, type RestoreAction } from "./HistoryRow";
@@ -203,6 +203,26 @@ function RunGroup({
const first = run.entries[0];
const who = whoChanged(first);
const dev = [first.device.name || first.device.id, first.device.os].filter(Boolean).join(" · ");
// What this run READ, joined on the session id its own ops carry — never on
// the note, which anyone can set to anything. Both the session and the
// device are required by the server, so a card only ever shows reads its
// own device reported.
const sid = first.session;
const did = first.device?.id;
const { data: reads } = useQuery({
queryKey: ["session-reads", apiBase, sid, did],
queryFn: () =>
getJSON<{ paths: string[] }>(
apiBase + "heat?session=" + encodeURIComponent(sid!) + "&device=" + encodeURIComponent(did!),
),
enabled: !!sid && !!did,
staleTime: 30_000,
});
const readPaths = new Set(reads?.paths ?? []);
const written = new Set(run.entries.map((e) => e.path));
// Read but never written: the half of the run that History could not show
// before, and usually the half that answers "what did it look at?".
const readOnly = [...readPaths].filter((p) => !written.has(p)).sort();
const times = run.entries.map((e) => new Date(e.time).getTime());
const span = fmtSpan(Math.min(...times), Math.max(...times));
// Distinct paths, not ops: repeat edits to one file must not inflate the
@@ -226,7 +246,8 @@ function RunGroup({
<NoteText text={run.note} />
</span>
<span className="hrun-meta">
{n} file{n === 1 ? "" : "s"} · {who}
{readPaths.size > 0 ? `read ${readPaths.size} · changed ${n}` : `${n} file${n === 1 ? "" : "s"}`} ·{" "}
{who}
{dev ? " · " + dev : ""}
</span>
<span className="hrun-time">{span}</span>
@@ -247,8 +268,26 @@ function RunGroup({
remove={remove}
restoreSha={restoreSha(run.idx[k])}
inRun
read={readPaths.has(e.path)}
/>
))}
{readOnly.length > 0 && (
<div className="hrun-reads">
<div className="hrun-reads-head">Read, not changed</div>
{readOnly.map((p) => (
<button key={p} type="button" className="hrun-read" onClick={() => onOpen(p)}>
<span className="hkind">read</span>
<span className="hpath">{p}</span>
</button>
))}
</div>
)}
{/* Not decoration: reads are recorded only for paths the project
still has, so a file this run read and then deleted shows its
write with no read. Saying so beats reading as a bug. */}
{sid && (
<div className="hrun-foot">Reads shown only for files the project still has.</div>
)}
</div>
)}
</div>
@@ -129,3 +129,65 @@ test("a run split across two pages groups into one card", () => {
assert.deepEqual(items[1].run?.entries.map((x) => x.path), ["a.md", "b.md", "c.md"]);
assert.deepEqual(items[1].run?.idx, [1, 2, 3]); // idx still addresses the flat feed
});
// ---- grouping on the session id (BEA-98) ----
const es = (path: string, session?: string, note = "claude-code session x", device = "mac-mini"): HistoryEntry => ({
time: "2026-07-29T14:02:00Z",
kind: "edit",
path,
note,
session,
device: { id: device },
});
test("legacy entries (no session) still group by note + device, unchanged", () => {
const items = groupRuns([e("a.md", "session-1"), e("b.md", "session-1")]);
assert.equal(items.length, 1);
assert.equal(items[0].run?.entries.length, 2);
assert.equal(items[0].run?.session, undefined);
});
test("one note, two sessions = two runs — a note cannot merge into someone's card", () => {
const items = groupRuns([
es("a.md", "sess-1"),
es("b.md", "sess-1"),
es("c.md", "sess-2"),
es("d.md", "sess-2"),
]);
assert.equal(items.length, 2);
assert.deepEqual(
items.map((i) => i.run?.session),
["sess-1", "sess-2"],
);
});
test("one session on two devices is still two runs", () => {
const items = groupRuns([
es("a.md", "sess-1", "note", "mac-mini"),
es("b.md", "sess-1", "note", "mac-mini"),
es("c.md", "sess-1", "note", "linux-box"),
es("d.md", "sess-1", "note", "linux-box"),
]);
assert.equal(items.length, 2);
});
test("a session-keyed run never merges with a note-keyed one", () => {
// The legacy rows carry the same note the session rows do; only the
// session-bearing ones may group together.
const items = groupRuns([
es("a.md", "sess-1"),
es("b.md", "sess-1"),
e("c.md", "claude-code session x"),
e("d.md", "claude-code session x"),
]);
assert.equal(items.length, 2);
assert.equal(items[0].run?.session, "sess-1");
assert.equal(items[1].run?.session, undefined);
});
test("a session with no note at all still forms a run", () => {
const items = groupRuns([es("a.md", "sess-1", ""), es("b.md", "sess-1", "")]);
assert.equal(items.length, 1);
assert.equal(items[0].run?.entries.length, 2);
});
+26 -16
View File
@@ -4,9 +4,10 @@ import type { HistoryEntry } from "../api/types";
Pure grouping for the history feed, no React: the run card's shape and the
one number in its header, unit-tested on node (`npm test`). */
// One run: the entries that share a (note, device), with the index each came
// from so diff lookups still address the flat feed.
export type Run = { note: string; entries: HistoryEntry[]; idx: number[] };
// One run: the entries that share a (session, device) — or a (note, device)
// for ops written before session ids existed — with the index each came from
// so diff lookups still address the flat feed.
export type Run = { note: string; session?: string; entries: HistoryEntry[]; idx: number[] };
export type Item = { run?: Run; i: number };
// How much of the project a run touched: distinct paths, not ops. A path
@@ -17,27 +18,36 @@ export function runFileCount(run: Run): number {
return new Set(run.entries.map((e) => e.path)).size;
}
/* Group key = note + device id, exact match. Deliberately simple, and it
guarantees a group never spans two journals one writer, one op range
which is what a later run-wide restore needs. Two devices that happen to
write the same note are two runs. Grouping spans the whole window rather
than only consecutive rows, so a run whose ops interleave with another
device's still reads as one thing; each group sits where its newest
member did, keeping the feed newest-first. */
/* Group key = session id + device id when the entry carries a session,
falling back to note + device id for ops written before Op.Session existed.
The session is preferred because it is the one of the two the writer cannot
choose: `bdrive sync --note` sets any note it likes, so grouping on the
note alone let a member forge a string that collides with a teammate's run
and merge into their card. The key still guarantees a group never spans two
journals one writer, one op range which is what a later run-wide
restore needs. Two devices that happen to share a note (or a session) are
two runs. Grouping spans the whole window rather than only consecutive
rows, so a run whose ops interleave with another device's still reads as
one thing; each group sits where its newest member did, keeping the feed
newest-first. */
export function groupRuns(entries: HistoryEntry[]): Item[] {
// NUL separator: it cannot occur in a note or a device id, so no pair of
// them can collide into one key.
const key = (e: HistoryEntry) => e.note + "\0" + (e.device?.id ?? "");
// NUL separator: it cannot occur in a note, a session id or a device id, so
// no pair of them can collide into one key. Written as "\0", never pasted
// as a literal NUL (BEA-70: a Bin diff on a .tsx is the tell).
// The "s"/"n" tag keeps a session-keyed group from ever colliding with a
// note-keyed one on a device that writes both.
const key = (e: HistoryEntry) =>
(e.session ? "s\0" + e.session : "n\0" + e.note) + "\0" + (e.device?.id ?? "");
const runs = new Map<string, Run>();
entries.forEach((e, i) => {
if (!e.note) return;
if (!e.note && !e.session) return;
const run = runs.get(key(e));
if (run) {
run.entries.push(e);
run.idx.push(i);
return;
}
runs.set(key(e), { note: e.note, entries: [e], idx: [i] });
runs.set(key(e), { note: e.note ?? "", session: e.session, entries: [e], idx: [i] });
});
// A run that touched one file is not worth a card: the row already shows
// its note, and wrapping it would say the same thing twice. Grouping earns
@@ -47,7 +57,7 @@ export function groupRuns(entries: HistoryEntry[]): Item[] {
const out: Item[] = [];
const carded = new Set<Run>();
entries.forEach((e, i) => {
const run = e.note ? runs.get(key(e)) : undefined;
const run = e.note || e.session ? runs.get(key(e)) : undefined;
if (!run || runFileCount(run) < 2) {
out.push({ i });
return;
+11
View File
@@ -780,6 +780,17 @@ a.ai-main:hover { color: var(--accent); }
/* Rows inside a card don't repeat the card's own border or note. */
.hrun-body { border-top: 1px solid var(--border); }
.hrun-body .hentry:last-child { border-bottom: none; }
/* "this run read it too" a quieter badge than the kind pill it follows,
because the change is still the headline of the row. */
.hread { flex: none; padding: 2px 6px; border-radius: 4px; font-size: 10px; text-transform: uppercase; letter-spacing: .06em; font-weight: 600; color: var(--text-dim); background: var(--hover); }
/* What the run read and did NOT change: same two columns as a change row so
the eye reads one list, dimmer because nothing moved. */
.hrun-reads { border-top: 1px solid var(--border); padding: 4px 0 6px; }
.hrun-reads-head { padding: 6px 14px 4px; font-size: 11px; text-transform: uppercase; letter-spacing: .06em; color: var(--text-faint); }
.hrun-read { display: flex; gap: 10px; align-items: center; width: 100%; padding: 5px 14px; border: none; background: none; font: inherit; text-align: left; cursor: pointer; }
.hrun-read:hover { background: rgba(255,255,255,.015); }
.hrun-read .hkind { color: var(--text-dim); background: var(--hover); }
.hrun-foot { padding: 8px 14px 10px; border-top: 1px solid var(--border); font-size: 11.5px; color: var(--text-faint); }
/* ---- restore / remove ---- */
.hrestore-btn, .hremove-btn { display: inline-flex; align-items: center; gap: 4px; margin-left: auto; padding: 2px 8px 2px 5px; border: 1px solid var(--border); border-radius: 5px; background: none; color: var(--text-faint); font: inherit; font-size: 12px; cursor: pointer; }
+7 -1
View File
@@ -44,6 +44,12 @@ type HistoryEntry struct {
Author string `json:"author,omitempty"` // offline/git fallback identity
Device historyDevice `json:"device"`
Note string `json:"note,omitempty"`
// Session is the agent session the op was committed during (hook-set,
// see journal.Op.Session). It is the run card's group key and the only
// place a session id is ever served: it is never enumerated, never a
// column in /heat's output, and never in ?by=device — it appears here,
// on the op that carries it, and is accepted as a ?session= filter INPUT.
Session string `json:"session,omitempty"`
}
// histLess is the display order of the history feed: newest wall-clock time
@@ -263,7 +269,7 @@ func (s *Server) handleHistory(v *volume, w http.ResponseWriter, r *http.Request
Time: op.Time.UTC().Format("2006-01-02T15:04:05Z"), Kind: kinds[i],
Path: op.Path, Size: op.Size, Blob: op.Blob,
User: op.User, UserName: op.UserName, Author: op.Author,
Device: dev, Note: op.Note,
Device: dev, Note: op.Note, Session: op.Session,
}, op})
}
// Truncation happens AFTER the sort: cutting during the walk above would
+9 -2
View File
@@ -27,8 +27,15 @@ func TestPermRankAndAtLeast(t *testing.T) {
// permHub builds an org hub where alice owns the org, bob and carol are plain
// members, and dave is in another org entirely. The project is alice's.
func permHub(t *testing.T) (h http.Handler, srv *Server, cookies map[string]*http.Cookie, p Project) {
h, srv, cookies, p, _ = permHubAt(t)
return
}
// permHubAt is permHub plus the storage root, for tests that seed a journal
// through newFakeRemoteAt.
func permHubAt(t *testing.T) (h http.Handler, srv *Server, cookies map[string]*http.Cookie, p Project, root string) {
t.Helper()
srv, _, _ = newHub(t, true, nil)
srv, _, root = newHub(t, true, nil)
auth, err := OpenBuiltinAuth(filepath.Join(t.TempDir(), "auth.json"), true, nil)
if err != nil {
t.Fatal(err)
@@ -75,7 +82,7 @@ func permHub(t *testing.T) (h http.Handler, srv *Server, cookies map[string]*htt
t.Fatal(err)
}
}
return h, srv, cookies, p
return h, srv, cookies, p, root
}
// Nothing changes for an existing hub: with no permission edits, every org
+208 -7
View File
@@ -27,6 +27,12 @@ import (
// /store/* sync traffic is replication, not reading, and is never counted;
// history /blob views are spelunking, not consumption, and aren't either.
//
// One exception to "never an event log": session reads (SessionRead), the
// per-session detail behind a History run card. They are a separate table
// with their own, shorter retention, and they never enter the ledger's
// in-memory bucket map — see SessionReadRepo for why that separation is the
// whole point.
//
// Privacy: rows are daily aggregation buckets, never an event log. The actor
// column (account email / device id / share token) exists only to count
// distinct readers and never appears in an API response — with exactly one
@@ -40,6 +46,15 @@ import (
// arbitrary string, and never someone else's machine. This route also never
// registers a device: registering the id it is about to judge is what turned
// the round-2 check into a one-request speed bump.
//
// A session id is identity-adjacent and gets the same ruling, written down
// before anything serves it: a session id appears ONLY in History responses,
// on the op that carries it, and as a ?session= filter INPUT. It is never
// enumerated — no "list sessions" response, no session column in /heat's
// output, nothing new in ?by=device. That is sound because the id is already
// visible to every project member inside Op.Note today, so serving it as its
// own field discloses nothing new, while refusing to enumerate keeps /heat
// identity-free exactly as documented above.
// Read kinds.
const (
@@ -69,6 +84,25 @@ func (s ReadStat) key() ReadStatKey {
return ReadStatKey{s.Project, s.Path, s.Day, s.Kind, s.Actor}
}
// SessionRead records that one agent session read one path, from one device.
// Not a count and not a bucket: the run card asks "did this session read this
// file?", and one row per (session, device, path) answers it with no
// aggregation. Device is always the hub-validated device the report arrived
// from, never anything the client put in the body.
type SessionRead struct {
Project string `json:"project"`
Session string `json:"session"`
Device string `json:"device"`
Path string `json:"path"`
Last time.Time `json:"last"`
}
type sessionReadKey struct{ Project, Session, Device, Path string }
func (s SessionRead) key() sessionReadKey {
return sessionReadKey{s.Project, s.Session, s.Device, s.Path}
}
// HeatEntry is the per-path aggregate the heat API returns. Counts only —
// never identities.
type HeatEntry struct {
@@ -90,6 +124,15 @@ const (
// DefaultReadRetentionDays is how long daily buckets keep per-day
// resolution before folding into the all-time row.
DefaultReadRetentionDays = 400
// DefaultSessionRetentionDays is how long per-session read detail is
// kept. Much shorter than the bucket retention: this is event-shaped
// data whose only consumer is a History run card, and a month covers a
// retro. Rows past it are deleted, not folded — the heat totals were
// never derived from them, so nothing is lost from any count.
DefaultSessionRetentionDays = 30
// sessionPruneEvery throttles the retention delete; it rides the same
// flush the buckets use rather than owning a goroutine.
sessionPruneEvery = time.Hour
)
// ReadLedger is the in-memory read-telemetry service over a ReadRepo, in the
@@ -100,19 +143,29 @@ type ReadLedger struct {
repo ReadRepo
retention time.Duration
// Session-read detail, optional (nil = off) and deliberately outside
// byKey: these rows are never loaded into memory in bulk, so Heat's full
// map scan and the boot load are unaffected by session cardinality. If a
// future change ever moves them into byKey, Heat (below) is what pays.
sessions SessionReadRepo
sessionRetention time.Duration
// scans counts ShareOpens passes over byKey. Tests assert one per
// project per list render — the "never one scan per share" rule is
// invisible in the response body, so this is the only thing that can
// catch the regression.
scans atomic.Int64
mu sync.Mutex
byKey map[ReadStatKey]ReadStat
dirty map[ReadStatKey]bool
pendingDel []ReadStatKey // retention deletions awaiting a successful flush
seen map[ReadStatKey]time.Time // debounce; Day field unused ("")
lastFlush time.Time
warned bool
mu sync.Mutex
byKey map[ReadStatKey]ReadStat
dirty map[ReadStatKey]bool
pendingDel []ReadStatKey // retention deletions awaiting a successful flush
seen map[ReadStatKey]time.Time // debounce; Day field unused ("")
pendingSess map[sessionReadKey]SessionRead
lastFlush time.Time
lastSessPrun time.Time
warned bool
sessWarned bool
}
// NewReadLedger loads the ledger and immediately folds buckets older than the
@@ -153,6 +206,29 @@ func OpenReadLedger(path string, retentionDays int) (*ReadLedger, error) {
return NewReadLedger(newFileReadRepo(path), retentionDays)
}
// OpenSessionReadRepo is the file-backed session-read store, for hubs
// running without a MetaStore (the historical JSON-files layout).
func OpenSessionReadRepo(path string) SessionReadRepo { return newFileSessionReadRepo(path) }
// WithSessions turns on per-session read detail (the data behind a History
// run card). Separate from the constructor so every existing caller — and
// every backend that has no session repo — keeps working with it off.
// retentionDays <= 0 means the default.
func (l *ReadLedger) WithSessions(repo SessionReadRepo, retentionDays int) *ReadLedger {
if l == nil || repo == nil {
return l
}
if retentionDays <= 0 {
retentionDays = DefaultSessionRetentionDays
}
l.mu.Lock()
defer l.mu.Unlock()
l.sessions = repo
l.sessionRetention = time.Duration(retentionDays) * 24 * time.Hour
l.pendingSess = map[sessionReadKey]SessionRead{}
return l
}
// Record counts one read. Nil-safe and never fails: telemetry must not break
// the page view (or sync cycle) that triggered it.
func (l *ReadLedger) Record(project, path, kind, actor string) {
@@ -180,6 +256,97 @@ func (l *ReadLedger) Record(project, path, kind, actor string) {
}
}
// RecordSession notes that one agent session read one path from one device.
// Nil-safe, off when no session repo is configured, and — like Record —
// never fails: telemetry must not break the sync cycle that reported it.
// Unlike Record it is NOT debounced: a row is a fact ("this session read this
// file"), not a count, so repeats are the same row rewritten.
func (l *ReadLedger) RecordSession(project, session, device, path string) {
if l == nil || project == "" || session == "" || device == "" || path == "" {
return
}
l.mu.Lock()
defer l.mu.Unlock()
if l.sessions == nil {
return
}
now := time.Now()
sr := SessionRead{Project: project, Session: session, Device: device, Path: path, Last: now.UTC()}
l.pendingSess[sr.key()] = sr
// Same throttle the buckets use. Record's own flush check sits behind its
// debounce return, so a report whose buckets are all debounced would
// otherwise leave these rows buffered indefinitely.
if now.Sub(l.lastFlush) >= readFlushEvery {
l.flushLocked()
}
}
// SessionPaths returns the paths one session read from one device, for the
// History run card. Both the session and the device are required by the
// caller (handleHeat): a session-only lookup would return rows a member
// reported under someone else's session id, which pinning to the validated
// device is what makes harmless.
func (l *ReadLedger) SessionPaths(project, session, device string) []string {
if l == nil || project == "" || session == "" || device == "" {
return nil
}
l.mu.Lock()
repo := l.sessions
// Flush first, so a card opened seconds after a sync sees that sync's
// reads instead of an empty list.
if repo != nil {
l.flushSessionsLocked()
}
l.mu.Unlock()
if repo == nil {
return nil
}
rows, err := repo.ListBySession(project, session, device)
if err != nil {
log.Printf("beardrive: session reads lookup failed: %v", err)
return nil
}
out := make([]string, 0, len(rows))
for _, r := range rows {
out = append(out, r.Path)
}
return out
}
// flushSessionsLocked persists buffered session rows and, at most hourly,
// deletes the ones past the session retention. Failures keep the buffer for
// the next attempt and log once — a read_sessions failure must never affect
// read_stats, so this is deliberately separate from persistLocked. Callers
// hold mu.
func (l *ReadLedger) flushSessionsLocked() {
if l.sessions == nil {
return
}
if len(l.pendingSess) > 0 {
batch := make([]SessionRead, 0, len(l.pendingSess))
for _, sr := range l.pendingSess {
batch = append(batch, sr)
}
if err := l.sessions.PutBatch(batch); err != nil {
if !l.sessWarned {
l.sessWarned = true
log.Printf("beardrive: session read flush failed (will retry): %v", err)
}
} else {
l.sessWarned = false
l.pendingSess = map[sessionReadKey]SessionRead{}
}
}
now := time.Now()
if now.Sub(l.lastSessPrun) < sessionPruneEvery {
return
}
l.lastSessPrun = now
if err := l.sessions.PruneBefore(now.UTC().Add(-l.sessionRetention)); err != nil {
log.Printf("beardrive: session read prune failed (will retry): %v", err)
}
}
// Heat aggregates reads per path for one project. since bounds the window
// (zero = all time, including retention folds); prefix "" means the whole
// project, otherwise paths under "<prefix>/".
@@ -358,6 +525,7 @@ func (l *ReadLedger) flushLocked() {
} else {
l.warned = false
}
l.flushSessionsLocked()
}
// compactLocked folds daily buckets older than the retention horizon into
@@ -516,6 +684,25 @@ func (s *Server) handleHeat(v *volume, w http.ResponseWriter, r *http.Request) {
}
_ = v
q := r.URL.Query()
// ?session=&device= is the run-card join: which paths that agent session
// read. Both are required — a session-only query would also return rows a
// member reported under someone else's session id, which pinning the row
// to the reporting device (handleReadReport) is what makes harmless. This
// is a filter INPUT only: nothing here or anywhere else enumerates
// sessions, and the response carries paths, no identities and no counts.
if session := q.Get("session"); session != "" || q.Get("device") != "" {
device := q.Get("device")
if session == "" || device == "" {
http.Error(w, "session and device must be given together", http.StatusBadRequest)
return
}
paths := s.Reads.SessionPaths(projectID(r), session, device)
if paths == nil {
paths = []string{} // an empty list, never a null the client must special-case
}
writeJSON(w, map[string]any{"paths": paths})
return
}
days := 30
if raw := q.Get("days"); raw != "" {
var err error
@@ -600,6 +787,10 @@ func (s *Server) handleReadReport(v *volume, w http.ResponseWriter, r *http.Requ
var req struct {
Reads []struct {
Path string `json:"path"`
// Session is the agent session the read happened in — a CLIENT
// string, so it is only ever stored alongside the device the hub
// validated below, never on its own. See the row write.
Session string `json:"session,omitempty"`
// Time is accepted for forward compatibility but buckets use
// server time: client clocks are unreliable and late flushes are
// telemetry noise, not data loss.
@@ -656,6 +847,16 @@ func (s *Server) handleReadReport(v *volume, w http.ResponseWriter, r *http.Requ
continue // no such file in this project: a read of nothing is not a read
}
s.Reads.Record(project, e.Path, ReadKindAgent, device)
// The session id is the one field here the hub cannot vouch for: it
// arrives in the body, so any member could report reads naming a
// teammate's session and paint files onto that teammate's run card.
// The row is therefore pinned to `device` — the id ownsDevice just
// validated — and the query side requires BOTH session and device, so
// a forged row can only ever be found under the forger's own device,
// which MayActAs guarantees is never someone else's.
if sess := trimText(e.Session, 128); sess != "" && journal.SafeText(sess) {
s.Reads.RecordSession(project, sess, device, e.Path)
}
n++
}
writeJSON(w, map[string]any{"accepted": n})
+4
View File
@@ -163,6 +163,10 @@ func TestSec_Store_AJournalsAuthorFieldsAreCheckedLikeItsNote(t *testing.T) {
{"author", "Alice\x1b[2Kx", "C0 escape in author"},
{"user_name", "Bob\u202egnp.exe", "bidi override in user_name"},
{"user_name", "Bob\u0085\u009bx", "C1 control in user_name"},
// Op.Session (BEA-98) is served by History and rendered next to the
// note, so it is the same class of peer-written free text.
{"session", "8f21e4\u202ex", "bidi override in session"},
{"session", "8f21e4\x1b[2Kx", "C0 escape in session"},
} {
op := map[string]any{
"device": bobDev, "path": "row-" + tc.field + ".md",
+224
View File
@@ -0,0 +1,224 @@
package webapp
import (
"encoding/json"
"net/http"
"net/http/httptest"
"path/filepath"
"strings"
"testing"
"time"
)
// sessionHub is permHub with read telemetry that also keeps session detail —
// the shape a served hub has (cmd/bdrive/web.go).
func sessionHub(t *testing.T) (http.Handler, *Server, map[string]*http.Cookie, Project, string) {
t.Helper()
h, srv, c, p, root := permHubAt(t)
reads, err := OpenReadLedger(filepath.Join(t.TempDir(), "reads.json"), 0)
if err != nil {
t.Fatal(err)
}
srv.Reads = reads.WithSessions(OpenSessionReadRepo(filepath.Join(t.TempDir(), "sessions.json")), 0)
return h, srv, c, p, root
}
func sessionPaths(t *testing.T, h http.Handler, p Project, c *http.Cookie, session, device string) []string {
t.Helper()
rec := doAs(t, h, "GET",
"/api/p/"+p.ID+"/heat?session="+session+"&device="+device, nil, c)
if rec.Code != 200 {
t.Fatalf("session heat: %d %s", rec.Code, rec.Body)
}
var out struct {
Paths []string `json:"paths"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatal(err)
}
return out.Paths
}
func reportRead(t *testing.T, h http.Handler, p Project, c *http.Cookie, device string, reads []map[string]string) *httptest.ResponseRecorder {
t.Helper()
return secfixDo(t, h, "POST", "/api/p/"+p.ID+"/reads",
map[string]any{"reads": reads}, c, map[string]string{"X-Bdrive-Device": device})
}
// The join, end to end: a session's reads come back for the session+device
// pair its own ops carry, and only for files the project actually has.
func TestSessionReadsRoundTrip(t *testing.T) {
h, _, c, p, root := sessionHub(t)
f := newFakeRemoteAt(t, filepath.Join(root, p.ID))
f.putAs("dev1", "alice@x.io", "Alice", "wiki/plan.md", "# plan")
f.putAs("dev1", "alice@x.io", "Alice", "wiki/spec.md", "# spec")
// dev1 is alice's: one sync registers it, as /store/* traffic does.
if rec := secfixSync(t, h, p.ID, c["alice"], "dev1", "laptop", "mac"); rec.Code != 200 {
t.Fatalf("alice sync: %d %s", rec.Code, rec.Body)
}
rec := reportRead(t, h, p, c["alice"], "dev1", []map[string]string{
{"path": "wiki/plan.md", "session": "8f21e4"},
{"path": "wiki/spec.md", "session": "8f21e4"},
{"path": "wiki/gone.md", "session": "8f21e4"}, // landmine 3: no such file, records nothing
{"path": "wiki/plan.md", "session": "other"}, // a different session, its own row
})
if rec.Code != 200 {
t.Fatalf("report: %d %s", rec.Code, rec.Body)
}
got := sessionPaths(t, h, p, c["alice"], "8f21e4", "dev1")
if len(got) != 2 || got[0] != "wiki/plan.md" || got[1] != "wiki/spec.md" {
t.Fatalf("session paths = %v, want plan.md + spec.md (gone.md is not in the project)", got)
}
if got := sessionPaths(t, h, p, c["alice"], "other", "dev1"); len(got) != 1 || got[0] != "wiki/plan.md" {
t.Fatalf("second session = %v, want only plan.md — sessions must not merge", got)
}
// A member who is not the reporting device still sees the run's reads:
// the card is project-wide, and this response carries no identities.
if got := sessionPaths(t, h, p, c["bob"], "8f21e4", "dev1"); len(got) != 2 {
t.Fatalf("member view = %v, want the same two paths", got)
}
// An unknown session is an empty list, never an error and never a hint
// that some other session exists.
if got := sessionPaths(t, h, p, c["alice"], "no-such-session", "dev1"); len(got) != 0 {
t.Fatalf("unknown session = %v, want empty", got)
}
}
// Landmine 1's read-half twin: the session id in a read report is a CLIENT
// string, so bob can report reads naming alice's session. The row is pinned
// to the device the hub validated — bob's, never alice's — and the query
// requires both, so his rows can never surface on her run card.
func TestSessionReadsCannotBePaintedOntoAnotherDevicesRun(t *testing.T) {
h, srv, c, p, root := sessionHub(t)
f := newFakeRemoteAt(t, filepath.Join(root, p.ID))
f.putAs("dev1", "alice@x.io", "Alice", "wiki/plan.md", "# plan")
f.putAs("dev1", "alice@x.io", "Alice", "payroll.md", "secret")
// alice-mbp is claimed by alice, as `bdrive login` claims it
// (DeviceRegistry.Bind) — the state in which MayActAs has something to
// refuse.
srv.Devices.Observe(DeviceInfo{ID: "alice-mbp", Name: "laptop", OS: "mac", User: "alice@x.io"})
if rec := secfixSync(t, h, p.ID, c["bob"], "bob-mbp", "laptop", "linux"); rec.Code != 200 {
t.Fatalf("bob sync: %d %s", rec.Code, rec.Body)
}
if rec := reportRead(t, h, p, c["alice"], "alice-mbp", []map[string]string{
{"path": "wiki/plan.md", "session": "8f21e4"},
}); rec.Code != 200 {
t.Fatalf("alice report: %d %s", rec.Code, rec.Body)
}
// bob reports under ALICE's session id, from his own device.
if rec := reportRead(t, h, p, c["bob"], "bob-mbp", []map[string]string{
{"path": "payroll.md", "session": "8f21e4"},
}); rec.Code != 200 {
t.Fatalf("bob report: %d %s", rec.Code, rec.Body)
}
// bob naming alice's DEVICE outright is refused by ownsDevice, so it
// records for nobody.
if rec := reportRead(t, h, p, c["bob"], "alice-mbp", []map[string]string{
{"path": "payroll.md", "session": "8f21e4"},
}); rec.Code != 200 {
t.Fatalf("bob's forged-device report: %d %s", rec.Code, rec.Body)
}
got := sessionPaths(t, h, p, c["alice"], "8f21e4", "alice-mbp")
if len(got) != 1 || got[0] != "wiki/plan.md" {
t.Fatalf("alice's run card = %v, want only her own read — bob painted onto it", got)
}
}
// The API shape: ?session= is a filter INPUT that requires its device, and
// the route is membership-gated exactly as /heat is.
func TestSessionHeatQueryContract(t *testing.T) {
h, _, c, p, _ := sessionHub(t)
base := "/api/p/" + p.ID + "/heat"
for _, u := range []string{base + "?session=8f21e4", base + "?device=dev1"} {
if rec := doAs(t, h, "GET", u, nil, c["alice"]); rec.Code != 400 {
t.Fatalf("GET %s: %d, want 400 (session and device are required together)", u, rec.Code)
}
}
// dave is in no org here: a non-member is walled out of the session
// query exactly as they are out of plain heat.
for _, u := range []string{base, base + "?session=8f21e4&device=dev1"} {
if rec := doAs(t, h, "GET", u, nil, c["dave"]); rec.Code != http.StatusForbidden {
t.Fatalf("outsider GET %s: %d, want 403", u, rec.Code)
}
}
}
// The privacy ruling, tested: nothing enumerates sessions. ?by=device output
// is byte-identical with session rows present, and plain /heat never grows a
// session column.
func TestSessionIdsAreNeverEnumerated(t *testing.T) {
h, _, c, p, root := sessionHub(t)
f := newFakeRemoteAt(t, filepath.Join(root, p.ID))
f.putAs("dev1", "alice@x.io", "Alice", "wiki/plan.md", "# plan")
if rec := secfixSync(t, h, p.ID, c["alice"], "dev1", "laptop", "mac"); rec.Code != 200 {
t.Fatalf("alice sync: %d %s", rec.Code, rec.Body)
}
if rec := reportRead(t, h, p, c["alice"], "dev1", []map[string]string{
{"path": "wiki/plan.md", "session": "8f21e4"},
}); rec.Code != 200 {
t.Fatalf("report: %d %s", rec.Code, rec.Body)
}
for _, body := range []string{
doAs(t, h, "GET", "/api/p/"+p.ID+"/heat?by=device", nil, c["alice"]).Body.String(),
doAs(t, h, "GET", "/api/p/"+p.ID+"/heat", nil, c["alice"]).Body.String(),
} {
if strings.Contains(body, "8f21e4") || strings.Contains(body, "session") {
t.Fatalf("a heat response enumerated a session: %s", body)
}
}
}
// Retention: session rows past the horizon are DELETED, not folded, and the
// path's heat totals — which were never derived from them — are unchanged.
func TestSessionReadRetentionPrunes(t *testing.T) {
repo := OpenSessionReadRepo(filepath.Join(t.TempDir(), "sessions.json"))
l, _ := openTestLedger(t, 0)
l.WithSessions(repo, 1) // one day
l.Record("p-1", "a.md", ReadKindAgent, "dev1")
l.RecordSession("p-1", "old", "dev1", "a.md")
l.RecordSession("p-1", "new", "dev1", "a.md")
// Age the "old" session past the horizon, then force a prune.
l.mu.Lock()
for k, sr := range l.pendingSess {
if k.Session == "old" {
sr.Last = time.Now().UTC().Add(-48 * time.Hour)
l.pendingSess[k] = sr
}
}
l.lastSessPrun = time.Time{}
l.flushSessionsLocked()
l.mu.Unlock()
if got, _ := repo.ListBySession("p-1", "old", "dev1"); len(got) != 0 {
t.Fatalf("expired session rows survived: %+v", got)
}
if got, _ := repo.ListBySession("p-1", "new", "dev1"); len(got) != 1 {
t.Fatalf("recent session rows = %+v, want the one row", got)
}
// The aggregate is untouched by any of it.
if e := l.Heat("p-1", "", time.Time{})["a.md"]; e.Agent != 1 {
t.Fatalf("heat after the session prune = %+v, want agent 1", e)
}
}
// With no session repo the ledger behaves exactly as before: recording is a
// no-op and a lookup is empty, never a panic.
func TestSessionReadsOffByDefault(t *testing.T) {
l, _ := openTestLedger(t, 0)
l.RecordSession("p-1", "s1", "dev1", "a.md")
if got := l.SessionPaths("p-1", "s1", "dev1"); len(got) != 0 {
t.Fatalf("session paths with no repo = %v, want none", got)
}
var nilLedger *ReadLedger
nilLedger.RecordSession("p-1", "s1", "dev1", "a.md")
if got := nilLedger.SessionPaths("p-1", "s1", "dev1"); got != nil {
t.Fatalf("nil ledger = %v", got)
}
}
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+2 -2
View File
@@ -5,8 +5,8 @@
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>BearDrive</title>
<link rel="icon" href="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32' fill='%23f5a623'><rect x='4' y='4' width='5.6' height='24'/><rect x='11.2' y='4' width='14.4' height='11.2'/><rect x='11.2' y='16.8' width='16.8' height='11.2'/></svg>">
<script type="module" crossorigin src="/assets/index-C64R_Rr_.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-BdCy9HmN.css">
<script type="module" crossorigin src="/assets/index-DRd_YZgy.js"></script>
<link rel="stylesheet" crossorigin href="/assets/index-Do25j1to.css">
</head>
<body>
<div id="root"></div>
+5 -1
View File
@@ -353,7 +353,11 @@ func journalOps(key string, tmp *os.File) ([]journal.Op, error) {
// and a C0 run in author is the "renders as nothing" shape its own doc
// comment names. DeviceName is absent on purpose: History serves the
// device REGISTRY's name, not the op's.
if !journal.SafeText(op.Note) || !journal.SafeText(op.Author) || !journal.SafeText(op.UserName) {
// Op.Session joins that list for the same reason: History serves it
// beside the note and the frontend groups run cards on it, so it is
// peer-written free text rendered in the audit surface.
if !journal.SafeText(op.Note) || !journal.SafeText(op.Author) ||
!journal.SafeText(op.UserName) || !journal.SafeText(op.Session) {
return nil, fmt.Errorf("journal carries invalid text")
}
}
@@ -47,6 +47,30 @@ lens — four views:
spotting an agent that never discovered the folder at all, which usually means
a missing [root pointer](/guides/shared-agent-memory/).
## One session, read and written
History groups an agent session's changes into a single **run card**. The card
also shows what that session *read*: files it read before changing them are
marked, and files it read without touching at all get their own **Read, not
changed** list underneath.
That's the question the heat map alone can't answer — *when my agent answered,
what did it actually look at, and was it the current version or the retired
one?*
Two things worth knowing:
- Reads are shown only for files the project still has. A file a run read and
then deleted appears as a change with no read. The card says so on screen.
- The join is on a session id the sync hook stamps, never on the run's note —
the note is free text anyone can set with `bdrive sync --note`, so joining on
it would let one person's changes attach to another person's card.
Per-session detail is kept for 30 days by default
(`reads.session_retention_days`, see [Hub config](/reference/hub-config/));
after that the run card shows changes only. Read *counts* are unaffected — they
come from the heat buckets, which have their own, much longer retention.
## Using it
A few things this surfaces that are otherwise guesswork:
@@ -67,6 +91,13 @@ distinct-reader counts, and last-read times. **Never who read what.**
already public via history. Human email addresses never appear in a heat
response.
A session id is treated the same way. It appears only in History, on the change
that carries it, and is accepted as a `?session=<id>&device=<id>` filter
**input** — both are required. Nothing enumerates sessions: no listing
endpoint, no session column in heat output, nothing new in `?by=device`. And a
session's reads are always recorded against the device the hub validated the
report came from, so nobody can paint files onto a teammate's run card.
Telemetry degrades silently: recording or flushing a read can never fail a
request or a sync cycle.
@@ -63,7 +63,8 @@ cloud credentials on the serving machine.
},
"reads": { // read heatmap telemetry (hub mode)
"enabled": true, // default true; aggregate counts only
"retention_days": 400 // daily buckets older than this fold into all-time totals
"retention_days": 400, // daily buckets older than this fold into all-time totals
"session_retention_days": 30 // how long History's run cards keep per-session read detail
},
"database": { "driver": "sqlite", "dsn": "/var/lib/bdrive/hub.db" }
}