mirror of
https://github.com/block/buzz.git
synced 2026-08-18 06:50:31 +02:00
Fixes #2637 — full analysis and reproduction there. ## Problem Audit entries are stamped and hashed with `Utc::now()` (nanoseconds), then stored in a `TIMESTAMPTZ` column (microseconds). `compute_hash` covers `created_at.to_rfc3339()`, and chrono emits 0/3/6/**9** fractional digits depending on the value — so the digest written at `service.rs:103` is computed over `…T12:00:00.123456789+00:00` while `verify_chain` recomputes over the `…T12:00:00.123456+00:00` that Postgres hands back. Every hash chain backed by a real database therefore fails verification at its first entry, on untampered data. That is not just a broken feature — it means a genuinely forged row is indistinguishable from the permanent baseline failure, so `HashMismatch` carries no signal. It is invisible in CI because all six chain tests are `#[ignore = "requires Postgres"]`, and the in-process `hash.rs` tests use a fixture timestamp of `2026-01-01T00:00:00Z` — zero sub-seconds, the one value where the bug cannot appear. ## Solution Reduce `created_at` to the stored precision *before* hashing, so the in-memory entry and the row are byte-identical: ```rust pub fn to_storage_precision(created_at: DateTime<Utc>) -> DateTime<Utc> { created_at.trunc_subsecs(6) } ``` `log_inner` is the only place that assigns `created_at` — every caller goes through `NewAuditEntry`, which carries no timestamp — so this is a single choke point. It is wrapped in a `log_timestamp()` helper purely so the invariant is assertable without a database. I chose truncation at the write path over the alternative (hashing a precision-independent encoding such as `timestamp_micros().to_be_bytes()`). Both fix the mismatch, but truncating keeps the existing hash preimage format and gives the stronger invariant: the `AuditEntry` returned from `log()` is now exactly what a later read returns. Truncation matches what actually happens on the wire — sqlx encodes `DateTime<Utc>` as microseconds since the Postgres epoch, truncating — so the value hashed is the value stored. ## Validation Toolchain note: built on Windows with the `x86_64-pc-windows-gnu` toolchain (no MSVC linker locally). **Before**, against Postgres 17 with `migrations/*` applied: ``` $ cargo test -p buzz-audit --lib -- --ignored --test-threads=1 test service::tests::chain_links_within_one_community ... FAILED test service::tests::chains_are_independent_per_community ... FAILED test service::tests::community_chain_starts_at_seq_1_with_null_prev ... ok test service::tests::cross_community_row_does_not_verify ... ok test service::tests::verify_detects_tampering_within_a_community ... FAILED test service::tests::verify_empty_range_is_false ... ok test result: FAILED. 3 passed; 3 failed ``` with `HashMismatch { seq: 2 }` / `HashMismatch { seq: 1 }` on untampered chains. **After**, same database: ``` test result: ok. 6 passed; 0 failed ``` `verify_detects_tampering_within_a_community` is the one to look at: it asserts `HashMismatch` lands on the *tampered* entry's `seq`. It was failing because verification already blew up on an earlier untampered row — so the assertion proving tamper detection works had never actually been exercised. It passes now. Also: - `cargo test -p buzz-audit --lib` (no Postgres) — 12 passed, 0 failed. - `cargo clippy -p buzz-audit --all-targets -- -D warnings` — clean. - `cargo fmt -p buzz-audit -- --check` — clean. ## New tests Three in `hash.rs`, none needing Postgres: - `storage_precision_drops_sub_microsecond_digits` — the helper's contract, and that it is idempotent so a re-read value is unchanged. - `nanosecond_timestamps_cannot_survive_a_database_round_trip` — asserts the digests **differ**. This is the trap itself, written down so the next person changing the hash preimage sees why the precision reduction is load-bearing. - `storage_precision_timestamps_survive_a_database_round_trip` — the invariant the write path must hold. Plus `log_timestamp_carries_no_sub_microsecond_digits` in `service.rs`, deliberately **not** `#[ignore]`d, so a regression on the write path is caught by `just test-unit` instead of only by Postgres-gated tests that normally never run. ## Compatibility Rows written before this stay unverifiable — they always were — so there is no migration. An operator relying on an existing chain has to re-anchor. ## Relationship to #2620 #2620 proposes a shared `verify_entries` walk (anchoring, seq contiguity, tail-truncation detection) plus a `buzz-admin audit verify` command. Its Postgres-free unit tests build entries in memory and would pass regardless, but its `#[ignore]` Postgres tests and the operator command itself would fail on every real chain until this lands. Worth taking this first so that work has a verifiable baseline — the two changes don't overlap in code. --------- Signed-off-by: Shani Singh <teamdeveloperworld@gmail.com>
273 lines
10 KiB
Rust
273 lines
10 KiB
Rust
use chrono::{DateTime, SubsecRound, Utc};
|
|
use sha2::{Digest, Sha256};
|
|
|
|
use crate::entry::AuditEntry;
|
|
use crate::error::AuditError;
|
|
|
|
/// The 32-byte sentinel hashed in place of `prev_hash` for a community's first
|
|
/// entry. Stored as `prev_hash = NULL`; hashed as all-zero bytes.
|
|
pub const GENESIS_HASH: [u8; 32] = [0u8; 32];
|
|
|
|
/// Reduce a timestamp to the precision the audit store round-trips.
|
|
///
|
|
/// `audit_log.created_at` is `TIMESTAMPTZ`, which Postgres keeps at microsecond
|
|
/// resolution. [`compute_hash`] covers `created_at.to_rfc3339()`, and that
|
|
/// string's sub-second digit count follows the value (chrono emits 0, 3, 6 or 9
|
|
/// digits), so a timestamp carrying nanoseconds hashes to a digest that can
|
|
/// never be recomputed from the stored row — the entry is written with one
|
|
/// preimage and verified against another.
|
|
///
|
|
/// Every `created_at` must therefore pass through here *before* it is hashed
|
|
/// and stored, so the in-memory entry and the row are byte-identical.
|
|
pub fn to_storage_precision(created_at: DateTime<Utc>) -> DateTime<Utc> {
|
|
created_at.trunc_subsecs(6)
|
|
}
|
|
|
|
/// SHA-256 over the entry's identity, chain, and context fields.
|
|
///
|
|
/// Field order is fixed — changing it invalidates all existing chains. The
|
|
/// `community_id` is hashed first so chain identity carries the tenant: an entry
|
|
/// cannot be lifted out of one community's chain and re-verified inside another.
|
|
///
|
|
/// `created_at` is normalized through [`to_storage_precision`] here rather than
|
|
/// hashed as given. Write paths truncate before storing so the row matches the
|
|
/// in-memory entry, but normalizing again at the single point that consumes the
|
|
/// value means no future caller can reintroduce the write/read preimage split
|
|
/// by forgetting to. Values already at storage precision are unaffected —
|
|
/// truncation is idempotent — so this does not change any digest.
|
|
///
|
|
/// `detail` is serialized via [`canonical_json`] (sorted keys) so the hash is
|
|
/// stable across machines and Rust versions. A serialization failure is a hard
|
|
/// error, never silently hashed as empty.
|
|
pub fn compute_hash(entry: &AuditEntry) -> Result<[u8; 32], AuditError> {
|
|
let mut hasher = Sha256::new();
|
|
// Tenant binding: community_id leads the hash.
|
|
hasher.update(entry.community_id.as_bytes());
|
|
hasher.update(entry.seq.to_be_bytes());
|
|
hasher.update(
|
|
to_storage_precision(entry.created_at)
|
|
.to_rfc3339()
|
|
.as_bytes(),
|
|
);
|
|
hasher.update(entry.action.as_str().as_bytes());
|
|
match &entry.actor_pubkey {
|
|
Some(pk) => {
|
|
hasher.update([1u8]); // presence tag — distinguishes Some(empty) from None
|
|
hasher.update(pk);
|
|
}
|
|
None => hasher.update([0u8]),
|
|
}
|
|
match &entry.object_id {
|
|
Some(id) => {
|
|
hasher.update([1u8]);
|
|
hasher.update(id.as_bytes());
|
|
}
|
|
None => hasher.update([0u8]),
|
|
}
|
|
hasher.update(canonical_json(&entry.detail)?.as_bytes());
|
|
match &entry.prev_hash {
|
|
Some(h) => hasher.update(h),
|
|
None => hasher.update(GENESIS_HASH),
|
|
}
|
|
Ok(hasher.finalize().into())
|
|
}
|
|
|
|
/// Serialize a JSON value with sorted object keys for deterministic output.
|
|
///
|
|
/// Propagates any scalar serialization error rather than substituting a
|
|
/// placeholder — a hash must never silently stand in an empty value for a real
|
|
/// payload.
|
|
fn canonical_json(value: &serde_json::Value) -> Result<String, serde_json::Error> {
|
|
use serde_json::Value;
|
|
use std::collections::BTreeMap;
|
|
|
|
match value {
|
|
Value::Object(map) => {
|
|
let sorted: BTreeMap<&str, &Value> = map.iter().map(|(k, v)| (k.as_str(), v)).collect();
|
|
let mut out = String::from("{");
|
|
let mut first = true;
|
|
for (k, v) in &sorted {
|
|
if !first {
|
|
out.push(',');
|
|
}
|
|
first = false;
|
|
out.push_str(&serde_json::to_string(k)?);
|
|
out.push(':');
|
|
out.push_str(&canonical_json(v)?);
|
|
}
|
|
out.push('}');
|
|
Ok(out)
|
|
}
|
|
Value::Array(arr) => {
|
|
let mut out = String::from("[");
|
|
let mut first = true;
|
|
for v in arr {
|
|
if !first {
|
|
out.push(',');
|
|
}
|
|
first = false;
|
|
out.push_str(&canonical_json(v)?);
|
|
}
|
|
out.push(']');
|
|
Ok(out)
|
|
}
|
|
other => serde_json::to_string(other),
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use crate::{action::AuditAction, entry::AuditEntry};
|
|
use chrono::Utc;
|
|
use uuid::Uuid;
|
|
|
|
fn sample_entry() -> AuditEntry {
|
|
AuditEntry {
|
|
community_id: Uuid::from_u128(1),
|
|
seq: 1,
|
|
hash: Vec::new(),
|
|
prev_hash: None,
|
|
action: AuditAction::EventCreated,
|
|
actor_pubkey: Some(vec![0xab; 32]),
|
|
object_id: Some("abc123".into()),
|
|
detail: serde_json::Value::Null,
|
|
created_at: chrono::DateTime::parse_from_rfc3339("2026-01-01T00:00:00Z")
|
|
.unwrap()
|
|
.with_timezone(&Utc),
|
|
}
|
|
}
|
|
|
|
/// A wall-clock instant carrying sub-microsecond digits, like `Utc::now()`
|
|
/// returns on Linux (`clock_gettime`, nanosecond resolution).
|
|
fn nanosecond_instant() -> chrono::DateTime<Utc> {
|
|
chrono::DateTime::from_timestamp_nanos(1_700_000_000_123_456_789)
|
|
}
|
|
|
|
/// What Postgres hands back for a `TIMESTAMPTZ`: microsecond resolution.
|
|
fn after_database_round_trip(ts: chrono::DateTime<Utc>) -> chrono::DateTime<Utc> {
|
|
ts.trunc_subsecs(6)
|
|
}
|
|
|
|
#[test]
|
|
fn deterministic() {
|
|
let entry = sample_entry();
|
|
assert_eq!(compute_hash(&entry).unwrap(), compute_hash(&entry).unwrap());
|
|
assert_eq!(compute_hash(&entry).unwrap().len(), 32);
|
|
}
|
|
|
|
#[test]
|
|
fn storage_precision_drops_sub_microsecond_digits() {
|
|
let stored = to_storage_precision(nanosecond_instant());
|
|
assert_eq!(stored.timestamp_subsec_nanos(), 123_456_000);
|
|
// Idempotent, so a stored value re-read from Postgres is unchanged.
|
|
assert_eq!(stored, after_database_round_trip(stored));
|
|
}
|
|
|
|
#[test]
|
|
fn rfc3339_sub_second_width_follows_the_value() {
|
|
// The underlying trap, pinned on the preimage rather than the digest:
|
|
// chrono emits 0/3/6/9 fractional digits depending on the value, so a
|
|
// nanosecond timestamp and its microsecond truncation are *different
|
|
// strings*. Hashing the untruncated value therefore produces a digest
|
|
// that cannot be recomputed from the stored row — which is what made
|
|
// every entry fail `verify_chain` with `HashMismatch`.
|
|
let ns = nanosecond_instant();
|
|
assert_eq!(ns.to_rfc3339(), "2023-11-14T22:13:20.123456789+00:00");
|
|
assert_eq!(
|
|
after_database_round_trip(ns).to_rfc3339(),
|
|
"2023-11-14T22:13:20.123456+00:00"
|
|
);
|
|
assert_ne!(ns.to_rfc3339(), after_database_round_trip(ns).to_rfc3339());
|
|
}
|
|
|
|
#[test]
|
|
fn compute_hash_normalizes_sub_microsecond_timestamps() {
|
|
// The enforcement point: even handed an untruncated `created_at`,
|
|
// `compute_hash` digests the storage-precision value, so a write path
|
|
// that forgot to truncate cannot split the write/read preimage.
|
|
let ns = nanosecond_instant();
|
|
let mut written = sample_entry();
|
|
written.created_at = ns;
|
|
let mut read_back = sample_entry();
|
|
read_back.created_at = after_database_round_trip(ns);
|
|
|
|
assert_eq!(
|
|
compute_hash(&written).unwrap(),
|
|
compute_hash(&read_back).unwrap()
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn storage_precision_timestamps_survive_a_database_round_trip() {
|
|
// The invariant the write path must hold: hash what will be stored, so
|
|
// recomputing from the row reproduces the digest.
|
|
let mut written = sample_entry();
|
|
written.created_at = to_storage_precision(nanosecond_instant());
|
|
let mut read_back = written.clone();
|
|
read_back.created_at = after_database_round_trip(read_back.created_at);
|
|
|
|
assert_eq!(
|
|
compute_hash(&written).unwrap(),
|
|
compute_hash(&read_back).unwrap()
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn community_id_is_part_of_identity() {
|
|
// The whole point: the same logical entry in two communities hashes
|
|
// differently, so a row can't be replayed across chains.
|
|
let a = sample_entry();
|
|
let mut b = a.clone();
|
|
b.community_id = Uuid::from_u128(2);
|
|
assert_ne!(compute_hash(&a).unwrap(), compute_hash(&b).unwrap());
|
|
}
|
|
|
|
#[test]
|
|
fn sensitive_to_each_field() {
|
|
let base = sample_entry();
|
|
let h0 = compute_hash(&base).unwrap();
|
|
|
|
let mut e = base.clone();
|
|
e.seq = 2;
|
|
assert_ne!(h0, compute_hash(&e).unwrap());
|
|
|
|
let mut e = base.clone();
|
|
e.action = AuditAction::EventDeleted;
|
|
assert_ne!(h0, compute_hash(&e).unwrap());
|
|
|
|
let mut e = base.clone();
|
|
e.actor_pubkey = Some(vec![0xcd; 32]);
|
|
assert_ne!(h0, compute_hash(&e).unwrap());
|
|
|
|
let mut e = base.clone();
|
|
e.object_id = Some("different".into());
|
|
assert_ne!(h0, compute_hash(&e).unwrap());
|
|
|
|
let mut e = base.clone();
|
|
e.detail = serde_json::json!({"key": "value"});
|
|
assert_ne!(h0, compute_hash(&e).unwrap());
|
|
|
|
let mut e = base.clone();
|
|
e.prev_hash = Some(vec![0xff; 32]);
|
|
assert_ne!(h0, compute_hash(&e).unwrap());
|
|
}
|
|
|
|
#[test]
|
|
fn presence_tag_distinguishes_none_from_empty() {
|
|
// Some(empty) must not collide with None — the presence tag prevents it.
|
|
let mut none = sample_entry();
|
|
none.actor_pubkey = None;
|
|
let mut empty = sample_entry();
|
|
empty.actor_pubkey = Some(Vec::new());
|
|
assert_ne!(compute_hash(&none).unwrap(), compute_hash(&empty).unwrap());
|
|
}
|
|
|
|
#[test]
|
|
fn canonical_json_key_order_is_stable() {
|
|
let a = serde_json::json!({"z": 1, "a": 2, "m": 3});
|
|
let b = serde_json::json!({"a": 2, "m": 3, "z": 1});
|
|
assert_eq!(canonical_json(&a).unwrap(), canonical_json(&b).unwrap());
|
|
}
|
|
}
|