Apply cargo fmt to the 11 files touched by the 9 reviewed multi-tenant
fix commits. CI Rust Lint (just fmt-check) flagged formatting drift my local
clippy --lib --tests run did not exercise; base a7d3817481 was fmt-clean, so
this only normalizes lines my own commits introduced. No logic change.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
A brand-new interval workflow on a cold engine has no in-memory last_fired
entry and no prior durable claim, so the scheduler resolves last = None.
interval_should_fire then reads last = now and suppresses the tick (correct:
wait a full interval), but the in-memory anchor is only written AFTER a won
claim, and no claim is attempted until the prefilter passes. Every subsequent
tick repeats with last = None, so the workflow suppresses forever.
Extract interval_prefilter_should_fire (free fn over the last_fired map + a
thin &self wrapper): on the cold-start None suppress path it seeds the anchor
to now so the next tick counts from a real anchor and fires after one interval.
It seeds ONLY when last was None; an existing Some anchor is mid-interval and
must elapse on its own, so it is never advanced. A due/firing tick passes
through without seeding (the post-claim path owns that write).
Unit tests (no Db/Postgres; pure in-memory anchor state):
- cold start seeds then fires after one interval
- mid-interval suppress does not advance an existing anchor
- a due fire passes through without seeding
Caught by Max in cold review of the scheduled-workflow lane; predates this
branch's claim work but lives on the exact lane being cleared.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
The scheduler's post-run `attach_scheduled_workflow_run` does `SET workflow_run_id`,
but neither schema/schema.sql nor the initial migration declared that column, so
every won scheduled claim would create+run, then the best-effort attach would hit
`column "workflow_run_id" does not exist` (PG 42703) and warn forever — the audit
link could never populate. Found by Max in cold review of 778a5d28c.
- Add nullable `workflow_run_id UUID` to scheduled_workflow_fires (schema + migration)
with a composite FK `(community_id, workflow_run_id) REFERENCES workflow_runs
(community_id, id)`. The FK uses ON DELETE NO ACTION, not SET NULL: community_id is
shared with the claim PK and is NOT NULL, so SET NULL is unimplementable (verified
against live PG: it raises a NOT NULL violation mid-cascade). NO ACTION blocks a
delete of a still-linked run cleanly; workflow_runs are not pruned today regardless.
- Rewrite the stale scheduled-fires schema comment that still claimed community is
'resolved server-side from workflow_id, never a caller-supplied claim parameter' —
contradicted by the S1 reconciliation: community is server provenance from
list_all_enabled_workflows(), passed explicitly, since id is not globally unique.
- Surface the interval-anchor read failure with a warn! instead of unwrap_or(None)
swallowing it (still fail-closed: a missing anchor suppresses the tick and retries).
- Add attach_links_run_to_claim_and_is_idempotent: proves the column populates on
attach and the IS NULL guard makes a second attach a no-op. Proven RED against the
pre-migration schema (the exact 42703 error), GREEN after — the regression that
would have caught this gap.
Verified vs live Postgres, serial: buzz-db 110 / buzz-workflow 145 / buzz-relay 414,
0 failed; clippy -D warnings clean on the trio.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
The scheduled-workflow fire path never called the durable claim
primitives (`claim_scheduled_workflow_fire` had zero callers outside the
DB wrappers and tests), so with N relay pods every pod that saw a due
cron/interval row created a run and executed the side effect — a
multi-pod duplicate-run bug. And the claim primitive itself was still
keyed by bare `workflow_id` (`WHERE w.id = $1`) despite the schema's
`(community_id, id)` workflow identity: with duplicate workflow UUIDs
across communities (which the schema explicitly allows and the Issue-4
confinement tests pin), a single `INSERT ... SELECT` matched every
community's row and fanned one claim across all of them.
Scope the claim to its community and wire the claim->run boundary into
the scheduler:
- `claim_scheduled_workflow_fire` takes `CommunityId` and binds
`WHERE w.community_id = $1 AND w.id = $2`. The community is server
provenance — the `workflow.community_id` from the global scan, never
client input. This reverses the earlier S1 "resolve-from-id-alone"
lock, which was written against a globally-unique-`workflow_id`
assumption the final `(community_id, id)` schema does not hold (and
which is unimplementable on that key). The surviving invariant is
"the claim community is server provenance, not client-controlled."
- `WorkflowEngine::run` now claims before `create_workflow_run`; the
loser skips before any run creation or side effect. The claim anchor
`scheduled_for` is deterministic across pods: the cron's own scheduled
instant (`cron_fire_instant`, not `now`) or the interval bucket
boundary (`interval_fire_instant`, floor to the interval). The
interval anchor is seeded from `latest_scheduled_workflow_fire` on the
first tick after restart so a process bounce can't double-fire within
an interval. `attach_scheduled_workflow_run` links the won claim to
its run for ops/audit forensics.
Tests:
- Rewrite the stale S1 comment block and `claim_for_workflow_in_other_
community_no_ops` (which encoded the now-false globally-unique
assumption) into `claim_confined_to_its_community`: a dup workflow
UUID in A and B claims independently (claiming A/id leaves B/id
claimable). Proven RED on the bare-`id` regression.
- `concurrent_same_window_claims_exactly_one_wins` and the prune-anchor
test updated to the scoped signature.
- New `cron_fire_instant` / `interval_fire_instant` unit tests pin the
deterministic, drift-stable claim anchors.
Full `cargo test -p buzz-db -p buzz-workflow -p buzz-relay` green vs
live Postgres (serial); clippy clean on the trio.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
`workflows`, `workflow_runs`, and `workflow_approvals` are all keyed
`(community_id, id|token)`, so the same UUID/token is structurally allowed
in two communities — exactly like channels and events. But the execution
and approval spine still fetched, listed, mutated, and posted by bare id,
so a webhook/manual trigger or NIP-09 deletion in community B could load,
drive, or erase community A's colliding workflow, and workflow side effects
were published under the deployment/default tenant instead of the run's own
community. This threads the owning community through every request-scoped
and run-scoped path so each lookup, write, and side effect is confined to
its tenant.
- `ActionSink::send_message` now takes the run's `community_id` as its first
parameter. `RelayActionSink` drops `bind_deployment_community(relay_url)` —
the Issue-4 root cause — and instead resolves the run community's host via
`lookup_community_host` to form a complete `TenantContext::resolved`, fail
closed if the community is unmapped. A workflow in B now posts into B.
- Executor (`dispatch_action`, `execute_run`, `execute_from_step`,
`execute_steps`) and engine (`finalize_run`, `on_event`) carry the run's
community; every `get_workflow_run` / `get_workflow` / `send_message` and
the post-store `on_event` call (from `dispatch_persistent_event`, which has
the bound `tenant`) are scoped. The interval `last_fired` DashMap is keyed
`(CommunityId, Uuid)` so duplicate workflow UUIDs across communities cannot
cross-suppress in memory.
- Webhook `/hooks/{id}` now binds its community from the request Host before
any lookup (`bind_community`), then `get_workflow(community, id)`. The host
— not the workflow row — determines the tenant, so a request to A's host
can only reach A's workflows; unmapped host and not-found both fail closed
with the same generic 404.
- WS manual trigger and `create_workflow` use `tenant.community()` as the
authoritative owner. `create_workflow` no longer resolves the community via
the ambiguous `community_of_channel(channel_id)`; it verifies the channel
exists *inside* the bound community via scoped `get_channel` (the same
guarantee the composite FK enforces, surfaced as a clean rejection).
- Approval grant/deny/resume handlers and the `buzz-db` approval methods
(`get_approval`, `get_approval_by_stored_hash`, `get_run_approvals`,
`update_approval`, `update_approval_by_stored_hash`, `create_approval`)
are scoped by community; `create_approval`'s INSERT now includes the
`community_id` NOT-NULL column it previously omitted. NIP-09 a-tag workflow
deletion (`delete_workflow`, `find_workflow_by_owner_and_name`) is scoped
to the request tenant.
Adds three `#[ignore]` Postgres regressions in `buzz-db::workflow`, each
verified green against live PG and red when the `community_id` predicate is
dropped: `workflow_lookup_is_confined_to_its_community` (dup workflow+channel
UUID in A/B; scoped get/list resolve only the bound community's row, cross
lookup is NotFound), `workflow_delete_is_confined_to_its_community` (deleting
A/id leaves B/id intact), and `approval_is_confined_to_its_community` (same
token in A/B; granting A leaves B pending). Full `cargo test -p buzz-db
-p buzz-workflow -p buzz-relay` green, clippy clean on the trio.
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr