mirror of
https://github.com/block/buzz.git
synced 2026-08-18 06:50:31 +02:00
+1
14fba21e57
14fba21e57
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com> Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Mari <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Quinn <96f056ad5f2305c8ddf637dc65d048aa4c12d7daeb8867690e34fca46b0ef64c@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Dawn <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: Sami <sami@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co>
67 lines
3.0 KiB
Markdown
67 lines
3.0 KiB
Markdown
# Buzz relay bus scaling harness
|
||
|
||
This harness gives reproducible evidence for the rewrite's Redis fan-out scaling claim:
|
||
|
||
- **old/global bus:** every relay pod receives every community's event;
|
||
- **new/community-scoped bus:** each pod retains only the server-resolved community topics for which it has local subscribers (`buzz:{community_id}:global` or `buzz:{community_id}:channel:{channel_id}`).
|
||
|
||
The measured default uses Redis PUB/SUB directly with simulated relay pod subscribers. It intentionally isolates the bus boundary: no DB ingest, websocket framing, client rendering, or relay business logic is included.
|
||
|
||
## Run the measured Redis harness
|
||
|
||
Start Redis locally, or point at an existing instance with `REDIS_URL`:
|
||
|
||
```bash
|
||
REDIS_URL=redis://127.0.0.1:6379/0 ./perf/relay_bus_scaling.py --mode redis
|
||
```
|
||
|
||
`--mode redis` is the default. The script uses only Python stdlib and speaks RESP directly; no Python Redis dependency is required.
|
||
|
||
Baseline scenario used for the PR summary:
|
||
|
||
```text
|
||
64 communities × 100 events/s, one subscribed community, all pods interested in that community, pods = 1,2,4
|
||
```
|
||
|
||
Measured output shape:
|
||
|
||
| pods | old global cluster ingress/s | old avg pod ingress/s | new scoped cluster ingress/s | new avg pod ingress/s | reduction | old irrelevant/pod | new irrelevant/pod |
|
||
|---:|---:|---:|---:|---:|---:|---:|---:|
|
||
| 1 | 6,400 | 6,400 | 100 | 100 | 64.0× | 98.44% | 0.00% |
|
||
| 2 | 12,800 | 6,400 | 200 | 100 | 64.0× | 98.44% | 0.00% |
|
||
| 4 | 25,600 | 6,400 | 400 | 100 | 64.0× | 98.44% | 0.00% |
|
||
|
||
The harness fails non-zero by default unless:
|
||
|
||
- observed reduction is at least 95% of the ideal `communities / subscribed_communities`; and
|
||
- scoped-mode irrelevant delivery is at most `--max-scoped-irrelevant-pct` (default `0.0`).
|
||
|
||
That makes the scaling claim load-bearing: if scoped subscribers are accidentally changed to receive the old global firehose, the assertion goes red.
|
||
|
||
## No-service model mode
|
||
|
||
For quick review without Redis:
|
||
|
||
```bash
|
||
./perf/relay_bus_scaling.py --mode model
|
||
```
|
||
|
||
Model mode prints the same contract using deterministic arithmetic. It is useful for docs and unit tests, but the PR evidence should cite `--mode redis` because that path measures actual Redis PUB/SUB delivery.
|
||
|
||
## Unit tests
|
||
|
||
```bash
|
||
python3 -m unittest discover -s perf -p 'test_*.py'
|
||
```
|
||
|
||
The unit tests pin the default 1/2/4-pod 64× contract and include a mutant row that represents scoped mode receiving irrelevant global-firehose traffic; that row must fail the assertion.
|
||
|
||
## Code provenance
|
||
|
||
The scoped Redis channel format corresponds to `buzz_pubsub::EventTopicKey::redis_channel()` in `crates/buzz-pubsub/src/topic.rs`:
|
||
|
||
- global: `buzz:{community_id}:global`
|
||
- channel: `buzz:{community_id}:channel:{channel_id}`
|
||
|
||
`retain_topic` / `release_topic` drive dynamic local Redis `SUBSCRIBE` interest. This harness measures that bus-bound property only. Live relay latency, DB capacity, and client rendering should be measured separately with a full stack because they include unrelated bottlenecks.
|