mirror of
https://github.com/block/buzz.git
synced 2026-08-18 06:50:31 +02:00
docs: rename Typesense references to Postgres FTS
Migrate prose and doc-comments to describe the Postgres FTS backend that replaced Typesense: README architecture diagram (3 boxes, Postgres now "events + FTS search"), ARCHITECTURE.md buzz-search section rewritten to the real API (SearchService::new(pool), search(&SearchQuery), ChannelScope) and the search_tsv generated-column mechanism (CASE WHEN kind IN (1059,30300,30622) THEN NULL, idx_events_search_tsv GIN), CONTRIBUTING step-6, VISION, AGENTS, TESTING (both), and the chart README. Comment-only edits in desktop and test-client files; drop the dead reindex-kind0 Justfile recipe (its binary no longer exists). Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
This commit is contained in:
npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
co-authored by
Tyler Longwell
Tyler Longwell
parent
9d9cbe1172
commit
c258393ebb
@@ -42,7 +42,7 @@ crates/
|
||||
buzz-db # Postgres event store and data access layer
|
||||
buzz-auth # Authentication and authorization
|
||||
buzz-pubsub # Redis pub/sub fan-out, presence, typing indicators
|
||||
buzz-search # Typesense-backed full-text search
|
||||
buzz-search # Postgres FTS full-text search
|
||||
buzz-audit # Hash-chain audit log
|
||||
buzz-media # Blossom/S3 media storage
|
||||
# Agent surface
|
||||
@@ -130,7 +130,7 @@ to be duplicated:
|
||||
|
||||
- `POST /events` — submit any signed event (same path the WebSocket uses).
|
||||
- `POST /query` — Nostr REQ filters over HTTP. NIP-50 `search` filters
|
||||
are routed to `buzz-search` (Typesense-backed) automatically.
|
||||
are routed to `buzz-search` (Postgres FTS) automatically.
|
||||
- `POST /count` — Nostr COUNT filters over HTTP.
|
||||
|
||||
If you find yourself reaching for a new REST endpoint, first check whether
|
||||
|
||||
+36
-19
@@ -64,8 +64,8 @@ Buzz is a Rust monorepo, licensed Apache 2.0 under Block, Inc.
|
||||
(multi-node fan-out wired; local-echo dedup via AppState.local_event_ids).
|
||||
|
||||
┌──────────────┐
|
||||
│ Typesense │ ← buzz-search (bounded worker queue)
|
||||
│ (full-text │
|
||||
│ Postgres │ ← buzz-search (FTS over the search_tsv
|
||||
│ (full-text │ generated column + GIN index)
|
||||
│ search) │
|
||||
└──────────────┘
|
||||
```
|
||||
@@ -80,7 +80,7 @@ buzz-core (zero I/O — types, verification, filter matching, kind registry)
|
||||
├── buzz-db (Postgres: events, channels, tokens, workflows, audit)
|
||||
├── buzz-auth (NIP-42, NIP-98, API tokens, scopes, rate limiting)
|
||||
├── buzz-pubsub (Redis pub/sub, presence, typing indicators)
|
||||
├── buzz-search (Typesense: index, query, delete)
|
||||
├── buzz-search (Postgres FTS: query, delete)
|
||||
├── buzz-audit (hash-chain tamper-evident log)
|
||||
└── buzz-workflow (YAML-as-code automation engine)
|
||||
│
|
||||
@@ -462,21 +462,32 @@ EXPIRE buzz:typing:{channel_id} 60
|
||||
|
||||
---
|
||||
|
||||
### buzz-search — Typesense Integration
|
||||
### buzz-search — Postgres FTS Integration
|
||||
|
||||
Full-text search via Typesense. All HTTP calls use `reqwest` with `X-TYPESENSE-API-KEY`. In multi-community mode, indexed documents and every query filter include `community_id`; the shared Typesense collection is infrastructure, not a cross-community result space.
|
||||
|
||||
**Collection schema (7 fields):** `id`, `content`, `kind` (int32), `pubkey` (facet), `channel_id` (facet, optional), `created_at` (int64, default sort), `tags_flat` (string[]).
|
||||
Full-text search via Postgres FTS. Events are searchable through the
|
||||
`events.search_tsv` generated `tsvector` column (populated on insert, indexed
|
||||
by a GIN index) — there is no separate search service or out-of-band indexer.
|
||||
Privacy-sensitive kinds are excluded at the storage level (the `search_tsv`
|
||||
`CASE WHEN kind IN (...)` yields `NULL`, which never matches `@@`). In
|
||||
multi-community mode every query filter includes `community_id`, so the shared
|
||||
`events` table is infrastructure, not a cross-community result space; the relay
|
||||
re-authorizes every candidate hit before returning it.
|
||||
|
||||
**Key behaviors:**
|
||||
- `ensure_collection()` is idempotent: handles 409 race condition (another process created it between check and create).
|
||||
- Tag flattening uses `\x1f` (ASCII unit separator) to avoid ambiguity with tag values containing colons (e.g., URLs in `r` tags).
|
||||
- Upsert indexing: `POST /documents?action=upsert` (single), `POST /documents/import?action=upsert` (batch JSONL).
|
||||
- `delete_event()` validates event ID (64-char hex) before constructing the URL — prevents path injection.
|
||||
- `delete_event()` is idempotent: 404 treated as success.
|
||||
- Permission filtering is **caller's responsibility** — `buzz-search` provides the `filter_by` mechanism but does not enforce access policy.
|
||||
- `SearchService::new(pool)` wraps a `PgPool`; `search(&SearchQuery)` runs a
|
||||
parameterized FTS query against the `events.search_tsv` GIN index and returns
|
||||
`SearchResult` (candidate `SearchHit`s).
|
||||
- `ChannelScope` makes the channel constraint explicit (`Any` /
|
||||
`ChannelLessOnly` / `Channels` / `ChannelsOrChannelLess`), closing the
|
||||
ambiguity the old `Option<Vec<Uuid>> + bool` matrix could not express.
|
||||
- Every query carries `community_id`; the FTS predicate is BitmapAnd-ed with
|
||||
the community-leading btree filters so a query never crosses tenants.
|
||||
- Permission filtering is **caller's responsibility** — `buzz-search` returns
|
||||
candidate hits; the relay re-authorizes each one (channel membership, `#p`,
|
||||
owner gates) before delivering it.
|
||||
|
||||
**Does NOT:** enforce channel membership or access control. Does NOT store events in Postgres.
|
||||
**Does NOT:** enforce channel membership or access control. Does NOT write
|
||||
events (indexing is the `search_tsv` generated column on the `events` insert).
|
||||
|
||||
---
|
||||
|
||||
@@ -652,7 +663,7 @@ pub enum AuthState { Pending { challenge: String }, Authenticated(AuthContext),
|
||||
| GET | `/api/channels/{channel_id}/messages` | List channel messages |
|
||||
| GET | `/api/channels/{channel_id}/threads/{event_id}` | Get message thread |
|
||||
| GET/POST | `/api/channels/{channel_id}/workflows` | List/create channel workflows |
|
||||
| GET | `/api/search` | Full-text search via Typesense |
|
||||
| GET | `/api/search` | Full-text search via Postgres FTS |
|
||||
| GET | `/api/agents` | List agent accounts |
|
||||
| GET/PUT | `/api/presence` | Presence status (bulk) / set presence |
|
||||
| GET | `/api/feed` | Personalized feed (mentions/needs-action/activity) |
|
||||
@@ -831,9 +842,8 @@ Docker Compose provides the full local development stack. All services include h
|
||||
|
||||
| Service | Image | Port | Purpose |
|
||||
|---------|-------|------|---------|
|
||||
| Postgres | `postgres:17-alpine` | 5432 | Primary event store — events, channels, tokens, workflows, audit |
|
||||
| Postgres | `postgres:17-alpine` | 5432 | Primary event store — events, channels, tokens, workflows, audit; full-text search (`search_tsv` GIN) |
|
||||
| Redis | `redis:7-alpine` | 6379 | Pub/sub fan-out, presence (SET EX), typing (sorted sets) |
|
||||
| Typesense | `typesense/typesense:27.1` | 8108 | Full-text search index |
|
||||
| Adminer | `adminer` | 8082 | DB web UI (dev only) |
|
||||
| MinIO | `minio/minio` | 9000 (API), 9001 (console) | S3-compatible object storage (media) |
|
||||
| Prometheus | `prom/prometheus` | 9090 | Metrics collection |
|
||||
@@ -859,9 +869,16 @@ Docker Compose provides the full local development stack. All services include h
|
||||
| `buzz:presence:{pubkey_hex}` | String | 90s | Online/away status (single-community form; shared multi-community Redis must scope by community) |
|
||||
| `buzz:typing:{channel_uuid}` | Sorted Set | 60s | Active typers (5s window; shared multi-community Redis must scope by community) |
|
||||
|
||||
### Typesense Collection
|
||||
### Full-Text Search (Postgres FTS)
|
||||
|
||||
Single collection (`events` by default, configurable via `TYPESENSE_COLLECTION`). Schema today: `id`, `content`, `kind` (int32), `pubkey` (facet), `channel_id` (facet, optional), `created_at` (int64, default sort), `tags_flat` (string[]). Multi-community mode adds faceted `community_id` and either prefixes document IDs with community or makes all upsert/delete/refetch paths carry community context.
|
||||
Search runs over the `events.search_tsv` generated `tsvector` column on the
|
||||
`events` table (no separate collection or service). The column is populated on
|
||||
insert — `to_tsvector('simple', content)` — and excludes privacy-sensitive
|
||||
kinds via `CASE WHEN kind IN (1059, 30300, 30622) THEN NULL`, so those rows are
|
||||
storage-level unsearchable (a `NULL` tsvector never matches `@@`). A GIN index
|
||||
(`idx_events_search_tsv`) backs the `@@` probe; in multi-community mode the
|
||||
community-leading btree filters BitmapAnd with the GIN probe so every query is
|
||||
fenced to its `community_id`.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+9
-6
@@ -43,7 +43,7 @@ unacceptable behavior to **conduct@buzz-relay.org**.
|
||||
| Node.js | 24+ | Required for desktop app commands and `just ci` |
|
||||
| pnpm | 10+ | Required for desktop app commands and `just ci` |
|
||||
| Flutter | 3.41+ | Required for mobile app — install via [flutter.dev](https://docs.flutter.dev/get-started/install) |
|
||||
| Docker | 24+ | For Postgres, Redis, Typesense |
|
||||
| Docker | 24+ | For Postgres, Redis, MinIO |
|
||||
| `just` | latest | Task runner — `cargo install just` |
|
||||
| `lefthook` | latest | Optional; run `lefthook install` for local Git hooks |
|
||||
| `sqlx` migrations | workspace crate | `just migrate` applies embedded migrations from `migrations/` |
|
||||
@@ -85,9 +85,9 @@ cached thereafter). You can also run `just bootstrap` independently at any time;
|
||||
it is safe to re-run.
|
||||
|
||||
`just setup` then starts Docker services (Postgres on `:5432`, Redis on `:6379`,
|
||||
Typesense on `:8108`, Adminer on `:8082`, Keycloak on `:8180` for local
|
||||
OAuth/OIDC testing, MinIO on `:9000` for media storage, and Prometheus on
|
||||
`:9090` for metrics) and runs all pending database migrations.
|
||||
Adminer on `:8082`, Keycloak on `:8180` for local OAuth/OIDC testing, MinIO on
|
||||
`:9000` for media storage, and Prometheus on `:9090` for metrics) and runs all
|
||||
pending database migrations.
|
||||
|
||||
### Running the Relay and Desktop App
|
||||
|
||||
@@ -380,8 +380,11 @@ for team access setup, onboarding, and the full repo inventory. See
|
||||
handler in `buzz-db/src/` (e.g., `buzz-db/src/my_feature.rs`) with
|
||||
the appropriate `INSERT` and `SELECT` queries.
|
||||
|
||||
6. **Index for search** (if applicable) — add the kind to the Typesense
|
||||
indexing logic in `buzz-search/src/index.rs`.
|
||||
6. **Index for search** (if applicable) — Postgres FTS indexes persisted
|
||||
events automatically via the `events.search_tsv` generated column. To
|
||||
exclude a privacy-sensitive kind from search, add it to the `CASE WHEN
|
||||
kind IN (...)` exclusion in the `search_tsv` definition (see the initial
|
||||
schema migration) rather than wiring a separate indexer.
|
||||
|
||||
7. **Audit** — the audit log captures all events automatically; no changes
|
||||
needed unless you need custom audit metadata.
|
||||
|
||||
@@ -427,13 +427,6 @@ migrate: _ensure-migrations
|
||||
|
||||
# ─── Utilities ────────────────────────────────────────────────────────────────
|
||||
|
||||
# Rebuild Typesense docs for all kind:0 (user profile) events.
|
||||
# Required once after deploying the indexer change that flattens kind:0 content
|
||||
# for searchability; new/updated profiles are indexed correctly automatically.
|
||||
# Safe to run repeatedly — Typesense upserts.
|
||||
reindex-kind0:
|
||||
cargo run --release -p buzz-relay --bin buzz-reindex-kind0
|
||||
|
||||
# Remove build artifacts
|
||||
clean:
|
||||
cargo clean
|
||||
|
||||
@@ -76,7 +76,7 @@ Yes, it's another AI-adjacent developer tool. We're sorry. The difference is wha
|
||||
|
||||
## Why Buzz is better
|
||||
|
||||
One community. One identity model. One event log. Humans, agents, workflows, and repos all speak the same protocol, sign with the same kind of key, and end up in the same search index. In the default self-hosted deployment, one relay hosts one community; in a hosted multi-tenant deployment, each community keeps that same semantic boundary even when the backend shares Postgres, Redis, Typesense, and object storage.
|
||||
One community. One identity model. One event log. Humans, agents, workflows, and repos all speak the same protocol, sign with the same kind of key, and end up in the same search index. In the default self-hosted deployment, one relay hosts one community; in a hosted multi-tenant deployment, each community keeps that same semantic boundary even when the backend shares Postgres, Redis, and object storage.
|
||||
|
||||
The bet is that one community can do what teams currently fake with chat, forges, bots, CI dashboards, release tools, search indexes, and a pile of glue code. Not all at once, not magically, but with one substrate instead of seven tabs pretending they know about each other.
|
||||
|
||||
@@ -153,12 +153,13 @@ For agents, set `BUZZ_PRIVATE_KEY` and use [`buzz-cli`](crates/buzz-cli) — JSO
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ buzz-relay │
|
||||
│ NIP-01 · NIP-42 auth · channel/DM/media/workflow/git REST · audit log │
|
||||
└───┬──────────────────┬──────────────────┬──────────────────┬────────────┘
|
||||
│ │ │ │
|
||||
┌──▼───────┐ ┌─────▼─────┐ ┌───────▼────┐ ┌────────▼────┐
|
||||
│ Postgres │ │ Redis │ │ Typesense │ │ S3/MinIO │
|
||||
│ (events) │ │ (pub/sub) │ │ (search) │ │ (Blossom) │
|
||||
└──────────┘ └───────────┘ └────────────┘ └─────────────┘
|
||||
└───┬──────────────────────────┬──────────────────────────┬──────────────┘
|
||||
│ │ │
|
||||
┌──▼───────────┐ ┌──────▼──────┐ ┌───────▼─────┐
|
||||
│ Postgres │ │ Redis │ │ S3/MinIO │
|
||||
│ (events + │ │ (pub/sub) │ │ (Blossom) │
|
||||
│ FTS search) │ └─────────────┘ └─────────────┘
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
A Rust workspace of focused crates. Single source of truth: the relay. See [ARCHITECTURE.md](ARCHITECTURE.md) for the full breakdown.
|
||||
@@ -168,7 +169,7 @@ A Rust workspace of focused crates. Single source of truth: the relay. See [ARCH
|
||||
|
||||
**Core protocol** — `buzz-core` (zero-I/O types, NIP-01 filters, Schnorr verify) · `buzz-relay` (Axum WS + REST)
|
||||
|
||||
**Services** — `buzz-db` (Postgres) · `buzz-auth` (NIP-42/98 Schnorr auth, rate limiting) · `buzz-pubsub` (Redis, presence, typing) · `buzz-search` (Typesense) · `buzz-audit` (hash-chain log). Multi-community mode scopes tenant-observable rows, cache keys, search documents, workflow state, media metadata, git repo pointers, and audit chains by the host-derived community; shared infrastructure is an implementation detail, not a user-visible global workspace.
|
||||
**Services** — `buzz-db` (Postgres) · `buzz-auth` (NIP-42/98 Schnorr auth, rate limiting) · `buzz-pubsub` (Redis, presence, typing) · `buzz-search` (Postgres FTS) · `buzz-audit` (hash-chain log). Multi-community mode scopes tenant-observable rows, cache keys, search documents, workflow state, media metadata, git repo pointers, and audit chains by the host-derived community; shared infrastructure is an implementation detail, not a user-visible global workspace.
|
||||
|
||||
**Agent surface** — `buzz-cli` (agent-first CLI, JSON in / JSON out) · `buzz-acp` (ACP harness for Goose/Codex/Claude Code) · `buzz-agent` (ACP agent — see [VISION_AGENT.md](VISION_AGENT.md)) · `buzz-dev-mcp` (shell + file-edit tools) · `buzz-workflow` (YAML automation) · `buzz-persona` (agent persona packs)
|
||||
|
||||
|
||||
+2
-3
@@ -34,8 +34,8 @@ just setup # start Docker services, run migrations
|
||||
```
|
||||
|
||||
> **Already running Buzz Desktop?** Desktop uses the same Docker container
|
||||
> names (`buzz-postgres`, `buzz-redis`, `buzz-typesense`) and the same
|
||||
> default ports (`:5432`, `:6379`, `:8108`). `just setup` will reuse those
|
||||
> names (`buzz-postgres`, `buzz-redis`) and the same
|
||||
> default ports (`:5432`, `:6379`). `just setup` will reuse those
|
||||
> services, so **your test relay writes into Desktop's database**. That's
|
||||
> fine for read/write smoke tests, but: `just reset` wipes Desktop's data
|
||||
> along with yours. If you need isolation, stop Desktop first or run the
|
||||
@@ -267,7 +267,6 @@ out of the box with `just setup` or `just relay`. Common overrides:
|
||||
| `RELAY_URL` | `ws://localhost:3000` | Advertised in NIP-11 / NIP-42 challenges. **Note: no `BUZZ_` prefix.** |
|
||||
| `DATABASE_URL` | `postgres://buzz:buzz_dev@localhost:5432/buzz` | |
|
||||
| `REDIS_URL` | `redis://localhost:6379` | |
|
||||
| `TYPESENSE_URL` | `http://localhost:8108` | |
|
||||
| `BUZZ_REQUIRE_AUTH_TOKEN` | `false` | When true, REST requires NIP-98 (no `X-Pubkey` fallback) |
|
||||
| `BUZZ_REQUIRE_RELAY_MEMBERSHIP` | `false` | When true, only pubkeys in `relay_members` can connect |
|
||||
| `BUZZ_AUTO_MIGRATE` | `false` | Opt in with `true`/`1`/`yes`/`on` to run embedded SQLx migrations on relay startup |
|
||||
|
||||
@@ -192,7 +192,7 @@ Not afterthoughts — ship blockers:
|
||||
| Throughput | ~600K events/day (~7/sec avg) |
|
||||
| Event store | Postgres 17, partitioned monthly |
|
||||
| Fan-out | Redis pub/sub, <50ms p99 |
|
||||
| Search | Typesense, permission-aware, full-text |
|
||||
| Search | Postgres FTS, permission-aware, full-text |
|
||||
| Audit | Hash-chain audit log, tamper-evident |
|
||||
| Accessibility | WCAG 2.1 AA minimum |
|
||||
|
||||
|
||||
@@ -14,7 +14,6 @@ Docker services running and healthy:
|
||||
docker compose ps
|
||||
# buzz-postgres healthy
|
||||
# buzz-redis healthy
|
||||
# buzz-typesense healthy
|
||||
```
|
||||
|
||||
If not running: `just setup` from the repo root.
|
||||
|
||||
@@ -1700,8 +1700,8 @@ mod search_fts {
|
||||
|
||||
/// Obligation: every search `filter` includes `community_id`; same
|
||||
/// channel/token in A and B return only same-community hits; deleting in A
|
||||
/// does not delete the B document. Postgres FTS (search_tsv/GIN), not
|
||||
/// Typesense.
|
||||
/// does not delete the B document. Enforced in Postgres FTS
|
||||
/// (search_tsv/GIN), not a separate search engine.
|
||||
///
|
||||
/// Shape (designed so the wire-observable property is a *single*
|
||||
/// per-community row whose content is the community's own):
|
||||
|
||||
@@ -963,11 +963,9 @@ async fn test_nip10_thread_reply_not_in_top_level() {
|
||||
}
|
||||
|
||||
/// Send a kind:1059 gift wrap AND a kind:9 message with the same unique content.
|
||||
/// Query Typesense directly to prove the gift wrap was NOT indexed while the
|
||||
/// kind:9 message WAS. This bypasses all relay-level filtering (channel_id, #p)
|
||||
/// and tests the actual indexing skip in dispatch_persistent_event.
|
||||
///
|
||||
/// Requires TYPESENSE_URL and TYPESENSE_API_KEY env vars (defaults to dev values).
|
||||
/// Use the relay NIP-50 search to prove the gift wrap was NOT indexed while
|
||||
/// the kind:9 message WAS, exercising the storage-level exclusion in the
|
||||
/// `events.search_tsv` generated column.
|
||||
#[tokio::test]
|
||||
#[ignore]
|
||||
async fn test_nip17_gift_wrap_not_searchable() {
|
||||
@@ -1069,7 +1067,7 @@ async fn test_nip50_search_relevance_order() {
|
||||
send_rest_message(&keys, &channel, &msg2).await;
|
||||
send_rest_message(&keys, &channel, &msg3).await;
|
||||
|
||||
// Wait for Typesense indexing.
|
||||
// Wait for FTS indexing.
|
||||
tokio::time::sleep(Duration::from_secs(3)).await;
|
||||
|
||||
let mut client = BuzzTestClient::connect(&url, &keys).await.expect("connect");
|
||||
@@ -1674,7 +1672,7 @@ async fn test_nipdv_search_rejects_third_party() {
|
||||
let snapshot_id = snapshot.id.to_hex();
|
||||
client_a.disconnect().await.expect("disconnect A");
|
||||
|
||||
// Give Typesense a beat (it must NOT have indexed the snapshot).
|
||||
// Give FTS a beat (it must NOT have indexed the snapshot).
|
||||
tokio::time::sleep(Duration::from_secs(3)).await;
|
||||
|
||||
// B issues a kindless search filter carrying A's snapshot id — the bypass
|
||||
|
||||
@@ -405,7 +405,7 @@ async fn test_search_returns_indexed_event() {
|
||||
|
||||
ws_client.disconnect().await.ok();
|
||||
|
||||
// Wait for the async search index to catch up. Typesense indexing is
|
||||
// Wait for the async search index to catch up. FTS indexing is
|
||||
// fire-and-forget (tokio::spawn), so we need a generous delay.
|
||||
tokio::time::sleep(Duration::from_secs(2)).await;
|
||||
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
# Buzz Helm Chart
|
||||
|
||||
[Buzz](https://github.com/block/buzz) is a Nostr-based messaging platform for human–agent collaboration: a single relay binary serving WebSocket + REST + web UI, backed by PostgreSQL, Redis, Typesense, and S3-compatible object storage.
|
||||
[Buzz](https://github.com/block/buzz) is a Nostr-based messaging platform for human–agent collaboration: a single relay binary serving WebSocket + REST + web UI, backed by PostgreSQL, Redis, and S3-compatible object storage.
|
||||
|
||||
This chart has two operating profiles selected by values:
|
||||
|
||||
| Profile | When | What you get |
|
||||
|---|---|---|
|
||||
| **Production** (default) | Self-hosted multi-tenant, regulated, or GitOps-managed | External managed Postgres/Redis/Typesense/S3, `secrets.existingSecret:`, no chart-side autogen, HA-capable (`replicaCount ≥ 2`) |
|
||||
| **Quickstart** (eval) | Eval, single-node, one-off demo | In-cluster Postgres + Redis + MinIO + Typesense subcharts/Deployments, chart auto-generates relay + service secrets, single replica |
|
||||
| **Production** (default) | Self-hosted multi-tenant, regulated, or GitOps-managed | External managed Postgres/Redis/S3, `secrets.existingSecret:`, no chart-side autogen, HA-capable (`replicaCount ≥ 2`) |
|
||||
| **Quickstart** (eval) | Eval, single-node, one-off demo | In-cluster Postgres + Redis + MinIO subcharts/Deployments, chart auto-generates relay + service secrets, single replica |
|
||||
|
||||
## Quickstart (eval only)
|
||||
|
||||
@@ -18,15 +18,13 @@ helm install buzz oci://ghcr.io/block/buzz/charts/buzz --version 0.1.0 \
|
||||
--set postgresql.enabled=true \
|
||||
--set redis.enabled=true \
|
||||
--set minio.enabled=true \
|
||||
--set typesense.enabled=true \
|
||||
--set relayUrl=wss://buzz.example.com \
|
||||
--set ownerPubkey=<64-char-hex-pubkey>
|
||||
```
|
||||
|
||||
This brings up **everything in-cluster** — Postgres, Redis, MinIO (with its
|
||||
bucket created by a post-install Job), and Typesense — and composes the relay's
|
||||
`BUZZ_S3_ENDPOINT` / `TYPESENSE_URL` plus autogenerated credentials
|
||||
automatically. No external services required. The `quickstart=true` flag is an
|
||||
This brings up **everything in-cluster** — Postgres, Redis, and MinIO (with
|
||||
its bucket created by a post-install Job) — and composes the relay's
|
||||
`BUZZ_S3_ENDPOINT` plus autogenerated credentials automatically. No external services required. The `quickstart=true` flag is an
|
||||
intent marker surfaced in NOTES.txt; the bundled services are opted in via the
|
||||
four `*.enabled` flags above (see `ci/quickstart-values.yaml` for the exact set
|
||||
CI installs). Eval-only: every bundled service is a single replica with no HA.
|
||||
@@ -50,7 +48,7 @@ See:
|
||||
| `relayUrl` | Public `wss://` URL clients connect to | Always |
|
||||
| `ownerPubkey` | 64-char lowercase hex Nostr pubkey of the relay operator | When `relay.requireRelayMembership=true` (default) |
|
||||
| `secrets.existingSecret` | Name of pre-created Secret | Production / GitOps |
|
||||
| `externalPostgresql.url` / `externalRedis.url` / `typesense.url` / `s3.endpoint` | External service URLs | Production — when the matching bundled service is disabled (the default) |
|
||||
| `externalPostgresql.url` / `externalRedis.url` / `s3.endpoint` | External service URLs | Production — when the matching bundled service is disabled (the default) |
|
||||
|
||||
The chart fails at `helm install` / `helm template` time with a clear message if any of these are missing or malformed (see `templates/_validate.tpl`).
|
||||
|
||||
@@ -81,17 +79,17 @@ Save these. Losing any of them is data loss. See NOTES.txt printed by `helm inst
|
||||
|
||||
## Honest limitations (v1)
|
||||
|
||||
- **Bundled MinIO + Typesense are eval-only.** The quickstart profile runs an
|
||||
in-cluster MinIO and Typesense (single replica, no HA, `lookup`-autogenerated
|
||||
credentials) so the relay starts with zero external services. Production
|
||||
leaves `minio.enabled` / `typesense.enabled` off and points `s3.endpoint` +
|
||||
`typesense.url` (or `BUZZ_S3_*` / `TYPESENSE_URL` in `existingSecret`) at
|
||||
managed S3-compatible storage and Typesense. The bundled Deployments are not
|
||||
GitOps-safe and are not intended for production traffic.
|
||||
- **Bundled MinIO is eval-only.** The quickstart profile runs an in-cluster
|
||||
MinIO (single replica, no HA, `lookup`-autogenerated credentials) so the
|
||||
relay starts with zero external object storage. Production leaves
|
||||
`minio.enabled` off and points `s3.endpoint` (or `BUZZ_S3_*` in
|
||||
`existingSecret`) at managed S3-compatible storage. The bundled Deployment is
|
||||
not GitOps-safe and is not intended for production traffic.
|
||||
- **Minimal-mode is not yet supported.** The relay's `BUZZ_PUBSUB=local` /
|
||||
`BUZZ_SEARCH=pg` / filesystem media paths are upstream work in progress —
|
||||
even quickstart currently stands up real Redis, Typesense, and S3 rather than
|
||||
the relay's single-node fallbacks.
|
||||
filesystem media paths are upstream work in progress — even quickstart
|
||||
currently stands up real Redis and S3 rather than the relay's single-node
|
||||
fallbacks. (Full-text search already runs in Postgres, so no separate search
|
||||
service is provisioned.)
|
||||
- **OCI publish to GHCR + cosign signing** is a follow-up PR. For now, install the chart from source: `helm install buzz ./deploy/charts/buzz` after cloning the repo.
|
||||
|
||||
## Development
|
||||
|
||||
@@ -195,14 +195,14 @@ pub async fn search_users(
|
||||
}
|
||||
|
||||
// NIP-50 full-text search on kind:0 profiles. The relay's HTTP bridge
|
||||
// intercepts the `search` field on POST /query and routes to Typesense
|
||||
// intercepts the `search` field on POST /query and routes to Postgres FTS
|
||||
// (see `crates/buzz-relay/src/api/bridge.rs::handle_bridge_search`),
|
||||
// so we get indexed, server-side search instead of fetching every kind:0
|
||||
// and scanning client-side. The old path was capped at 2000 kind:0 events
|
||||
// by the relay's HTTP bridge limit, which silently hid users on busy relays.
|
||||
//
|
||||
// We over-fetch (limit=50, which the bridge accepts up to 500) and re-rank
|
||||
// locally because Typesense scores BM25 against the whole kind:0 JSON
|
||||
// locally because the relay scores FTS rank against the whole kind:0 JSON
|
||||
// `content` blob, where a hit in `display_name` is not weighted any higher
|
||||
// than a substring hit in `about`. Re-ranking ≤50 results client-side is
|
||||
// cheap and keeps display ordering predictable for autocomplete.
|
||||
|
||||
@@ -77,7 +77,7 @@ pub fn list_user_search_results(events: &[Event], limit: usize) -> SearchUsersRe
|
||||
/// Rank and truncate kind:0 events from a NIP-50 search response for the
|
||||
/// member-picker / DM-recipient autocomplete.
|
||||
///
|
||||
/// The relay returns results scored by Typesense BM25 against the whole kind:0
|
||||
/// The relay returns results scored by Postgres FTS rank against the whole kind:0
|
||||
/// JSON content blob. That ranking is fine as a recall mechanism but not as
|
||||
/// final ordering — a user whose `display_name` *is* the query should always
|
||||
/// rank above someone whose `about` happens to mention it. We re-rank with a
|
||||
@@ -87,7 +87,7 @@ pub fn list_user_search_results(events: &[Event], limit: usize) -> SearchUsersRe
|
||||
/// - field priority: display_name (or name) > nip05 > pubkey hex
|
||||
///
|
||||
/// `limit` clamps the output. Pubkey de-duplication keeps only the
|
||||
/// highest-scoring result per pubkey (Typesense should already return one doc
|
||||
/// highest-scoring result per pubkey (the relay should already return one doc
|
||||
/// per event id, and kind:0 is a NIP-16 replaceable event so stale rows are
|
||||
/// soft-deleted in the DB and filtered out before reaching us — this is
|
||||
/// defense in depth in case both somehow slip through).
|
||||
|
||||
@@ -57,7 +57,7 @@ export function useChannelFind({ channelId, messages }: UseChannelFindOptions) {
|
||||
return found;
|
||||
}, [messages, query]);
|
||||
|
||||
// Relay-backed search: full history via Typesense.
|
||||
// Relay-backed search: full history via Postgres FTS.
|
||||
const relaySearch = useSearchMessagesQuery(debouncedQuery, {
|
||||
channelId: channelId ?? undefined,
|
||||
enabled: isOpen && debouncedQuery.length >= MIN_QUERY_LENGTH,
|
||||
@@ -66,7 +66,7 @@ export function useChannelFind({ channelId, messages }: UseChannelFindOptions) {
|
||||
|
||||
// Merge: start with client-side matches, then supplement with relay hits
|
||||
// that are loaded in the timeline but were missed by exact substring match
|
||||
// (e.g. Typesense stemming). Only loaded messages are kept so the match
|
||||
// (e.g. FTS stemming). Only loaded messages are kept so the match
|
||||
// count stays accurate relative to what's visible on screen.
|
||||
const loadedMessageIds = React.useMemo(
|
||||
() => new Set(messages.map((m) => m.id)),
|
||||
|
||||
Reference in New Issue
Block a user