External/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-08-18 06:50:31 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat: NIP-50 search, NIP-10 threads, NIP-17 DMs, Sprout DM discovery (#74)

Browse Source
This commit is contained in:
tlongwell-block
2026-03-16 11:15:06 -04:00
committed by GitHub
parent 10d8ce9747
commit 39e856fd85
13 changed files with 1826 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files
NOSTR.md
+24 -8
View File
@@ -55,7 +55,7 @@ mysql -u sprout -psprout_dev sprout -e \
| **Admin delete event (kind:9005)** | ✅ | Event author can always delete own. Otherwise owner/admin required. Target must be in same channel. |
| **Group deletion (kind:9008)** | ✅ | Owner only. |
| **Leave group (kind:9022)** | ✅ | Any member. Last-owner guard prevents orphaned groups. |
| **Group metadata (kind:39000)** | ✅ | Relay-signed; always `d`, `name`, `closed` tags; `about` only if description non-empty; `private` if applicable |
| **Group metadata (kind:39000)** | ✅ | Relay-signed; always `d`, `name`, `closed` tags; `about` only if description non-empty; `private` if applicable; `hidden` for DM channels |
| **Group admins (kind:39001)** | ✅ | Relay-signed; `d` tag + `p` tags with roles (`owner`, `admin`) |
| **Group members (kind:39002)** | ✅ | Relay-signed; `d` tag + `p` tags for all members |
| **Membership notifications** | ✅ | kind:44100 (added) / kind:44101 (removed); relay-signed, global scope |
@@ -64,6 +64,10 @@ mysql -u sprout -psprout_dev sprout -e \
| **NIP-42 authentication** | ✅ | Proactive challenge; optional pubkey allowlist |
| **NIP-11 relay info** | ✅ | `GET /` with `Accept: application/nostr+json` |
| **Blossom media** | ✅ | `PUT /media/upload` (BUD-02), `GET /media/{sha256}.{ext}` (BUD-01) |
| **NIP-50 search** | ✅ | One-shot search REQs: `{"search":"query","kinds":[9],"#h":["<uuid>"]}` → relevance-sorted results → EOSE. Not registered as persistent subscriptions. |
| **NIP-10 threads** | ✅ | WS-submitted replies with `["e","<root>","","reply"]` tags create `thread_metadata` atomically. Visible in REST thread queries. Unknown parents rejected. |
| **NIP-17 DMs (gift wrap)** | ✅ | kind:1059 accepted with ephemeral signing keys. Stored globally (channel_id=None). Delivered via `#p`-filtered subscriptions. Not indexed in search. |
| **DM discovery** | ✅ | DM creation emits kind:39000 (with `hidden` tag) + kind:44100 membership notifications. NIP-29 clients discover DMs via standard group discovery flow. |
| **Edits (kind:40003)** | ⚠️ | Works on the wire but Sprout-only — no standard NIP-29 client renders these |
| **Rich content (kind:40002)** | ⚠️ | Works on the wire but Sprout-only — no standard NIP-29 client renders these |
@@ -74,9 +78,7 @@ mysql -u sprout -psprout_dev sprout -e \
| **Create invite (kind:9009)** | ⚠️ | Accepted and stored, but side-effect handler is deferred (no-op with warning log) |
| **Join request (kind:9021)** | ⚠️ | Accepted and stored, but side-effect handler is deferred (no-op with warning log) |
| **Group roles (kind:39003)** | ❌ | Defined in kind registry but not emitted by the relay |
| **Threads** | ⚠️ | Threading is REST-only (`parent_event_id`); no WebSocket-native thread model |
| **NIP-50 search** | ❌ | Sprout uses Typesense; not exposed via NIP-50 |
| **DMs** | ❌ | NIP-04/NIP-44 not implemented |
| **DMs** | ⚠️ | NIP-17 gift wraps supported; NIP-04/NIP-44 not implemented. kind:10050 (DM relay list) deferred. |
### Pubkey Allowlist
@@ -102,7 +104,7 @@ All discovery events include a `d` tag set to the channel UUID (NIP-29 addressab
| Kind | Tags | Content |
|------|------|---------|
| **39000** | `d=<uuid>`, `name`, `closed` (always); `about` (if description non-empty); `private` (if applicable) | Group metadata. **Note:** `closed` is always emitted per NIP-29 convention (Sprout channels require explicit membership), but open channels are still readable/writable by non-members at runtime. The tag reflects the membership model, not access enforcement. |
| **39000** | `d=<uuid>`, `name`, `closed` (always); `about` (if description non-empty); `private` (if applicable); `hidden` (DM channels only) | Group metadata. **Note:** `closed` is always emitted per NIP-29 convention (Sprout channels require explicit membership), but open channels are still readable/writable by non-members at runtime. The tag reflects the membership model, not access enforcement. |
| **39001** | `d=<uuid>`, `p` tags with role label (`owner`, `admin`) | Admin list |
| **39002** | `d=<uuid>`, `p` tags for all members | Member list |
@@ -166,6 +168,19 @@ nak event -k 5 -c "reason" --tag "h=<channel-uuid>" --tag "e=<message-event-id>"
# Create a group
nak event -k 9007 --tag "name=my-channel" --tag "visibility=open" \
--auth --sec <privkey> ws://localhost:3000
# Search messages (NIP-50)
nak req -k 9 --tag "h=<channel-uuid>" --search "search query" -l 20 \
--auth --sec <privkey> ws://localhost:3000
# Reply to a message (NIP-10 threading)
nak event -k 9 -c "Reply text" --tag "h=<channel-uuid>" \
--tag "e=<parent-event-id>;;reply" \
--auth --sec <privkey> ws://localhost:3000
# Fetch gift-wrapped DMs (NIP-17)
nak req -k 1059 --tag "p=<your-hex-pubkey>" \
--auth --sec <privkey> ws://localhost:3000
```
### Tested Clients (Direct)
@@ -173,7 +188,8 @@ nak event -k 9007 --tag "name=my-channel" --tag "visibility=open" \
| Client | Platform | Evidence | Notes |
|--------|----------|:--------:|-------|
| **SproutTestClient** | Rust (repo) | Automated E2E | Full NIP-29 flow: discovery (39000/39001/39002), kind:9 send/receive, reactions, deletions, h-tag enforcement |
| **nak** | CLI | Manual (anecdotal) | Used during development; not automated in CI |
| **E2E nostr interop** | Rust (repo) | Automated E2E | NIP-50 search (3 tests), NIP-10 threads (3 tests), NIP-17 gift wraps (3 tests), DM discovery (1 test) |
| **nak** | CLI | Manual (verified) | kind:9 send/recv, NIP-50 search, NIP-10 thread replies, group discovery |
**Not verified in-repo** (anecdotal / expected based on NIP-29 support):
- **Chachi** (Web/Mobile) — NDK-based; NIP-29 native
@@ -254,8 +270,8 @@ curl -X POST http://localhost:4869/admin/guests \
| **Inbound deletions (kind:5)** | ❌ | Blocked by proxy policy; not yet implemented |
| **DMs (NIP-04/NIP-44)** | ❌ | Proxy only handles NIP-28 channel events |
| **User profiles (kind:0)** | ❌ | Profiles managed via REST API or kind:0 (direct path) |
| **NIP-10 reply threading** | ⚠️ | `#e` reply tags preserved but threading is REST-only in Sprout |
| **NIP-50 search** | ❌ | Sprout uses Typesense; not exposed via NIP-50 |
| **NIP-10 reply threading** | ⚠️ | Threading works on direct path; proxy preserves `#e` tags but does not translate thread metadata |
| **NIP-50 search** | ❌ | Available on direct path only (ws://relay:3000); not proxied |
| **File uploads (NIP-94/96)** | ❌ | Use Blossom on the relay directly (Path 1) |
| **Relay lists / Outbox (NIP-65)** | ❌ | Single-relay architecture |
crates/sprout-db/src/event.rs
+68 -10
View File
@@ -31,6 +31,9 @@ pub struct EventQuery {
pub limit: Option<i64>,
/// Number of events to skip (for pagination).
pub offset: Option<i64>,
/// Restrict to events with a `p` tag mentioning this hex pubkey.
/// Joins against `event_mentions` table (indexed).
pub p_tag_hex: Option<String>,
}
/// Insert a Nostr event. Rejects AUTH and ephemeral kinds.
@@ -94,21 +97,42 @@ pub async fn insert_event(
/// Uses `QueryBuilder` for dynamic filter composition — avoids string concatenation
/// while keeping all user values in bind parameters.
pub async fn query_events(pool: &MySqlPool, q: &EventQuery) -> Result<Vec<StoredEvent>> {
// kinds:[] means "match no kinds" — return empty immediately.
if q.kinds.as_deref().is_some_and(|k| k.is_empty()) {
return Ok(vec![]);
}
let limit_val = q.limit.unwrap_or(100).min(1000);
let offset_val = q.offset.unwrap_or(0);
let mut qb: QueryBuilder<sqlx::MySql> = QueryBuilder::new(
"SELECT id, pubkey, created_at, kind, tags, content, sig, received_at, channel_id \
FROM events WHERE deleted_at IS NULL",
);
let mut qb: QueryBuilder<sqlx::MySql> = if let Some(ref p_hex) = q.p_tag_hex {
// Join against event_mentions for #p-filtered queries (indexed).
let mut b = QueryBuilder::new(
"SELECT e.id, e.pubkey, e.created_at, e.kind, e.tags, e.content, \
e.sig, e.received_at, e.channel_id \
FROM events e \
INNER JOIN event_mentions m ON e.id = m.event_id \
WHERE e.deleted_at IS NULL AND m.pubkey_hex = ",
);
b.push_bind(p_hex.to_ascii_lowercase());
b
} else {
QueryBuilder::new(
"SELECT id, pubkey, created_at, kind, tags, content, sig, received_at, channel_id \
FROM events WHERE deleted_at IS NULL",
)
};
// Use unqualified column names when no join, qualified when joined.
let col_prefix = if q.p_tag_hex.is_some() { "e." } else { "" };
if let Some(ch) = q.channel_id {
qb.push(" AND channel_id = ")
qb.push(format!(" AND {col_prefix}channel_id = "))
.push_bind(ch.as_bytes().to_vec());
}
if let Some(ks) = q.kinds.as_deref().filter(|k| !k.is_empty()) {
qb.push(" AND kind IN (");
qb.push(format!(" AND {col_prefix}kind IN ("));
let mut sep = qb.separated(", ");
for k in ks {
sep.push_bind(*k);
@@ -117,16 +141,19 @@ pub async fn query_events(pool: &MySqlPool, q: &EventQuery) -> Result<Vec<Stored
}
if let Some(ref pk) = q.pubkey {
qb.push(" AND pubkey = ").push_bind(pk.clone());
qb.push(format!(" AND {col_prefix}pubkey = "))
.push_bind(pk.clone());
}
if let Some(s) = q.since {
qb.push(" AND created_at >= ").push_bind(s);
qb.push(format!(" AND {col_prefix}created_at >= "))
.push_bind(s);
}
if let Some(u) = q.until {
qb.push(" AND created_at <= ").push_bind(u);
qb.push(format!(" AND {col_prefix}created_at <= "))
.push_bind(u);
}
qb.push(" ORDER BY created_at DESC LIMIT ")
qb.push(format!(" ORDER BY {col_prefix}created_at DESC LIMIT "))
.push_bind(limit_val);
qb.push(" OFFSET ").push_bind(offset_val);
@@ -347,6 +374,37 @@ pub async fn get_event_by_id_including_deleted(
}
}
/// Batch-fetch non-deleted events by their raw 32-byte IDs.
///
/// Returns events in arbitrary order — callers reorder as needed.
/// Uses a single `WHERE id IN (...)` query regardless of input size.
pub async fn get_events_by_ids(pool: &MySqlPool, ids: &[&[u8]]) -> Result<Vec<StoredEvent>> {
if ids.is_empty() {
return Ok(vec![]);
}
debug_assert!(ids.len() <= 500, "batch fetch should be bounded by caller");
let mut qb: QueryBuilder<sqlx::MySql> = QueryBuilder::new(
"SELECT id, pubkey, created_at, kind, tags, content, sig, received_at, channel_id \
FROM events WHERE deleted_at IS NULL AND id IN (",
);
let mut sep = qb.separated(", ");
for id in ids {
sep.push_bind(id.to_vec());
}
qb.push(")");
let rows = qb.build().fetch_all(pool).await?;
let mut out = Vec::with_capacity(rows.len());
for row in rows {
if let Some(ev) = row_to_stored_event(row)? {
out.push(ev);
}
}
Ok(out)
}
/// Parameters for [`insert_event_with_thread_metadata`].
#[derive(Debug)]
pub struct ThreadMetadataParams<'a> {
crates/sprout-db/src/lib.rs
+7
View File
@@ -228,6 +228,13 @@ impl Db {
event::get_event_by_id_including_deleted(&self.pool, id_bytes).await
}
/// Batch-fetch non-deleted events by their raw ID bytes.
///
/// Returns events in arbitrary order — callers reorder as needed.
pub async fn get_events_by_ids(&self, ids: &[&[u8]]) -> Result<Vec<StoredEvent>> {
event::get_events_by_ids(&self.pool, ids).await
}
/// Atomically insert an event and its thread metadata in one transaction.
///
/// Prevents the race where a concurrent delete between separate insert calls
crates/sprout-relay/src/api/canvas.rs
+1 -4
View File
@@ -57,11 +57,8 @@ pub async fn get_canvas(
let q = EventQuery {
channel_id: Some(channel_id),
kinds: Some(vec![KIND_CANVAS as i32]),
pubkey: None,
since: None,
until: None,
limit: Some(1),
offset: None,
..Default::default()
};
let events = state
crates/sprout-relay/src/api/dms.rs
+47 -1
View File
@@ -16,7 +16,11 @@ use nostr::util::hex as nostr_hex;
use serde::Deserialize;
use uuid::Uuid;
use crate::handlers::side_effects::emit_system_message;
use sprout_core::kind::KIND_MEMBER_ADDED_NOTIFICATION;
use crate::handlers::side_effects::{
emit_group_discovery_events, emit_membership_notification, emit_system_message,
};
use crate::state::AppState;
use super::{api_error, extract_auth_context, internal_error};
@@ -126,6 +130,26 @@ pub async fn open_dm_handler(
{
tracing::warn!("Failed to emit system message: {e}");
}
// Emit NIP-29 group discovery events so Nostr clients can find this DM.
if let Err(e) = emit_group_discovery_events(&state, channel.id).await {
tracing::warn!(channel = %channel.id, "DM discovery emission failed: {e}");
}
// Notify each participant so their Nostr client learns about the new DM.
for participant in &all_bytes {
if let Err(e) = emit_membership_notification(
&state,
channel.id,
participant,
&self_bytes,
KIND_MEMBER_ADDED_NOTIFICATION,
)
.await
{
tracing::warn!("DM membership notification failed: {e}");
}
}
}
// Resolve participant display names.
@@ -236,6 +260,28 @@ pub async fn add_dm_member_handler(
.await
.map_err(|e| internal_error(&format!("db error: {e}")))?;
if was_created {
// Emit NIP-29 group discovery events for the new expanded DM.
if let Err(e) = emit_group_discovery_events(&state, new_channel.id).await {
tracing::warn!(channel = %new_channel.id, "DM discovery emission failed: {e}");
}
// Notify each participant about the new DM.
for participant_bytes in &all_bytes {
if let Err(e) = emit_membership_notification(
&state,
new_channel.id,
participant_bytes,
&self_bytes,
KIND_MEMBER_ADDED_NOTIFICATION,
)
.await
{
tracing::warn!("DM membership notification failed: {e}");
}
}
}
let participants = resolve_participants(&state, new_channel.id).await;
let status = if was_created {
crates/sprout-relay/src/handlers/event.rs
+257 -12
View File
@@ -2,6 +2,7 @@
use std::sync::Arc;
use chrono::Utc;
use hex;
use tracing::{debug, error, info, warn};
@@ -10,7 +11,7 @@ use sprout_audit::{AuditAction, NewAuditEntry};
use sprout_core::event::StoredEvent;
use sprout_core::kind::{
event_kind_u32, is_ephemeral, is_workflow_execution_kind, KIND_AUTH, KIND_CANVAS,
KIND_DELETION, KIND_FORUM_COMMENT, KIND_FORUM_POST, KIND_FORUM_VOTE,
KIND_DELETION, KIND_FORUM_COMMENT, KIND_FORUM_POST, KIND_FORUM_VOTE, KIND_GIFT_WRAP,
KIND_MEMBER_ADDED_NOTIFICATION, KIND_MEMBER_REMOVED_NOTIFICATION, KIND_PRESENCE_UPDATE,
KIND_STREAM_MESSAGE, KIND_STREAM_MESSAGE_BOOKMARKED, KIND_STREAM_MESSAGE_DIFF,
KIND_STREAM_MESSAGE_EDIT, KIND_STREAM_MESSAGE_PINNED, KIND_STREAM_MESSAGE_SCHEDULED,
@@ -69,13 +70,17 @@ pub(crate) async fn dispatch_persistent_event(
);
}
let search = Arc::clone(&state.search);
let stored_for_search = stored_event.clone();
tokio::spawn(async move {
if let Err(e) = search.index_event(&stored_for_search).await {
error!(event_id = %stored_for_search.event.id.to_hex(), "Search index failed: {e}");
}
});
// Skip search indexing for NIP-17 gift wraps — content is ciphertext,
// and indexing would leak #p tag metadata into the search index.
if kind_u32 != KIND_GIFT_WRAP {
let search = Arc::clone(&state.search);
let stored_for_search = stored_event.clone();
tokio::spawn(async move {
if let Err(e) = search.index_event(&stored_for_search).await {
error!(event_id = %stored_for_search.event.id.to_hex(), "Search index failed: {e}");
}
});
}
let audit = Arc::clone(&state.audit);
let audit_event_id = event_id_hex.clone();
@@ -106,7 +111,8 @@ pub(crate) async fn dispatch_persistent_event(
.iter()
.any(|t| t.as_slice().first().map(|s| s.as_str()) == Some("sprout:workflow"));
if !is_workflow_execution_kind(kind_u32) && !is_relay_workflow_msg {
if !is_workflow_execution_kind(kind_u32) && !is_relay_workflow_msg && kind_u32 != KIND_GIFT_WRAP
{
let workflow_engine = Arc::clone(&state.workflow_engine);
let workflow_event = stored_event.clone();
tokio::spawn(async move {
@@ -161,8 +167,11 @@ pub async fn handle_event(event: Event, conn: Arc<ConnectionState>, state: Arc<A
// Enforce that the event's pubkey matches the authenticated identity.
// Without this, a user authenticated as key A could submit events signed by key B.
// Exception: proxy:submit scope allows submitting events on behalf of shadow pubkeys.
if event.pubkey != auth_pubkey && !has_proxy_scope {
// Exceptions: proxy:submit scope, and NIP-17 gift wraps (kind:1059) which use
// ephemeral one-time signing keys by design. The relay still knows the submitter
// via NIP-42 auth for rate limiting and abuse prevention.
let is_gift_wrap = kind_u32 == KIND_GIFT_WRAP;
if event.pubkey != auth_pubkey && !has_proxy_scope && !is_gift_wrap {
conn.send(RelayMessage::ok(
&event_id_hex,
false,
@@ -288,6 +297,11 @@ pub async fn handle_event(event: Event, conn: Arc<ConnectionState>, state: Arc<A
return;
}
}
} else if is_gift_wrap {
// NIP-17 gift wraps are always global (channel_id = None).
// A client-supplied h-tag is ignored — storing gift wraps channel-scoped
// would let any channel subscriber read them, bypassing #p gating.
None
} else {
extract_channel_id(&event)
};
@@ -394,7 +408,35 @@ pub async fn handle_event(event: Event, conn: Arc<ConnectionState>, state: Arc<A
}
}
let (stored_event, was_inserted) = match state.db.insert_event(&event, channel_id).await {
// ── NIP-10 thread resolution for channel-scoped content kinds ─────────
// Resolve ancestry from e-tags before storage so thread_metadata is populated
// atomically with the event insert (prevents race with concurrent deletes).
let thread_meta = if requires_h_channel_scope(kind_u32) {
if let Some(ch_id) = channel_id {
match resolve_nip10_thread_meta(&event, ch_id, &state).await {
Ok(meta) => meta,
Err(msg) => {
conn.send(RelayMessage::ok(
&event_id_hex,
false,
&format!("invalid: {msg}"),
));
return;
}
}
} else {
None
}
} else {
None
};
let thread_params = thread_meta.as_ref().map(|m| m.as_params());
let (stored_event, was_inserted) = match state
.db
.insert_event_with_thread_metadata(&event, channel_id, thread_params)
.await
{
Ok(result) => result,
Err(sprout_db::DbError::AuthEventRejected) => {
conn.send(RelayMessage::ok(
@@ -714,6 +756,209 @@ fn requires_h_channel_scope(kind: u32) -> bool {
)
}
/// Owned thread metadata — bridges the async resolve step and the borrowed
/// [`sprout_db::event::ThreadMetadataParams`] expected by the DB layer.
struct ThreadMetadataOwned {
event_id: Vec<u8>,
event_created_at: chrono::DateTime<Utc>,
channel_id: uuid::Uuid,
parent_event_id: Vec<u8>,
parent_event_created_at: chrono::DateTime<Utc>,
root_event_id: Vec<u8>,
root_event_created_at: chrono::DateTime<Utc>,
depth: i32,
broadcast: bool,
}
impl ThreadMetadataOwned {
fn as_params(&self) -> sprout_db::event::ThreadMetadataParams<'_> {
sprout_db::event::ThreadMetadataParams {
event_id: &self.event_id,
event_created_at: self.event_created_at,
channel_id: self.channel_id,
parent_event_id: Some(&self.parent_event_id),
parent_event_created_at: Some(self.parent_event_created_at),
root_event_id: Some(&self.root_event_id),
root_event_created_at: Some(self.root_event_created_at),
depth: self.depth,
broadcast: self.broadcast,
}
}
}
/// Resolve NIP-10 thread ancestry from e-tags on a WebSocket-submitted event.
///
/// Returns:
/// `Ok(None)` — not a reply (no NIP-10 markers), use plain insert
/// `Ok(Some(meta))` — reply with resolved ancestry, use insert_event_with_thread_metadata
/// `Err(msg)` — validation failure, reject event with OK false
async fn resolve_nip10_thread_meta(
event: &nostr::Event,
channel_id: uuid::Uuid,
state: &AppState,
) -> Result<Option<ThreadMetadataOwned>, String> {
// Scan e-tags for NIP-10 positional markers.
let mut root_hex: Option<String> = None;
let mut reply_hex: Option<String> = None;
for tag in event.tags.iter() {
let parts = tag.as_slice();
if parts.len() >= 4 && parts[0] == "e" {
let hex_val = &parts[1];
let marker = &parts[3];
if hex_val.len() == 64 && hex_val.chars().all(|c| c.is_ascii_hexdigit()) {
match marker.as_str() {
"root" => root_hex = Some(hex_val.to_string()),
"reply" => reply_hex = Some(hex_val.to_string()),
_ => {}
}
}
}
}
// No NIP-10 markers → not a reply, proceed with plain insert.
if root_hex.is_none() && reply_hex.is_none() {
return Ok(None);
}
// NIP-10: "reply" marker is required to indicate a reply. "root"-only
// is a thread-context reference, not a reply — skip thread metadata.
let (root_hex, parent_hex) = match (root_hex, reply_hex) {
(Some(r), Some(p)) => (r, p), // nested reply: root + parent
(None, Some(p)) => (p.clone(), p), // direct reply: single "reply" = both
(Some(_), None) | (None, None) => return Ok(None), // not a reply
};
// Decode and look up parent event.
let parent_bytes =
hex::decode(&parent_hex).map_err(|_| "invalid parent event ID hex".to_string())?;
let parent_event = state
.db
.get_event_by_id(&parent_bytes)
.await
.map_err(|e| format!("db error looking up parent: {e}"))?
.ok_or_else(|| "reply parent not found".to_string())?;
// Verify parent belongs to the same channel.
match parent_event.channel_id {
Some(parent_ch) if parent_ch != channel_id => {
return Err("parent event belongs to a different channel".to_string());
}
None => return Err("parent event has no channel association".to_string()),
_ => {}
}
let parent_created =
chrono::DateTime::from_timestamp(parent_event.event.created_at.as_u64() as i64, 0)
.unwrap_or_else(Utc::now);
let client_root_bytes =
hex::decode(&root_hex).map_err(|_| "invalid root event ID hex".to_string())?;
let parent_meta = state
.db
.get_thread_metadata_by_event(&parent_bytes)
.await
.map_err(|e| format!("db error looking up thread metadata: {e}"))?;
let (final_root_bytes, root_created, depth) = match parent_meta {
Some(meta) => {
let effective_root = meta.root_event_id.unwrap_or_else(|| parent_bytes.clone());
// Reject if client-supplied root diverges from server-resolved root.
// This prevents stored thread_metadata from contradicting wire-visible e-tags.
if client_root_bytes != effective_root {
return Err("root tag does not match thread ancestry".to_string());
}
let root_ts = if let Ok(Some(root_ev)) = state.db.get_event_by_id(&effective_root).await
{
chrono::DateTime::from_timestamp(root_ev.event.created_at.as_u64() as i64, 0)
.unwrap_or(parent_created)
} else {
parent_created
};
let depth = meta.depth + 1;
if depth > 100 {
return Err("thread depth limit exceeded".to_string());
}
(effective_root, root_ts, depth)
}
None => {
// Parent has no thread metadata — either it's a top-level message (the root)
// or a legacy WS reply created before NIP-10 thread resolution was added.
// Fall back to parsing the parent's own e-tags to find its root.
//
// Check "root" marker first, then "reply" — Sprout's REST emitter uses a
// single ["e", <root>, "", "reply"] tag for direct replies (no separate
// "root" tag), so we must handle both forms.
let parent_root = parent_event
.event
.tags
.iter()
.find_map(|t| {
let parts = t.as_slice();
if parts.len() >= 4 && parts[0] == "e" && parts[3] == "root" {
hex::decode(&parts[1]).ok().filter(|b| b.len() == 32)
} else {
None
}
})
.or_else(|| {
// Direct-reply form: single "reply" tag = root reference
parent_event.event.tags.iter().find_map(|t| {
let parts = t.as_slice();
if parts.len() >= 4 && parts[0] == "e" && parts[3] == "reply" {
hex::decode(&parts[1]).ok().filter(|b| b.len() == 32)
} else {
None
}
})
})
.unwrap_or_else(|| parent_bytes.clone());
if client_root_bytes != parent_root {
return Err("root tag does not match thread ancestry".to_string());
}
// depth=1 if parent is root, depth=2 if parent is a legacy reply.
// depth=2 is an approximation for deeper legacy chains (no metadata to
// determine true depth), but correct for the common direct-reply case.
let depth = if parent_root == parent_bytes { 1 } else { 2 };
// Look up actual root event for its timestamp (don't use parent_created).
let root_created = if parent_root != parent_bytes {
if let Ok(Some(root_ev)) = state.db.get_event_by_id(&parent_root).await {
chrono::DateTime::from_timestamp(root_ev.event.created_at.as_u64() as i64, 0)
.unwrap_or(parent_created)
} else {
parent_created
}
} else {
parent_created
};
(parent_root, root_created, depth)
}
};
let broadcast = event.tags.iter().any(|t| {
let parts = t.as_slice();
parts.len() >= 2 && parts[0] == "broadcast" && parts[1] == "1"
});
let event_created_at = chrono::DateTime::from_timestamp(event.created_at.as_u64() as i64, 0)
.unwrap_or_else(Utc::now);
Ok(Some(ThreadMetadataOwned {
event_id: event.id.as_bytes().to_vec(),
event_created_at,
channel_id,
parent_event_id: parent_bytes,
parent_event_created_at: parent_created,
root_event_id: final_root_bytes,
root_event_created_at: root_created,
depth,
broadcast,
}))
}
#[cfg(test)]
mod tests {
use super::requires_h_channel_scope;
crates/sprout-relay/src/handlers/req.rs
+306 -30
View File
@@ -8,7 +8,9 @@ use tracing::{debug, warn};
use hex;
use nostr::Filter;
use sprout_core::filter::filters_match;
use sprout_core::kind::{KIND_MEMBER_ADDED_NOTIFICATION, KIND_MEMBER_REMOVED_NOTIFICATION};
use sprout_core::kind::{
KIND_GIFT_WRAP, KIND_MEMBER_ADDED_NOTIFICATION, KIND_MEMBER_REMOVED_NOTIFICATION,
};
use sprout_db::EventQuery;
use sprout_auth::Scope;
@@ -77,37 +79,51 @@ pub async fn handle_req(
let channel_id = extract_channel_id_from_filters(&filters);
// Enforce #p filter for membership notification subscriptions.
//
// ── NIP-50 search: intercept BEFORE #p gating ────────────────────────────
// Search filters are one-shot (not registered as persistent subscriptions).
// They never deliver gift wraps (not indexed) or membership notifications
// (global, no channel_id), so the #p gate below is irrelevant for them.
// Intercepting here lets clients send `{"search":"foo"}` without `kinds`.
let has_search = filters.iter().any(|f| f.search.is_some());
if has_search {
if filters.iter().any(|f| f.search.is_none()) {
conn.send(RelayMessage::closed(
&sub_id,
"error: mixed search and non-search filters not supported",
));
return;
}
handle_search_req(&sub_id, &filters, &accessible_channels, &conn, &state).await;
return;
}
// ── #p gating for globally-stored sensitive kinds ─────────────────────────
// Only applies to GLOBAL subscriptions (channel_id = None). Channel-scoped
// subscriptions can never receive globally-stored membership events — the
// fan_out() invariant in subscription.rs prevents it.
//
// We use the resolved subscription scope (channel_id) rather than per-filter
// #h presence to prevent mixed-filter bypass: a client could send
// [{#h:..., kinds:[44100]}, {authors:[...]}] which resolves to global scope
// but would skip the #p check if we only looked at per-filter #h tags.
// subscriptions can never receive globally-stored events — the fan_out()
// invariant in subscription.rs prevents it.
const P_GATED_KINDS: [u32; 3] = [
KIND_MEMBER_ADDED_NOTIFICATION,
KIND_MEMBER_REMOVED_NOTIFICATION,
KIND_GIFT_WRAP,
];
if channel_id.is_none() {
let authed_pubkey_hex = hex::encode(&pubkey_bytes);
let p_tag = nostr::SingleLetterTag::lowercase(nostr::Alphabet::P);
for filter in &filters {
let can_match_membership = filter.kinds.as_ref().is_none_or(|ks| {
ks.iter().any(|k| {
let ku = k.as_u16() as u32;
ku == KIND_MEMBER_ADDED_NOTIFICATION || ku == KIND_MEMBER_REMOVED_NOTIFICATION
})
let can_match_p_gated = filter.kinds.as_ref().is_none_or(|ks| {
ks.iter()
.any(|k| P_GATED_KINDS.contains(&(k.as_u16() as u32)))
});
if can_match_membership {
let p_tag = nostr::SingleLetterTag::lowercase(nostr::Alphabet::P);
// ALL #p values must match the authenticated pubkey — prevents
// a client from sneaking in a victim's pubkey alongside their own
// (e.g. #p:[my_key, victim_key]) to receive the victim's notifications.
if can_match_p_gated {
let has_matching_p = filter.generic_tags.get(&p_tag).is_some_and(|values| {
!values.is_empty() && values.iter().all(|v| *v == authed_pubkey_hex)
});
if !has_matching_p {
conn.send(RelayMessage::closed(
&sub_id,
"restricted: membership notifications require #p matching your pubkey",
"restricted: p-gated events require #p matching your pubkey",
));
return;
}
@@ -116,8 +132,6 @@ pub async fn handle_req(
}
// Check channel access BEFORE registering the subscription.
// Registering first would allow non-members to receive live fan-out events
// from private channels before the access check fires.
if let Some(ch_id) = channel_id {
if !accessible_channels.contains(&ch_id) {
conn.send(RelayMessage::closed(
@@ -147,7 +161,25 @@ pub async fn handle_req(
let mut total_sent: usize = 0;
for filter in &filters {
let params = filter_to_query_params(filter, channel_id);
// Use per-filter #h channel scope when available, falling back to the
// subscription-level channel_id. This prevents unrelated accessible-channel
// rows from consuming the LIMIT when filters target specific channels but
// the subscription is global (multiple distinct #h values across filters).
let per_filter_channel = {
let h = nostr::SingleLetterTag::lowercase(nostr::Alphabet::H);
filter
.generic_tags
.get(&h)
.and_then(|vs| {
if vs.len() == 1 {
vs.iter().next()?.parse::<uuid::Uuid>().ok()
} else {
None
}
})
.or(channel_id)
};
let params = filter_to_query_params(filter, per_filter_channel);
let filter_events = state.db.query_events(&params).await;
@@ -161,12 +193,10 @@ pub async fn handle_req(
};
for stored in &events {
if !seen_ids.insert(stored.event.id) {
continue;
}
// Apply full NIP-01 filter matching (handles fields not in the DB query).
if !filters_match(&filters, stored) {
// Per-filter NIP-01 matching — use the current filter only, not the
// full filter set. OR semantics across filters are handled by the outer
// loop (each filter gets its own DB query).
if !filters_match(std::slice::from_ref(filter), stored) {
continue;
}
@@ -176,6 +206,12 @@ pub async fn handle_req(
}
}
// Dedup AFTER acceptance — an event that fails filter A's constraints
// must remain eligible for filter B (NIP-01 OR semantics).
if !seen_ids.insert(stored.event.id) {
continue;
}
let msg = RelayMessage::event(&sub_id, &stored.event);
if !conn.send(msg) {
return;
@@ -194,6 +230,193 @@ pub async fn handle_req(
);
}
/// Handle a NIP-50 search REQ: query Typesense, fetch full events, deliver results, EOSE.
/// Search subscriptions are one-shot — no persistent subscription is registered.
/// Maximum Typesense pages to fetch per filter (prevents unbounded loops).
const MAX_SEARCH_PAGES: u32 = 10;
async fn handle_search_req(
sub_id: &str,
filters: &[Filter],
accessible_channels: &[uuid::Uuid],
conn: &ConnectionState,
state: &AppState,
) {
if accessible_channels.is_empty() {
conn.send(RelayMessage::eose(sub_id));
return;
}
let all_channels_filter = {
let ids: Vec<String> = accessible_channels
.iter()
.map(|id| id.to_string())
.collect();
format!("channel_id:=[{}]", ids.join(","))
};
let mut seen_ids: HashSet<nostr::EventId> = HashSet::new();
for filter in filters {
let search_text = match &filter.search {
Some(s) if !s.is_empty() => s.clone(),
_ => continue,
};
let limit = filter
.limit
.map(|l| (l as u32).min(MAX_HISTORICAL_LIMIT as u32))
.unwrap_or(MAX_HISTORICAL_LIMIT as u32);
if limit == 0 {
continue; // NIP-01: limit 0 means "no results from this filter"
}
// Push as many NIP-01 constraints into Typesense as possible so
// post-filtering is a correction step, not the primary filter.
//
// If the filter has a #h tag, push the specific channel(s) into Typesense
// instead of the full accessible set. This prevents cross-channel hits from
// consuming pagination budget and causing under-fetch.
// If the filter has #h, intersect with accessible channels. If all #h
// values are invalid/inaccessible, skip the filter entirely (match nothing)
// rather than broadening to all channels.
let h_tag = nostr::SingleLetterTag::lowercase(nostr::Alphabet::H);
let channel_scope =
if let Some(vs) = filter.generic_tags.get(&h_tag).filter(|vs| !vs.is_empty()) {
let valid: Vec<String> = vs
.iter()
.filter_map(|v| v.parse::<uuid::Uuid>().ok())
.filter(|id| accessible_channels.contains(id))
.map(|id| id.to_string())
.collect();
if valid.is_empty() {
continue; // all #h values invalid/inaccessible — skip filter
}
format!("channel_id:=[{}]", valid.join(","))
} else {
all_channels_filter.clone()
};
let mut filter_parts = vec![channel_scope];
if let Some(ref kinds) = filter.kinds {
if !kinds.is_empty() {
let kind_vals: Vec<String> = kinds.iter().map(|k| k.as_u16().to_string()).collect();
filter_parts.push(format!("kind:=[{}]", kind_vals.join(",")));
}
}
if let Some(ref authors) = filter.authors {
if !authors.is_empty() {
let author_vals: Vec<String> = authors.iter().map(|a| a.to_hex()).collect();
filter_parts.push(format!("pubkey:=[{}]", author_vals.join(",")));
}
}
if let Some(since) = filter.since {
filter_parts.push(format!("created_at:>={}", since.as_u64()));
}
if let Some(until) = filter.until {
filter_parts.push(format!("created_at:<={}", until.as_u64()));
}
let filter_by = filter_parts.join(" && ");
// Paginate: keep fetching pages until we've emitted `limit` results
// or exhausted the search result set. This ensures post-filtering
// doesn't silently reduce the result count below the requested limit.
let mut emitted: u32 = 0;
// Always fetch full pages (100) regardless of limit — post-filtering
// may discard many hits, so we need headroom to fill the requested limit.
let per_page: u32 = 100;
for page in 1..=MAX_SEARCH_PAGES {
if emitted >= limit {
break;
}
let search_query = sprout_search::SearchQuery {
q: search_text.clone(),
filter_by: Some(filter_by.clone()),
sort_by: None, // Typesense default = relevance (text_match score)
page,
per_page,
};
let search_result = match state.search.search(&search_query).await {
Ok(r) => r,
Err(e) => {
warn!(sub_id = %sub_id, "NIP-50 search failed: {e}");
break;
}
};
let page_empty = search_result.hits.is_empty();
let exhausted = (page as u64) * (per_page as u64) >= search_result.found;
let hit_ids: Vec<Vec<u8>> = search_result
.hits
.into_iter()
.filter(|h| h.channel_id.is_some())
.filter_map(|h| hex::decode(&h.event_id).ok())
.filter(|bytes| bytes.len() == 32)
.collect();
if !hit_ids.is_empty() {
let id_refs: Vec<&[u8]> = hit_ids.iter().map(|b| b.as_slice()).collect();
let events = match state.db.get_events_by_ids(&id_refs).await {
Ok(evs) => evs,
Err(e) => {
warn!(sub_id = %sub_id, "NIP-50 batch fetch failed: {e}");
break;
}
};
let event_map: std::collections::HashMap<[u8; 32], &sprout_core::StoredEvent> =
events
.iter()
.map(|ev| (ev.event.id.to_bytes(), ev))
.collect();
for hit_id in &hit_ids {
if emitted >= limit {
break;
}
let id_array: [u8; 32] = match hit_id.as_slice().try_into() {
Ok(a) => a,
Err(_) => continue,
};
let stored = match event_map.get(&id_array) {
Some(ev) => ev,
None => continue,
};
// NIP-01 post-filtering against THIS filter only (not OR of all filters).
if !filters_match(std::slice::from_ref(filter), stored) {
continue;
}
if let Some(ch_id) = stored.channel_id {
if !accessible_channels.contains(&ch_id) {
continue;
}
}
// Dedup AFTER acceptance — an event that fails filter A's constraints
// must remain eligible for filter B (NIP-01 OR semantics).
if !seen_ids.insert(stored.event.id) {
continue;
}
if !conn.send(RelayMessage::event(sub_id, &stored.event)) {
return;
}
emitted += 1;
}
}
if page_empty || exhausted {
break;
}
}
}
conn.send(RelayMessage::eose(sub_id));
}
/// Convert a single NIP-01 filter into an [`EventQuery`] for the database.
///
/// Each filter is queried independently so that per-filter `limit` and time
@@ -222,12 +445,37 @@ fn filter_to_query_params(filter: &Filter, channel_id: Option<uuid::Uuid>) -> Ev
.map(|l| (l as i64).min(MAX_HISTORICAL_LIMIT))
.unwrap_or(MAX_HISTORICAL_LIMIT);
// Push single-author filter into SQL (EventQuery.pubkey is Option<Vec<u8>>).
// Multi-author filters fall through to in-memory filters_match post-filtering.
let pubkey = filter.authors.as_ref().and_then(|authors| {
if authors.len() == 1 {
authors.iter().next().map(|pk| pk.serialize().to_vec())
} else {
None
}
});
// Push single-value #p tag into SQL via event_mentions join.
// This is critical for gift-wrap (kind:1059) and membership notification
// queries where >500 events for other recipients would otherwise push
// the caller's events past the LIMIT before post-filtering.
let p_tag = nostr::SingleLetterTag::lowercase(nostr::Alphabet::P);
let p_tag_hex = filter.generic_tags.get(&p_tag).and_then(|values| {
if values.len() == 1 {
values.iter().next().map(|v| v.to_string())
} else {
None
}
});
EventQuery {
channel_id,
kinds,
pubkey,
since,
until,
limit: Some(limit),
p_tag_hex,
..Default::default()
}
}
@@ -324,4 +572,32 @@ mod tests {
];
assert_eq!(extract_channel_id_from_filters(&filters), Some(channel_id));
}
#[test]
fn test_search_filter_detection() {
let search_filter = Filter::new().search("hello world");
let filters = [search_filter];
assert!(filters.iter().any(|f| f.search.is_some()));
}
#[test]
fn test_mixed_search_and_non_search_detection() {
let search_filter = Filter::new().search("hello");
let plain_filter = Filter::new();
let filters = [search_filter, plain_filter];
let has_search = filters.iter().any(|f| f.search.is_some());
let has_non_search = filters.iter().any(|f| f.search.is_none());
assert!(has_search && has_non_search, "should detect mixed filters");
}
#[test]
fn test_all_search_filters_not_mixed() {
let f1 = Filter::new().search("hello");
let f2 = Filter::new().search("world");
let filters = [f1, f2];
let has_search = filters.iter().any(|f| f.search.is_some());
let has_non_search = filters.iter().any(|f| f.search.is_none());
assert!(has_search);
assert!(!has_non_search, "all-search filters should not be mixed");
}
}
crates/sprout-relay/src/handlers/side_effects.rs
+5
View File
@@ -452,6 +452,11 @@ pub async fn emit_group_discovery_events(
if channel.visibility == "private" {
tags.push(Tag::parse(&["private"])?);
}
// NIP-29 hidden tag: hint to clients not to show DMs in public group lists.
// Not a security boundary — access control is handled by channel-scoped storage.
if channel.channel_type == "dm" {
tags.push(Tag::parse(&["hidden"])?);
}
// Sprout channels always require explicit membership
tags.push(Tag::parse(&["closed"])?);
emit_addressable_discovery_event(
crates/sprout-relay/src/nip11.rs
+1 -1
View File
@@ -56,7 +56,7 @@ impl RelayInfo {
description: "Sprout — private team communication relay".to_string(),
pubkey: None,
contact: None,
supported_nips: vec![1, 11, 25, 29, 42],
supported_nips: vec![1, 10, 11, 17, 25, 29, 42, 50],
software: "https://github.com/sprout-rs/sprout".to_string(),
version: env!("CARGO_PKG_VERSION").to_string(),
limitation: Some(RelayLimitation {
crates/sprout-search/src/query.rs
+32 -9
View File
@@ -84,6 +84,11 @@ pub struct SearchResult {
pub page: u32,
}
#[derive(Debug, Deserialize)]
struct TypesenseMultiSearchResponse {
results: Vec<TypesenseSearchResponse>,
}
#[derive(Debug, Deserialize)]
struct TypesenseSearchResponse {
found: u64,
@@ -116,12 +121,6 @@ pub async fn search(
collection_name: &str,
query: &SearchQuery,
) -> Result<SearchResult, SearchError> {
let url = format!(
"{}/collections/{}/documents/search",
base_url, collection_name
);
let params = query.to_query_params();
debug!(
q = %query.q,
page = query.page,
@@ -130,10 +129,29 @@ pub async fn search(
"Executing search"
);
// Typesense GET search has a 4000-char query string limit. When filter_by
// contains hundreds of channel UUIDs, the URL exceeds this. Use the
// /multi_search POST endpoint which accepts the same params in a JSON body.
let url = format!("{}/multi_search", base_url);
let mut search_params = serde_json::json!({
"collection": collection_name,
"q": query.q,
"query_by": "content",
"page": query.page,
"per_page": query.per_page,
});
if let Some(ref filter) = query.filter_by {
search_params["filter_by"] = serde_json::Value::String(filter.clone());
}
if let Some(ref sort) = query.sort_by {
search_params["sort_by"] = serde_json::Value::String(sort.clone());
}
let body = serde_json::json!({ "searches": [search_params] });
let resp = client
.get(&url)
.post(&url)
.header("X-TYPESENSE-API-KEY", api_key)
.query(&params)
.json(&body)
.send()
.await?;
@@ -143,7 +161,12 @@ pub async fn search(
return Err(SearchError::Api { status, body });
}
let ts_resp: TypesenseSearchResponse = resp.json().await?;
// multi_search wraps results: {"results": [<search_response>]}
let wrapper: TypesenseMultiSearchResponse = resp.json().await?;
let ts_resp = wrapper.results.into_iter().next().ok_or(SearchError::Api {
status: 200,
body: "empty multi_search results".into(),
})?;
parse_response(ts_resp)
}
crates/sprout-test-client/tests/e2e_nostr_interop.rs
+1065
View File
File diff suppressed because it is too large Load Diff
crates/sprout-test-client/tests/e2e_relay.rs
+10 -4
View File
@@ -473,7 +473,12 @@ async fn test_auth_event_kind_rejected() {
client.disconnect().await.expect("disconnect");
}
/// NIP-11 max_subscriptions (100) must be enforced; 101st REQ gets CLOSED.
/// NIP-11 max_subscriptions must be enforced; (limit+1)th REQ gets CLOSED.
///
/// The relay's MAX_SUBSCRIPTIONS is 1024. Opening 1024 subs in a test is slow,
/// so we open a smaller batch and verify the NIP-11 advertised limit matches
/// the actual enforcement constant. The full-limit test is covered by the
/// NIP-11 assertion below (which verifies the advertised value is 1024).
#[tokio::test]
#[ignore]
async fn test_subscription_limit_enforced() {
@@ -483,7 +488,8 @@ async fn test_subscription_limit_enforced() {
.await
.expect("connect");
for i in 0..100 {
// Open 1024 subscriptions (the relay's MAX_SUBSCRIPTIONS).
for i in 0..1024 {
let sid = format!("limit-sub-{i}");
let filter = Filter::new().kind(Kind::Custom(9));
client
@@ -574,8 +580,8 @@ async fn test_nip11_relay_info() {
let limitation = body.get("limitation").expect("Missing 'limitation' field");
assert_eq!(
limitation.get("max_subscriptions").and_then(|v| v.as_u64()),
Some(100),
"limitation.max_subscriptions must be 100"
Some(1024),
"limitation.max_subscriptions must be 1024"
);
assert!(
limitation
crates/sprout-test-client/tests/e2e_rest_api.rs
+3 -2
View File
@@ -469,8 +469,9 @@ async fn test_search_returns_indexed_event() {
ws_client.disconnect().await.ok();
// Wait briefly for the search index to catch up.
tokio::time::sleep(Duration::from_millis(500)).await;
// Wait for the async search index to catch up. Typesense indexing is
// fire-and-forget (tokio::spawn), so we need a generous delay.
tokio::time::sleep(Duration::from_secs(2)).await;
// The unique_token is UUID simple format (hex only) — safe to use directly in the URL.
let url = format!("{}/api/search?q={unique_token}", relay_http_url());