feat(rag): hybrid retrieval (vector + full-text), retire HyDE

Recall no longer depends on a per-query HyDE LLM call — it comes from the index.
Each chunks_<type> table gets a generated `tsv` column + GIN index (migration
031; the engine CREATE TABLE matches so fresh tables get it too).
VectorStore.hybrid_search fuses pgvector cosine with Postgres full-text in one
query: score = min(1, cosine + 0.3 * normalized_ts_rank). A vector-only match
keeps its cosine score (so decisions/reviewer thresholds are unchanged), a
keyword match adds a bounded boost (the recall win), and a keyword-only match
stays low. Empty/garbage query text degrades to pure vector.

HyDE is removed from the search hot path: _compute_query_embedding now embeds
the query directly, and _generate_hyde_passage / rag_use_hyde /
IndexConfig.use_hyde are deleted. So a search is one local embed + one indexed
SQL — no LLM round-trip. The raw query text is threaded through the
embed-once + concurrent fan-out (search_with_embedding(embedding, query_text)).

Verified live via a real pgvector round-trip: vector ranking + keyword boost +
[0,1] scores + empty-query fallback all correct. Adds wiring + fan-out unit
tests; the fusion SQL itself is verified live (needs pgvector, not gated in CI).
This commit is contained in:
Renn F
2026-06-15 06:37:27 +02:00
parent d7aee91b39
commit de82e06b3c
7 changed files with 264 additions and 96 deletions
@@ -0,0 +1,36 @@
"""The plugin search path routes to hybrid_search with the query text.
The full-text half of hybrid retrieval only works if the raw query text is
threaded all the way to ``VectorStore.hybrid_search``. Mocks resolve any
attribute, so this guards against the wiring silently regressing to pure-vector
``store.search`` or dropping ``query_text``.
"""
from __future__ import annotations
from unittest.mock import AsyncMock
import pytest
from roboco.services.optimal_brain.indexes.standards import StandardsIndexPlugin
@pytest.mark.asyncio
async def test_search_with_embedding_calls_hybrid_with_query_text(
monkeypatch: pytest.MonkeyPatch,
) -> None:
plugin = StandardsIndexPlugin()
store = AsyncMock()
store.hybrid_search = AsyncMock(return_value=[])
monkeypatch.setattr(plugin, "_store", store)
monkeypatch.setattr(plugin, "_initialized", True)
outcome = await plugin.search_with_embedding(
[0.1, 0.2, 0.3], "claim a task", top_k=4
)
assert outcome.success
store.hybrid_search.assert_awaited_once()
call = store.hybrid_search.call_args
assert call.args[0] == [0.1, 0.2, 0.3] # embedding
assert call.args[1] == "claim a task" # raw query text drives full-text half
store.search.assert_not_awaited() # not the pure-vector path