Files
roboco/docs/rag/tools/kb-tools.md
T
4fb0059556 fix: journals/learnings never reached the RAG corpus + git-readonly slug 404s (#339)
* fix(rag): per-index chunk floors — journals and learnings were never indexed

The global 200-char garbage floor (sized for code/doc chunks) discarded
every templated journal note and most distilled org-memory lessons,
silently: ingest returned success with zero chunks, so agent journals
and learnings were never retrievable via RAG. IndexConfig now carries a
per-type min_chunk_length (journals 40, learnings 80, others unchanged).

* fix(mcp): git-readonly tools default project_slug from the container env

Agents 404ed /api/git/status with 'Project not found: roboco' — the
tools made the LLM supply the slug and six doc examples taught a slug
that matches no registered project. The tools now fall back to the
ROBOCO_PROJECT_SLUG the orchestrator already injects, and the stale
examples are corrected.

* feat(rag): startup backfill re-ingests zero-chunk journals and learnings

Before the per-index chunk-floor fix, ingest() returned success with
chunk_count=0 for undersized content: every historical journal entry and
distilled learning below the (then-global) 200-char floor was durably
recorded in journal_entries but silently never got a chunks_journals /
chunks_learnings row, and no exception meant the existing dead-letter
(rag_index_failures) never saw it either.

Extends the startup reconcile (roboco/api/app.py _reconcile_rag_indexes)
with a new pass: backfill_unindexed_journals (roboco/services/
rag_index_failures.py) queries journal_entries for rows missing from each
vector table and re-ingests them through the same live code paths
(_reindex_journal_entry / record_learning).

Journals and learnings are backfilled independently since a LEARNING entry
can clear the (lower) JOURNALS floor while still failing the (higher)
LEARNINGS floor — a learning's doc_source is a content hash, not the entry
id, so presence there is checked by hashing each candidate the same way
LearningsIndexPlugin.record_learning does and batch-querying chunks_learnings
for those exact sources.

Bounded to 200 rows per pass per boot (converges over restarts on a larger
backlog) and best-effort per row (one failure never aborts the pass). Rows
still under the current floor are excluded by a length filter in the SELECT
so they are never retried forever, and private entries are excluded from
the JOURNALS pass exactly like the live indexing path.

* test(rag): scope backfill assertions to their own rows

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-07-08 16:03:00 +02:00

180 lines
3.7 KiB
Markdown

# Knowledge Base Tools
## Search and Query
| Tool | Purpose |
|------|---------|
| `roboco_kb_search` | Semantic search |
| `roboco_rag_query` | AI-synthesized answer |
| `roboco_ask_mentor` | Conversational help |
| `roboco_kb_stats` | Index statistics |
## Semantic Search
```python
roboco_kb_search(
query="rate limiting redis",
top_k=5,
project="roboco-api",
index_types=["code", "docs"]
)
```
## AI-Generated Answers
```python
roboco_rag_query(
query="How does authentication work?",
top_k=5
)
```
## Mentor (Conversational)
```python
response = roboco_ask_mentor(
question="How do I handle auth?",
domain="coding"
)
# Follow-up
roboco_ask_mentor(
question="What about refresh tokens?",
conversation_id=response["conversation_id"]
)
```
## Documentation Writing (Documenter, Cell PM)
```python
# Write/update documentation (auto-dedup via RAG)
roboco_docs_write({
"task_id": "task-uuid",
"filename": "api-endpoints.md",
"doc_type": "api", # api, qa, guide, readme, changelog, architecture, design
"title": "API Endpoints",
"content": "# API Endpoints\n\n..."
})
# List docs for a task
roboco_docs_list(task_id="task-uuid")
# Read a doc
roboco_docs_read(path="backend/api/endpoints.md")
```
**SMART DEDUPLICATION**: `roboco_docs_write` searches RAG for similar existing docs. If high-similarity match found, updates instead of creating duplicate.
## Bulk Indexing
```python
# Index code (PM, Developer)
roboco_kb_index_code(
sources=["src/**/*.py"],
project="roboco-api"
)
# Index docs (PM, Documenter) - for bulk/explicit indexing
# Note: roboco_docs_write() auto-indexes when writing
roboco_kb_index_docs(
sources=["docs/**/*.md"],
project="roboco-api"
)
```
## Error Tracking
```python
# Search for similar errors
roboco_search_error(
error_message="Redis connection timed out",
context="startup"
)
# Record solution
roboco_record_error_solution(
error_message="Redis connection timed out",
solution="Added retry with backoff",
worked=True
)
```
## Decision Tracking
```python
# Check for similar decisions
roboco_check_decision(topic="session storage")
# Record decision
roboco_record_decision(params={
topic: "Session storage",
decision: "Use Redis",
rationale: "Sub-ms reads"
})
```
## Standards & Validation
### Get Standards
```python
roboco_get_standards(domain="coding", language="python")
```
**Domains:** `coding`, `security`, `workflow`, `architecture`
### Validate Action (LLM-Based)
Uses LLM to check code/context against organizational standards.
```python
result = roboco_validate_action(
action_type="create_endpoint",
context="""
def create_user(email, password):
user = User(email=email, password=password)
db.add(user)
return user
"""
)
```
**Returns:**
```json
{
"allowed": false,
"violations": [
{
"rule_id": "SEC-001",
"rule_title": "Password Hashing",
"message": "Password stored in plaintext",
"severity": "error",
"suggestion": "Hash password with bcrypt before storage"
}
],
"warnings": [...],
"relevant_standards": [...]
}
```
**How it works:**
1. Searches KB for relevant standards based on `action_type`
2. Sends standards + context to LLM for analysis
3. Returns structured violations with fix suggestions
4. Falls back to heuristic matching if LLM unavailable
**Action types:** `create_endpoint`, `add_dependency`, `database_migration`, `auth_change`, `file_upload`, `external_api`
### Code Review
```python
roboco_review_code(
code="def handle(...):",
file_path="src/api/auth.py",
change_type="modify" # add, modify, delete
)
```
**Returns:** Score (0-100), comments by severity, approval status