External/roboco
1
0
Fork 0
mirror of https://github.com/rennf93/roboco.git synced 2026-08-03 07:23:24 +02:00
Code Issues Packages Projects Releases Wiki Activity
Files
9cbfb5f0ddd56e8007dad2405819587311e27aa3
roboco/pyproject.toml
T
Renzo FGitHubRenn F
 303c2db289 Fix: rate limit real probe (#110) 
* fix(rate-limit): real provider liveness probe instead of time-based stub

The rate-limit recovery sweeper cleared a provider and resumed parked agents
purely on elapsed time — _do_probe was a stub that always returned True once
the retry_after window passed, so it never confirmed the provider had actually
stopped rate-limiting us. Under a sustained limit that resumes agents straight
into another 429, re-parking them: avoidable churn.

Make the probe real. _do_probe now issues a free, unmetered liveness call —
Anthropic GET /v1/models or Ollama GET /api/tags — and treats any non-429
response as the limit having lifted. A 429 keeps the provider parked; a
network error keeps it parked too (retry next sweep). When the provider can't
be probed (no API key, or an unrecognized provider), it falls back to the
prior time-expiry optimism rather than stranding agents. _probe_target keeps
URL/header resolution separate and testable, and _do_probe stays a
monkeypatchable boundary so the existing sweep tests are unaffected.

Also drop two acceptance-criteria-number labels from comments in this file.

* chore(rate-limit): clear merged gate debt in rate-limit tests + deps lint

The rate-limit PR landed with ruff violations the full gate flags but the
authors' runs missed: test_rate_limit_sweep.py was unformatted, and
test_rate_limit_tracker.py had unsorted/unused imports and magic-value
comparisons. Format the sweep test, drop the dead imports, and bind the
magic comparison values to locals. Also strip acceptance-criteria-number
labels from comments/docstrings across the three rate-limit test files
(leaving genuine acceptance_criteria=[...] test data untouched), and add
api/deps.py to the PLC0415 per-file-ignore — it is the DI wiring hub and
defers a couple of service imports to call time to avoid import cycles,
the same rationale already applied to api/routes, runtime, and services.

* fix(rate-limit): resolve redis type errors in RateLimitStateTracker

A cold mypy run (the gate's true state — prior passes were warm-cache only)
flagged four redis-typing errors in rate_limit_tracker.py that the merge
missed: three unused type:ignore[type-arg] on redis.Redis, and an
aclose() the bundled redis type stub doesn't expose.

Drop the now-unused ignores, and close the scan client via
'async with redis.from_url(...) as r:' instead of a finally-block
aclose(). The context manager closes the client on exit using the modern
redis.asyncio API — no deprecated close(), no stub-missing aclose(), no
suppression. Extend the test's redis mock to model the async
context-manager protocol so it returns itself on enter.

* test(prompter): pass route='main_pm' in the product main-PM routing test

Pre-existing master failure, unrelated to the rate-limit work. The test is
named ...product_routes_to_main_pm and asserts team=MAIN_PM, but called
confirm_live_draft without a route, so it got the 'board' default — which
assigns the Product Owner and yields team=BOARD by design (the board-review
path keeps the root at team=board until the CEO approves). The Main-PM path
is selected with route='main_pm', exactly as the sibling
...main_pm_route_assigns_main_pm test does. Add the missing kwarg so the test
verifies the path it names; behaviour under test is unchanged.

* Updated uv.lock

* refactor(complexity): bring all rank-C blocks under the xenon B ceiling

The full quality gate's xenon step (--max-absolute B --max-modules A
--max-average A) failed on eight rank-C blocks plus the extraction module
average — debt the rate-limit and token-analytics merges deferred. Reduce
each by extracting cohesive helpers, behaviour unchanged:

- orchestrator._probe_one_provider: split into _too_early_to_probe,
  _on_probe_success, _on_probe_failure, _parked_agents_for.
- rate_limit_tracker.list_rate_limited_providers: extract _read_rate_limited_entry
  and a _decode helper.
- trigger_filter.decide_spawn: extract _stale_trigger_decision (drops the
  PLR0911 suppression too).
- ollama_embedder (embed_query, _embed_batch_sync, aembed_query,
  _embed_batch_async): share _rl_backoff / _map_embed_error / _log_429 /
  _sleep_connect_retry / _asleep_connect_retry; remove a dead post-loop guard
  in aembed_query.
- mentor._synthesize_answer: extract _select_system_prompt and
  _answer_from_response.
- indexes/base.ask: extract the 429-retried LLM call into _ask_llm.
- extraction.__init__: extract _compile_patterns so the module average
  lands at rank A.

xenon now exits 0; rate-limit, optimal_brain, extraction, and events suites
all green.

* chore(deps): drop obsolete types-redis stub; honor redis 8.0 inline types

types-redis 4.6 (typed for redis 4.x) shadowed redis 8.0's own inline types,
which both masked real annotation mismatches in stream_bus.py and forced
awkward workarounds elsewhere. The stale stub is why the mypy gate only ever
passed warm-cached: a cold run under the wrong stub disagreed with the code.

Remove types-redis (and its orphaned transitive stubs) so mypy uses redis's
shipped types. That surfaces that xreadgroup/xclaim return bytes-keyed records
while _handle_message is annotated str — the code already decodes bytes
defensively, so this is an annotation gap, not a runtime bug. Make the types
honest: cast each result to its concrete shape and decode the stream name and
message id to str at the dispatch boundary via a _to_str helper.

mypy roboco/ is now clean cold (247 files) against redis's real types; events
suite green.

* Updated uv.lock

* fix(workspace): install the dev extra so agents can run make quality

Agent workspaces were set up with plain `uv sync`, which installs only the
project's default dependency group (pytest) — not the `dev` *extra* where the
gate tools live (ruff, mypy, xenon, radon, vulture, bandit, deptry). So an
agent's .venv had pytest but no linters, and `make quality` died immediately
on `ruff: command not found`. Agents literally could not lint, type-check, or
complexity-check their own work, which is how format/mypy/xenon debt merged
unseen. Sync the `dev` extra (`uv sync --extra dev`) so the workspace gets the
full toolchain the setup's own docstring already promised.

* fix(panel): rate-limit endpoint shape + websocket path

Two panel-facing breakages from the rate-limit rework:

- GET /api/system/rate-limits returned a raw list, but the panel store reads
  response.entries — so `r.entries is not iterable` crashed the banner sync on
  page load. Return the panel's contract: a { entries: [...] } envelope whose
  items are camelCase {provider, affectedAgents, hitAt, resumeAt,
  retryAfterSeconds}, derived from the raw Redis state (resumeAt = hitAt +
  retryAfter).
- The rate-limit websocket hook passed "/ws/system" while getWebSocketUrl()
  already supplies the "/ws" base, producing the doubled "/ws/ws/system" URL.
  Pass "/system" to match the agents/channels/notifications hooks.

Note: the backend /ws/system endpoint itself does not yet exist (the rework
shipped the panel hook only); the REST fix keeps the banner correct on load
and reconnect until that endpoint is built.

* test(workspace): assert uv sync installs the dev extra

Follow the workspace setup change: the dependency-install command is now
`uv sync --extra dev` so the agent workspace gets the lint/type/complexity
toolchain. Update the three assertions that pinned the old `uv sync`.

* feat(ws): add /ws/system stream and bridge rate-limit events to the panel

The rate-limit rework shipped the panel's websocket hook but no backend: there
was no /ws/system endpoint and nothing forwarded RATE_LIMIT_HIT/LIFTED to a
socket, so the banner got no live updates.

Build the missing half:
- ConnectionManager grows a system-wide connection set with connect_system /
  broadcast_system, and disconnect() now clears it.
- A /ws/system websocket endpoint (operator stream, no per-agent keying) with
  the same connected + ping/pong lifecycle as the other streams.
- websocket_bridge subscribes RATE_LIMIT_HIT/LIFTED and forwards each to
  broadcast_system tagged with the type the panel switches on. Both events
  ride the same StreamEventBus singleton, and the subscriptions register
  before start_listening(), so the consumer reads their streams.

Pairs with the panel hook now passing '/system' (getWebSocketUrl supplies the
'/ws' base). Covered by handler, manager, and endpoint-lifecycle tests.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-11 18:16:20 +02:00

452 lines
15 KiB
TOML
Raw Blame History

[project]
name = "roboco"
version = "0.1.0"
description = "AI Agents Company - A virtual organization of AI agents functioning as a software development workforce"
authors = [
{name = "Renzo Franceschini", email = "rennf93@users.noreply.github.com"}
]
readme = "README.md"
license = { text = "AGPL-3.0-or-later" }
requires-python = ">=3.10,<3.15"
dependencies = [
# Core
"pydantic",
"pydantic-settings",
# API
"fastapi",
"uvicorn[standard]",
"websockets",
# Database
"sqlalchemy[asyncio]",
"asyncpg", # PostgreSQL async driver
"alembic", # Migrations
# Cache/Queue
"redis",
"hiredis", # Redis performance
# RAG (piragi with PostgreSQL/pgvector backend)
"piragi[postgres]",
# AI/LLM
"anthropic",
"openai", # For embeddings
"tiktoken", # Token counting
"python-toon", # Token-efficient LLM serialization
# MCP (Model Context Protocol)
"mcp",
# Utilities
"httpx",
"python-multipart",
"python-jose[cryptography]", # JWT
"passlib[bcrypt]", # Password hashing
"tenacity", # Retry logic
"structlog", # Structured logging
# Streaming
"sse-starlette", # Server-Sent Events for A2A streaming
# Direct imports (promoted from transitive)
"cryptography", # utils/crypto.py — Fernet-encrypted project git tokens
# Build-time pin (not imported). Verified empirically: removing this
# entry causes uv to ignore the [tool.uv.sources.torch] CPU-only
# redirect and pull ~2.5GB of unused CUDA wheels. The entry has to
# appear in direct deps for the source override to bind. deptry's
# DEP002 ignore for `torch` below documents the same reality.
"torch",
"claude-agent-sdk>=0.2.94",
]
[project.optional-dependencies]
dev = [
# Testing
"pytest",
"pytest-asyncio",
"pytest-cov",
"pytest-xdist",
"httpx", # For TestClient
"factory-boy",
"faker",
# Code Quality
"ruff",
"mypy",
"vulture",
"bandit",
"pip-audit",
"radon",
"xenon",
"deptry",
"import-linter",
# Type Stubs
"types-passlib",
"types-python-jose",
# Development
"ipython",
"rich",
]
docs = [
"mkdocs",
"mkdocs-material",
"mkdocstrings[python]",
]
[project.scripts]
roboco = "roboco.cli:main"
roboco-bootstrap = "roboco.bootstrap:cli"
[project.urls]
Homepage = "https://github.com/rennf93/roboco"
Documentation = "https://roboco.dev/docs"
Repository = "https://github.com/rennf93/roboco"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["roboco"]
[tool.hatch.metadata]
allow-direct-references = true
# =============================================================================
# uv: pin torch to the CPU-only wheel index
# =============================================================================
# `piragi` transitively depends on torch. Our stack uses Ollama over HTTP for
# all embeddings/LLM, so torch is never actually loaded at runtime — but uv
# would otherwise resolve torch from PyPI, which bundles the full NVIDIA CUDA
# stack (~2.5GB across nvidia-cublas, cudnn, cufft, cusolver, nccl, etc.).
# The CPU-only index provides a ~200MB torch wheel and drops every CUDA dep.
[[tool.uv.index]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[tool.uv.sources]
torch = [{ index = "pytorch-cpu" }]
# =============================================================================
# uv: raise the floor on vulnerable transitive dependencies
# =============================================================================
# pyjwt is pulled in transitively (by mcp and msal). 2.12.1 carries four
# disclosed CVEs that are fixed in 2.13.0; constrain the floor so the resolver
# picks a patched release while leaving the direct dependents' own ranges intact.
[tool.uv]
constraint-dependencies = ["pyjwt>=2.13.0"]
# =============================================================================
# RUFF Configuration
# =============================================================================
[tool.ruff]
target-version = "py313"
src = ["src"]
exclude = ["vulture_whitelist.py", ".venv", "alembic"]
[tool.ruff.lint]
select = [
"E", # pycodestyle errors
"W", # pycodestyle warnings
"F", # Pyflakes
"I", # isort
"B", # flake8-bugbear
"C4", # flake8-comprehensions
"UP", # pyupgrade
"ARG", # flake8-unused-arguments
"SIM", # flake8-simplify
"TCH", # flake8-type-checking
"PTH", # flake8-use-pathlib
"PL", # Pylint
"RUF", # Ruff-specific
]
# Lazy imports to avoid circular dependencies
[tool.ruff.lint.per-file-ignores]
# MCP tool surfaces ARE the LLM-facing contract — every parameter must be
# top-level + typed so the SDK exposes it as a discrete schema field with
# enum constraints. Bundling into a dataclass would hide enum hints from
# the LLM and bring back invented values like nature='standard'.
"roboco/mcp/**/*.py" = ["PLC0415", "PLR0913"]
"roboco/services/*.py" = ["PLC0415"]
# Gateway methods are typed verb surfaces — agent-facing kwargs reflect the
# verb contract (session topic, channel, relationship type, etc.). Bundling
# into a dataclass hides the field-by-field schema the LLM needs at the
# tool layer; we accept the >5 kwarg signatures here for the same reason
# they're accepted in `roboco/mcp/**`.
"roboco/services/gateway/**/*.py" = ["PLC0415", "PLR0913"]
"roboco/api/routes/*.py" = ["PLC0415"]
# deps.py is the DI wiring hub; it defers a few service imports to call time
# to avoid import cycles with the modules it wires (same rationale as above).
"roboco/api/deps.py" = ["PLC0415"]
"roboco/runtime/*.py" = ["PLC0415"]
# The intake driver/entrypoint lazily import the heavy `claude-agent-sdk` (and
# uvicorn) so the modules import without those installed and don't pay the cost
# until a live container runs them — same rationale as the dirs above.
"roboco/agent_sdk/*.py" = ["PLC0415"]
# Lifecycle validators: foundation/_validate_lifecycle is imported from the
# bottom of policy/lifecycle.py at module-load time, so the validators must
# defer their inverse imports until call time to avoid a cycle.
"roboco/foundation/_validate_lifecycle.py" = ["PLC0415"]
# Test fixtures that reload modules to test env-var-at-import-time behavior
"tests/unit/mcp_servers/*.py" = ["PLC0415"]
# =============================================================================
# MyPy Configuration
# =============================================================================
[tool.mypy]
python_version = "3.13"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true
disallow_untyped_decorators = true
no_implicit_optional = true
strict_optional = true
warn_redundant_casts = true
warn_unused_ignores = true
warn_no_return = true
warn_unreachable = true
exclude = ["vulture_whitelist.py", ".venv", "alembic"]
plugins = ["pydantic.mypy"]
[[tool.mypy.overrides]]
module = [
"redis.*",
"anthropic.*",
"tiktoken.*",
"piragi.*",
"toon.*",
"sse_starlette.*",
"asyncpg.*",
"claude_agent_sdk.*", # third-party SDK, ships no type stubs
]
ignore_missing_imports = true
[tool.pydantic-mypy]
init_forbid_extra = true
init_typed = true
warn_required_dynamic_aliases = true
# =============================================================================
# Pytest Configuration
# =============================================================================
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
python_files = ["test_*.py"]
asyncio_default_fixture_loop_scope = "function"
addopts = "--cov=roboco --cov-report=term-missing"
[tool.coverage.run]
# Coverage core. Python 3.12+ supports sys.monitoring; the legacy `pytrace`
# core loses trace events across `await` boundaries on 3.13, under-counting
# every async route by ~30%. `sysmon` is the only core that gives accurate
# coverage for async route handlers.
core = "sysmon"
# Modules excluded from the coverage gate because they require live
# infrastructure (Ollama, real audio/video stacks, real workspaces,
# Docker daemon, Claude Code CLI) that the unit-coverage gate does not
# provision. They're covered by dedicated integration runs (smoke tests
# on the NAS) rather than the unit-coverage threshold.
omit = [
# RAG / proactive context — needs Ollama + real embedding model.
"roboco/services/proactive.py",
"roboco/services/optimal.py",
"roboco/services/optimal_brain/*",
# Audio transcription — needs Whisper-class model.
"roboco/services/transcription.py",
# Agent classes — instantiated by the orchestrator when spawning
# Docker containers running the Claude CLI. No usable unit-test
# surface; integration coverage is the smoke run.
"roboco/agents/*",
"roboco/agent_sdk/*",
# Container orchestration — drives Docker daemon, agent spawning,
# health/dispatch loops. Covered by smoke runs.
"roboco/runtime/orchestrator.py",
# MCP server entry points — modeled around the Claude CLI's STDIO
# MCP transport and only meaningful when running inside an agent
# container with the orchestrator reachable. Their _post() bridges
# ARE unit-tested separately (test_envelope_on_4xx, test_flow_server,
# test_do_server) — just not via direct module import here.
"roboco/mcp/*",
# WebSocket route — needs a running ASGI app + WS client.
"roboco/api/websocket.py",
# Event stream bus — wraps Redis Streams; tested via integration only.
"roboco/events/stream_bus.py",
# GitService — runs `git` subprocesses against per-agent workspaces.
# Unit-testable surface is < 5% of the module; covered via the
# _StubGit-based real-DB integration test + live smoke runs.
"roboco/services/git.py",
"roboco/services/workspace.py",
# Notification delivery — Redis Streams + push to MCP transport.
"roboco/services/notification_delivery.py",
# CLI entry point.
"roboco/cli.py",
# Auto-generated migrations.
"alembic/*",
]
markers = [
"asyncio: mark tests as async",
"slow: marks tests as slow",
"integration: marks tests as integration tests",
"unit: marks tests as unit tests",
]
# =============================================================================
# Vulture Configuration
# =============================================================================
[tool.vulture]
paths = ["roboco", "tests", "vulture_whitelist.py"]
exclude = ["**/conftest.py", ".venv", "alembic"]
min_confidence = 100
ignore_decorators = [
"@app.route",
"@app.get",
"@app.post",
"@app.put",
"@app.delete",
"@app.patch",
"@pytest.fixture",
"@pytest.mark.*",
"@validator",
"@root_validator",
"@field_validator",
"@model_validator",
]
ignore_names = [
"test_*",
"Test*",
"cleanup_*",
"mock_*",
"expected_*",
"exc_*",
"__*__",
"Config",
"Field",
]
sort_by_size = true
# =============================================================================
# Bandit Configuration
# =============================================================================
[tool.bandit]
skips = []
exclude_dirs = ["tests", ".venv", "vulture_whitelist.py", "alembic"]
severity = "medium"
# =============================================================================
# Radon Configuration
# =============================================================================
[tool.radon]
exclude = "tests/*,.venv/*,vulture_whitelist.py,alembic/*"
cc_min = "C"
mi_min = "A"
no_assert = false
show_closures = true
total_average = true
# =============================================================================
# Xenon Configuration
# =============================================================================
[tool.xenon]
max_absolute = "B"
max_modules = "A"
max_average = "A"
exclude = ["tests/*", ".venv/*", "vulture_whitelist.py", "alembic/*"]
ignore = []
# =============================================================================
# Deptry Configuration
# =============================================================================
[tool.deptry]
exclude = ["tests", ".venv", "vulture_whitelist.py", "alembic"]
extend_exclude = ["conftest.py", "setup.py"]
known_first_party = ["roboco"]
[tool.deptry.per_rule_ignores]
# DEP002: Dependencies not directly imported but used at runtime or via CLI
DEP002 = [
# Runtime server/driver dependencies (used by frameworks, not imported)
"uvicorn",
"websockets",
"asyncpg",
"hiredis",
"python-multipart",
# Database migrations (CLI tool)
"alembic",
# Auth libraries (used via passlib[bcrypt], python-jose[cryptography])
"python-jose",
"passlib",
# LLM utilities (embeddings/token counting)
"openai",
"tiktoken",
# Retry logic (used in production services)
"tenacity",
# Build-time pin, not imported — see the comment next to the torch
# entry in [project.dependencies] for the full rationale. Removing
# it from deps caused uv to pull the full CUDA stack.
"torch",
# Dev tools (CLI, not imported)
"pytest",
"pytest-asyncio",
"pytest-cov",
"pytest-xdist",
"factory-boy",
"faker",
"ruff",
"mypy",
"vulture",
"bandit",
"pip-audit",
"radon",
"xenon",
"deptry",
# CLI tool — invoked as `lint-imports` from the Makefile / quality gate
"import-linter",
"ipython",
"rich",
# Type stubs (used by mypy)
"types-passlib",
"types-python-jose",
# Documentation (CLI tools)
"mkdocs",
"mkdocs-material",
"mkdocstrings",
]
# DEP003: Starlette is a transitive dep of FastAPI, but BaseHTTPMiddleware is needed
DEP003 = ["starlette"]
[dependency-groups]
dev = [
"pytest>=9.0.3",
"pytest-asyncio>=1.3.0",
"pytest-cov>=7.1.0",
]
# =============================================================================
# Import Linter (architectural boundaries)
# =============================================================================
[tool.importlinter]
root_package = "roboco"
[[tool.importlinter.contracts]]
name = "Gateway layer must not import from API routes or MCP servers"
type = "forbidden"
source_modules = ["roboco.services.gateway"]
forbidden_modules = ["roboco.api.routes", "roboco.mcp"]
[[tool.importlinter.contracts]]
name = "Services must not import from API routes"
type = "forbidden"
source_modules = ["roboco.services"]
forbidden_modules = ["roboco.api.routes"]
# =============================================================================
# Roboco Commit Validator
# =============================================================================
[tool.roboco.commits]
subject_min_chars = 20
banned_words = ["wip", "tmp", "asdf", "oops", "fix", "update", "change", "stuff", "things"]
prefer_conventional = true
Reference in New Issue View Git Blame Copy Permalink