mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
* fix(gateway): push the branch before QA handoff so reviewers see the latest commits The commit content tool commits locally without pushing; only open_pr pushed the branch. On the first submission that was fine, but a fix committed while addressing needs_revision never reached origin (open_pr is skipped once the PR exists), so QA — which reviews the remote PR branch — re-reviewed the stale remote and re-failed the task on every cycle, a loop that never converged. i_am_done now pushes the task branch (idempotent; a no-op when nothing is unpushed) as part of the shared submit gate, covering both the normal and resume-from-verifying paths. A push failure blocks the handoff with a clear remediation rather than parking the task in awaiting_qa with commits that exist only in the developer's local workspace. * fix(orchestrator): don't reap a stale claim while the agent's container is alive The stale-claim reaper released any claimed/in_progress task whose last_heartbeat_at exceeded the TTL. The heartbeat only updates on certain gateway calls, so a developer deep in a long edit/test cycle outran the TTL and had its claim reaped mid-work — churning the task and risking a double spawn against the still-running container. The reaper now skips a task whose assignee still holds a live (ACTIVE) agent instance, trusting container liveness — the ground truth — over the heartbeat proxy. The check is defensive on missing fields so a heartbeat-only caller (and the reaper's existing unit tests) behave exactly as before. * fix(gateway): refuse to unblock a task while a dependency is unfinished A PM unblock on a dependency-gated task moved it straight to in_progress, overriding the dependency — letting a dependent proceed without its upstream's work (e.g. a frontend task built before its UX design lands). A dependency block is meant to clear on its own via _unblock_dependents the moment the upstream reaches a terminal state. unblock now refuses while any dependency is still non-terminal, returning a clear remediation that the block resolves automatically. Manual unblock remains available for genuine, non-dependency blockers. * fix(gateway): release a dependency-blocked claim to pending instead of looping A task that reached claimed/in_progress with an unfinished dependency was left in that state when the claim guard rejected, so the orchestrator's respawn loop kept reviving its assignee — which could make no progress — burning work for nothing. The claim guard now releases such a task back to pending. claimed -> blocked is not a legal transition, so pending — held by the dispatch dependency filter — is the lifecycle-correct resting state: the respawn loop ignores pending tasks, and _unblock_dependents re-dispatches it once the upstream reaches a terminal state. release_dependency_blocked_claim shares a _force_unclaim_to_pending core with unclaim_for_reaper so both record a truthful work-session abandon reason. * feat(security): warn at startup in header-trust mode + document the auth posture When ROBOCO_AGENT_AUTH_REQUIRED is not enabled the API accepts the X-Agent-Id / X-Agent-Role headers without a signed token, so any client that can reach it may act as any role (including 'ceo'). The API now logs a clear warning at startup in this mode, and the README gains a Security section documenting the auth posture and how to harden it. Acceptable only on a trusted private network — do not expose the API to untrusted networks. * fix(workspace): scope the refresh fetch to current + default branch ensure_workspace's healthy short-circuit ran an all-refs 'git fetch origin' to keep every origin/<branch> ref current. On a monorepo with many accumulated feature/* branches that exceeds the refresh timeout, the fetch silently fails, and the workspace keeps a stale base — so an agent builds on an out-of-date branch. The refresh now fetches only the workspace's current branch and the repo's default branch (resolved via origin/HEAD), with --no-tags --prune: it transfers near-nothing and can't time out. Readers need their own branch and the default; the integration branch is refreshed at branch-creation time. * fix(git): refresh a dependency-blocked task's branch off the current integration tip A cross-cell dependent (e.g. a frontend task waiting on the UX design) was branched off a base captured before its upstream merged into the integration branch, and the branch was never re-synced — so the agent built on a stale snapshot with none of the upstream's work. Two changes close the gap: - release_dependency_blocked_claim now clears branch_name, so the re-claim (after the dependency clears) re-runs branch creation. - create_branch, when the branch is already on disk with no commits of its own, resets it onto the freshly-pulled base — the dependent now builds on the current integration tip. A branch carrying real commits is left untouched, so no work is discarded; the cell->leaf cascade carries the upstream down to the dev branch automatically. * refactor(gateway): drop the sibling-sequence claim guard Sibling sequence no longer gates a claim. Cross-cell ordering is enforced by task dependencies — a cell task that depends on another is held until its upstream reaches a terminal state, a stronger, status-aware gate than the sequence-number check. That check was dormant in practice anyway: every fan-out child carries sequence 0, on which the guard short-circuited. `sequence` stays a sibling-ordering / dispatch-priority field (list_pending ordering and the panel). Removes sibling_sequence_guard and its _earlier_blocking_sibling helper, the now-unused skip_sequence parameter threaded through the claim verbs, and the sibling fetch that fed it. * feat(gateway): sort a cross-cell dependent after its upstream When the frontend cell task is wired to depend on its UX/UI sibling, set its sequence to the upstream's sequence + 1 so it sorts after the design it waits on — list_pending ordering and the panel now show UX ahead of the implementation it gates, in either delegation order. Adds TaskService.set_sequence (the sibling-ordering field is a service write; it carries no claim-gating semantics — dependencies gate claims). * feat(gateway): make the backend cell depend on UX too UX/UI design defines the screens and API contracts both implementation cells build against, so the backend cell — not just the frontend — waits on the UX/UI cell task in a product fan-out and sorts after it. Wires in either delegation order: a backend task delegated after UX gets the dependency directly; a UX task delegated after a still-pending backend sibling retro-wires it. Mirrors the existing frontend wiring (_depend_backend_on_ux and _depend_pending_backends_on_ux). Backend is held by the same dependency gate, so it costs no extra dispatch churn. * fix(websocket): forward notification acks instead of logging them incomplete The bridge handler serves both notification.sent and notification.acked, but acked events carry `agent_id` (the acking agent) rather than `recipient_id`, so every acknowledgement tripped the missing-field guard and logged "Incomplete notification event" instead of reaching the panel. Accept either field as the recipient. * feat(api): hint the full UUID when a truncated task id fails validation Agents copy the 8-character task prefix the system shows them (the commit prefix, task summaries) and send it as task_id, which fails UUID validation with an opaque "invalid length" 422 and wastes a call. The request-validation handler now detects a task_id UUID error and attaches a `remediate` hint telling the agent to retry with the full 36-character UUID from its task envelope. * fix(audit): record the blocked transition when a task is escalated Escalation sets a task to blocked by writing task.status directly, which bypassed the validated transition helper and so never emitted a task.blocked audit row — the lifecycle moved but the Auditor saw nothing. Extract the audit emit from the central transition helper into _emit_status_transition_audit and call it from the escalate path, capturing the prior status and outgoing owner before reassignment so the row is attributed correctly. * fix(docs): stop doubling the docs path so design specs index into RAG The documenter sometimes hands a doc path already rooted at docs/, and joining it onto DOCS_BASE_PATH (/app/docs) produced /app/docs/docs/..., so the file was never found and the spec never indexed — the frontend cell could not retrieve the UX design over RAG. Normalize the path before joining: trust an absolute path, otherwise strip a single redundant leading docs/ segment. * feat(security): let the control panel authenticate in secure mode With ROBOCO_AGENT_AUTH_REQUIRED=true every request must carry a valid HMAC token, which locked the human control panel out — it sends role headers but no token. nginx, the only trusted hop between the browser and the API, now injects the CEO token on /api and /ws, so the browser never holds the signing secret. The injected value is just the existing per-agent token issued for the CEO identity (issue_panel_token), so the token-verification path is unchanged. An empty value (dev/header-trust mode) renders to no header. `make panel-token` prints the value; set it as ROBOCO_PANEL_AGENT_TOKEN in .env before enabling secure mode. .env.example and the README Security section document the flow. * chore(compose): consolidate the two compose files into one docker-compose.yml and docker-compose.yaml had diverged: .yml — the file Docker actually uses — carried ROBOCO_PUBLIC_BASE_URL but was missing the /app/manifests bind-mount, while .yaml had the manifests mount but not the base URL. Merge the union into docker-compose.yml and delete the duplicate so there is one source of truth and no "multiple config files" warning. This activates the manifests mount in the deployed file: without it the orchestrator writes per-agent tool manifests to its ephemeral container fs, they never reach the host for the daemon to bind-mount, and agents fall back to all-verbs registration. Drop the stale .yaml reference from the config.py docstring, the labeler, and the CI path filters. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
384 lines
12 KiB
Python
384 lines
12 KiB
Python
"""
|
|
API Middleware
|
|
|
|
Request/response middleware for logging, error handling, and correlation IDs.
|
|
"""
|
|
|
|
import time
|
|
import uuid
|
|
from collections.abc import Callable, Sequence
|
|
from typing import Any, cast
|
|
|
|
import structlog
|
|
from fastapi import FastAPI, HTTPException, Request, Response
|
|
from fastapi import status as http_status
|
|
from fastapi.exceptions import RequestValidationError
|
|
from fastapi.responses import JSONResponse
|
|
from starlette.middleware.base import BaseHTTPMiddleware
|
|
|
|
from roboco.api.schemas.common import ErrorCode
|
|
from roboco.exceptions import (
|
|
AuthenticationError,
|
|
InvalidStateError,
|
|
NotFoundError,
|
|
PermissionDeniedError,
|
|
RobocoError,
|
|
ValidationError,
|
|
)
|
|
from roboco.services.base import (
|
|
ConflictError as ServiceConflictError,
|
|
)
|
|
from roboco.services.base import (
|
|
NotFoundError as ServiceNotFoundError,
|
|
)
|
|
from roboco.services.base import (
|
|
ServiceError,
|
|
ServiceUnavailableError,
|
|
)
|
|
from roboco.services.base import (
|
|
UnauthorizedError as ServiceUnauthorizedError,
|
|
)
|
|
from roboco.services.base import (
|
|
ValidationError as ServiceValidationError,
|
|
)
|
|
|
|
logger = structlog.get_logger()
|
|
|
|
|
|
# =============================================================================
|
|
# CORRELATION ID MIDDLEWARE
|
|
# =============================================================================
|
|
|
|
|
|
class CorrelationIdMiddleware(BaseHTTPMiddleware):
|
|
"""
|
|
Adds a correlation ID to each request for tracing.
|
|
|
|
The correlation ID is:
|
|
- Extracted from X-Correlation-ID header if present
|
|
- Generated if not present
|
|
- Added to response headers
|
|
- Bound to the logger context
|
|
"""
|
|
|
|
async def dispatch(self, request: Request, call_next: Callable) -> Response:
|
|
# Get or generate correlation ID
|
|
correlation_id = request.headers.get("X-Correlation-ID")
|
|
if not correlation_id:
|
|
correlation_id = str(uuid.uuid4())
|
|
|
|
# Store in request state for access in handlers
|
|
request.state.correlation_id = correlation_id
|
|
|
|
# Bind to structlog context
|
|
structlog.contextvars.clear_contextvars()
|
|
structlog.contextvars.bind_contextvars(
|
|
correlation_id=correlation_id,
|
|
path=request.url.path,
|
|
method=request.method,
|
|
)
|
|
|
|
# Process request
|
|
response = cast("Response", await call_next(request))
|
|
|
|
# Add correlation ID to response
|
|
response.headers["X-Correlation-ID"] = correlation_id
|
|
|
|
return response
|
|
|
|
|
|
# =============================================================================
|
|
# REQUEST LOGGING MIDDLEWARE
|
|
# =============================================================================
|
|
|
|
|
|
class RequestLoggingMiddleware(BaseHTTPMiddleware):
|
|
"""
|
|
Logs request/response details with timing.
|
|
"""
|
|
|
|
async def dispatch(self, request: Request, call_next: Callable) -> Response:
|
|
start_time = time.perf_counter()
|
|
|
|
# Log request
|
|
logger.info(
|
|
"Request started",
|
|
path=request.url.path,
|
|
method=request.method,
|
|
query_params=dict(request.query_params),
|
|
)
|
|
|
|
try:
|
|
response = cast("Response", await call_next(request))
|
|
duration_ms = (time.perf_counter() - start_time) * 1000
|
|
|
|
# Log response
|
|
logger.info(
|
|
"Request completed",
|
|
status_code=response.status_code,
|
|
duration_ms=round(duration_ms, 2),
|
|
)
|
|
|
|
# Add timing header
|
|
response.headers["X-Response-Time-Ms"] = str(round(duration_ms, 2))
|
|
|
|
return response
|
|
|
|
except Exception as e:
|
|
duration_ms = (time.perf_counter() - start_time) * 1000
|
|
logger.exception(
|
|
"Request failed",
|
|
duration_ms=round(duration_ms, 2),
|
|
error=str(e),
|
|
)
|
|
raise
|
|
|
|
|
|
# =============================================================================
|
|
# EXCEPTION HANDLERS
|
|
# =============================================================================
|
|
|
|
|
|
def get_status_code(exc: RobocoError) -> int:
|
|
"""Map exception type to HTTP status code."""
|
|
status_map = {
|
|
NotFoundError: 404,
|
|
ValidationError: 422,
|
|
InvalidStateError: 409,
|
|
PermissionDeniedError: 403,
|
|
AuthenticationError: 401,
|
|
}
|
|
|
|
for exc_type, status in status_map.items():
|
|
if isinstance(exc, exc_type):
|
|
return status
|
|
|
|
# Default for other RobocoError subclasses
|
|
return 400
|
|
|
|
|
|
async def roboco_exception_handler(request: Request, exc: Exception) -> JSONResponse:
|
|
"""Handle RobocoError exceptions."""
|
|
roboco_exc = cast("RobocoError", exc)
|
|
status_code = get_status_code(roboco_exc)
|
|
|
|
# Add correlation ID to error details
|
|
correlation_id = getattr(request.state, "correlation_id", None)
|
|
if correlation_id:
|
|
roboco_exc.details["correlation_id"] = correlation_id
|
|
|
|
logger.warning(
|
|
"Handled exception",
|
|
error_code=roboco_exc.code,
|
|
error_message=roboco_exc.message,
|
|
status_code=status_code,
|
|
)
|
|
|
|
return JSONResponse(
|
|
status_code=status_code,
|
|
content=roboco_exc.to_dict(),
|
|
)
|
|
|
|
|
|
# `roboco.services.base.ServiceError` is a parallel exception hierarchy that
|
|
# does NOT inherit from `RobocoError` (it extends `Exception` directly), so
|
|
# `roboco_exception_handler` never sees it and the requests fall through to
|
|
# `generic_exception_handler` as 500s. Map its subclasses to the same status
|
|
# codes used in the RobocoError handler so route-layer try/except blocks can
|
|
# surface clean 4xx codes whether the service raises from `roboco.exceptions`
|
|
# or `roboco.services.base`.
|
|
_SERVICE_ERROR_STATUS: dict[type[ServiceError], int] = {
|
|
ServiceNotFoundError: 404,
|
|
ServiceValidationError: 422,
|
|
ServiceConflictError: 409,
|
|
ServiceUnauthorizedError: 403,
|
|
ServiceUnavailableError: 503,
|
|
}
|
|
|
|
|
|
async def service_exception_handler(request: Request, exc: Exception) -> JSONResponse:
|
|
"""Handle `roboco.services.base.ServiceError` and subclasses."""
|
|
svc_exc = cast("ServiceError", exc)
|
|
status_code = 500
|
|
for exc_type, mapped_status in _SERVICE_ERROR_STATUS.items():
|
|
if isinstance(svc_exc, exc_type):
|
|
status_code = mapped_status
|
|
break
|
|
|
|
correlation_id = getattr(request.state, "correlation_id", None)
|
|
details = dict(svc_exc.details)
|
|
if correlation_id:
|
|
details["correlation_id"] = correlation_id
|
|
|
|
logger.warning(
|
|
"Handled exception",
|
|
error_type=type(svc_exc).__name__,
|
|
error_message=svc_exc.message,
|
|
status_code=status_code,
|
|
)
|
|
|
|
return JSONResponse(
|
|
status_code=status_code,
|
|
content={
|
|
"error": type(svc_exc).__name__,
|
|
"message": svc_exc.message,
|
|
"details": details,
|
|
},
|
|
)
|
|
|
|
|
|
async def generic_exception_handler(request: Request, exc: Exception) -> JSONResponse:
|
|
"""Handle unexpected exceptions."""
|
|
correlation_id = getattr(request.state, "correlation_id", None)
|
|
|
|
logger.exception(
|
|
"Unhandled exception",
|
|
error=str(exc),
|
|
error_type=type(exc).__name__,
|
|
)
|
|
|
|
return JSONResponse(
|
|
status_code=http_status.HTTP_500_INTERNAL_SERVER_ERROR,
|
|
content={
|
|
"error": {
|
|
"code": ErrorCode.INTERNAL_ERROR,
|
|
"message": "An internal error occurred",
|
|
"details": {
|
|
"correlation_id": correlation_id,
|
|
},
|
|
}
|
|
},
|
|
)
|
|
|
|
|
|
# Map HTTP status codes to string error codes
|
|
_HTTP_TO_ERROR_CODE: dict[int, str] = {
|
|
400: ErrorCode.INVALID_INPUT,
|
|
401: ErrorCode.NOT_AUTHORIZED,
|
|
403: ErrorCode.ACCESS_DENIED,
|
|
404: ErrorCode.NOT_FOUND,
|
|
409: ErrorCode.INVALID_INPUT, # Conflict
|
|
422: ErrorCode.INVALID_INPUT, # Validation error
|
|
500: ErrorCode.INTERNAL_ERROR,
|
|
}
|
|
|
|
|
|
async def http_exception_handler(request: Request, exc: Exception) -> JSONResponse:
|
|
"""
|
|
Handle FastAPI HTTPException with standardized error format.
|
|
|
|
Converts HTTP status codes to string error codes for consistency with MCP.
|
|
"""
|
|
http_exc = cast("HTTPException", exc)
|
|
correlation_id = getattr(request.state, "correlation_id", None)
|
|
|
|
# Map status code to error code
|
|
error_code = _HTTP_TO_ERROR_CODE.get(http_exc.status_code, ErrorCode.INTERNAL_ERROR)
|
|
|
|
logger.warning(
|
|
"HTTP exception",
|
|
status_code=http_exc.status_code,
|
|
error_code=error_code,
|
|
detail=http_exc.detail,
|
|
)
|
|
|
|
response_content: dict = {
|
|
"error": {
|
|
"code": error_code,
|
|
"message": str(http_exc.detail),
|
|
}
|
|
}
|
|
|
|
if correlation_id:
|
|
response_content["error"]["details"] = {"correlation_id": correlation_id}
|
|
|
|
return JSONResponse(
|
|
status_code=http_exc.status_code,
|
|
content=response_content,
|
|
)
|
|
|
|
|
|
# =============================================================================
|
|
# SETUP FUNCTION
|
|
# =============================================================================
|
|
|
|
|
|
def _uuid_field_remediation(errors: Sequence[Any]) -> str | None:
|
|
"""Spell out the fix when a truncated id is sent where a UUID is required.
|
|
|
|
Agents routinely copy the 8-character task prefix the system shows them
|
|
(e.g. the ``[cee99ecc]`` commit prefix) and send it as ``task_id``, which
|
|
fails UUID validation with an opaque "invalid length" message and wastes a
|
|
call. Detect that case and hand back an actionable remediation instead.
|
|
"""
|
|
for err in errors:
|
|
if not isinstance(err, dict):
|
|
continue
|
|
loc = err.get("loc") or ()
|
|
field = loc[-1] if loc else None
|
|
if field == "task_id" and "uuid" in str(err.get("type", "")).lower():
|
|
return (
|
|
"Use the FULL 36-character task UUID, not the 8-character short "
|
|
"form shown in commit prefixes or summaries. The full id is in "
|
|
"the `task_id` field of the envelope returned by give_me_work "
|
|
"or your most recent verb."
|
|
)
|
|
return None
|
|
|
|
|
|
async def request_validation_handler(request: Request, exc: Exception) -> JSONResponse:
|
|
"""Log the rejected body before returning the standard 422 response.
|
|
|
|
FastAPI's default 422 returns validation details to the client but
|
|
nothing lands in server logs. During smoke tests this leaves us
|
|
blind to which field actually broke. Log the body + the per-field
|
|
errors so the next 422 is debuggable in one log scan.
|
|
|
|
When the failure is a truncated ``task_id`` (the recurring agent mistake),
|
|
add a ``remediate`` hint so the agent knows to retry with the full UUID.
|
|
"""
|
|
rve = cast("RequestValidationError", exc)
|
|
body = rve.body if isinstance(rve.body, str | bytes | dict | list) else None
|
|
errors = rve.errors()
|
|
logger.warning(
|
|
"Request validation failed",
|
|
path=request.url.path,
|
|
method=request.method,
|
|
body=body,
|
|
errors=errors,
|
|
)
|
|
content: dict[str, Any] = {"detail": errors, "body": body}
|
|
remediate = _uuid_field_remediation(errors)
|
|
if remediate is not None:
|
|
content["remediate"] = remediate
|
|
return JSONResponse(
|
|
status_code=http_status.HTTP_422_UNPROCESSABLE_ENTITY,
|
|
content=content,
|
|
)
|
|
|
|
|
|
def setup_middleware(app: FastAPI) -> None:
|
|
"""
|
|
Setup all middleware for the application.
|
|
|
|
Order matters:
|
|
1. CorrelationIdMiddleware - first to set correlation ID
|
|
2. RequestLoggingMiddleware - logs with correlation ID
|
|
|
|
Exception handler priority:
|
|
1. RequestValidationError - 422s; log body + per-field errors
|
|
2. HTTPException - most common, converts to string error codes
|
|
3. RobocoError - custom domain exceptions
|
|
4. Exception - catch-all for unexpected errors
|
|
"""
|
|
# Exception handlers (order: specific to general)
|
|
app.add_exception_handler(RequestValidationError, request_validation_handler)
|
|
app.add_exception_handler(HTTPException, http_exception_handler)
|
|
app.add_exception_handler(RobocoError, roboco_exception_handler)
|
|
app.add_exception_handler(ServiceError, service_exception_handler)
|
|
app.add_exception_handler(Exception, generic_exception_handler)
|
|
|
|
# Middleware (added in reverse order due to LIFO)
|
|
app.add_middleware(RequestLoggingMiddleware)
|
|
app.add_middleware(CorrelationIdMiddleware)
|