mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
* test: align phase1 smoke mock with the armed team-match gate The8e5f84c4sweep fixed 13 test files' inconsistent-team mocks but ran only the gateway/foundation/runtime subsets; the full gate caught this integration mock whose parent task carried an auto-generated MagicMock team and died on not_authorized before the incomplete_input assertion. * fix(runtime): attribute every agent.spawned audit to its dispatcher A rogue spawner could not be identified live (2026-07-02): agent.spawned rows carry container/model but not which dispatch loop launched them. spawn_agent now takes spawned_by, stamps it into the spawned/spawn_failed audit details, every call site passes its loop name, and an AST sweep test holds future callers to it. * fix(api): admin-complete refuses when the task's PR is still open PATCH status=completed on a task with an OPEN PR stranded its commits unmerged (bit the CEO twice live 2026-07-02). The override now refuses with the PR number/URL and the consequence before the generic hatch text; force:true stays the deliberate, audited escape. * fix(panel): awaiting_ceo_approval offers the working ceo-approve path The header's only approve action was Approve & Merge (POST /approve-and-merge, no notes) which 400s NO_PR on a branchless MegaTask umbrella — the CEO's approve button just failed. Primary action is now Approve & Complete via the CeoApproveDialog (POST /ceo-approve, notes >=20 chars, proven live); Approve & Merge stays for PR-bearing tasks. * test: stop leaking self-heal + rate-limit state into live Redis Two test files wrote real keys into a developer's localhost Redis: self-heal originate tests left self_heal:notified:* (2h TTL) and the i_am_blocked rate-limited tests left a NO-TTL 'anthropic rate-limited' tracker blob — order/state-dependent poison for anything reading the real tracker, and the prime suspect class for the one-off test_self_heal_engine full-run failure (not reproduced in 5x dir runs, adversarial orders, and a green full gate). Both files now point the computed redis_url at an unreachable port; the engines' fail-open paths keep every assertion intact. Leaked keys scrubbed live. * docs: changelog + map delta for the leak-fix batch; mypy-clean attribution test The attribution test's direct method assignments tripped the full gate's mypy (method-assign) — switched to the house monkeypatch idiom, no suppressions. * fix(gate): clear the ten xenon C-ranks; isolate all tests from live Redis Master CI has been red at the phase1 smoke test, so neither CI nor a local full gate had reached the xenon step since the team-match sweep — whose inline 'agent_team=str(agent.team) if ...' kwarg pushed nine verb bodies from B(10) to C(11-12) unseen. A shared actor_context_fields() (_protocol.py) computes (actor_slug, agent_team) once per verb, restoring all nine to B with zero behavior change; the new admin-complete override helper extraction does the same for routes/tasks.py. tests/conftest.py gains an autouse fixture pointing the computed redis_url at an unreachable port for every test — the root fix for the three families caught writing live-Redis keys (self-heal dedupe, rate-limit tracker, notification purpose-dedupe); no test uses a real Redis, and every production path is fail-open by design. * refactor(runtime): delete the never-wired dispatch-time spawn cooldown _safe_spawn / gateway_pre_spawn_check / trigger_filter had no caller in the repo's entire history (87ef42bfonly flipped the flag). Its five rules are superseded: provider parking runs inside spawn_agent, claim freshness is the guards+reaper, runaway respawns are the progress-aware breaker + notification cooldown; the per-task cooldown rule would queue-stall every normal stage handoff if wired today. gateway_triggers table kept inert. Ratified by the CEO over wiring it. * build: serialize uv — gate recipes never implicitly sync the venv Every uv run re-syncs implicitly, so a background make quality plus any foreground uv run raced two writers on one .venv and tore site-packages apart (the recurring rich/pip/bandit ImportError corruption; bit twice today, four times on 2026-07-02's first session). UV_NO_SYNC=1 is now exported Makefile-wide and quality/quality-fast/gate depend on one explicit up-front sync step. * fix(git): PR/merge/branch REST calls honor github_api_base_url Fifteen sites hardcoded https://api.github.com while the CI-run and open-PR-list calls already read settings.github_api_base_url — a GHE or test override silently applied to half the surface. One _api_base() helper keeps them uniform; default behavior unchanged. * ci: split the monolith — backend CI, Panel CI, E2E Smoke ci.yml keeps its file name and the backend quality job only (self-heal / ci-watch / release-readiness default to the ci.yml workflow); the panel job moves to panel-ci.yml scoped to panel/**, and the new scripted-agent lifecycle smoke gets e2e-smoke.yml + a make e2e-smoke target (env-gated out of the default pytest run). Trade: a panel-only red now lands on Panel CI, which the ci.yml-pinned watch engines don't see. * feat(tests): e2e lifecycle smoke harness — scripted agents, real gates tests/e2e_smoke stands up the real API (flow/do routers + middleware on uvicorn) over the ephemeral test Postgres, a local bare origin standing in for GitHub, and a fake GitHub REST layer whose merges are real git merges. A deterministic driver reloads the real MCP flow/do modules per agent and walks claim (real clone + worktree) -> tracing-gap -> note -> plan gate -> commit -> PR -> the full i_am_done ladder -> QA verdicts -> documenter -> awaiting_pm_review in ~5s. Runs via make e2e-smoke + its own CI workflow; skipped (env-gated) in the default suite. The freeze-lift condition's first half: scenario 1 green. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
269 lines
10 KiB
Python
269 lines
10 KiB
Python
"""Self-heal task origination against a real Postgres DB.
|
|
|
|
The loop opens a fix task only when ``self_heal_originate_enabled``, dedupes one
|
|
open task per regression fingerprint, honors the per-cycle and rolling open-task
|
|
caps, and creates the task PENDING + assigned to the Main PM agent +
|
|
``confirmed_by_human=False`` so it is HELD for the CEO's Approve-&-Start (it
|
|
does not dispatch autonomously). Crucially the loop NEVER calls start / approve
|
|
/ merge / deploy — asserted here.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import TYPE_CHECKING
|
|
from unittest.mock import AsyncMock
|
|
|
|
import pytest
|
|
from roboco.config import settings as cfg
|
|
from roboco.db.tables import AgentTable, ProjectTable
|
|
from roboco.foundation import identity as _foundation
|
|
from roboco.models.base import AgentRole, AgentStatus, TaskStatus, Team
|
|
from roboco.services.notification import NotificationService
|
|
from roboco.services.self_heal_engine import SelfHealEngine
|
|
from roboco.services.task import TaskService, get_task_service
|
|
from roboco.services.telemetry import TelemetrySample
|
|
|
|
if TYPE_CHECKING:
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
|
|
SYSTEM_UUID = _foundation.AGENTS["system"].uuid
|
|
MAIN_PM_UUID = _foundation.AGENTS["main-pm"].uuid
|
|
SLUG = "roboco"
|
|
ONE = 1
|
|
|
|
|
|
class _FakeSource:
|
|
def __init__(self, samples: list[TelemetrySample]) -> None:
|
|
self._samples = samples
|
|
|
|
async def fetch(self) -> list[TelemetrySample]:
|
|
return list(self._samples)
|
|
|
|
|
|
def _breach(signal: str) -> TelemetrySample:
|
|
return TelemetrySample(
|
|
signal_name=signal,
|
|
value=1.0,
|
|
threshold=1.0,
|
|
window="latest_completed_run",
|
|
repo_hint=SLUG,
|
|
observed_at="2026-06-17T00:00:00Z",
|
|
raw_ref="https://github.com/x/roboco/actions/runs/1",
|
|
detail=f"{signal} concluded 'failure'",
|
|
)
|
|
|
|
|
|
async def _seed_project(session: AsyncSession, slug: str = SLUG) -> None:
|
|
"""Seed the system agent (FK target for created_by) + RoboCo's own project.
|
|
|
|
The system agent has a FIXED foundation uuid (origination hardcodes it as
|
|
created_by). Another test in the full suite may have already committed it
|
|
into the session-scoped DB, so get-or-create by id — a plain insert collides
|
|
on pk_agents (green in isolation, red in the full CI run).
|
|
"""
|
|
if await session.get(AgentTable, SYSTEM_UUID) is None:
|
|
session.add(
|
|
AgentTable(
|
|
id=SYSTEM_UUID,
|
|
name="System",
|
|
slug=f"system-{slug}",
|
|
role=AgentRole.SYSTEM,
|
|
team=None,
|
|
status=AgentStatus.ACTIVE,
|
|
model_config={},
|
|
system_prompt="system",
|
|
capabilities=[],
|
|
permissions={},
|
|
metrics={},
|
|
)
|
|
)
|
|
await session.flush()
|
|
# The loop assigns the fix task to the Main PM agent (an FK to agents.id), so
|
|
# that row must exist — get-or-create by its fixed foundation uuid.
|
|
if await session.get(AgentTable, MAIN_PM_UUID) is None:
|
|
session.add(
|
|
AgentTable(
|
|
id=MAIN_PM_UUID,
|
|
name="Main PM",
|
|
slug="main-pm",
|
|
role=AgentRole.MAIN_PM,
|
|
team=Team.MAIN_PM,
|
|
status=AgentStatus.ACTIVE,
|
|
model_config={},
|
|
system_prompt="pm",
|
|
capabilities=[],
|
|
permissions={},
|
|
metrics={},
|
|
)
|
|
)
|
|
await session.flush()
|
|
session.add(
|
|
ProjectTable(
|
|
name="RoboCo",
|
|
slug=slug,
|
|
git_url="https://github.com/x/roboco.git",
|
|
default_branch="master",
|
|
protected_branches=["master"],
|
|
assigned_cell=Team.BACKEND,
|
|
created_by=SYSTEM_UUID,
|
|
is_active=True,
|
|
)
|
|
)
|
|
await session.flush()
|
|
|
|
|
|
def _enable(monkeypatch: pytest.MonkeyPatch, **overrides: object) -> None:
|
|
monkeypatch.setattr(cfg, "self_heal_enabled", True)
|
|
monkeypatch.setattr(cfg, "self_heal_originate_enabled", True)
|
|
monkeypatch.setattr(cfg, "self_heal_max_open_tasks", 5)
|
|
monkeypatch.setattr(cfg, "self_heal_max_per_cycle", 5)
|
|
for key, value in overrides.items():
|
|
monkeypatch.setattr(cfg, key, value)
|
|
# Keep notification a no-op (its own session/IO is out of scope here).
|
|
monkeypatch.setattr(NotificationService, "send_ack_notification", AsyncMock())
|
|
# Point the notify-dedupe at an unreachable Redis: _already_notified fails
|
|
# open and _mark_notified swallows, so these tests neither read from nor
|
|
# leak `self_heal:notified:*` keys (2h TTL) into a developer's live Redis.
|
|
# (redis_url is a computed property — patch its inputs.)
|
|
monkeypatch.setattr(cfg, "redis_host", "127.0.0.1")
|
|
monkeypatch.setattr(cfg, "redis_port", 1)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_disabled_originate_creates_no_task(
|
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
await _seed_project(db_session)
|
|
_enable(monkeypatch, self_heal_originate_enabled=False)
|
|
engine = SelfHealEngine(db_session, source=_FakeSource([_breach("ci:roboco")]))
|
|
obs = await engine.run_cycle()
|
|
assert len(obs) == ONE # detected + notified
|
|
assert await get_task_service(db_session).list_open_self_heal_tasks() == []
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_originate_creates_pending_main_pm_assigned_task(
|
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
await _seed_project(db_session)
|
|
_enable(monkeypatch)
|
|
engine = SelfHealEngine(db_session, source=_FakeSource([_breach("ci:roboco")]))
|
|
await engine.run_cycle()
|
|
|
|
open_tasks = await get_task_service(db_session).list_open_self_heal_tasks()
|
|
assert len(open_tasks) == ONE
|
|
task = open_tasks[0]
|
|
assert task.status == TaskStatus.PENDING
|
|
# Assigned to the Main PM agent up front (not just team=main_pm) so that, once
|
|
# the CEO approves it, the orchestrator dispatches it straight to that agent.
|
|
assert task.assigned_to == MAIN_PM_UUID
|
|
# held for the CEO's Approve-&-Start — NOT auto-confirmed: the orchestrator
|
|
# + give_me_work keep it out of dispatch until approve_and_start flips this.
|
|
assert task.confirmed_by_human is False
|
|
assert task.team == Team.MAIN_PM
|
|
assert task.source == "self_heal"
|
|
assert task.acceptance_criteria # non-empty (AC-guardrail)
|
|
assert (task.orchestration_markers or {}).get("self_heal_fp")
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_dedupe_no_second_task_same_fingerprint(
|
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
await _seed_project(db_session)
|
|
_enable(monkeypatch)
|
|
src = _FakeSource([_breach("ci:roboco")])
|
|
await SelfHealEngine(db_session, source=src).run_cycle()
|
|
await SelfHealEngine(db_session, source=src).run_cycle() # same fingerprint
|
|
open_tasks = await get_task_service(db_session).list_open_self_heal_tasks()
|
|
assert len(open_tasks) == ONE
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_per_cycle_cap_limits_origination(
|
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
await _seed_project(db_session)
|
|
_enable(monkeypatch, self_heal_max_per_cycle=1)
|
|
# Two distinct regressions in one cycle, cap = 1 → only one task opens.
|
|
src = _FakeSource([_breach("ci:roboco:a"), _breach("ci:roboco:b")])
|
|
await SelfHealEngine(db_session, source=src).run_cycle()
|
|
assert len(await get_task_service(db_session).list_open_self_heal_tasks()) == ONE
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_open_task_cap_blocks_further_origination(
|
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
await _seed_project(db_session)
|
|
_enable(monkeypatch, self_heal_max_open_tasks=1)
|
|
await SelfHealEngine(
|
|
db_session, source=_FakeSource([_breach("ci:roboco:a")])
|
|
).run_cycle()
|
|
# A different regression next cycle, but one task is already open → blocked.
|
|
await SelfHealEngine(
|
|
db_session, source=_FakeSource([_breach("ci:roboco:b")])
|
|
).run_cycle()
|
|
assert len(await get_task_service(db_session).list_open_self_heal_tasks()) == ONE
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_unresolved_project_is_notify_only(
|
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
await _seed_project(db_session)
|
|
_enable(monkeypatch)
|
|
ghost = TelemetrySample(
|
|
signal_name="ci:ghost",
|
|
value=1.0,
|
|
threshold=1.0,
|
|
window="latest_completed_run",
|
|
repo_hint="not-a-registered-project",
|
|
observed_at="",
|
|
raw_ref="",
|
|
)
|
|
# No crash, no task — the regression can't be tied to a repo.
|
|
await SelfHealEngine(db_session, source=_FakeSource([ghost])).run_cycle()
|
|
assert await get_task_service(db_session).list_open_self_heal_tasks() == []
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_loop_never_starts_or_approves(
|
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
await _seed_project(db_session)
|
|
_enable(monkeypatch)
|
|
approve = AsyncMock()
|
|
ceo_approve = AsyncMock()
|
|
monkeypatch.setattr(TaskService, "approve_and_start", approve)
|
|
monkeypatch.setattr(TaskService, "ceo_approve", ceo_approve)
|
|
await SelfHealEngine(
|
|
db_session, source=_FakeSource([_breach("ci:roboco")])
|
|
).run_cycle()
|
|
|
|
approve.assert_not_awaited()
|
|
ceo_approve.assert_not_awaited()
|
|
open_tasks = await get_task_service(db_session).list_open_self_heal_tasks()
|
|
assert len(open_tasks) == ONE
|
|
assert open_tasks[0].status == TaskStatus.PENDING # never advanced by the loop
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_originated_task_is_held_for_ceo_approve_and_start(
|
|
db_session: AsyncSession, monkeypatch: pytest.MonkeyPatch
|
|
) -> None:
|
|
"""The opened task is HELD (confirmed_by_human=False) for the CEO's
|
|
Approve-&-Start — the PM dispatcher does NOT pick it up autonomously; the
|
|
CEO must approve_and_start it first (F059)."""
|
|
await _seed_project(db_session)
|
|
_enable(monkeypatch)
|
|
await SelfHealEngine(
|
|
db_session, source=_FakeSource([_breach("ci:roboco")])
|
|
).run_cycle()
|
|
task = (await get_task_service(db_session).list_open_self_heal_tasks())[0]
|
|
assert task.confirmed_by_human is False # held for the CEO — no autonomous dispatch
|
|
assert task.status == TaskStatus.PENDING # not advanced by the loop
|
|
assert (
|
|
task.assigned_to == MAIN_PM_UUID
|
|
) # straight to the Main PM agent once approved
|