mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
* feat(forge): Phase 2 — Gitea provider, per-call routing, host registry Gitea support lands behind the Phase-1 seam: - GiteaProvider (services/forge/gitea.py): Gitea v1 transport addressed by instance host (api base from the project's git_url). Where Gitea's wire contract diverges from GitHub's, the provider adapts responses back into the shapes GitService already classifies (ShapedResponse): `token` auth scheme, duplicate-PR 409→422 with the "already exists" text GitService keys on, commit statuses reshaped into check_runs / workflow_runs envelopes, APPROVE→APPROVED review mapping, Do-keyed POST merge, merge-method repo keys, label-color '#' prefix, client-side head/base PR filtering. Deliberate postures per the spec: zero-workflows fail-open (statuses-free repo → no_ci_configured) and merge_branch as a shaped 501 (env-sync cascade lands on missing_ref; the shared local-git fallback is Phase-2.1). - ForgeRouter (services/forge/router.py): GitService._forge now routes per call from RepoRef.host — every existing call site unchanged in shape. RepoRef gains an optional host; _parse_git_url returns the host-stamped ref and it is threaded through GitService/release executor instead of being rebuilt from strings (helpers re-signatured to take RepoRef). - Host registry (services/forge/registry.py): in-memory host→provider map, self-healing — ProjectService.get/get_by_slug re-register on every read; provider_for resolves gitea projects by git_url host. - Registration validation now accepts git_provider="gitea"; GitLab remains recognized-but-rejected. Panel: the read-only Forge badge becomes a real picker (Auto-detect / GitHub-GHE / Gitea / GitLab disabled). Plain git (clone/fetch/push) needs no changes — the Basic-auth extraheader works on Gitea unchanged. Gates: mypy 392 files, xenon A, full unit suite 6356 green, integration suite 2257 green. * feat(forge): live-Gitea contract suite + scheme support + slash-safe refs Hardening from running the provider against a real dockerized Gitea 1.22.6 (the spec's Phase-2 contract suite, now committed as the env-gated tests/e2e_smoke/test_gitea_live.py — self-seeding: creates its own repo, pushes real commits, and drives PR open → duplicate reshape → list/filter → diff → review → labels → commit-status CI reshapes → squash merge → branch delete → release, plus a live verification of the x-access-token Basic-auth git-CLI claim). Two real findings fixed: - Branch refs weren't URL-encoded — every RoboCo branch carries slashes (feature/backend/...), and Gitea's router 404s on the extra path segments. list_ci_runs + delete_branch_ref now quote the ref (regression-pinned in the unit suite). - The API base hardcoded https; a LAN instance serving plain http is a real deployment shape. GiteaProvider gains a scheme (recorded per host by the registry from the project's git_url). ShapedResponse moves to forge/shaping.py (shared by the upcoming GitLab transport, which needs its text override for diff reassembly). * feat(forge): Phase 3 GitLab provider + Phase 2.1 local-merge fallback GitLabProvider (services/forge/gitlab.py): GitLab v4 transport addressed by host+scheme, subgroup-safe (the MR project path packs into RepoRef.owner, URL-encoded per call). Adapters translate MR semantics into the GitHub shapes GitService classifies: iid→number, source/target_branch→head/base with a merged bool, per-file diffs reassembled into unified-diff text (ShapedResponse text override, 3-page cap), approve-vs-note review routing (GitLab has no request-changes verb), pipelines/statuses reshaped into workflow_runs/check_runs, merge-method repo-key mapping, duplicate-MR 409→422. Reviewer mirroring is skipped (needs numeric ids RoboCo doesn't store); provisioning stays Phase 4. gitlab.com now auto-detects at registration like github.com; self-hosted GitLab sets the provider explicitly (panel picker enabled). Phase 2.1: neither Gitea nor GitLab has GitHub's server-side merges API — their merge_branch returns a shaped 501 and GitService.sync_env_branch now runs the shared local-git fallback (_local_merge_branch: throwaway clone → ancestor check → merge → push; a conflict aborts with the remote untouched; same status vocabulary as the merges-API path). Also aligns the whole tree with the full gate's tests/-scoped mypy (provider-test responder typing, e2e_smoke's stale owner/repo shapes). Gates: mypy 1229 files clean, xenon A, unit suite 6393 green, forge suites 85 green, panel typecheck/lint clean. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
323 lines
13 KiB
Python
323 lines
13 KiB
Python
"""Git-workflow smoke scenarios for the 0.19.0 hardening batch.
|
|
|
|
Cross-layer wiring for three findings that the brief deferred here:
|
|
|
|
- H11 — the clone/fetch PAT rides a per-call ``-c http.extraheader=
|
|
Authorization: Basic <base64>`` config, never URL-embedded into argv
|
|
(``/proc/<pid>/cmdline`` exposure class). The smoke env uses local-protocol
|
|
origins (tokenless), so a true HTTPS exercise isn't feasible; this mirrors
|
|
``tests/unit/services/test_workspace_read_clone.py::
|
|
test_sync_read_clone_with_token_uses_extraheader_not_url`` — patching
|
|
``subprocess.run`` and asserting the argv shape against a fake HTTPS URL +
|
|
token. The real production path is exercised through the same
|
|
``WorkspaceService._sync_read_clone`` static method driven here.
|
|
- M37 — two concurrent ``WorkSessionService.merge_pr`` calls on the same
|
|
ACTIVE work_session serialize via ``FOR UPDATE`` so only one writes
|
|
``merged_by`` / ``pr_merged_at``. Drives the real service against the e2e
|
|
Postgres (separate sessions on ``asyncio.gather``), seeded with project +
|
|
agents + task + ACTIVE work_session.
|
|
- M38 — ``GitService._pr_is_merged`` returns ``None`` on ``httpx.HTTPError``
|
|
(indeterminate, not False), and the caller (``_merge_with_retry``) treats
|
|
``None`` as "assume merged" — falling through instead of raising
|
|
``MergeConflictError`` and respawning the PM against an already-merged PR.
|
|
Drives ``_pr_is_merged`` with a mocked ``httpx.AsyncClient`` that raises,
|
|
then drives ``_merge_with_retry`` with ``_pr_is_merged`` stubbed to ``None``.
|
|
|
|
H8 (the ``rebase_onto_base`` clean-tree gate) is NOT exercised here — a real
|
|
clean-tree gate needs a dirty agent worktree, which the harness's
|
|
session-scoped workspace contamination makes infeasible without polluting
|
|
other scenarios. H8 is unit-covered by
|
|
``tests/unit/services/test_git_rebase.py``.
|
|
|
|
Deviations from a true end-to-end exercise (noted): the smoke harness's git
|
|
origin is local-protocol (tokenless), so H11 cannot exercise a real
|
|
HTTPS-authenticated clone/fetch — the argv-shape assertion against a fake
|
|
HTTPS URL + token is the strongest feasible check and matches the unit test.
|
|
M38's ``_pr_is_merged`` network call is mocked because the smoke harness's
|
|
fake GitHub router does not model the merged-state GET deterministically
|
|
across transient failure; the unit-covered caller path (``_merge_with_retry``
|
|
fall-through) is exercised directly. The full-suite session-scoped workspace
|
|
contamination across e2e_smoke files is a pre-existing harness limitation
|
|
(documented in the module README) and is out of scope — each scenario here
|
|
passes in isolation.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import base64
|
|
import subprocess
|
|
from pathlib import Path
|
|
from typing import TYPE_CHECKING, Any
|
|
from unittest.mock import AsyncMock, MagicMock, patch
|
|
from uuid import UUID, uuid4
|
|
|
|
import httpx
|
|
import pytest
|
|
from roboco.db.tables import (
|
|
AgentTable,
|
|
ProjectTable,
|
|
TaskTable,
|
|
WorkSessionTable,
|
|
)
|
|
from roboco.models import AgentRole, AgentStatus, Team
|
|
from roboco.models.base import Complexity, TaskNature, TaskStatus, TaskType
|
|
from roboco.models.work_session import WorkSessionStatus
|
|
from roboco.services.forge import RepoRef
|
|
from roboco.services.git import GitService
|
|
from roboco.services.work_session import get_work_session_service
|
|
from roboco.services.workspace import WorkspaceService
|
|
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
|
|
|
if TYPE_CHECKING:
|
|
from tests.e2e_smoke.harness import E2EStack
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# H11 — clone/fetch PAT injected via http.extraheader, not URL-embedded
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_h11_pat_not_in_argv(e2e_stack: E2EStack) -> None:
|
|
"""H11: a private-repo refresh injects the PAT via
|
|
``-c http.extraheader=Authorization: Basic <base64>``, never URL-embedded
|
|
into the fetch argv. The smoke origin is local-protocol (tokenless), so
|
|
this drives ``WorkspaceService._sync_read_clone`` with a fake HTTPS URL +
|
|
token and a ``subprocess.run`` capture — the same argv-shape contract the
|
|
production path honors (mirrors the unit test)."""
|
|
clone = e2e_stack.root / "h11-clone"
|
|
clone.mkdir()
|
|
token = "ghp_SMOKEARGV"
|
|
git_url = "https://github.com/o/r"
|
|
captured: list[list[str]] = []
|
|
|
|
def _fake_run(argv: list[str], **_kw: object) -> subprocess.CompletedProcess[str]:
|
|
captured.append(list(argv))
|
|
return subprocess.CompletedProcess(argv, 0, "", "")
|
|
|
|
with patch("roboco.services.workspace.subprocess.run", side_effect=_fake_run):
|
|
WorkspaceService._sync_read_clone(clone, git_url, "master", token)
|
|
|
|
fetch_argv = next(a for a in captured if "fetch" in a)
|
|
# Token never appears in argv (no /proc/<pid>/cmdline leak).
|
|
assert token not in fetch_argv
|
|
assert f"https://{token}@" not in fetch_argv
|
|
# The bare URL is the fetch ref (no token embedded in the URL).
|
|
assert git_url in fetch_argv
|
|
# The basic-auth extraheader carries the encoded token.
|
|
expected = base64.b64encode(f"x-access-token:{token}".encode()).decode()
|
|
assert f"http.extraheader=Authorization: Basic {expected}" in fetch_argv
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# M37 — concurrent merge_pr calls serialize: exactly one writes merged_by
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
async def _seed_active_session(session: AsyncSession) -> tuple[UUID, UUID, UUID]:
|
|
"""Seed project + 3 agents + task + ACTIVE work_session; return
|
|
(ws_id, merger_a_id, merger_b_id). The two merger agents exist so the
|
|
merged_by FK is satisfied."""
|
|
worker = AgentTable(
|
|
id=uuid4(),
|
|
name="m37-worker",
|
|
slug=f"m37-worker-{uuid4().hex[:8]}",
|
|
role=AgentRole.DEVELOPER,
|
|
team=Team.BACKEND,
|
|
status=AgentStatus.ACTIVE,
|
|
model_config={},
|
|
system_prompt="dev",
|
|
capabilities=[],
|
|
permissions={},
|
|
metrics={},
|
|
)
|
|
merger_a = AgentTable(
|
|
id=uuid4(),
|
|
name="m37-merger-a",
|
|
slug=f"m37-merger-a-{uuid4().hex[:8]}",
|
|
role=AgentRole.CELL_PM,
|
|
team=Team.BACKEND,
|
|
status=AgentStatus.ACTIVE,
|
|
model_config={},
|
|
system_prompt="pm",
|
|
capabilities=[],
|
|
permissions={},
|
|
metrics={},
|
|
)
|
|
merger_b = AgentTable(
|
|
id=uuid4(),
|
|
name="m37-merger-b",
|
|
slug=f"m37-merger-b-{uuid4().hex[:8]}",
|
|
role=AgentRole.CELL_PM,
|
|
team=Team.BACKEND,
|
|
status=AgentStatus.ACTIVE,
|
|
model_config={},
|
|
system_prompt="pm",
|
|
capabilities=[],
|
|
permissions={},
|
|
metrics={},
|
|
)
|
|
for agent in (worker, merger_a, merger_b):
|
|
session.add(agent)
|
|
await session.flush()
|
|
project = ProjectTable(
|
|
id=uuid4(),
|
|
name="m37-proj",
|
|
slug=f"m37-proj-{uuid4().hex[:6]}",
|
|
git_url="git@x:y/z.git",
|
|
assigned_cell=Team.BACKEND,
|
|
created_by=worker.id,
|
|
)
|
|
session.add(project)
|
|
await session.flush()
|
|
tid = uuid4()
|
|
session.add(
|
|
TaskTable(
|
|
id=tid,
|
|
title="m37 task",
|
|
description="d",
|
|
acceptance_criteria=["done"],
|
|
status=TaskStatus.IN_PROGRESS,
|
|
priority=2,
|
|
task_type=TaskType.CODE,
|
|
nature=TaskNature.TECHNICAL,
|
|
estimated_complexity=Complexity.LOW,
|
|
team=Team.BACKEND,
|
|
confirmed_by_human=True,
|
|
project_id=project.id,
|
|
created_by=worker.id,
|
|
branch_name="feature/x",
|
|
)
|
|
)
|
|
await session.flush()
|
|
ws_id = uuid4()
|
|
session.add(
|
|
WorkSessionTable(
|
|
id=ws_id,
|
|
project_id=project.id,
|
|
task_id=tid,
|
|
agent_id=worker.id,
|
|
branch_name="feature/x",
|
|
base_branch="master",
|
|
target_branch="master",
|
|
status=WorkSessionStatus.ACTIVE,
|
|
)
|
|
)
|
|
await session.flush()
|
|
await session.commit()
|
|
return ws_id, UUID(str(merger_a.id)), UUID(str(merger_b.id))
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_m37_concurrent_merge_pr_single_write(e2e_stack: E2EStack) -> None:
|
|
"""M37: two concurrent ``merge_pr`` calls on the same ACTIVE work_session
|
|
serialize via ``FOR UPDATE`` — only one writes ``merged_by`` /
|
|
``pr_merged_at``; the other blocks on the row lock, then sees COMPLETED
|
|
and no-ops. Drives the real ``WorkSessionService`` against the e2e
|
|
Postgres (two separate sessions on ``asyncio.gather``)."""
|
|
engine = create_async_engine(e2e_stack.db_url, future=True)
|
|
factory = async_sessionmaker(
|
|
bind=engine, class_=AsyncSession, expire_on_commit=False
|
|
)
|
|
try:
|
|
async with factory() as seed_sess:
|
|
ws_id, merger_a_id, merger_b_id = await _seed_active_session(seed_sess)
|
|
|
|
async def _call(merger: UUID) -> Any:
|
|
async with factory() as sess:
|
|
svc = get_work_session_service(sess)
|
|
result = await svc.merge_pr(ws_id, merger)
|
|
await sess.commit()
|
|
return result, merger
|
|
|
|
# Both fire together: without FOR UPDATE each session's SELECT sees
|
|
# ACTIVE and both write their own merger. With FOR UPDATE one
|
|
# caller's SELECT blocks on the other's row lock until it commits,
|
|
# then reads COMPLETED and no-ops — exactly one merger is recorded.
|
|
# Which caller wins the lock is non-deterministic, so the assertions
|
|
# below check the invariant, not the winner.
|
|
res_a, res_b = await asyncio.gather(_call(merger_a_id), _call(merger_b_id))
|
|
finally:
|
|
await engine.dispose()
|
|
|
|
a_row, a_merger = res_a
|
|
b_row, b_merger = res_b
|
|
assert a_row is not None
|
|
assert b_row is not None
|
|
# Both resolved COMPLETED; both report the same committed merger.
|
|
assert a_row.status == b_row.status == WorkSessionStatus.COMPLETED
|
|
winner = a_row.merged_by
|
|
assert winner in (a_merger, b_merger)
|
|
assert b_row.merged_by == winner
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# M38 — _pr_is_merged returns None on HTTPError; caller assumes merged
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _httpx_raising_client() -> MagicMock:
|
|
"""An AsyncClient whose GET raises httpx.HTTPError (network indeterminate)."""
|
|
fake_client = MagicMock()
|
|
fake_client.__aenter__ = AsyncMock(return_value=fake_client)
|
|
fake_client.__aexit__ = AsyncMock(return_value=False)
|
|
fake_client.get = AsyncMock(side_effect=httpx.HTTPError("network indeterminate"))
|
|
return fake_client
|
|
|
|
|
|
def _git_service() -> GitService:
|
|
"""A GitService with a MagicMock session — _pr_is_merged / _merge_with_retry
|
|
don't touch the DB in the paths exercised here."""
|
|
session = MagicMock()
|
|
session.execute = AsyncMock()
|
|
session.commit = AsyncMock()
|
|
session.rollback = AsyncMock()
|
|
session.flush = AsyncMock()
|
|
return GitService(session)
|
|
|
|
|
|
def _bind(svc: GitService, name: str, value: object) -> None:
|
|
object.__setattr__(svc, name, value)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_m38_pr_is_merged_returns_none_on_httperror() -> None:
|
|
"""M38(a): on ``httpx.HTTPError`` the lookup is indeterminate -> ``None``,
|
|
not ``False``. A clean ``False`` would make the caller raise
|
|
``MergeConflictError`` and respawn the PM against an already-merged PR."""
|
|
svc = _git_service()
|
|
with patch(
|
|
"roboco.services.git.httpx.AsyncClient",
|
|
return_value=_httpx_raising_client(),
|
|
):
|
|
out = await svc._pr_is_merged(RepoRef("acme", "repo"), 11, "tok")
|
|
assert out is None
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_m38_merge_with_retry_none_does_not_raise_conflict() -> None:
|
|
"""M38(b): the caller (``_merge_with_retry``) treats ``None`` as "assume
|
|
merged" — it falls through (returns the failed resp) instead of raising
|
|
``MergeConflictError``, so an indeterminate gh call doesn't respawn the
|
|
PM against an already-merged PR. A real ``False`` still raises."""
|
|
svc = _git_service()
|
|
_bind(svc, "log", MagicMock())
|
|
_bind(svc, "_first_allowed_merge_method", AsyncMock(return_value=None))
|
|
# 405 -> disambiguation path -> _pr_is_merged returns None -> fall through.
|
|
resp_405 = MagicMock(is_success=False, status_code=405, text="not allowed")
|
|
_bind(svc, "_call_merge_api", AsyncMock(return_value=resp_405))
|
|
_bind(svc, "_pr_is_merged", AsyncMock(return_value=None))
|
|
_bind(svc, "_sync_target_branch", AsyncMock())
|
|
|
|
ctx = GitService._MergeContext(
|
|
repo_ref=RepoRef("acme", "repo"),
|
|
pr_number=11,
|
|
git_token="tok",
|
|
workspace=Path("/tmp/ws"),
|
|
target="feature/main_pm/root1",
|
|
)
|
|
# None must NOT raise — indeterminate falls through, not conflict.
|
|
out = await svc._merge_with_retry(ctx)
|
|
assert out is resp_405
|