mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
[a1f52d13] Rebuild evidence-assembly timeout fix on current main (#796)
* [a1f52d13] feat(gateway): re-apply evidence-assembly timeout fix on current main
Rebuilds the bounded-timeout fix for claim_review/evidence()/roboco_git_diff
that PR #765 carried as contaminated pre-Board-Programs file copies. The
rebase restored the current-main baseline; this commit re-applies ONLY the
validated timeout-fix hunks on top of that baseline:
- config.py: evidence_assembly_timeout_seconds (90.0s) and
conventions_validator_timeout_seconds (45s) Settings fields
- envelope.py: Envelope.gateway_timeout classmethod for structured
timeout errors naming the stalled component
- git.py: GitService.diff_and_files (single resolution, concurrent
diff+--name-only subprocesses), settings-backed conventions-validator
timeout, and changed_files passthrough to conventions_check_for_task
- qa.py: claim_review bounded-timeout wrapper around
_build_qa_claim_evidence, split into _qa_git_and_conventions and
_qa_db_reads concurrent segments (pass_review signature and gate
untouched)
- content_actions.py: evidence() bounded wrapper with _evidence_git_and_fetch
and _evidence_db_reads concurrent split (all other methods untouched)
- git routes: GET /diff bounded asyncio.wait_for guard with 504 on trip
The pre-existing methods (sync_env_branch, open_sync_pr, pass_review with
criteria_verified gate, all propose_*/request_render methods) are preserved
and untouched. The test files already exist on current main.
* [a1f52d13] fix(git-routes): wrap workspace resolution in bounded timeout for GET /diff
The bounded asyncio.wait_for must cover the workspace resolution step too,
not just the diff+stat subprocesses — the integration test makes
get_workspace slow (not _run_git), so the timeout guard needs to enclose
the entire resolve-and-diff path to return a structured 504 on trip.
* [a1f52d13] test: update mock setups from git.diff/list_changed_files to diff_and_files
Tests that exercise claim_review and evidence() paths need their git
mocks to return a tuple from diff_and_files instead of separate diff
and list_changed_files return values, matching the new combined accessor
introduced by the evidence-assembly timeout fix.
* [a1f52d13] test: configure diff_and_files mock in lifecycle parity test _make_deps
The spec-parity test test_claim_review_matches_spec[qa-awaiting_qa] failed
with ValueError at qa.py:330 because _make_deps set git=AsyncMock() without
configuring diff_and_files.return_value, so the await resolved to a MagicMock
instead of a (diff_summary, files_changed) 2-tuple. Adds the same
return_value = ("", []) configuration that commit 014a0d03 applied to the
other 4 test files.
* [a1f52d13] docs(evidence-assembly-timeout): correct GET /diff guard scope and _qa_git_and_conventions timing field
---------
Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech>
Co-authored-by: Backend Documenter <be-doc@roboco.tech>
This commit is contained in:
co-authored by
Backend Developer 2
Backend Documenter
parent
89254f796c
commit
ce89a04d26
@@ -22,7 +22,7 @@ A new `settings.evidence_assembly_timeout_seconds` (default 90s, well under the
|
||||
|
||||
- `ContentActions.evidence()` — wraps the `asyncio.gather` of the git-diff coroutine and `_evidence_db_reads`.
|
||||
- `QAMixin.claim_review()` — wraps the call to `_build_qa_claim_evidence`; the claim itself (`qa_claim` + `mark_evidence_inspected`) has already committed by this point, so a trip here means only evidence assembly is slow, not that the claim failed.
|
||||
- `GET /api/git/diff` (`roboco/api/routes/git.py`, `roboco_git_diff`) — wraps the diff + `--stat` subprocess pair.
|
||||
- `GET /api/git/diff` (`roboco/api/routes/git.py`, `roboco_git_diff`) — wraps workspace resolution *plus* the diff + `--stat` subprocess pair in one bounded guard, so a slow workspace fetch trips the 504 too — not just a slow diff. (The workspace resolution is enclosed because an unbounded `get_workspace` is the step the integration test makes slow; leaving it outside the guard would let the whole request hang past the timeout anyway.)
|
||||
|
||||
On a trip, each site returns a structured `Envelope.gateway_timeout(component, timeout_seconds, remediate)` (new classmethod on `roboco/services/gateway/envelope.py`'s `Envelope`) naming which segment stalled (e.g. `"git diff/fetch or a journal/ancestor/findings DB read"`) and a remediation hint, instead of the outer 120s rollback with no indication of which piece stuck. The route's HTTP path raises a `504 Gateway Timeout` with the equivalent detail message.
|
||||
|
||||
@@ -40,7 +40,7 @@ Each segment logs its own duration (structlog `.info()` calls) so a slow request
|
||||
- `GitService.diff_and_files` logs `resolve_ms` (workspace/token/head/base resolution) and `diff_ms` (the concurrent diff subprocesses) as "evidence diff_and_files timing".
|
||||
- `conventions_check_for_task` logs `total_ms` plus whether it reused the passed-in file list, as "conventions_check_for_task timing".
|
||||
- `evidence()` logs `git_diff_and_fetch_ms` ("evidence git diff/fetch timing") and the DB-reads batch logs `db_reads_ms` ("evidence db reads timing").
|
||||
- `claim_review`'s two segment helpers (`_qa_git_and_conventions`, `_qa_db_reads`) log `git_diff_and_fetch_ms`/`conventions_ms` ("claim_review git+conventions timing") and `db_reads_ms` ("claim_review db reads timing") respectively.
|
||||
- `claim_review`'s two segment helpers (`_qa_git_and_conventions`, `_qa_db_reads`) log `git_diff_and_fetch_ms` ("claim_review git+conventions timing", covering the whole git+conventions block as one measurement) and `db_reads_ms` ("claim_review db reads timing") respectively.
|
||||
|
||||
## Related files
|
||||
|
||||
|
||||
+229
-45
@@ -33,6 +33,8 @@ from roboco.api.deps import CurrentAgentContext, DbSession
|
||||
from roboco.api.schemas.git import (
|
||||
BranchInfo,
|
||||
CommitInfo,
|
||||
GitBranchCleanupRequest,
|
||||
GitBranchCleanupResponse,
|
||||
GitBranchListResponse,
|
||||
GitCheckoutRequest,
|
||||
GitCheckoutResponse,
|
||||
@@ -45,6 +47,7 @@ from roboco.api.schemas.git import (
|
||||
GitDiffResponse,
|
||||
GitFetchRequest,
|
||||
GitFetchResponse,
|
||||
GitFileContentResponse,
|
||||
GitLogResponse,
|
||||
GitMergePRRequest,
|
||||
GitMergePRResponse,
|
||||
@@ -89,6 +92,43 @@ _LOG_FORMAT_PARTS = 5
|
||||
# bubbling as 500 Internal Server Errors with no `detail`.
|
||||
_TranslatableError = (ServiceError, GitError)
|
||||
|
||||
# Cap an unbounded whole-file read so a huge file can't flood the panel.
|
||||
_FILE_MAX_LINES = 2000
|
||||
|
||||
|
||||
def _compute_file_range(
|
||||
*,
|
||||
total: int,
|
||||
line: int | None,
|
||||
context: int,
|
||||
start: int | None,
|
||||
end: int | None,
|
||||
) -> tuple[int, int, bool]:
|
||||
"""Resolve the (start, end, truncated) slice for a file-content read.
|
||||
|
||||
Explicit ``start``/``end`` win; else ``line`` centers a context window;
|
||||
else the whole file. Whichever branch resolves the window, it is capped
|
||||
at ``_FILE_MAX_LINES`` lines afterward. Returns 1-based inclusive
|
||||
[start, end] and whether the slice is shorter than the file.
|
||||
"""
|
||||
if start is not None and end is not None:
|
||||
s, e_ = start, end
|
||||
elif line is not None:
|
||||
s = max(1, line - context)
|
||||
e_ = min(total, line + context)
|
||||
else:
|
||||
s, e_ = 1, total
|
||||
|
||||
s = max(1, min(s, total))
|
||||
e_ = max(s, min(e_, total))
|
||||
|
||||
truncated = e_ < total
|
||||
if e_ - s + 1 > _FILE_MAX_LINES:
|
||||
e_ = s + _FILE_MAX_LINES - 1
|
||||
truncated = True
|
||||
return s, e_, truncated
|
||||
|
||||
|
||||
# Roles permitted to rebase branches via the /rebase endpoint.
|
||||
# Rebase is a history-rewriting operation that should be authorised only by
|
||||
# PM-level or CEO-level callers. Developers are intentionally excluded:
|
||||
@@ -206,6 +246,18 @@ async def get_git_log(
|
||||
if not branch:
|
||||
branch = await git_service.get_current_branch(workspace)
|
||||
|
||||
# This is the CALLER's own clone, which is never the branch's own
|
||||
# author when inspecting another agent's task (QA/PM/documenter
|
||||
# reading a dev's branch) — a local ref left over from an earlier
|
||||
# inspection can be pinned stale (behind, or diverged after a
|
||||
# routine rebase force-push) while origin has since moved. Resolve
|
||||
# through _resolve_head_ref (fetch + prefer origin) instead of the
|
||||
# bare branch name so this reads the same authoritative tip diff()/
|
||||
# read_file_at_branch() do, not whatever this clone happened to
|
||||
# have on disk from the last time it looked.
|
||||
token = await git_service._token_for_branch(branch)
|
||||
head_ref = await git_service._resolve_head_ref(workspace, branch, token=token)
|
||||
|
||||
# Get log with format. Don't raise if the branch doesn't exist in
|
||||
# this workspace yet — that's a normal race (branch created in a
|
||||
# different agent's clone, not yet fetched here). Return empty.
|
||||
@@ -216,7 +268,7 @@ async def get_git_log(
|
||||
log_format = "%H%x1f%h%x1f%s%x1f%an%x1f%aI"
|
||||
log_result = await git_service._run_git(
|
||||
workspace,
|
||||
["log", f"--format={log_format}", f"-n{limit}", branch],
|
||||
["log", f"--format={log_format}", f"-n{limit}", head_ref],
|
||||
check=False,
|
||||
)
|
||||
if log_result.returncode != 0:
|
||||
@@ -253,6 +305,28 @@ async def get_git_log(
|
||||
)
|
||||
|
||||
|
||||
def _parse_branch_line(line: str) -> tuple[str, bool, str | None] | None:
|
||||
"""Classify one `%(refname)|%(objectname:short)` line as (name, is_remote,
|
||||
last_commit), or None for skippable entries (blank, origin/HEAD, other ref
|
||||
namespaces). Full refname, not `:short` — a remote-tracking ref shortens to
|
||||
`origin/<branch>`, indistinguishable from a local branch literally named
|
||||
that; classify on the `refs/heads/` vs `refs/remotes/` prefix instead.
|
||||
"""
|
||||
if not line:
|
||||
return None
|
||||
parts = line.split("|")
|
||||
ref = parts[0]
|
||||
last_commit = parts[1] if len(parts) > 1 else None
|
||||
if ref.startswith("refs/heads/"):
|
||||
return ref.removeprefix("refs/heads/"), False, last_commit
|
||||
if ref.startswith("refs/remotes/"):
|
||||
_remote_name, _, name = ref.removeprefix("refs/remotes/").partition("/")
|
||||
if not name or name == "HEAD":
|
||||
return None # origin/HEAD is a symbolic pointer, not a branch
|
||||
return name, True, last_commit
|
||||
return None
|
||||
|
||||
|
||||
@router.get("/branches", response_model=GitBranchListResponse)
|
||||
async def list_branches(
|
||||
db: DbSession,
|
||||
@@ -268,8 +342,12 @@ async def list_branches(
|
||||
workspace = await git_service.get_workspace(project_slug, agent.agent_id)
|
||||
current_branch = await git_service.get_current_branch(workspace)
|
||||
|
||||
# Get branches
|
||||
args = ["branch", "--format=%(refname:short)|%(objectname:short)"]
|
||||
if include_remote:
|
||||
# Self-heal orphaned remote-tracking refs (branches deleted
|
||||
# upstream via the forge API) before listing them.
|
||||
await git_service.prune_remote_best_effort(workspace)
|
||||
|
||||
args = ["branch", "--format=%(refname)|%(objectname:short)"]
|
||||
if include_remote:
|
||||
args.append("-a")
|
||||
|
||||
@@ -279,16 +357,10 @@ async def list_branches(
|
||||
|
||||
branches = []
|
||||
for line in branch_result.stdout.strip().split("\n"):
|
||||
if not line:
|
||||
parsed = _parse_branch_line(line)
|
||||
if parsed is None:
|
||||
continue
|
||||
parts = line.split("|")
|
||||
name = parts[0]
|
||||
last_commit = parts[1] if len(parts) > 1 else None
|
||||
|
||||
is_remote = name.startswith("remotes/")
|
||||
if is_remote:
|
||||
name = name.replace("remotes/origin/", "")
|
||||
|
||||
name, is_remote, last_commit = parsed
|
||||
branches.append(
|
||||
BranchInfo(
|
||||
name=name,
|
||||
@@ -315,49 +387,50 @@ async def get_git_diff(
|
||||
) -> GitDiffResponse:
|
||||
"""Get git diff for a project.
|
||||
|
||||
The diff + stat subprocesses run inside a bounded
|
||||
``settings.evidence_assembly_timeout_seconds`` guard — well under the
|
||||
outer gateway verb budget — so a genuinely slow diff computation (a
|
||||
huge working-tree change, a hung git process) returns a structured 504
|
||||
naming the slow component instead of hanging indefinitely.
|
||||
The diff + stat subprocess pair is wrapped in a bounded
|
||||
``evidence_assembly_timeout_seconds`` guard so a genuinely slow diff
|
||||
computation returns a structured 504 instead of hanging indefinitely.
|
||||
"""
|
||||
project_slug = await _resolve_project_slug(project_slug, db)
|
||||
git_service = get_git_service(db)
|
||||
|
||||
async def _run_diff_and_stat() -> tuple[Any, Any]:
|
||||
workspace = await git_service.get_workspace(project_slug, agent.agent_id)
|
||||
|
||||
args = ["diff"]
|
||||
if staged:
|
||||
args.append("--staged")
|
||||
if file_path:
|
||||
args.extend(["--", file_path])
|
||||
diff_res = await git_service._run_git(workspace, args)
|
||||
|
||||
# Count files changed
|
||||
stat_args = ["diff", "--stat"]
|
||||
if staged:
|
||||
stat_args.append("--staged")
|
||||
stat_res = await git_service._run_git(workspace, stat_args)
|
||||
return diff_res, stat_res
|
||||
timeout = settings.evidence_assembly_timeout_seconds
|
||||
|
||||
try:
|
||||
# Wrap workspace resolution + diff + stat subprocesses in one bounded
|
||||
# timeout so a slow workspace fetch or diff computation returns a
|
||||
# structured 504 instead of hanging indefinitely.
|
||||
async def _resolve_and_diff() -> tuple[Any, Any]:
|
||||
workspace = await git_service.get_workspace(project_slug, agent.agent_id)
|
||||
|
||||
args = ["diff"]
|
||||
if staged:
|
||||
args.append("--staged")
|
||||
if file_path:
|
||||
args.extend(["--", file_path])
|
||||
|
||||
stat_args = ["diff", "--stat"]
|
||||
if staged:
|
||||
stat_args.append("--staged")
|
||||
|
||||
diff_result, stat_result = await asyncio.gather(
|
||||
git_service._run_git(workspace, args),
|
||||
git_service._run_git(workspace, stat_args),
|
||||
)
|
||||
return diff_result, stat_result
|
||||
|
||||
diff_result, stat_result = await asyncio.wait_for(
|
||||
_run_diff_and_stat(),
|
||||
timeout=settings.evidence_assembly_timeout_seconds,
|
||||
_resolve_and_diff(), timeout=timeout
|
||||
)
|
||||
except TimeoutError as e:
|
||||
except _TranslatableError as e:
|
||||
raise _translate_error(e) from e
|
||||
except TimeoutError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_504_GATEWAY_TIMEOUT,
|
||||
detail=(
|
||||
"diff computation exceeded the "
|
||||
f"{settings.evidence_assembly_timeout_seconds:.0f}s bounded "
|
||||
"timeout — retry, or scope the request with file_path to a "
|
||||
"single file"
|
||||
f"git diff timed out after {timeout}s — the bounded "
|
||||
"evidence_assembly_timeout_seconds guard tripped"
|
||||
),
|
||||
) from e
|
||||
except _TranslatableError as e:
|
||||
raise _translate_error(e) from e
|
||||
) from None
|
||||
|
||||
files_changed = stat_result.stdout.count("\n") - 1 if stat_result.stdout else 0
|
||||
|
||||
@@ -370,6 +443,60 @@ async def get_git_diff(
|
||||
)
|
||||
|
||||
|
||||
@router.get("/file", response_model=GitFileContentResponse)
|
||||
async def get_git_file(
|
||||
*,
|
||||
db: DbSession,
|
||||
agent: CurrentAgentContext,
|
||||
branch: str = Query(..., description="Task branch holding the file"),
|
||||
path: str = Query(..., description="Repo-relative file path"),
|
||||
line: int | None = Query(default=None, ge=1, description="Target line"),
|
||||
context: int = Query(default=10, ge=0, le=100, description="Context around line"),
|
||||
start: int | None = Query(default=None, ge=1, description="Explicit start line"),
|
||||
end: int | None = Query(default=None, ge=1, description="Explicit end line"),
|
||||
) -> GitFileContentResponse:
|
||||
"""Return a file's content at a branch tip, optionally sliced to a range.
|
||||
|
||||
Reads straight out of the branch with ``git show`` (``read_file_at_branch``)
|
||||
so a reviewer/CEO can read a finding's source lines without a workspace
|
||||
mount. A missing file or bad ref yields 404. When `line` is given the
|
||||
slice is centered on it (`line - context` .. `line + context`); explicit
|
||||
`start`/`end` override. With none of the three, the whole file returns
|
||||
(capped at 2000 lines to bound the payload — `truncated` flags the cut).
|
||||
"""
|
||||
git_service = get_git_service(db)
|
||||
|
||||
try:
|
||||
content = await git_service.read_file_at_branch(
|
||||
branch_name=branch, path=path, actor_agent_id=agent.agent_id
|
||||
)
|
||||
except _TranslatableError as e:
|
||||
raise _translate_error(e) from e
|
||||
|
||||
if content is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"File not found at {branch}:{path}",
|
||||
)
|
||||
|
||||
all_lines = content.splitlines()
|
||||
total = len(all_lines)
|
||||
|
||||
s, e_, truncated = _compute_file_range(
|
||||
total=total, line=line, context=context, start=start, end=end
|
||||
)
|
||||
|
||||
sliced = all_lines[s - 1 : e_]
|
||||
return GitFileContentResponse(
|
||||
branch=branch,
|
||||
path=path,
|
||||
content="\n".join(sliced),
|
||||
start_line=s,
|
||||
total_lines=total,
|
||||
truncated=truncated,
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# WRITE ENDPOINTS
|
||||
# =============================================================================
|
||||
@@ -688,7 +815,7 @@ async def rebase_branch(
|
||||
try:
|
||||
workspace = await git_service.get_workspace(project_slug, agent.agent_id)
|
||||
conflict, conflicted_files = await git_service.rebase(
|
||||
workspace, data.target_branch
|
||||
workspace, data.target_branch, project_slug
|
||||
)
|
||||
except _TranslatableError as e:
|
||||
raise _translate_error(e) from e
|
||||
@@ -698,3 +825,60 @@ async def rebase_branch(
|
||||
conflict=conflict,
|
||||
conflicted_files=conflicted_files,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/branches/cleanup", response_model=GitBranchCleanupResponse)
|
||||
@guard_deco.rate_limit(requests=5, window=60)
|
||||
@guard_deco.max_request_size(size_bytes=65536)
|
||||
@guard_deco.block_clouds()
|
||||
@guard_deco.content_type_filter(["application/json"])
|
||||
async def cleanup_stale_branches(
|
||||
data: GitBranchCleanupRequest,
|
||||
db: DbSession,
|
||||
agent: CurrentAgentContext,
|
||||
) -> GitBranchCleanupResponse:
|
||||
"""Sweep a project's terminal-task branches (PM/CEO only).
|
||||
|
||||
Deletes the remote + local branch of every completed/cancelled task in
|
||||
the project (capped per call — see ``GitService.cleanup_stale_branches``),
|
||||
skipping the default branch and any environment-ladder rung so a live
|
||||
integration/prod branch is never touched. Role-gated identically to
|
||||
``/rebase`` — a history-affecting bulk operation shouldn't be open to
|
||||
developers either. Purely a read + external-git-op endpoint: no task rows
|
||||
are mutated, so there's nothing for this request to commit.
|
||||
"""
|
||||
if agent.role not in _REBASE_ALLOWED_ROLES:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_403_FORBIDDEN,
|
||||
detail=(
|
||||
f"BRANCH_CLEANUP_ROLE_RESTRICTED: Role '{agent.role}' is not "
|
||||
"permitted to sweep branches. Only CEO and PM roles (cell_pm, "
|
||||
"main_pm) may use this endpoint."
|
||||
),
|
||||
)
|
||||
project_slug = await _resolve_project_slug(data.project_slug, db)
|
||||
git_service = get_git_service(db)
|
||||
|
||||
try:
|
||||
(
|
||||
remote_deleted,
|
||||
local_deleted,
|
||||
skipped,
|
||||
errors,
|
||||
truncated,
|
||||
next_cursor,
|
||||
) = await git_service.cleanup_stale_branches(
|
||||
project_slug, after_task_id=data.after_cursor
|
||||
)
|
||||
except _TranslatableError as e:
|
||||
raise _translate_error(e) from e
|
||||
|
||||
return GitBranchCleanupResponse(
|
||||
project_slug=project_slug,
|
||||
remote_deleted=remote_deleted,
|
||||
local_deleted=local_deleted,
|
||||
skipped=skipped,
|
||||
errors=errors,
|
||||
truncated=truncated,
|
||||
next_cursor=next_cursor,
|
||||
)
|
||||
|
||||
+23
-25
@@ -38,7 +38,7 @@ class Settings(BaseSettings):
|
||||
# ==========================================================================
|
||||
# Application
|
||||
# ==========================================================================
|
||||
app_version: str = "0.27.0"
|
||||
app_version: str = "0.28.0"
|
||||
debug: bool = False
|
||||
environment: str = Field(
|
||||
default="development", pattern="^(development|staging|production)$"
|
||||
@@ -1649,6 +1649,28 @@ class Settings(BaseSettings):
|
||||
"legitimate verbs are unaffected."
|
||||
),
|
||||
)
|
||||
evidence_assembly_timeout_seconds: float = Field(
|
||||
default=90.0,
|
||||
ge=1.0,
|
||||
description=(
|
||||
"Bounded inner-work timeout for the evidence-assembly segment of "
|
||||
"claim_review / evidence() / roboco_git_diff — git diff/fetch, "
|
||||
"conventions validation, and the DB reads that accompany them. "
|
||||
"Kept comfortably below flow_verb_timeout_seconds so a slow "
|
||||
"assembly returns a structured gateway_timeout envelope naming "
|
||||
"the stalled component instead of hitting the outer 120s rollback."
|
||||
),
|
||||
)
|
||||
conventions_validator_timeout_seconds: int = Field(
|
||||
default=45,
|
||||
ge=1,
|
||||
description=(
|
||||
"Timeout for the conventions-validator subprocess "
|
||||
"(python -m roboco.conventions check), kept comfortably below "
|
||||
"evidence_assembly_timeout_seconds so a hung validator can't by "
|
||||
"itself exhaust the outer verb's budget."
|
||||
),
|
||||
)
|
||||
flow_verb_slow_timeout_seconds: int = Field(
|
||||
default=900,
|
||||
ge=1,
|
||||
@@ -1706,30 +1728,6 @@ class Settings(BaseSettings):
|
||||
"repository; set this to sandbox smoke-test projects."
|
||||
),
|
||||
)
|
||||
evidence_assembly_timeout_seconds: float = Field(
|
||||
default=90.0,
|
||||
ge=1.0,
|
||||
description=(
|
||||
"Bounded inner-work timeout for the evidence-assembly segment of "
|
||||
"claim_review / evidence() / roboco_git_diff (git fetch, diff "
|
||||
"computation, conventions validation, DB reads). Well under "
|
||||
"flow_verb_timeout_seconds (120s default) so a genuinely slow "
|
||||
"sub-component returns a structured, named gateway_timeout "
|
||||
"envelope instead of the whole verb hitting the outer server-side "
|
||||
"rollback with no indication of which segment stalled."
|
||||
),
|
||||
)
|
||||
conventions_validator_timeout_seconds: int = Field(
|
||||
default=45,
|
||||
ge=5,
|
||||
description=(
|
||||
"Timeout in seconds for the conventions-validator subprocess "
|
||||
"(``python -m roboco.conventions check``) run inside "
|
||||
"claim_review / i_am_done / pr_pass. Kept comfortably below "
|
||||
"evidence_assembly_timeout_seconds so a hung validator can't by "
|
||||
"itself exhaust the outer verb's whole server-side budget."
|
||||
),
|
||||
)
|
||||
|
||||
# ==========================================================================
|
||||
# Agent Guardrails (per-session budgets, loop detection, SLAs)
|
||||
|
||||
@@ -49,16 +49,21 @@ from roboco.foundation.policy import tracing as _tr
|
||||
from roboco.foundation.policy.content import (
|
||||
ContentValidationError,
|
||||
markers,
|
||||
validate_findings,
|
||||
)
|
||||
from roboco.services.content_notes import apply_structured_note
|
||||
from roboco.services.gateway.choreographer import findings as findings_lib
|
||||
from roboco.services.gateway.choreographer._protocol import actor_context_fields
|
||||
from roboco.services.gateway.choreographer.collision import build_collision_context
|
||||
from roboco.services.gateway.envelope import Envelope
|
||||
from roboco.services.gateway.evidence_builder import build_evidence_for_task
|
||||
|
||||
logger = structlog.get_logger()
|
||||
|
||||
# Cap on one criteria_verified entry's `evidence` — mirrors Finding.fix's cap
|
||||
# (roboco.foundation.policy.content.models._FINDING_FIX_CAP): a pointer
|
||||
# (file:line, screenshot ref, test name), not a transcript.
|
||||
_CRITERION_EVIDENCE_CAP = 500
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from uuid import UUID
|
||||
|
||||
@@ -177,37 +182,32 @@ class QAMixin(_Base):
|
||||
t = await self.task.qa_claim(qa_agent_id, task_id)
|
||||
await self.task.mark_evidence_inspected(task_id)
|
||||
|
||||
# Bounded-timeout wrapper around evidence assembly: the claim itself
|
||||
# (qa_claim + mark_evidence_inspected) has already committed by this
|
||||
# point, so a trip here means only evidence assembly is slow. The
|
||||
# agent gets a structured gateway_timeout envelope pointing at
|
||||
# evidence(task_id) (which re-runs the slow path but not the claim)
|
||||
# instead of the outer 120s rollback with no indication of which
|
||||
# piece stuck.
|
||||
timeout = settings.evidence_assembly_timeout_seconds
|
||||
try:
|
||||
ev = await asyncio.wait_for(
|
||||
self._build_qa_claim_evidence(qa_agent_id, t, task_id),
|
||||
timeout=settings.evidence_assembly_timeout_seconds,
|
||||
timeout=timeout,
|
||||
)
|
||||
except TimeoutError:
|
||||
# The claim itself already committed (qa_claim + mark_evidence_
|
||||
# inspected above) — well under flow_verb_timeout_seconds's own
|
||||
# ceiling. Only evidence assembly (git diff/fetch, conventions
|
||||
# validation, or a DB read) is slow, so name it and point at the
|
||||
# cheap fallback instead of a bare 504 that looks like the whole
|
||||
# claim failed.
|
||||
return await self._emit_rejection(
|
||||
Envelope.gateway_timeout(
|
||||
component=(
|
||||
"evidence assembly (git diff/fetch, conventions "
|
||||
"validation, or a journal/findings DB read)"
|
||||
),
|
||||
timeout_seconds=settings.evidence_assembly_timeout_seconds,
|
||||
remediate=(
|
||||
"the claim already succeeded (status is awaiting_qa, "
|
||||
"you are the claimant) — call evidence(task_id) to "
|
||||
"fetch the PR diff/files/findings directly instead "
|
||||
"of retrying claim_review"
|
||||
),
|
||||
context_briefing=briefing,
|
||||
).with_introspection(task=t, role=role_str),
|
||||
agent_id=qa_agent_id,
|
||||
task_id=task_id,
|
||||
verb="claim_review",
|
||||
)
|
||||
return Envelope.gateway_timeout(
|
||||
component="git diff/fetch or a journal/ancestor/findings DB read",
|
||||
timeout_seconds=timeout,
|
||||
remediate=(
|
||||
"the claim itself already committed — call "
|
||||
"evidence(task_id) to re-fetch the PR diff, "
|
||||
"journal highlights, and findings without re-running "
|
||||
"the claim; or retry claim_review after a short wait"
|
||||
),
|
||||
task_id=str(task_id),
|
||||
context_briefing=briefing,
|
||||
).with_introspection(task=t, role=role_str)
|
||||
return Envelope.ok(
|
||||
status=str(t.status),
|
||||
task_id=str(task_id),
|
||||
@@ -217,55 +217,155 @@ class QAMixin(_Base):
|
||||
).with_introspection(task=t, role=role_str)
|
||||
|
||||
async def _qa_convention_findings(
|
||||
self,
|
||||
qa_agent_id: UUID,
|
||||
t: Any,
|
||||
*,
|
||||
changed_files: list[str] | None = None,
|
||||
self, qa_agent_id: UUID, t: Any
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Convention-validator findings on the task's changed files (flag-gated).
|
||||
|
||||
Empty when the subsystem is off; a validator that could not run surfaces
|
||||
a single explicit ``could_not_run`` entry rather than being dropped, so
|
||||
QA never mistakes a silent failure for a clean diff.
|
||||
|
||||
``changed_files``, when given, is passed through to
|
||||
``conventions_check_for_task`` so it skips its own redundant
|
||||
``list_changed_files`` call (a third re-derivation of the same list
|
||||
on one claim_review request).
|
||||
"""
|
||||
if not settings.conventions_enabled:
|
||||
return []
|
||||
result = await self.git.conventions_check_for_task(
|
||||
qa_agent_id, t, changed_files=changed_files
|
||||
)
|
||||
result = await self.git.conventions_check_for_task(qa_agent_id, t)
|
||||
if result.get("could_not_run"):
|
||||
reason = result.get("reason") or "validator could not run"
|
||||
return [{"could_not_run": True, "reason": reason}]
|
||||
return list(result.get("findings", []))
|
||||
|
||||
@staticmethod
|
||||
def _qa_video_context(t: Any) -> dict[str, Any] | None:
|
||||
"""QA-facing artifact context for a video-authoring task.
|
||||
|
||||
None for every non-video task. Carries the composition id (from the
|
||||
``video_draft`` marker) + the latest ``request_render`` preview so QA
|
||||
is pointed at the rendered artifact instead of the source alone.
|
||||
"""
|
||||
if getattr(t, "source", None) != markers.VIDEO_TASK_SOURCE:
|
||||
return None
|
||||
draft = markers.get_video_draft(t) or {}
|
||||
return {
|
||||
"composition_id": draft.get("composition_id"),
|
||||
"render_preview": markers.get_render_preview(t),
|
||||
"note": (
|
||||
"This task ships a rendered video. Call request_render to "
|
||||
"render the PR branch state, then Read every returned frame "
|
||||
"image — verify each acceptance criterion's scene appears "
|
||||
"fully and legibly. Do not pass on source reading alone."
|
||||
),
|
||||
}
|
||||
|
||||
async def _build_qa_claim_evidence(
|
||||
self, qa_agent_id: UUID, t: Any, task_id: UUID
|
||||
) -> Any:
|
||||
"""Assemble the inline evidence payload returned by claim_review.
|
||||
|
||||
Bundles files_changed + pr_diff_summary (both from git, the
|
||||
authoritative source) + journal_highlights so the QA agent has
|
||||
the full PR context up-front and can't miss a piece.
|
||||
|
||||
files_changed comes from ``git.diff_and_files`` (a single call
|
||||
that resolves the shared workspace/token/head/base state once and
|
||||
runs the diff + --name-only subprocesses concurrently) instead of
|
||||
the legacy ``work_session.files_modified`` path. The git+conventions
|
||||
segment and the DB-reads segment run concurrently via
|
||||
``asyncio.gather`` so the git fetch overlaps the DB queries instead
|
||||
of running serially.
|
||||
"""
|
||||
git_coro = self._qa_git_and_conventions(qa_agent_id, t)
|
||||
db_coro = self._qa_db_reads(task_id, t)
|
||||
(
|
||||
(diff_summary, files_changed, convention_findings),
|
||||
(
|
||||
journal_highlights,
|
||||
parent_context,
|
||||
open_findings,
|
||||
prior_findings,
|
||||
),
|
||||
) = await asyncio.gather(git_coro, db_coro)
|
||||
|
||||
# The collision map: surfaced siblings (same parent) that would
|
||||
# collide with this task, with the declared-vs-actual drift (QA has
|
||||
# the real touched files in hand). Best-effort — a fetch failure omits
|
||||
# the block rather than breaking claim_review.
|
||||
collision_context: list[dict[str, Any]] | None = None
|
||||
try:
|
||||
if t.parent_task_id:
|
||||
siblings = await self.task.get_subtasks(t.parent_task_id)
|
||||
collision_context = build_collision_context(
|
||||
task=t, siblings=siblings, actual_files=files_changed
|
||||
)
|
||||
except Exception as exc: # best-effort enrichment, never breaks the verb
|
||||
logger.warning(
|
||||
"qa_collision_context_skip", task_id=str(t.id), error=str(exc)
|
||||
)
|
||||
return build_evidence_for_task(
|
||||
t,
|
||||
journal_highlights=journal_highlights,
|
||||
files_changed=files_changed,
|
||||
pr_diff_summary=diff_summary,
|
||||
convention_findings=convention_findings,
|
||||
revision_findings=open_findings,
|
||||
prior_findings=prior_findings,
|
||||
parent_context=parent_context,
|
||||
collision_context=collision_context,
|
||||
video_context=self._qa_video_context(t),
|
||||
)
|
||||
|
||||
async def _qa_git_and_conventions(
|
||||
self, qa_agent_id: UUID, t: Any
|
||||
) -> tuple[str, list[str], list[dict[str, Any]]]:
|
||||
"""Git diff/fetch + conventions validation segment of claim_review.
|
||||
|
||||
Returns ``(diff_summary, files_changed, convention_findings)``.
|
||||
Uses ``diff_and_files`` (single resolution, concurrent subprocesses)
|
||||
and passes the file list straight into ``conventions_check_for_task``
|
||||
so the changed-file list is computed once, not three times.
|
||||
"""
|
||||
git_start = time.monotonic()
|
||||
files_changed: list[str] = []
|
||||
diff_summary = ""
|
||||
convention_findings: list[dict[str, Any]] = []
|
||||
if t.branch_name:
|
||||
diff_summary, files_changed = await self.git.diff_and_files(
|
||||
branch_name=t.branch_name, actor_agent_id=qa_agent_id
|
||||
)
|
||||
# Pass the file list through so conventions_check_for_task
|
||||
# doesn't trigger a redundant list_changed_files call.
|
||||
if not settings.conventions_enabled:
|
||||
convention_findings = []
|
||||
else:
|
||||
result = await self.git.conventions_check_for_task(
|
||||
qa_agent_id,
|
||||
t,
|
||||
changed_files=files_changed,
|
||||
)
|
||||
if result.get("could_not_run"):
|
||||
reason = result.get("reason") or "validator could not run"
|
||||
convention_findings = [{"could_not_run": True, "reason": reason}]
|
||||
else:
|
||||
convention_findings = list(result.get("findings", []))
|
||||
git_ms = (time.monotonic() - git_start) * 1000
|
||||
logger.info(
|
||||
"claim_review git+conventions timing",
|
||||
git_diff_and_fetch_ms=round(git_ms, 1),
|
||||
task_id=str(t.id),
|
||||
)
|
||||
return diff_summary, files_changed, convention_findings
|
||||
|
||||
async def _qa_db_reads(
|
||||
self, task_id: UUID, t: Any
|
||||
) -> tuple[list[dict[str, Any]], list[dict[str, Any]], list[Any], list[Any]]:
|
||||
"""The four DB reads behind claim_review's evidence, run sequentially
|
||||
as ONE coroutine.
|
||||
) -> tuple[Any, Any, list[Any], list[Any]]:
|
||||
"""DB-reads segment of claim_review — runs concurrently with git work.
|
||||
|
||||
They stay sequential AMONG THEMSELVES on purpose: they all read
|
||||
through the request-scoped ``self.task.session`` / evidence_repo, a
|
||||
single ``AsyncSession`` bound to one DBAPI connection — asyncpg
|
||||
cannot serve two in-flight queries on the same connection at once
|
||||
(``asyncpg.exceptions`` / SQLAlchemy's own
|
||||
``IllegalStateChangeError: ... concurrent operations are not
|
||||
permitted``, confirmed against a live Postgres while building this
|
||||
fix; opening a second ad-hoc session per read would silently diverge
|
||||
from the request's transaction and break every existing test that
|
||||
injects a mocked ``evidence_repo``). Bundling them as one coroutine
|
||||
lets ``_build_qa_claim_evidence`` still `asyncio.gather` this WHOLE
|
||||
batch against the independent git + conventions-validator work
|
||||
below — the actual "parallelize the independent reads" win, without
|
||||
the same-connection hazard.
|
||||
Returns ``(journal_highlights, parent_context, open_findings,
|
||||
prior_findings)``. Reads stay sequential *within* this coroutine on
|
||||
purpose: they all read through the same request-scoped AsyncSession,
|
||||
and asyncpg cannot serve two in-flight queries on one connection at
|
||||
once (IllegalStateChangeError) — the win is git-work-vs-DB-work
|
||||
overlap via ``asyncio.gather``, not fanning out the DB reads.
|
||||
"""
|
||||
t0 = time.monotonic()
|
||||
db_start = time.monotonic()
|
||||
journal_highlights = await self.evidence_repo.journal_highlights_for_task(
|
||||
task_id
|
||||
)
|
||||
@@ -280,90 +380,14 @@ class QAMixin(_Base):
|
||||
prior_findings = await findings_lib.full_ledger_for_task(
|
||||
self.task.session, t.id
|
||||
)
|
||||
db_ms = (time.monotonic() - db_start) * 1000
|
||||
logger.info(
|
||||
"claim_review db reads timing",
|
||||
task_id=str(task_id),
|
||||
db_reads_ms=round((time.monotonic() - t0) * 1000.0),
|
||||
db_reads_ms=round(db_ms, 1),
|
||||
task_id=str(t.id),
|
||||
)
|
||||
return journal_highlights, parent_context, open_findings, prior_findings
|
||||
|
||||
async def _qa_git_and_conventions(
|
||||
self, qa_agent_id: UUID, t: Any
|
||||
) -> tuple[str, list[str], list[dict[str, Any]]]:
|
||||
"""The git diff/fetch + conventions-validator segment, as ONE
|
||||
coroutine so it can run concurrently with the DB-reads batch.
|
||||
|
||||
The conventions check needs ``files_changed`` from the git segment,
|
||||
so within this coroutine it still runs AFTER git — but that chain as
|
||||
a whole now overlaps the independent DB reads (item d of the
|
||||
claim_review timeout fix), instead of the old strictly-serial
|
||||
diff -> list_changed_files -> conventions(list_changed_files again)
|
||||
-> DB reads chain.
|
||||
"""
|
||||
files_changed: list[str] = []
|
||||
diff_summary = ""
|
||||
git_t0 = time.monotonic()
|
||||
if t.branch_name:
|
||||
diff_summary, files_changed = await self.git.diff_and_files(
|
||||
branch_name=t.branch_name
|
||||
)
|
||||
git_ms = (time.monotonic() - git_t0) * 1000.0
|
||||
conv_t0 = time.monotonic()
|
||||
convention_findings = await self._qa_convention_findings(
|
||||
qa_agent_id, t, changed_files=files_changed
|
||||
)
|
||||
conv_ms = (time.monotonic() - conv_t0) * 1000.0
|
||||
logger.info(
|
||||
"claim_review git+conventions timing",
|
||||
task_id=str(getattr(t, "id", None)),
|
||||
git_diff_and_fetch_ms=round(git_ms),
|
||||
conventions_ms=round(conv_ms),
|
||||
)
|
||||
return diff_summary, files_changed, convention_findings
|
||||
|
||||
async def _build_qa_claim_evidence(
|
||||
self, qa_agent_id: UUID, t: Any, task_id: UUID
|
||||
) -> Any:
|
||||
"""Assemble the inline evidence payload returned by claim_review.
|
||||
|
||||
Bundles files_changed + pr_diff_summary (both from git, the
|
||||
authoritative source) + journal_highlights so the QA agent has
|
||||
the full PR context up-front and can't miss a piece.
|
||||
|
||||
files_changed comes from the combined ``diff_and_files``
|
||||
accessor instead of ``work_session.files_modified``. The legacy
|
||||
``add_files_modified`` HTTP path that populated files_modified
|
||||
is not called by the gateway ``commit()``, so the work_session
|
||||
list was always empty — QA saw no files even on real PRs.
|
||||
|
||||
The git+conventions segment and the DB-reads segment run
|
||||
concurrently via ``asyncio.gather`` (each internally sequential per
|
||||
``_qa_db_reads``'s docstring); per-segment timing is logged inside
|
||||
each helper (item a of the claim_review timeout fix).
|
||||
"""
|
||||
(
|
||||
(diff_summary, files_changed, convention_findings),
|
||||
(
|
||||
journal_highlights,
|
||||
parent_context,
|
||||
open_findings,
|
||||
prior_findings,
|
||||
),
|
||||
) = await asyncio.gather(
|
||||
self._qa_git_and_conventions(qa_agent_id, t),
|
||||
self._qa_db_reads(task_id, t),
|
||||
)
|
||||
return build_evidence_for_task(
|
||||
t,
|
||||
journal_highlights=journal_highlights,
|
||||
files_changed=files_changed,
|
||||
pr_diff_summary=diff_summary,
|
||||
convention_findings=convention_findings,
|
||||
revision_findings=open_findings,
|
||||
prior_findings=prior_findings,
|
||||
parent_context=parent_context,
|
||||
)
|
||||
|
||||
async def _verify_qa_owner(
|
||||
self, qa_agent_id: UUID, task_id: UUID, verb: str
|
||||
) -> tuple[Envelope | None, Any]:
|
||||
@@ -502,13 +526,13 @@ class QAMixin(_Base):
|
||||
def _qa_ac_coverage_check(
|
||||
cls, task: Any, ac_verdicts: list[str] | None
|
||||
) -> Envelope | None:
|
||||
"""Per-acceptance-criterion verification gate for pass_review.
|
||||
"""Legacy count-only per-AC gate — superseded by ``criteria_verified``.
|
||||
|
||||
QA may not pass a task until it has recorded a verification for EVERY
|
||||
acceptance criterion. A single gestalt "looks good" approval is how a
|
||||
silently-unbuilt criterion slips through; requiring one verdict per
|
||||
criterion forces QA to check each individually. If a criterion does not
|
||||
hold, the QA fails the review instead of passing a partial.
|
||||
No longer wired into ``pass_review`` (a count of arbitrary strings
|
||||
never verified they actually named the right criterion — the live gap
|
||||
``_validate_criteria_verified`` closes). Kept for ``ac_verdicts``'
|
||||
existing callers/tests; ``ac_verdicts`` itself still folds into the
|
||||
persisted notes when supplied.
|
||||
"""
|
||||
criteria = list(getattr(task, "acceptance_criteria", None) or [])
|
||||
if not criteria:
|
||||
@@ -620,36 +644,181 @@ class QAMixin(_Base):
|
||||
verb=verb,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _parse_criterion_entry(entry: Any, idx: int) -> tuple[str, str] | Envelope:
|
||||
"""Validate one ``criteria_verified`` entry into a (criterion, evidence)
|
||||
pair, or an ``Envelope`` rejection on any structural problem."""
|
||||
criterion = entry.get("criterion") if isinstance(entry, dict) else None
|
||||
evidence = entry.get("evidence") if isinstance(entry, dict) else None
|
||||
if not isinstance(criterion, str) or not criterion.strip():
|
||||
return Envelope.invalid_state(
|
||||
message=f"criteria_verified[{idx}] is missing a `criterion` string",
|
||||
remediate=(
|
||||
"each entry needs {criterion, evidence} naming one "
|
||||
"acceptance criterion"
|
||||
),
|
||||
)
|
||||
if not isinstance(evidence, str) or not evidence.strip():
|
||||
return Envelope.invalid_state(
|
||||
message=(
|
||||
f"criteria_verified[{idx}] ({criterion!r}) is missing `evidence`"
|
||||
),
|
||||
remediate=(
|
||||
"state concrete evidence: file:line, screenshot ref, "
|
||||
"rendered-frame path, test name"
|
||||
),
|
||||
)
|
||||
if len(evidence) > _CRITERION_EVIDENCE_CAP:
|
||||
return Envelope.invalid_state(
|
||||
message=(
|
||||
f"criteria_verified[{idx}] evidence exceeds "
|
||||
f"{_CRITERION_EVIDENCE_CAP} chars"
|
||||
),
|
||||
remediate="keep evidence concise — a pointer, not a transcript",
|
||||
)
|
||||
return criterion.strip(), evidence.strip()
|
||||
|
||||
@classmethod
|
||||
def _parse_criteria_verified_entries(
|
||||
cls, criteria_verified: list[dict[str, Any]]
|
||||
) -> tuple[list[tuple[str, str]], Envelope | None]:
|
||||
"""Shape + soup validation for every ``criteria_verified`` entry.
|
||||
|
||||
Pure parsing — AC matching/coverage is the caller's job. Split out
|
||||
of ``_validate_criteria_verified`` to keep its return count under
|
||||
the complexity bound.
|
||||
"""
|
||||
pairs: list[tuple[str, str]] = []
|
||||
for idx, entry in enumerate(criteria_verified):
|
||||
parsed = cls._parse_criterion_entry(entry, idx)
|
||||
if isinstance(parsed, Envelope):
|
||||
return [], parsed
|
||||
pairs.append(parsed)
|
||||
soup = cls._free_text_soup(
|
||||
checks=(("criteria_verified.evidence", [e for _, e in pairs], 8),)
|
||||
)
|
||||
if soup is not None:
|
||||
return [], soup
|
||||
return pairs, None
|
||||
|
||||
@classmethod
|
||||
def _validate_criteria_verified(
|
||||
cls, t: Any, criteria_verified: list[dict[str, Any]] | None
|
||||
) -> tuple[list[tuple[str, str]], Envelope | None]:
|
||||
"""Mandatory per-AC verification gate for pass_review.
|
||||
|
||||
Returns ``(pairs, rejection)`` — ``pairs`` is ``[]`` and ``rejection``
|
||||
non-None on any failure: none supplied (lists every AC verbatim),
|
||||
a malformed or soupy entry, an entry naming a criterion absent from
|
||||
the task (names the valid criteria), or a task AC left uncovered
|
||||
(names the gap). No task ACs imposes no requirement (mirrors the
|
||||
legacy ``_qa_ac_coverage_check``). A gestalt "looks good" is no
|
||||
longer enough — every criterion needs its own matched, evidenced
|
||||
entry, or QA must call ``fail_review`` instead of passing a partial.
|
||||
"""
|
||||
criteria = list(getattr(t, "acceptance_criteria", None) or [])
|
||||
if not criteria:
|
||||
return [], None
|
||||
if not criteria_verified:
|
||||
return [], Envelope.invalid_state(
|
||||
message=(
|
||||
"pass_review needs criteria_verified naming every "
|
||||
f"acceptance criterion; none supplied. Unverified: {criteria!r}"
|
||||
),
|
||||
remediate=(
|
||||
"re-run the review and call pass_review with "
|
||||
"criteria_verified=[{criterion, evidence}, ...] — stamp "
|
||||
"EACH criterion with concrete evidence (file:line, "
|
||||
"screenshot ref, rendered-frame path, test name)"
|
||||
),
|
||||
)
|
||||
pairs, bad = cls._parse_criteria_verified_entries(criteria_verified)
|
||||
if bad is not None:
|
||||
return [], bad
|
||||
provided = [c for c, _ in pairs]
|
||||
if unknown := findings_lib.unmatched_criteria(t, provided):
|
||||
return [], Envelope.invalid_state(
|
||||
message=(
|
||||
f"criteria_verified names criteria not on this task: {unknown!r}"
|
||||
),
|
||||
remediate=(
|
||||
"each entry's criterion must match one of the task's "
|
||||
f"acceptance criteria (by id or exact text): {criteria!r}"
|
||||
),
|
||||
)
|
||||
if uncovered := findings_lib.uncovered_acceptance_criteria(t, provided):
|
||||
return [], Envelope.invalid_state(
|
||||
message=(
|
||||
"criteria_verified is missing these acceptance criteria: "
|
||||
f"{uncovered!r}"
|
||||
),
|
||||
remediate=(
|
||||
"stamp every criterion with concrete evidence, or call "
|
||||
"fail_review with the specific gap if one does not hold"
|
||||
),
|
||||
)
|
||||
return pairs, None
|
||||
|
||||
@staticmethod
|
||||
def _render_criteria_verified(pairs: list[tuple[str, str]]) -> list[str]:
|
||||
"""One '[AC] <criterion> — verified: <evidence>' line per entry.
|
||||
|
||||
Style-matched to the findings ledger's '[F-<id8>] ...' bracket-tag
|
||||
rendering (``findings_lib.render_finding_line``).
|
||||
"""
|
||||
return [
|
||||
f"[AC] {criterion} — verified: {evidence}" for criterion, evidence in pairs
|
||||
]
|
||||
|
||||
@classmethod
|
||||
def _merge_criteria_verified_into_notes(
|
||||
cls, notes: str, pairs: list[tuple[str, str]]
|
||||
) -> str:
|
||||
"""Fold the per-AC verification lines into the persisted QA notes.
|
||||
|
||||
Mirrors ``_merge_ac_verdicts_into_notes`` — keeps the per-criterion
|
||||
verification in the audit trail (qa_notes) so PM/CEO see exactly how
|
||||
QA verified each acceptance criterion.
|
||||
"""
|
||||
lines = cls._render_criteria_verified(pairs)
|
||||
if not lines:
|
||||
return notes
|
||||
return f"{notes}\n\n" + "\n".join(lines)
|
||||
|
||||
async def _qa_pass_final_gates(
|
||||
self,
|
||||
qa_agent_id: UUID,
|
||||
task_id: UUID,
|
||||
t: Any,
|
||||
role_str: str,
|
||||
ac_verdicts: list[str] | None,
|
||||
) -> Envelope | None:
|
||||
"""AC-coverage + toolchain-runnability gates for pass_review.
|
||||
criteria_verified: list[dict[str, Any]] | None,
|
||||
) -> tuple[Envelope | None, list[tuple[str, str]]]:
|
||||
"""Per-AC verification + toolchain-runnability gates for pass_review.
|
||||
|
||||
Returns the first rejection (already emitted), else None. QA must not
|
||||
PASS on a workspace that cannot run the suite — that is a source-read
|
||||
Returns ``(rejection, pairs)`` — the first emitted rejection (else
|
||||
None) and the validated ``criteria_verified`` (criterion, evidence)
|
||||
pairs for the caller to render into notes. QA must not PASS on a
|
||||
workspace that cannot run the suite — that is a source-read
|
||||
"verification"; fail_review is unaffected.
|
||||
"""
|
||||
ac_rejection = self._qa_ac_coverage_check(t, ac_verdicts)
|
||||
if ac_rejection is not None:
|
||||
return await self._emit_rejection(
|
||||
ac_rejection.with_introspection(task=t, role=role_str),
|
||||
pairs, bad = self._validate_criteria_verified(t, criteria_verified)
|
||||
if bad is not None:
|
||||
rejection = await self._emit_rejection(
|
||||
bad.with_introspection(task=t, role=role_str),
|
||||
agent_id=qa_agent_id,
|
||||
task_id=task_id,
|
||||
verb="pass_review",
|
||||
)
|
||||
return rejection, []
|
||||
if toolchain := await self._toolchain_broken_guard(qa_agent_id, t):
|
||||
return await self._emit_rejection(
|
||||
rejection = await self._emit_rejection(
|
||||
toolchain.with_introspection(task=t, role=role_str),
|
||||
agent_id=qa_agent_id,
|
||||
task_id=task_id,
|
||||
verb="pass_review",
|
||||
)
|
||||
return None
|
||||
return rejection, []
|
||||
return None, pairs
|
||||
|
||||
async def pass_review(
|
||||
self,
|
||||
@@ -657,6 +826,7 @@ class QAMixin(_Base):
|
||||
task_id: UUID,
|
||||
notes: str,
|
||||
ac_verdicts: list[str] | None = None,
|
||||
criteria_verified: list[dict[str, Any]] | None = None,
|
||||
) -> Envelope:
|
||||
"""QA passes the task; transitions awaiting_qa → awaiting_documentation.
|
||||
|
||||
@@ -671,6 +841,16 @@ class QAMixin(_Base):
|
||||
The composed atomic ``qa_pass`` is then dispatched through
|
||||
``VerbRunner.run_intent``, after which the verb body reassigns
|
||||
the documenter for handoff.
|
||||
|
||||
``criteria_verified`` ({criterion, evidence} entries) is the
|
||||
mandatory per-AC verification gate (``_validate_criteria_verified``):
|
||||
every one of the task's acceptance criteria must be named by exactly
|
||||
one entry — matched by AC id or exact text, the same match
|
||||
``fail_review``'s findings ledger uses for its own ``criterion``
|
||||
field — carrying substantive evidence, or the pass is refused. A
|
||||
gestalt "looks good" is no longer enough; QA must walk each
|
||||
criterion. ``ac_verdicts`` (legacy, count-only) still folds into the
|
||||
persisted notes when supplied but no longer gates the pass.
|
||||
"""
|
||||
rejection, t = await self._verify_qa_owner(qa_agent_id, task_id, "pass_review")
|
||||
if rejection is not None:
|
||||
@@ -696,18 +876,22 @@ class QAMixin(_Base):
|
||||
soup_checks=(("notes", notes, 8),),
|
||||
):
|
||||
return gate_rejection
|
||||
if rej := await self._qa_pass_final_gates(
|
||||
qa_agent_id, task_id, t, role_str, ac_verdicts
|
||||
):
|
||||
return rej
|
||||
final_rejection, criteria_pairs = await self._qa_pass_final_gates(
|
||||
qa_agent_id, task_id, t, role_str, criteria_verified
|
||||
)
|
||||
if final_rejection is not None:
|
||||
return final_rejection
|
||||
|
||||
briefing = await self._briefing_for(qa_agent_id, task_id)
|
||||
merged_notes = self._merge_criteria_verified_into_notes(
|
||||
self._merge_ac_verdicts_into_notes(notes, ac_verdicts), criteria_pairs
|
||||
)
|
||||
spec_ctx = spec_module.Context(
|
||||
actor_id=qa_agent_id,
|
||||
actor_slug=getattr(agent, "slug", None) if agent is not None else None,
|
||||
agent_team=str(agent.team) if agent is not None and agent.team else None,
|
||||
original_developer_slug=_extract_original_developer(t),
|
||||
notes=self._merge_ac_verdicts_into_notes(notes, ac_verdicts),
|
||||
notes=merged_notes,
|
||||
)
|
||||
self._store_qa_note(t, notes, ac_verdicts, passed=True)
|
||||
runner = self._verb_runner()
|
||||
@@ -803,16 +987,9 @@ class QAMixin(_Base):
|
||||
)
|
||||
if cap := findings_lib.findings_count_guard(raw):
|
||||
return [], cap
|
||||
try:
|
||||
validated = validate_findings(raw)
|
||||
except ContentValidationError as exc:
|
||||
return [], Envelope.invalid_state(
|
||||
message=f"malformed finding: {exc.field} — {exc.reason}",
|
||||
remediate=(
|
||||
"each finding needs expected + actual (file/line/severity/"
|
||||
"criterion/fix/evidence optional)"
|
||||
),
|
||||
)
|
||||
validated, bad = findings_lib.validate_or_reject(raw)
|
||||
if bad is not None:
|
||||
return [], bad
|
||||
if unknown := findings_lib.unknown_finding_criteria(t, validated):
|
||||
return [], findings_lib.criterion_mismatch_rejection(t, unknown)
|
||||
return validated, None
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -245,19 +245,27 @@ class Envelope:
|
||||
component: str,
|
||||
timeout_seconds: float,
|
||||
remediate: str,
|
||||
task_id: str | None = None,
|
||||
context_briefing: dict[str, Any] | None = None,
|
||||
) -> Envelope:
|
||||
"""A bounded inner-work timeout tripped (e.g. evidence assembly),
|
||||
well under the outer server-side ``flow_verb_timeout_seconds``
|
||||
rollback (``api.middleware``'s own ``gateway_timeout``). Names the
|
||||
slow ``component`` so the agent isn't left guessing which segment
|
||||
(git fetch, diff computation, conventions validation, a DB read)
|
||||
stalled, instead of a bare 504 that reads as the whole verb failing.
|
||||
"""A bounded inner-work timeout tripped — the evidence-assembly segment
|
||||
(git diff/fetch, conventions validation, or a DB read) exceeded
|
||||
``evidence_assembly_timeout_seconds`` without completing.
|
||||
|
||||
Distinct from the outer ``flow_verb_timeout_seconds`` rollback: that
|
||||
one cancels the whole request transaction with no indication of which
|
||||
piece stuck. This returns a structured, named error so the agent (and
|
||||
the audit log) sees exactly which component stalled and can retry or
|
||||
escalate accordingly. On a trip the verb's own state change (e.g.
|
||||
``qa_claim``) has already committed, so the remediate points at
|
||||
``evidence(task_id)`` rather than re-running the whole verb.
|
||||
"""
|
||||
return cls(
|
||||
error="gateway_timeout",
|
||||
task_id=task_id,
|
||||
message=(
|
||||
f"{component} exceeded the {timeout_seconds:.0f}s bounded timeout"
|
||||
f"evidence assembly timed out after {timeout_seconds}s — "
|
||||
f"the {component} segment did not complete in time"
|
||||
),
|
||||
remediate=remediate,
|
||||
context_briefing=context_briefing or {},
|
||||
|
||||
+1501
-585
File diff suppressed because it is too large
Load Diff
@@ -32,6 +32,9 @@ def _make_deps(task_svc: Any = None) -> ChoreographerDeps:
|
||||
"audit": AsyncMock(),
|
||||
"evidence_repo": AsyncMock(),
|
||||
}
|
||||
# diff_and_files returns a (diff_summary, files_changed) 2-tuple; without
|
||||
# this the await resolves to a MagicMock and qa.py:330 unpacking fails.
|
||||
base["git"].diff_and_files.return_value = ("", [])
|
||||
repo = base["evidence_repo"]
|
||||
for method in (
|
||||
"list_unread_a2a",
|
||||
|
||||
@@ -41,10 +41,12 @@ def _over_cap_project() -> MagicMock:
|
||||
|
||||
|
||||
def _make_deps(task_svc: AsyncMock, **overrides: Any) -> ChoreographerDeps:
|
||||
git = AsyncMock()
|
||||
git.diff_and_files.return_value = ("", [])
|
||||
base: dict[str, Any] = {
|
||||
"task": task_svc,
|
||||
"work_session": AsyncMock(),
|
||||
"git": AsyncMock(),
|
||||
"git": git,
|
||||
"a2a": AsyncMock(),
|
||||
"journal": AsyncMock(),
|
||||
"audit": AsyncMock(),
|
||||
|
||||
@@ -37,7 +37,7 @@ def _make_deps(**overrides: AsyncMock) -> ContentActionsDeps:
|
||||
else:
|
||||
git = AsyncMock()
|
||||
git.commit.return_value = {"sha": "abc12345"}
|
||||
git.diff.return_value = ""
|
||||
git.diff_and_files.return_value = ("", [])
|
||||
|
||||
a2a = overrides.get("a2a", AsyncMock())
|
||||
journal = overrides.get("journal", AsyncMock())
|
||||
@@ -270,7 +270,7 @@ async def test_evidence_blocks_when_not_assignee() -> None:
|
||||
task_svc = AsyncMock()
|
||||
task_svc.get.return_value = task_obj
|
||||
git_svc = AsyncMock()
|
||||
git_svc.diff.return_value = ""
|
||||
git_svc.diff_and_files.return_value = ("", [])
|
||||
workspace_svc = AsyncMock()
|
||||
deps = _make_deps(task=task_svc, git=git_svc, workspace=workspace_svc)
|
||||
ca = ContentActions(deps)
|
||||
@@ -279,7 +279,7 @@ async def test_evidence_blocks_when_not_assignee() -> None:
|
||||
body = env.as_dict()
|
||||
assert body["error"] == "not_authorized"
|
||||
workspace_svc.fetch_branch_for_inspection.assert_not_awaited()
|
||||
git_svc.diff.assert_not_awaited()
|
||||
git_svc.diff_and_files.assert_not_awaited()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -410,7 +410,7 @@ async def test_evidence_unassigned_task_allows_inspection() -> None:
|
||||
task_svc = AsyncMock()
|
||||
task_svc.get.return_value = task_obj
|
||||
git_svc = AsyncMock()
|
||||
git_svc.diff.return_value = "diff content"
|
||||
git_svc.diff_and_files.return_value = ("diff content", [])
|
||||
workspace_svc = AsyncMock()
|
||||
deps = _make_deps(task=task_svc, git=git_svc, workspace=workspace_svc)
|
||||
ca = ContentActions(deps)
|
||||
@@ -446,8 +446,7 @@ async def test_evidence_allows_dependency_inspection() -> None:
|
||||
task_svc.get.return_value = target
|
||||
task_svc.list_assigned_for_agent.return_value = [callers_task]
|
||||
git_svc = AsyncMock()
|
||||
git_svc.diff.return_value = ""
|
||||
git_svc.list_changed_files.return_value = []
|
||||
git_svc.diff_and_files.return_value = ("", [])
|
||||
workspace_svc = AsyncMock()
|
||||
deps = _make_deps(task=task_svc, git=git_svc, workspace=workspace_svc)
|
||||
ca = ContentActions(deps)
|
||||
|
||||
@@ -7,7 +7,7 @@ Bug:
|
||||
commit's delta, even when GitHub showed a multi-commit change set.
|
||||
|
||||
Fix:
|
||||
Pull files via ``git.list_changed_files(branch_name=...)`` (no base
|
||||
Pull files via ``git.diff_and_files(branch_name=...)`` (no base
|
||||
→ full diff vs parent branch). Pull diff with ``base=None`` so the
|
||||
full PR diff comes through. Both use git as the authoritative source
|
||||
instead of the legacy ``work_session.files_modified`` field, which
|
||||
@@ -65,8 +65,10 @@ async def test_evidence_populates_files_changed_from_git() -> None:
|
||||
task_svc = AsyncMock()
|
||||
task_svc.get.return_value = _task_with_pr(task_id, commits=["abc", "def"])
|
||||
git_svc = AsyncMock()
|
||||
git_svc.diff.return_value = "diff --git a/README.md b/README.md\n+added line\n"
|
||||
git_svc.list_changed_files.return_value = ["README.md", "docs/guide.md"]
|
||||
git_svc.diff_and_files.return_value = (
|
||||
"diff --git a/README.md b/README.md\n+added line\n",
|
||||
["README.md", "docs/guide.md"],
|
||||
)
|
||||
workspace_svc = AsyncMock()
|
||||
evidence_repo = AsyncMock()
|
||||
evidence_repo.journal_highlights_for_task.return_value = []
|
||||
@@ -80,12 +82,12 @@ async def test_evidence_populates_files_changed_from_git() -> None:
|
||||
assert body["error"] is None
|
||||
assert body["evidence"]["files_changed"] == ["README.md", "docs/guide.md"]
|
||||
assert "diff --git" in body["evidence"]["pr_diff_summary"]
|
||||
git_svc.list_changed_files.assert_awaited_once()
|
||||
git_svc.diff_and_files.assert_awaited_once()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_evidence_uses_full_pr_diff_not_head_minus_one() -> None:
|
||||
"""git.diff must be called with base=None (full PR diff vs parent),
|
||||
"""diff_and_files must be called with base=None (full PR diff vs parent),
|
||||
not base='HEAD~1' (only the last commit)."""
|
||||
agent_id = uuid4()
|
||||
task_id = uuid4()
|
||||
@@ -94,8 +96,7 @@ async def test_evidence_uses_full_pr_diff_not_head_minus_one() -> None:
|
||||
# task.commits was non-empty, masking earlier commits' changes.
|
||||
task_svc.get.return_value = _task_with_pr(task_id, commits=["sha1", "sha2", "sha3"])
|
||||
git_svc = AsyncMock()
|
||||
git_svc.diff.return_value = "full diff"
|
||||
git_svc.list_changed_files.return_value = []
|
||||
git_svc.diff_and_files.return_value = ("full diff", [])
|
||||
workspace_svc = AsyncMock()
|
||||
evidence_repo = AsyncMock()
|
||||
evidence_repo.journal_highlights_for_task.return_value = []
|
||||
@@ -105,13 +106,13 @@ async def test_evidence_uses_full_pr_diff_not_head_minus_one() -> None:
|
||||
)
|
||||
await ca.evidence(agent_id=agent_id, task_id=task_id)
|
||||
|
||||
git_svc.diff.assert_awaited_once()
|
||||
call_kwargs = git_svc.diff.await_args.kwargs
|
||||
git_svc.diff_and_files.assert_awaited_once()
|
||||
call_kwargs = git_svc.diff_and_files.await_args.kwargs
|
||||
# Pre-fix bug: kwargs['base'] would be 'HEAD~1' for any multi-commit
|
||||
# branch. Post-fix: base is omitted (or explicitly None).
|
||||
base = call_kwargs.get("base")
|
||||
assert base in (None, ""), (
|
||||
f"git.diff must use full-PR diff (base=None), got base={base!r}"
|
||||
f"diff_and_files must use full-PR diff (base=None), got base={base!r}"
|
||||
)
|
||||
assert call_kwargs.get("branch_name") == "feature/backend/abc12345--def67890"
|
||||
|
||||
@@ -125,8 +126,7 @@ async def test_evidence_populates_journal_highlights() -> None:
|
||||
task_svc = AsyncMock()
|
||||
task_svc.get.return_value = _task_with_pr(task_id, commits=["abc"])
|
||||
git_svc = AsyncMock()
|
||||
git_svc.diff.return_value = ""
|
||||
git_svc.list_changed_files.return_value = []
|
||||
git_svc.diff_and_files.return_value = ("", [])
|
||||
workspace_svc = AsyncMock()
|
||||
evidence_repo = AsyncMock()
|
||||
highlights = [
|
||||
@@ -179,5 +179,4 @@ async def test_evidence_no_branch_skips_git_calls() -> None:
|
||||
assert body["error"] is None
|
||||
assert body["evidence"]["files_changed"] == []
|
||||
assert body["evidence"]["pr_diff_summary"] == ""
|
||||
git_svc.diff.assert_not_awaited()
|
||||
git_svc.list_changed_files.assert_not_awaited()
|
||||
git_svc.diff_and_files.assert_not_awaited()
|
||||
|
||||
@@ -107,8 +107,7 @@ async def test_claim_review_evidence_carries_prior_findings(
|
||||
task_svc.list_paused_for_agent.return_value = []
|
||||
task_svc.qa_claim.return_value = t_claimed
|
||||
git_svc = AsyncMock()
|
||||
git_svc.diff.return_value = ""
|
||||
git_svc.list_changed_files.return_value = []
|
||||
git_svc.diff_and_files.return_value = ("", [])
|
||||
deps = _make_deps(task=task_svc, git=git_svc)
|
||||
c = Choreographer(deps)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user