2c403c77a2
* fix(git): don't delete a branch that still has open dependent PRs
Root cause of the run-zombifying "integration branch gone from origin" wedge.
_delete_remote_branch_best_effort deleted a merged PR's head branch
unconditionally, so:
- merging a cell->root PR deleted the cell branch while a sibling leaf PR was
still targeting it as base, and
- the CEO's root->master merge deleted the feature/main_pm/{root} integration
branch.
The dependent PRs lost their base, every later git op against the vanished
branch failed, and the task zombified (a51c3d31 only made the post-merge sync
non-fatal; this removes the cause).
The remote-branch delete chokepoint (the single path all merge/close/cancel
deletions funnel through) now first checks _branch_has_open_dependents: any OPEN
PR targeting the branch as its base marks it an active integration target and
preserves it. Fails safe (any error => keep the branch; cleanup is best-effort,
stranding is not). True leaf branches with no open dependents are still cleaned
up. Adds 6 unit tests for the guard + the probe.
* fix(git): recover a drifted shared clone on resume instead of BRANCH_MISMATCH
A dev/documenter/QA clone is shared across that agent's tasks. On a
respawn/resume it can sit on a sibling task's branch, or a re-provisioned clone
can lack the task branch as a local ref (commits only on origin). The
fresh-claim path git-resets the clone clean, but resume deliberately
short-circuits before it (_dev_reentry), so the agent's next commit hit
_assert_on_task_branch's BRANCH_MISMATCH, failed, and the task wedged in a
blocked respawn loop (the documenter that could never land its doc commit).
_assert_on_task_branch now recovers instead of only rejecting: fetch + checkout
the task branch (recreating a missing local ref from origin via `git branch
<b> origin/<b>`), and raise only when the switch genuinely can't happen
(uncommitted changes block it). Never discards work — checkout, not reset — so
a resumed agent's unpushed commits are preserved. Updates the RAG troubleshooting
+ developer docs to describe the auto-recovery. Adds 5 unit tests.
* fix(runtime): re-adopt running agent containers on restart (no double-spawn)
An orchestrator restart loses the in-memory _instances registry while the agent
containers keep running. The reaper already had a Docker-liveness fallback
(_assignee_container_running), but the spawn gate (_is_agent_active) did not, so
right after a restart it saw a live agent as inactive and could launch a second
container onto work the forgotten-but-running one was already doing.
start() now calls _readopt_running_agents() after _reconcile_orphan_claims_on_startup
and before the dispatcher/reaper loops launch: it probes each known agent slug's
container (AGENT_IMAGES, reusing _inspect_container_state — the same docker
inspect the reaper uses) and registers a minimal AgentInstance(state=ACTIVE) for
any that is running and not already tracked. Inert when nothing runs (cold start
unchanged); best-effort (a probe error leaves that slot for the reaper's own
fallback). This is the gateway-health spec's Task 4 / the orchestrator-state
spec's Phase 3 (_instances reconcile). Adds 4 unit tests.
* fix(git): treat an already-merged PR as idempotent success on merge
A merge PUT against an already-merged PR returns the same 405 as a genuine
"not mergeable" conflict, so _merge_with_retry raised MergeConflictError and the
completion path tried to rebase / close-superseded / escalate a PR that had
already landed (a prior cycle, a sibling, or the CEO merged it) — the
cell_pm_complete block<->unblock respawn loop.
_merge_with_retry now disambiguates before raising: a new _pr_is_merged probe
(GET the PR, check merged==true) returns success on an already-merged PR so
completion proceeds idempotently; a genuinely-unmerged 405 still raises the
conflict. Best-effort probe (False on any error → falls through to the existing
conflict handling). Adds 4 unit tests.
---------
Co-authored-by: Renn F <rennf93@users.noreply.github.com>
5.5 KiB
Git Error Troubleshooting
Missing Git Token
Error: Project requires a git token for HTTPS repositories (also surfaces as WorkspaceError during clone)
Cause: No encrypted GitHub PAT on projects.git_token_encrypted for this project.
Fix:
- Open the project's settings tab in the panel
- Paste a GitHub Personal Access Token with
reposcope - Save — the panel encrypts and stores it; the API never returns the plaintext
Notes:
- Each project has its own token (no global fallback)
- Tokens are encrypted at rest with Fernet
- The token is injected only at the MCP layer (commit / clone / PR ops);
.git/configis scrubbed post-clone so a leaked PAT from there is not a recovery path
Workspace Not Found
Error: Workspace does not exist
Cause: Workspace not cloned yet (or ROBOCO_WORKSPACE_AUTO_CLONE is false and no manual clone has run).
Fix:
- If
ROBOCO_WORKSPACE_AUTO_CLONE=true(default), the first MCP verb that touches the workspace will trigger the clone. Just call your next verb (i_will_work_on,commit, etc.). - Otherwise check
ROBOCO_WORKSPACE_CLONE_TIMEOUTand the orchestrator logs for a stuck clone.
BRANCH_MISMATCH
Error envelope: Workspace is on '<other-branch>' but task requires '<task-branch>'
Cause: You're trying to act on task A while your workspace is still on task B's branch — usually after a respawn/resume on a shared clone, or a re-provisioned clone that no longer has your task's branch locally.
Auto-recovery first: a commit-time check now recovers this for you — it re-checks out your task's branch (recreating a missing local ref from origin) before acting, without ever discarding your local commits. So you normally won't see this error on resume at all; just continue your work.
When you still see it: the only case left is uncommitted changes blocking the switch (the clone can't move off its current branch). Then:
commit(message=..., files=...)your in-progress work, or escalate viai_am_blockedif the changes aren't yours / are conflicting.- Re-call your role's claim verb on the intended task:
i_will_work_on(task_id)(devs),i_will_plan(task_id, plan)(PMs),claim_doc_task(task_id)(documenters),claim_review(task_id)(QA).
Don't checkout by hand — there is no roboco_git_checkout tool.
NO_COMMITS on open_pr
Cause: No commits on the task yet — the choreographer has nothing to open a PR over.
Fix: commit(message=..., files=...) at least once, then call open_pr(task_id) again.
Branch Behind Its Base / master
Symptom: roboco_git_status shows behind > 0 against the base branch when you go to submit.
Cause: The base (cell branch or master) advanced after your branch was cut, so your branch is stale.
Fix: You have no rebase verb — do not create a rebase task or improvise with shell git. Bringing the branch current is a platform/PM action. Escalate: devs i_am_blocked(reason="branch behind base — needs rebase"); PMs escalate_up(...).
src refspec does not match any (during open_pr)
Error: src refspec '<branch>' does not match any
Cause: A re-provisioned or shared clone can be missing the local branch ref even though the commits are already on origin.
Fix: Just call open_pr(task_id) again — the gateway recovers the ref from origin, or returns an explicit "unclaim and re-claim to rebuild" instruction. Follow whichever the envelope gives you; don't switch branches by hand.
Updates were rejected / non-fast-forward (on push)
Cause: Your branch is behind its remote, so the push can't fast-forward.
Fix: Escalate rather than improvise — devs i_am_blocked(...), PMs escalate_up(...). There is no agent-layer pull/rebase to reconcile it.
NO_PR on pass / fail
Cause: The PR was never created — usually because open_pr(task_id) did not run cleanly.
Fix: Roll back to the dev: have them re-call open_pr(task_id) after fixing whatever blocked the PR opening (see PR Creation Failed, below). QA cannot create the PR.
PR Creation Failed (during open_pr)
Causes:
- Nothing to push — no commits on the branch
- Branch is on the workspace but not pushed yet (rare; the choreographer pushes during
commit, but a stale workspace can drift) - Project has no git token configured
- The GitHub repo doesn't allow PRs from your branch (rare; usually org-level branch protection)
Fix:
- Verify commits exist with
roboco_git_log(project_slug=...) - Verify the project has a git token (Missing Git Token, above)
- If the task is in a stuck state,
unclaim(task_id)and re-claimto rebuild the branch
FORCE_PUSH_FORBIDDEN
Cause: Force-push is CEO-only. Anyone else attempting it (typically because their branch diverged) is denied.
Fix: unclaim(task_id) and re-claim it. The choreographer rebuilds the branch from the parent's HEAD; replay your commits with commit(...).
Merge Conflicts on complete
Cause: The leaf PR conflicts with the parent branch (cell branch or master).
Fix: This currently surfaces as an error envelope from complete. The recovery path:
- PM
unblock(task_id, restore=False)— frees the task back to the dev - Dev re-claims, the choreographer rebuilds the branch off the latest parent, and they replay their commits
- Dev
open_pragain - QA re-runs
pass(orfailif the rebase changed behaviour) - PM
completeagain
We don't expose a "resolve conflicts in place" path at the agent layer — rebuilds via the lifecycle are the recovery.