diff --git a/AGENTS.md b/AGENTS.md index 01457f7..10dacc7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -44,7 +44,7 @@ Drivers know apps, adapters know vendors, `local/` knows this project. Module map for `manager/core/` (dependencies flow strictly left to right): ``` -config → state / reports → taskfiles → events / github / drive / sync → agents → watch / httpd → board.py +config → state / reports → taskfiles → events / github / drive / sync → agents → phases → watch / httpd → board.py ``` - `config.py` — paths, stages, settings, prompt/adapter/driver resolution @@ -57,6 +57,8 @@ config → state / reports → taskfiles → events / github / drive / sync → - `drive.py` — runs the project driver, tracks the one live drive - `sync.py` — origin/main as the shared board: push on move, pull on a beat - `agents.py` — headless work/review jobs, launched through the adapter +- `phases.py` — a phase run: its own branch, its cards merged into it one + at a time. A beat, not an agent; it holds no registry of where a phase is - `watch.py` — 2s disk poller narrating moves made outside the API, and the gate that keeps a move a pull applied from triggering anything - `httpd.py` — HTTP routes, the SSE stream, serving the page @@ -468,6 +470,12 @@ resolution commit, and refuses semantic ones, naming the collision for a human to settle. GitHub computes mergeability lazily, so an UNKNOWN reading keeps the chip's last state rather than flapping. +A phase member's PR is opened against **its phase's branch**, not `main`: +its branch was cut from there, so that is the only base whose diff is the +member's own work — and a PR into `main` carrying a whole phase is exactly +the merge this design refuses to make. Every other card, the phase card +itself included, opens into `main` as it always did. + The card wears that verdict in the design system's state colours: approved → pine (`--calm`) border and an `approved` pill; changes asked → terracotta (`--alarm`) and a `changes asked` pill; otherwise it stays the @@ -475,7 +483,10 @@ neutral `waiting on you`. Tool chips (CI, copilot, PR, drive) are destinations, not statuses: they live in the card's footer row, never squeezed into the author row — `CI ✓` (pine), `CI ✕` (terracotta), `◌` while in flight — with hover actions staying in the status pill's slot. -Merging remains yours — the board never merges. +Merging into `main` remains yours — the board never merges into `main`. It +does merge into a branch of its own: a phase's integration branch is the +board's, and merging into it is bookkeeping in the same family as +committing a move (see "A phase runs itself, on a branch of its own"). The agent's first duty is to judge whether the task is actionable. If the task still has open questions — unresolved decisions only its author can @@ -793,7 +804,83 @@ that would otherwise surface later as a runner behaving oddly. `**Depends on:**` is the other half, and it guards rather than orders: the list says what runs next, a member's dependencies say whether it *may*. The board parses the numbers out of the line and shows them; acting on them -belongs to whatever runs a phase. +belongs to the runner below. + +### A phase runs itself, on a branch of its own + +Running a phase works its list into a single integration branch. Starting +one cuts `phase/` from the newest `origin/main` it can see — the +same rule and the same timeout a task branch is cut by — and gives it a +worktree beside the task worktrees. From there each member is branched +**from the phase's tip**, run headless exactly as **▸ start work** runs any +card, and merged back into the phase branch when its checks are green; +then the next one starts. At the end one PR, from the phase branch into +`main`, for a human. + +That is why the branch exists. Members of a phase are related by +definition, so card two branched from `main` could not see card one's work +while card one sat unmerged in `review/` — it would conflict, or quietly +build the same thing twice. Gating on a merge into `main` would fix the +branch point and destroy the point, because `main` is merged by a person +and the phase would stall on every card. So the human gate moves from every +card to the phase boundary, and the promise survives intact: the board +merges into a branch it created, inside a scope you opened, and `main` +still waits for your click. + +**The runner is a beat, not an agent.** Everything it decides is already +structured state — a card's stage, a PR's CI verdict, whether one branch is +contained in another — so an agent paid to poll would be the wrong tool at +the wrong price. It is a plain thread (`BOARD_PHASE_INTERVAL`, 30s), silent +when no phase is running. + +**The beat is stateless.** Each pass recomputes which members are finished, +which is first unfinished and what that one needs; it holds no registry of +where a phase *is*. Two durable things carry the memory instead, and both +are things the board already writes: **git**, where a member is finished +when its branch is contained in the phase branch, and **the phase card**, +which grows a `## Phase log` section the runner adds one line to per +decision — a run started, a member started, a member merged, a halt. The +log is the record a person reads, and the only thing that can tell "this +member has run and it ended badly" from "the phase has not reached it yet"; +without it a restarted board would relaunch a run that died. So a restart +resumes a phase by looking, and the same logic answers "what now?" whether +the last event was a launch, a merge or a crash. + +**Advance on green.** A member is finished when its card reaches `review/` +and its checks are not against it. Green is read from the same PR poll the +board already runs: red halts the phase, running holds it, and a member +with no checks at all advances — a project without CI must not deadlock +every phase it runs. A member with no branch at all that is already in +`review/` or `done/` is simply finished; there is nothing to bring. + +**Halt, never skip.** Five conditions stop a phase, each already a visible +state on the card: a member that exits `NOT READY`, a run that exits +non-zero, a clean exit that committed nothing, CI red, and a merge into the +phase branch that is not mechanical. A phase that stepped over a failed +card would build the rest on a foundation that never landed. The halt is +written into the log and then held — said once, not once a beat — and +nothing retries by itself. Running the phase again is a person's decision, +and it is what appends the line that clears the halt. A member whose +`**Depends on:**` names something unfinished is a *wait*, not a halt: the +phase idles until the dependency lands (merged, for a card inside the +phase; `done/`, for one outside). + +**Merges are additive, always.** Nothing here rebases and nothing +force-pushes. `main` is merged into the phase branch on every beat, so a +phase that runs for hours does not drift into one enormous conflict at the +end; a conflict there halts the phase like any other, aborted cleanly, with +the colliding files named. When every member is in, the branch is pushed, a +PR into `main` is opened with the member list as its body, the `**PR:**` +line is written into the phase card and the card moves to `review/` — where +the existing apparatus applies unchanged: the CI chip, **◔ review PR**, +**⚑ copilot**, and drag-to-`done/` for **merge & clean up**. + +**One board runs it.** State syncs; reactions don't, so the phase card's +**Assignee** is where "who runs it" is written down — the same claim that +gates starting work. A replica renders the phase and advances nothing. + +Members run one at a time. Running independent members in parallel, and +any UI beyond the API and the ticker, are separate cards. An optional **Assignee** line records who holds the card: diff --git a/manager/core/.env.example b/manager/core/.env.example index 1a2385e..ca16723 100644 --- a/manager/core/.env.example +++ b/manager/core/.env.example @@ -124,6 +124,14 @@ BOARD_COMMIT_MOVES= BOARD_SYNC= BOARD_SYNC_INTERVAL=30 +# Seconds between passes of the phase runner: a phase card being run +# (**Type:** Phase, in in-progress/) has its list worked into its own +# `phase/` branch one card at a time. Each pass recomputes what the +# phase needs from the cards and from git and does the one next thing, so +# raising this only makes a phase slower to notice, never wrong. Nothing +# runs at all while no phase is being run. +BOARD_PHASE_INTERVAL=30 + # Seconds between disk polls of the stage directories. BOARD_WATCH_INTERVAL=2 diff --git a/manager/core/agents.py b/manager/core/agents.py index 09d7f21..3c6e1ad 100644 --- a/manager/core/agents.py +++ b/manager/core/agents.py @@ -22,8 +22,17 @@ import config import events import reports import state -from taskfiles import (actor_name, append_to_task, find_stage_of, move_task, - read_task, set_assignee) +from taskfiles import (actor_name, append_to_task, collect, find_stage_of, + move_task, read_task, set_assignee) + +# A phase runs on an integration branch of its own — phases.py owns the +# behaviour, the name lives here because this is where a launch decides +# what to branch from. +PHASE_BRANCH_PREFIX = "phase/" + + +def phase_branch(phase_file: str) -> str: + return PHASE_BRANCH_PREFIX + phase_file[:-3] def _report_of(record: dict, text: str | None = None) -> str: @@ -201,7 +210,38 @@ def _fresh_branch_point() -> tuple[str | None, str | None]: return "origin/main", None -def _claim_for_launch(filename: str, stage: str, takeover: bool) -> None: +def phase_branch_point(filename: str) -> tuple[str | None, str | None]: + """Where a *member of a running phase* branches from: the phase's own + branch, not main. + + That is the whole reason a phase has a branch. Related cards run one + after another, so card two branched from main could not see card one's + work while card one sat unmerged in review/ — it would conflict, or + quietly build the same thing twice. Branching from the phase tip is + what makes the list add up. + + (None, None) for a card in no phase, or one whose phase has not been + started — then the ordinary fresh branch point applies. + """ + phase = None + try: + for stage in collect()["stages"]: + for task in stage["tasks"]: + if task["file"] == filename: + phase = task.get("phase") + except OSError: + return None, None + if not phase: + return None, None + branch = phase_branch(phase["file"]) + exists = subprocess.run(["git", "-C", str(config.REPO), "rev-parse", + "--verify", "--quiet", branch], capture_output=True) + if exists.returncode != 0: + return None, None + return branch, f"branched from {branch}, the phase's own branch" + + +def claim_for_launch(filename: str, stage: str, takeover: bool = False) -> None: """One agent per task is a board-memory rule; across machines the card file is the only thing every board can see, so the claim is what gates a launch here. @@ -238,7 +278,7 @@ def start_agent(filename: str, stage: str, takeover: bool = False) -> dict: # Moving a card to in-progress is the commitment; only then does work start. _validate(filename, stage, {"in-progress"}, "work starts from in-progress/ — move the card there first") - _claim_for_launch(filename, stage, takeover) + claim_for_launch(filename, stage, takeover) stem = filename[:-3] branch = f"task/{stem}" @@ -267,7 +307,11 @@ def start_agent(filename: str, stage: str, takeover: bool = False) -> dict: base = _git("merge-base", "main", branch).stdout.strip() result = _git("worktree", "add", str(worktree), branch) else: - point, base_note = _fresh_branch_point() + # A phase member starts from its phase's tip; everything else + # from the newest main this checkout can see. + point, base_note = phase_branch_point(filename) + if point is None: + point, base_note = _fresh_branch_point() if point: base = _git("rev-parse", point).stdout.strip() result = _git("worktree", "add", "--no-track", "-b", branch, diff --git a/manager/core/board.py b/manager/core/board.py index 556ec46..9bf289d 100755 --- a/manager/core/board.py +++ b/manager/core/board.py @@ -14,6 +14,7 @@ task files, but the tasks work as a plain folder kanban without it. See events.py hook payloads → displayable events, session registry sync.py origin/main as the shared board: push on move, pull on a beat agents.py headless work/review agents: launch, reap, stop, diff + phases.py a phase run: its own branch, its members merged into it watch.py 2s disk poller narrating moves made outside the API httpd.py HTTP routes, SSE stream, the page itself .prompts/ agent prompt templates (read fresh on every launch) @@ -34,6 +35,7 @@ import drive import events import github import httpd +import phases import state import sync import watch @@ -66,6 +68,7 @@ def main() -> None: threading.Thread(target=watch.watcher, daemon=True).start() threading.Thread(target=github.poller, daemon=True).start() threading.Thread(target=github.reconcile, daemon=True).start() + threading.Thread(target=phases.beat, daemon=True).start() if config.SYNC: # Team mode: board commits publish themselves and a beat pulls what # the other boards published. Off, neither thread nor hook exists. diff --git a/manager/core/config.py b/manager/core/config.py index 1563f6d..3e80f24 100644 --- a/manager/core/config.py +++ b/manager/core/config.py @@ -175,6 +175,11 @@ SYNC_INTERVAL = float(setting("BOARD_SYNC_INTERVAL", "30")) # tasks/ stays a hand job. COMMIT_MOVES = flag("BOARD_COMMIT_MOVES") or SYNC +# How often the phase runner takes a pass: recompute what each running +# phase needs and do the one next thing. A beat, not an agent — it costs a +# handful of git commands and nothing at all when no phase is running. +PHASE_INTERVAL = float(setting("BOARD_PHASE_INTERVAL", "30")) + WATCH_INTERVAL = float(setting("BOARD_WATCH_INTERVAL", "2")) EVENTS_CAP = int(setting("BOARD_EVENTS_CAP", "800")) BOARD_EVENTS_CAP = int(setting("BOARD_HISTORY_CAP", "300")) diff --git a/manager/core/github.py b/manager/core/github.py index f498b82..b17d105 100644 --- a/manager/core/github.py +++ b/manager/core/github.py @@ -27,7 +27,8 @@ import config import drive as drive_mod import reports import state -from taskfiles import STATUS_RE, commit_edit, find_stage_of, move_task, read_task +from taskfiles import (STATUS_RE, collect, commit_edit, find_stage_of, + move_task, read_task) PR_STATE: dict[str, dict] = {} # filename -> {verdict, detail, url, ts} _OPENING: set[str] = set() # filenames with a PR-open in flight @@ -52,6 +53,75 @@ def _branch_exists(branch: str) -> bool: return _run(["git", "rev-parse", "--verify", "--quiet", branch]).returncode == 0 +def branch_of(filename: str) -> str: + """The branch a card's work lives on. Ordinary cards get `task/`; + a phase runs on `phase/`, its own integration branch (phases.py), + and from review/ onwards it is reviewed, driven and completed through + the same apparatus as any other card.""" + phase = f"phase/{filename[:-3]}" + return phase if _branch_exists(phase) else f"task/{filename[:-3]}" + + +def _woven(filename: str, stage: str) -> dict: + """The card as the board shows it. A card's phase is *derived* across + the whole board (`taskfiles.weave_phases`), and both what a PR is based + on and what a phase's PR says depend on that reading, so a lone + `read_task` is not enough here.""" + for group in collect()["stages"]: + for task in group["tasks"]: + if task["file"] == filename: + return task + return read_task(config.TASKS / stage / filename, stage) + + +def _remote_has_branch(rname: str, branch: str) -> bool: + """Whether the remote already carries this branch. Sync only ever + fetches origin/main, so a phase branch the running board pushed is not a + local ref on any other board even when it is published — asking the + remote directly is the only way another board sees it.""" + return bool(_run(["git", "ls-remote", "--heads", rname, branch], + timeout=30).stdout.strip()) + + +def _pr_base(task: dict) -> str: + """What a PR is opened against. A phase member's branch was cut from + its phase's branch, so that is the only base whose diff is the member's + own work — a PR into main would carry the whole phase, and invite a + merge into main that this board exists not to make. Everything else, + the phase card included, goes into main. + + The phase branch is the board's own and is published as the phase runs, + so a board that did not run the phase may know it only through the + remote — that still makes it the base, not main.""" + phase = task.get("phase") + if phase: + branch = f"phase/{phase['file'][:-3]}" + if _branch_exists(branch): + return branch + rname = remote() + if rname and _remote_has_branch(rname, branch): + return branch + return "main" + + +def _pr_body(filename: str, task: dict) -> str: + """The PR's body. A phase's PR is the one a whole run produces, so it + says what is in it: the member list, in the order it ran.""" + if task.get("isPhase"): + cards = "\n".join(f"- {m['number']} — {m['title']}" + for m in task.get("members") or []) + return (f"Phase: `{filename}` — tracked in `.task-manager/tasks/review/`.\n\n" + f"Opened by the board when every card in the phase had been " + f"merged into its branch.\n\n## Cards in this phase\n\n" + f"{cards or '_none_'}\n") + body = (f"Task: `{filename}` — tracked in `.task-manager/tasks/review/`.\n\n" + f"Opened by the board when the card moved to review.") + summary = _agent_report(filename) + if summary: + body += f"\n\n## Agent summary\n\n{summary}" + return body + + def _write_pr_line(filename: str, url: str) -> None: """The url joins the header — and in team mode commits itself, so the gate that stops a second board opening a second PR travels to the other @@ -115,14 +185,14 @@ def open_pr_now(filename: str) -> str: def _open_pr(filename: str) -> str: - branch = f"task/{filename[:-3]}" + branch = branch_of(filename) if not _branch_exists(branch): # nothing to publish — a hand-moved card without agent work raise _Quiet(f"{filename} has no {branch} branch — nothing to open a PR from") stage = find_stage_of(filename) if stage != "review": raise _Quiet(f"{filename} is not in review/ — PRs open from there") - task = read_task(config.TASKS / stage / filename, stage) + task = _woven(filename, stage) if task.get("pr"): raise _Quiet(f"{filename} already has a PR: {task['pr']}") @@ -132,24 +202,29 @@ def _open_pr(filename: str) -> str: ("no git remote configured" if rname is None else "gh is not installed")) - # The PR's diff is computed against the remote main — refuse to open one - # that would drag unpushed main commits along with it. - _run(["git", "fetch", rname, "main"], timeout=120) - ahead = _run(["git", "rev-list", "--count", f"{rname}/main..main"]).stdout.strip() - if ahead.isdigit() and int(ahead) > 0: - raise ValueError(f"won't open a PR for {filename}: main is {ahead} commits " - f"ahead of {rname} — push main first, then move the card again") + base = _pr_base(task) + if base == "main": + # The PR's diff is computed against the remote main — refuse to open one + # that would drag unpushed main commits along with it. + _run(["git", "fetch", rname, "main"], timeout=120) + ahead = _run(["git", "rev-list", "--count", f"{rname}/main..main"]).stdout.strip() + if ahead.isdigit() and int(ahead) > 0: + raise ValueError(f"won't open a PR for {filename}: main is {ahead} commits " + f"ahead of {rname} — push main first, then move the card again") + elif _branch_exists(base): + # A phase branch is the board's own: publish it so the member's PR + # has a base on the remote to be opened against. When only the + # remote carries it — a member PR opened from a board that did not + # run the phase — it is already there, and there is nothing local + # to push. + _run(["git", "push", "-u", rname, base], timeout=180) push = _run(["git", "push", "-u", rname, branch], timeout=180) if push.returncode != 0: raise ValueError(f"push failed for {branch}: {push.stderr.strip()[:140]}") - body = (f"Task: `{filename}` — tracked in `.task-manager/tasks/review/`.\n\n" - f"Opened by the board when the card moved to review.") - summary = _agent_report(filename) - if summary: - body += f"\n\n## Agent summary\n\n{summary}" - result = _run([config.GH_BIN, "pr", "create", "--head", branch, "--base", "main", + body = _pr_body(filename, task) + result = _run([config.GH_BIN, "pr", "create", "--head", branch, "--base", base, "--title", task["title"], "--body", body], timeout=120) if result.returncode != 0: # The rare double-fire: two attempts crossed and GitHub already has @@ -417,7 +492,7 @@ def complete_task(filename: str, stage: str) -> dict: def _complete(filename: str, stage: str) -> dict: """The steps themselves, run under the claim complete_task holds.""" stem = filename[:-3] - branch = f"task/{stem}" + branch = branch_of(filename) # a phase's own branch, or task/ # 1. the app must not keep running code that is about to be merged away d = drive_mod.DRIVE @@ -512,11 +587,17 @@ def _merge_on_origin(filename: str, stage: str, branch: str) -> None: def task_branches() -> list[str]: - """Stems of all task/* branches — the UI uses this to say honestly - whether a review card has work attached.""" + """Stems of all task/* and phase/* branches — the UI uses this to say + honestly whether a card has work attached, and a phase card's work is + its own integration branch.""" result = _run(["git", "for-each-ref", "--format=%(refname:short)", - "refs/heads/task/"]) - return [ref[len("task/"):] for ref in result.stdout.split() if ref.startswith("task/")] + "refs/heads/task/", "refs/heads/phase/"]) + stems = [] + for ref in result.stdout.split(): + for prefix in ("task/", "phase/"): + if ref.startswith(prefix): + stems.append(ref[len(prefix):]) + return stems def reconcile() -> None: diff --git a/manager/core/httpd.py b/manager/core/httpd.py index 6edd014..379d9e9 100644 --- a/manager/core/httpd.py +++ b/manager/core/httpd.py @@ -17,6 +17,7 @@ import config import drive import events import github +import phases import state import sync import taskfiles @@ -34,6 +35,9 @@ def state_payload() -> dict: "sessions": sessions, "agents": agents.list_public(), "prs": github.public_state(), + # what the last pass of the phase runner saw: per running phase + # card, its branch and each member's state + "phases": phases.public_state(), "drive": drive.public(), "hasDriver": config.driver_path() is not None, "branches": github.task_branches(), @@ -190,6 +194,13 @@ class Handler(BaseHTTPRequestHandler): payload = self._read_body() agent = agents.start_pr_fix(payload["file"], payload["stage"]) self._json(200, {"agent": agent}) + elif path == "/api/phase/run": + payload = self._read_body() + # takeover carries the same meaning it does for a launch: + # the deliberate second click on someone else's card + self._json(200, {"phase": phases.start_phase( + payload["file"], payload["stage"], + bool(payload.get("takeover")))}) elif path == "/api/pr/open": payload = self._read_body() self._json(200, {"url": github.open_pr_now(payload["file"])}) diff --git a/manager/core/phases.py b/manager/core/phases.py new file mode 100644 index 0000000..58213ea --- /dev/null +++ b/manager/core/phases.py @@ -0,0 +1,605 @@ +"""A phase runs itself, on a branch of its own. + +A phase is a card that lists its cards (`taskfiles.weave_phases`). Running +one means working that list into a single integration branch: `phase/` +cut from the newest main, each member branched from the phase's tip, run +headless, merged back when its checks are green, and the next one started. +At the end one PR into `main`, for a human. The board never merges into +`main` — a phase branch is the board's own, and merging into it is +bookkeeping in the same family as committing a move. + +**The runner is a beat, not an agent.** Everything it decides is already +structured state — a card's stage, a PR's CI verdict, whether one branch is +contained in another — so an agent paid to poll would be the wrong tool at +the wrong price. + +**The beat is stateless.** On each pass it recomputes, from disk and from +git, which members are finished, which is first unfinished and what that +one needs. It holds no registry of where a phase "is": a restarted board +resumes a phase by looking, and the same logic answers "what now?" whether +the last event was a launch, a merge or a crash. Two things carry the +memory, and both are durable: + +- **git** — a member is merged when its branch is contained in the phase + branch. That is what makes a restart safe from repeating a merge. +- **the phase card** — a `## Phase log` section the runner appends one line + to per decision (a run started, a member started, a member merged, a + halt). It is the record a person reads, and the only thing that can tell + "the phase already started this member and its run ended badly" from "the + phase has not reached this member yet". Without it a restart would + silently relaunch a run that died. + +**Halt, never skip.** Five conditions stop a phase, each of them already a +visible state on the card: a member that declined (`NOT READY`), a run that +exited non-zero, a clean exit that committed nothing, CI red, and a merge +into the phase branch that is not mechanical. A phase that steps over a +failed card builds the rest on a foundation that never landed. Halting is +recorded in the log and nothing retries by itself; running the phase again +is a person's decision and appends the line that clears the halt. + +**One board runs it.** The actor rule decides, and the phase card's +**Assignee** is where it is written down — the same claim that gates +starting work. A replica renders the phase and advances nothing. +""" + +from __future__ import annotations + +import re +import subprocess +import threading +import time +from pathlib import Path + +import agents +import config +import github +import state +import taskfiles + +LOG_HEADING = "Phase log" + +# One log line: `-