Files
roboco/docs/optional/autonomous-maintenance.md
T
153723406e Feat/autonomous maintenance (#264)
* feat(ci-watch): config flags

Default-off CI-watch config (mirrors self_heal_*): ci_watch_enabled,
ci_watch_default_workflow (ci.yml), ci_watch_interval_seconds (1800),
ci_watch_max_open_tasks (3), ci_watch_max_per_cycle (1). Registers
ci_watch_enabled in the panel FEATURE_FLAGS. 4 tests.

* feat(ci-watch): per-project ci_watch_enabled/workflow (migration 048)

Adds projects.ci_watch_enabled (bool NOT NULL default false) +
projects.ci_watch_workflow (varchar null) — the per-project opt-in for
multi-repo CI-watch. ProjectTable + Pydantic Project fields + migration 048
(off 047_ws_single_active). Real upgrade->downgrade->upgrade chain verified
against a throwaway Postgres; 2 ORM round-trip tests.

* feat(runtime): prune dangling agent images in the background sweeper

Every agent-image rebuild orphans the prior build's layers as an untagged
<none> image; across deploys these pile up (the operator hit ~80). The sweeper
now runs 'docker image prune -f --filter dangling=true' (dangling only — a
tagged image or one backing a running container is never dangling), throttled
to settings.image_prune_interval_seconds (default 6h) and gated by
image_prune_enabled (default on). Best-effort: any failure is logged, never
raised into the sweeper. Mirrors the transcript-retention prune. 4 tests.

* feat(ci-watch): source tag + open-task dedupe query

CI_WATCH_SOURCE='ci_watch' + TaskService.list_open_ci_watch_tasks(git_url=None):
non-terminal ci_watch tasks (the dedupe + open-cap basis), optionally scoped to
one repo by git_url — a monorepo registers several cell-projects on one git_url,
so dedupe keys on the repo, not the slug. 2 real-PG tests.

* feat(ci-watch): multi-project CI telemetry fan-out

MultiProjectCITelemetrySource.fetch(projects) reuses the hardened per-project
get_latest_ci_conclusion for each opted-in project (passing its ci_watch_workflow
or the configured default). Per-project isolation: a GitHub error or absent
signal yields NO sample (unknown, never read as green) and never aborts the
sweep; only a real conclusion yields a sample (fail→breach, pass→non-breach).
self-heal source untouched. 3 tests + self-heal regression green.

* feat(ci-watch): engine — fan-out, originate, dedupe, cap

CiWatchEngine.run_cycle(projects) mirrors SelfHealEngine: assess via
MultiProjectCITelemetrySource, open one PENDING ci_watch fix task per red repo
(team=main_pm, assigned_to=main-pm, confirmed_by_human=True so it dispatches
without an Approve-&-Start — the fe029fe3 lesson), never starts/approves/merges.
Dedupe per git_url (monorepo → one fix task per repo) + per-cycle/rolling caps.
Default-off; disabled → no-op. 5 real-PG tests (red→one task, dedupe, cap,
green/none→nothing, disabled).

* feat(ci-watch): orchestrator loop tick + watch-set loader

_ci_watch_loop (registered in start(), cancelled in stop(), separate from the
untouched self-heal loop): dormant unless ci_watch_enabled; each interval loads
the watch set (ci_watch_enabled projects, collapsed one-per-repo via the
existing _projects_one_per_repo) and runs CiWatchEngine.run_cycle, committing
opened tasks. _run_ci_watch_cycle extracted for testing; loud warning when
enabled-but-empty. confirmed_by_human=True on the originated task means it
dispatches without an Approve-&-Start (no stranding, the fe029fe3 lesson).
5 tests (disabled no-op, watch-set filter+one-per-repo, empty warn, engine run).

* docs(ci-watch): CHANGELOG + CLAUDE.md for multi-repo CI-watch

Document CI-watch (Added) in the CHANGELOG and the Self-Healing & Feature Flags
section of CLAUDE.md — it generalizes self-heal to opted-in projects, reuses the
hardened per-project CI lookup, never auto-merges, default-off. Adds the
ci_watch_enabled flag to the feature-flags enumeration.

* feat(dep-update): config flags

Default-off dep-update config (mirrors self_heal_*/ci_watch_*): dep_update_enabled,
dep_update_interval_seconds (604800 = weekly), dep_update_max_open_tasks (3),
dep_update_max_per_cycle (1). Registers dep_update_enabled in FEATURE_FLAGS. 4 tests.

* feat(dep-update): per-project dep_update_command/paths (migration 049)

Adds projects.dep_update_command (varchar null) + dep_update_paths (varchar[]
null) — the per-project opt-in for the dependency-update bot. ProjectTable +
Pydantic Project fields + migration 049 (off 048_ci_watch_project_cols). Real
upgrade->downgrade->upgrade chain verified on a throwaway Postgres; 2 ORM tests.

* feat(dep-update): source tag + open-task dedupe query

DEP_UPDATE_SOURCE='dep_update' + TaskService.list_open_dep_update_tasks(git_url=None):
non-terminal dep_update tasks (dedupe + open-cap basis), optionally scoped to one
repo by git_url (monorepo → one open dependency-update task per repo). 2 real-PG
tests.

* feat(dep-update): read-only lockfile-diff probe

WorkspaceService.dry_upgrade_changes_lockfile(project): clones the project's
read clone into a throwaway dir (--no-hardlinks, so the read clone is never
mutated), runs project.dep_update_command (no shell, shlex.split), and reports
whether any lockfile path (dep_update_paths or inferred uv.lock/pnpm-lock.yaml)
is dirty. Fail-safe: null/failing command → False (don't originate on a broken
probe), logged; throwaway always removed; never commits/pushes. 5 real-git tests.

* feat(dep-update): engine — detect, originate, dedupe, cap

DepUpdateEngine.run_cycle(projects) mirrors SelfHealEngine/CiWatchEngine: for
each opted-in project (dep_update_command set) with updates available (the
read-only probe), open one PENDING dep_update task (team=main_pm, assigned-to
main-pm, confirmed_by_human=True), never starts/approves/merges. Cheap checks
(command, per-git_url dedupe) before the expensive probe; per-cycle + rolling
caps. Default-off; disabled → no-op. 6 real-PG tests.

* feat(dep-update): weekly orchestrator loop tick

_dep_update_loop (registered in start(), cancelled in stop(), separate from the
self-heal + CI-watch loops): dormant unless dep_update_enabled; each interval
(default weekly) loads projects with a dep_update_command (one-per-repo) and runs
DepUpdateEngine.run_cycle, committing opened tasks. _run_dep_update_cycle
extracted for testing; loud warning when enabled-but-no-commands. Refactored
stop() to cancel background tasks via a shared _cancel_background_task loop
(keeps it under xenon B as the loop count grows). 4 loop tests.

Task 7 (anti-stranding dispatch guard) is satisfied by construction: no
dispatcher skip targets source='dep_update', and the engine sets
confirmed_by_human=True (the fe029fe3 lesson), asserted in the engine tests —
so the originated task dispatches via the assigned-PM path, never stranded.

* docs(dep-update): CHANGELOG + CLAUDE.md for the dependency-update bot

Document the dep-update bot (Added) in the CHANGELOG and the Self-Healing &
Feature Flags section of CLAUDE.md — read-only lockfile-diff probe, never
auto-merges, per-project opt-in via dep_update_command, default-off. Adds the
dep_update_enabled flag to the feature-flags enumeration.

* feat(ci-watch): route fix-task notification to the project's cell PM

On opening a fix task, CiWatchEngine notifies the red project's own cell PM
(resolved from project.assigned_cell via foundation AGENTS — e.g. BACKEND →
be-pm), not the CEO, once per project per cycle. Best-effort: a notification
failure never rolls back the origination. Adds _cell_pm_slug_for +
_notify_cell_pm. 1 real-PG test (asserts to_agent='be-pm', not 'ceo').

* feat(ci-watch,dep-update): expose per-project opt-ins in the project API

Add ci_watch_enabled/ci_watch_workflow + dep_update_command/dep_update_paths to
ProjectUpdate, ProjectUpdateRequest, the PATCH route mapping, ProjectResponse,
and project_to_response — so the panel edit-project dialog can read + set the
per-project autonomy opt-ins (the columns were unreachable through the API
before). Also threads the previously-dropped quality_command through the update
route. 1 real-PG update round-trip test.

* feat(ci-watch,dep-update): panel project-edit fields for the per-project opt-ins

Adds an 'Autonomous Maintenance' section to the edit-project dialog: a CI-watch
enable switch + workflow input, and a dependency-update command + lockfile-paths
input (comma-separated → list). Threads the four fields through the Project /
ProjectUpdate TS types and the mock-mode create fixture. The global on/off
toggles already live in Settings → Feature Flags; these are the per-project
opt-ins. panel tsc --noEmit + eslint green.

* docs(0.12): CI-watch + dep-update bot + image-prune across user docs + RAG

New docs/optional/autonomous-maintenance.md (mirrors self-heal.md) covering both
engines; optional/index rows; panel settings + projects-and-products notes for
the Feature Flags toggles + the edit-project Autonomous Maintenance fields;
resilience note for the dangling-image prune; env-reference + RAG config-reference
tables for all ROBOCO_CI_WATCH_* / ROBOCO_DEP_UPDATE_* / ROBOCO_IMAGE_PRUNE_*
vars; mkdocs nav entry. reflow-check green; prompts unchanged (operator-facing,
not agent-facing).

* chore(release): 0.12.0

Cut [Unreleased] -> [0.12.0] (CI-watch + dep-update bot + image-prune housekeeping
+ the post-0.11.1 run-hardening fixes). Bumps all 8 canonical version refs to
0.12.0 (pyproject / uv.lock roboco pkg / panel package.json / __init__ /
config.app_version + the README / deployment / agent-image-tag examples).

* fix(pr-review): repo-scope external-PR dedupe (no duplicate review on a monorepo)

external_review_task_exists keyed on (project_id, pr, head_sha), but a monorepo
registers several cell-projects on one git_url and the poll already collapses to
one canonical project per repo — so once a review task was re-pointed to a
sibling project, the next poll (checking the canonical project) no longer saw it
and opened a second review of the same PR (observed: PR #131 reviewed once on
guard-core-saas-frontend, once on -backend). Dedupe now spans every project
sharing the PR's repo (git_url); re-review on a new head SHA still works; a
genuinely different repo with the same PR number is independent. 3 real-PG tests.

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
2026-06-25 21:11:36 +02:00

7.1 KiB

Autonomous maintenance

RoboCo can keep your projects healthy on their own schedule: it can watch each opted-in project's CI and open a fix task when it goes red, and it can periodically check whether a dependency upgrade would change a project's lockfiles and open an "update dependencies" task when it would. Both are per-project, both are off by default, and neither ever auto-merges — every task they open rides the normal dev → QA → PR review → CEO merge pipeline, exactly like any other task.

These two engines generalize the self-healing CI loop, which watches only RoboCo's own repository. Multi-repo CI-watch extends the same idea to any project you opt in.

Multi-repo CI-watch

CI-watch assesses each opted-in project's CI on its default branch. On a red conclusion it opens one fix task into that project and notifies that project's cell PM. It never starts that task, never merges it, and never deploys.

What it does

On each pass (ROBOCO_CI_WATCH_INTERVAL_SECONDS, default 1800s) the engine checks each opted-in project's latest CI conclusion on its default branch. On a red conclusion it opens one fix task into that project and notifies the project's cell PM. The pass is resilient by construction:

  • A missing CI signal is treated as "unknown", never a false green — an absent run never masks a real failure.
  • One repo's GitHub error never aborts the sweep — the engine moves on to the next project.
  • Origination is bounded and deduped per repo: a monorepo's cell-projects share one fix task, and the caps below stop a flapping CI from flooding the backlog.
flowchart TD
    A[Interval tick] --> B{For each opted-in project}
    B --> C{CI conclusion on default branch}
    C -->|green| B
    C -->|unknown / missing| B
    C -->|red| D[Open ONE fix task · notify the cell PM]
    D -.never auto.-> E[never merges · never deploys]

!!! danger "It never merges or deploys" A CI-watch fix task is an ordinary task. It flows through the normal delivery lifecycle — QA, the in-path PR-review gate, and your merge — exactly like work you create yourself. The engine never approves, merges, or deploys on its own.

Bounds on origination

Setting Default Meaning
ROBOCO_CI_WATCH_MAX_PER_CYCLE 1 Most fix tasks the sweep may open in one cycle.
ROBOCO_CI_WATCH_MAX_OPEN_TASKS 3 Rolling cap on concurrently-open CI-watch fix tasks per repo; the engine originates nothing more while this many are still open.

Enable it

=== "Panel"

**Settings → Feature Flags** carries the global **"Multi-repo CI-watch"** toggle. Then opt each project in from the **edit-project dialog → "Autonomous Maintenance" section**: turn on `ci_watch_enabled` and optionally set `ci_watch_workflow` (the workflow file to scope the CI signal to, default `ci.yml`).

!!! note "Takes effect on the next backend restart"
    The feature flag persists in the settings store and applies on the **next backend restart**. The per-project fields apply on the next sweep.

=== "Environment"

```bash
ROBOCO_CI_WATCH_ENABLED=true                 # global switch
# ROBOCO_CI_WATCH_INTERVAL_SECONDS=1800      # default
# ROBOCO_CI_WATCH_MAX_OPEN_TASKS=3           # default
# ROBOCO_CI_WATCH_MAX_PER_CYCLE=1            # default
```

The per-project opt-in (`ci_watch_enabled`, `ci_watch_workflow`) lives on the project, not in env — set it in the edit-project dialog.

Dependency-update bot

The dependency-update bot periodically checks whether a dependency upgrade would change a project's lockfiles and, if so, opens one "update dependencies" task into that project. Detection is read-only: nothing in the real repo is ever mutated.

What it does

On each pass (ROBOCO_DEP_UPDATE_INTERVAL_SECONDS, default 604800s — weekly) the bot runs the project's dep_update_command (e.g. uv lock --upgrade / pnpm update) in a throwaway clone and diffs the lockfiles. If the lockfiles would change it opens one "update dependencies" task; otherwise it opens nothing.

  • Detection is read-only: the command runs in a throwaway clone and only the lockfiles are diffed. The real repo is never mutated — nothing is committed or pushed.
  • It is fail-safe: a null dep_update_command or a command that fails opens nothing.
  • Origination is bounded and deduped per repo, with the caps below.
flowchart TD
    A[Interval tick] --> B{For each opted-in project}
    B --> C[Run dep_update_command in a throwaway clone]
    C --> D{lockfiles change?}
    D -->|no / null / fails| B
    D -->|yes| E[Open ONE update-dependencies task]
    E -.never auto.-> F[never merges · never deploys]

!!! danger "It never merges or deploys" The update-dependencies task is an ordinary task — QA, the in-path PR-review gate, and your merge all apply. The bot only ever detects and opens; it never commits, pushes, merges, or deploys.

Bounds on origination

Setting Default Meaning
ROBOCO_DEP_UPDATE_MAX_PER_CYCLE 1 Most update-dependencies tasks the bot may open in one cycle.
ROBOCO_DEP_UPDATE_MAX_OPEN_TASKS 3 Rolling cap on concurrently-open update-dependencies tasks per repo.

Enable it

=== "Panel"

**Settings → Feature Flags** carries the global **"Dependency-update bot"** toggle. Then opt each project in from the **edit-project dialog → "Autonomous Maintenance" section**: set `dep_update_command` (e.g. `uv lock --upgrade` / `pnpm update`) and optionally `dep_update_paths` (comma-separated lockfile paths to watch; defaults to inferring `uv.lock` / `pnpm-lock.yaml`).

!!! note "Takes effect on the next backend restart"
    The feature flag persists in the settings store and applies on the **next backend restart**. The per-project fields apply on the next sweep.

=== "Environment"

```bash
ROBOCO_DEP_UPDATE_ENABLED=true                  # global switch
# ROBOCO_DEP_UPDATE_INTERVAL_SECONDS=604800     # default (weekly)
# ROBOCO_DEP_UPDATE_MAX_OPEN_TASKS=3            # default
# ROBOCO_DEP_UPDATE_MAX_PER_CYCLE=1             # default
```

The per-project opt-in (`dep_update_command`, `dep_update_paths`) lives on the project, not in env — set it in the edit-project dialog.

What changes when each is on

  • With CI-watch on, a background sweep polls each opted-in project's CI on the configured interval; on a red conclusion a fix task appears in that project's backlog (bounded by the caps above) and its cell PM is notified. With the global flag off, nothing polls.
  • With the dependency-update bot on, a background sweep checks each opted-in project's lockfiles on the configured interval; when an upgrade would change them, an update-dependencies task appears. With the global flag off, nothing runs and no throwaway clone is made.

Next

Self-healing CI for the single-repo loop these generalize · Task lifecycle for what an opened task does once you start it · Environment reference for the full env list · back to Optional subsystems.