feat(conventions): generalize defaults, backfill old projects, adopt the standard in-repo

Harden the architectural-conventions standard so it works out-of-the-box on any project and resolves for projects that predate it, and make RoboCo pass its own gate. General defaults (apply to every project, not just one with a tuned file): - The auto-scan excludes test and documentation trees (tests/, docs/) — those legitimately define fixtures and aren't enforced code. - Helper placement seeds at warn, not block: `helper` matches any top-level function, too blunt a signal to hard-block a route file's small private glue. Misplaced model/route/component stay block; the body-level thin_routes check remains the real fat-handler guard. - thin_routes no longer counts transaction-lifecycle calls (commit/flush/ refresh) as data access — an explicit `db.commit()` after delegating to a service is a valid pattern. - no_lint_suppressions exempts a small allowlist of structurally-unavoidable framework codes (ruff TC001-TC003, pydantic prop-decorator); bare or other suppressions still flag. - CLAUDE.md rule-lifting skips bare common-word tokens that would match everywhere (e.g. "commit"), keeping only specific identifiers. - The ambient prompt block lists only constrained modules and truncates at a line boundary with a "+N more" pointer instead of cutting mid-line. Backfill: the standard previously read the committed file + repo scan from project.workspace_path, a field only a manual API call set — so an older project (or one whose workspace was cleared) showed an empty "missing" map no matter what was pushed. The service now ensures a dedicated, default-branch read clone on demand (WorkspaceService.ensure_read_clone) and resolves from it, persisting the resolved path + real HEAD. The panel tab, the spawn-time ambient block, and the per-task constraints all resolve the committed standard with no manual setup. Adopt in-repo: relocate the inline request/response models from the system and *_live route modules into roboco/api/schemas/ so the codebase passes its own placement gate, and ship a canonical .roboco/conventions.yml. no_models_in_routes and modular_cohesion are now clean and enforced at block. Docs updated across the user guide, the agent-facing RAG standard, the developer and pr_reviewer role prompts, CLAUDE.md, and the changelog. New unit tests cover the scan exclusions, helper-warn, the suppression allowlist, the commit exemption, and the resolve/backfill path; the conventions + project integration suites pass against Postgres.
2026-08-03 07:23:24 +02:00 · 2026-06-22 18:15:19 +02:00
parent 0c4d1c119f
commit 17ec52d1b7
26 changed files with 637 additions and 260 deletions
@@ -1,14 +1,16 @@
 # Architectural conventions for RoboCo.
 # Repo-canonical: this file overlays the auto-derived scan, and every consumer
 # (the validator, the per-task constraints, the ambient prompt block) reads the
 # merged effective map. Edit freely; the panel's Conventions tab round-trips it.
 #
-# Convention vs. claude-agent-runway: RoboCo keeps `no_inline_comments` at WARN,
+# This file overlays the auto-derived scan: every consumer (the validator, the
-# not block. The codebase deliberately documents intent in-line and in module /
+# per-task constraints, the spawn-time ambient block) reads the merged effective
-# function docstrings, so comments are welcome and never strand a task; the gate
+# map, so only project-specific divergences need to live here. The scan already
-# only nudges. The runway "no lint suppressions" rule is honoured but currently
+# excludes tests/ and docs/, maps the backend (roboco/) + frontend (panel/)
-# WARN (see the rule note below) until the ~18 existing pydantic-required
+# layers, and defaults misplaced-helper to warn — so this file is mostly a small
-# suppressions migrate to pyproject config, at which point it returns to block.
+# set of module declarations plus the rule-level policy below.
 #
 # Posture: BLOCK every boundary the codebase already honors (a model or helper
 # in a route, a route in a service, a model/route in a panel component, a hook
 # returning JSX — all refused), WARN the boundaries that still carry debt so a
 # task is never stranded on pre-existing code.
 version: 1
 languages:
@@ -22,11 +24,11 @@ modules:
    forbidden:
      - route
  - path: roboco/api
-    purpose: API package wiring — app, deps, middleware, websocket (helpers welcome)
+    purpose: API package wiring — app, deps, middleware, websocket
    forbidden:
      - model
  - path: roboco/api/routes
-    purpose: HTTP routes only — thin handlers that delegate to services
+    purpose: HTTP routes — thin handlers that delegate to services
    forbidden:
      - model
      - helper
@@ -34,11 +36,6 @@ modules:
    purpose: API request / response schemas
    forbidden:
      - route
  - path: roboco/api/utils
    purpose: API-layer helpers / utilities
    forbidden:
      - route
      - component
  - path: roboco/mcp/schemas
    purpose: MCP tool input / output schemas
    forbidden:
@@ -70,11 +67,6 @@ modules:
    forbidden:
      - component
      - route
  - path: panel/src/lib/stores
    purpose: client-side state management
    forbidden:
      - component
      - route
  - path: panel/src/lib/api
    purpose: typed API client
    forbidden:
@@ -84,80 +76,27 @@ modules:
    forbidden:
      - route
      - component
  - path: panel/lib
    purpose: shared frontend helpers / utilities
    forbidden:
      - route
      - component
  # --- Neutralized: tests + docs own no placement rules ------------------------
  # The auto-scan treats any dir named api/models/services/utils/schemas/routes
  # as an enforceable code module. Under tests/ and docs/ that is wrong: test
  # modules legitimately define fixtures, fakes, and helper functions, and docs/
  # is the MkDocs site (markdown, never classified). These overrides switch off
  # placement there so a test or docs task is never stranded. Longest-prefix wins,
  # so each derived sub-path is neutralized explicitly.
  - path: tests/unit/api
    purpose: tests — any kind may be defined
    forbidden: []
  - path: tests/unit/api/routes
    purpose: tests — any kind may be defined
    forbidden: []
  - path: tests/unit/api/schemas
    purpose: tests — any kind may be defined
    forbidden: []
  - path: tests/unit/models
    purpose: tests — any kind may be defined
    forbidden: []
  - path: tests/unit/services
    purpose: tests — any kind may be defined
    forbidden: []
  - path: tests/unit/utils
    purpose: tests — any kind may be defined
    forbidden: []
  - path: docs/api
    purpose: documentation site — not code
    forbidden: []
  - path: docs/models
    purpose: documentation site — not code
    forbidden: []
 rules:
-  # Posture: BLOCK every boundary this codebase already honors (real teeth, zero
+  # Comments are welcome in RoboCo — intent is documented in-code and in
-  # false-strands), WARN the boundaries that still carry debt so a task is never
+  # docstrings. The rule only nudges (full-line comments are never flagged).
  # stranded on pre-existing code. Every placement rule not named here inherits
  # the derived BLOCK level — e.g. a route in roboco/services, a model in a panel
  # component, or a hook returning JSX is refused outright.
  #
  # Hygiene
  no_lint_suppressions:
    # ~18 live in roboco/ today, most pydantic-required (computed_field
    # prop-decorator, TC003 runtime types). WARN until they migrate to pyproject
    # per-file-ignores / mypy overrides, then this returns to block.
    level: warn
  no_inline_comments:
    # RoboCo documents intent in-code; comments are welcome and never block.
    level: warn
-  # Placement debt — clean these up, then promote back to block
+  # A handful of unavoidable framework suppressions are auto-allowed
-  no_models_in_routes:
+  # (TC001-003, pydantic prop-decorator); the few remaining inline E402 / E501 /
-    # Inline request/response models in *_live.py belong in roboco/api/schemas.
+  # arg-type suppressions stay warn until they migrate to pyproject config.
-    level: warn
+  no_lint_suppressions:
  no_helpers_in_routes:
    # Local _-prefixed route helpers; tighten once they move to services/utils.
    level: warn
  # Modularity
  modular_cohesion:
    # Same root as no_models_in_routes (a file with both a model and a route).
    level: warn
  # RoboCo routes deliberately call db.commit() (get_db auto-commit is unreliable
  # under BaseHTTPMiddleware); explicit commits no longer count as data access,
  # so this stays advisory for the few routes that still read/write directly.
  thin_routes:
    # RoboCo routes deliberately call db.commit() (get_db auto-commit is
    # unreliable under BaseHTTPMiddleware) and the check counts commit as data
    # access — so this stays advisory rather than blocking the commit convention.
    level: warn
  # A few panel components still fetch inline; extract into hooks, then promote.
  thin_components:
    level: warn
-  god_class:
+  # Everything else inherits the derived BLOCK level — no models/helpers in
-    level: warn
+  # routes, no routes in services, no models/routes in panel components, etc.
 custom: []
 waivers: []
@@ -8,17 +8,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 ### Added
- **Architectural Conventions Standard — a per-project, repo-canonical architecture map that gates where code may live.** Beyond the `make`-style checks (syntax, types, tests), each project can carry a `.roboco/conventions.yml` declaring which definition *kinds* belong in which modules, a toggleable rule set, custom regex rules, and waivers — so an agent can no longer land a Pydantic model inside a router, a helper in a route file, or a lint suppression. A tree-sitter validator CLI (Python + TypeScript) classifies every changed definition and emits findings; a `block`-level finding refuses a developer's `i_am_done` and the in-path PR gate's `pr_pass` with the offending `file:line` and a fix hint, and findings surface in QA's review evidence. The file is auto-scaffolded on first clone, editable from a per-project Conventions tab in the panel, and a false positive is cleared by a waiver committed in the branch and reviewed in the PR. Gated by `ROBOCO_CONVENTIONS_ENABLED` (default off) and fully inert when off.
+- **Architectural Conventions Standard — a per-project, repo-canonical architecture map that gates where code may live.** Beyond the `make`-style checks (syntax, types, tests), each project can carry a `.roboco/conventions.yml` declaring which definition *kinds* belong in which modules, a toggleable rule set, custom regex rules, and waivers — so an agent can no longer land a Pydantic model inside a router or a lint suppression (a misplaced *helper* — any top-level function — warns rather than blocks). A tree-sitter validator CLI (Python + TypeScript) classifies every changed definition and emits findings; a `block`-level finding refuses a developer's `i_am_done` and the in-path PR gate's `pr_pass` with the offending `file:line` and a fix hint, and findings surface in QA's review evidence. The auto-derived defaults exclude test and documentation trees, count an explicit `db.commit()` in a route as legitimate (not a fat-route violation), and exempt a small allowlist of structurally-unavoidable framework suppressions (ruff `TC001`–`TC003`, pydantic `prop-decorator`). The committed file and repo scan are read from a dedicated project-level read clone the service ensures on demand — so the standard resolves even for a project created before it existed, with no manual workspace configuration. The file is auto-scaffolded on first clone, editable from a per-project Conventions tab in the panel, and a false positive is cleared by a waiver committed in the branch and reviewed in the PR. Gated by `ROBOCO_CONVENTIONS_ENABLED` (default off) and fully inert when off.
 - **Agent runtime toolchain matching — agents build each target project under the Python that project actually requires.** The agent image bakes one interpreter, but the projects RoboCo builds don't all share it, so a self-gate could pass against the wrong runtime. The workspace now resolves each target's Python from its `requires-python` / `.python-version`, provisions the clone with `uv sync --extra dev --python <version>` (fetching the interpreter on demand), and records a `.git/.roboco-toolchain` marker. A guard refuses a developer's `i_am_done`, QA's `pass_review`, and the PR gate's `pr_pass` when the suite cannot be collected under the provisioned interpreter, so "verifying by reading source" can't masquerade as a passing gate. Gated by `ROBOCO_TOOLCHAIN_MATCH_ENABLED` (default off).
 - **Provider overload circuit-break — a persistent model-API overload parks the provider instead of crash-retrying into it.** A sustained 529/500/503 (the SDK already retries transient ones) now trips the same park-and-probe break as a rate limit: the spawn gate queues further work for that provider and a background loop revives it when the overload lifts, instead of respawning the agent straight back into the failure and burning tokens. Gated by `ROBOCO_OVERLOAD_BREAK_ENABLED` (default on).
 - **Structured content standard with obligated note sections.** Every agent-authored handoff (developer, QA, documenter, PR-reviewer, auditor, PM resumption) is now a validated structured model persisted as the source of truth, with the legacy text column derived from it through a single chokepoint. An anti-soup guard rejects filler and all-token-noise free-text across the flow and content verbs, structured PR-review findings render a generated GitHub comment, and each role's note section is obligated at its lifecycle transition the way journals already were.
 ### Changed
 - **RoboCo adopts its own architectural standard.** The repo now ships a canonical `.roboco/conventions.yml`, and the inline request/response models that lived in the `system` and `*_live` route modules were relocated to `roboco/api/schemas/` so the codebase passes its own placement gate (`no_models_in_routes` / `modular_cohesion` are now clean and enforced at `block`).
 - **RoboCo's own `requires-python` floor is raised to `>=3.13`.** The codebase imports `tomllib` (3.11+) and runs on 3.13; the previous `>=3.10` floor made the toolchain resolver provision the self-hosted build at 3.10, where the suite cannot even be collected. Agent gate containers now also receive the test-database connection, so an agent's `make quality` runs the real, DB-backed suite instead of a coverage-collapsing unit-only subset.
 ### Fixed
 - **The conventions standard now resolves for projects created before it existed.** It previously read the committed `.roboco/conventions.yml` and the repo scan from `project.workspace_path` — a field only a manual API call ever set — so an older project (or one whose workspace was cleared) showed an empty "missing" map no matter what was pushed. The service now ensures a dedicated, default-branch read clone on demand and reads from it, persisting the resolved path + HEAD (the backfill). The panel tab, the spawn-time ambient block, and the per-task constraints all resolve the committed standard with no manual setup.
 - **The conventions ambient prompt block no longer truncates mid-line.** It now lists only modules that actually constrain a kind, and when the list would exceed its budget it trims at a line boundary with a `+N more` pointer instead of cutting a module in half.
 - **The toolchain gate no longer passes silently on an unverifiable workspace.** A `broken` interpreter still blocks; an `unknown` status — the smoke could not confirm the suite is collectable — now emits a warning when the gate proceeds, instead of slipping through unseen.
 - **The crypto tests are hermetic.** The Fernet round-trip tests supply their own key instead of depending on `ROBOCO_ENCRYPTION_KEY` in the environment, so they pass in any gate container without the production secret being injected.
 - **`ollama-init` is best-effort and gates startup on the models being present**, so a slow or unreachable model registry can no longer down a fully-cached deployment.
@@ -377,9 +377,9 @@ Agent backends are pluggable. `roboco/llm/providers/` defines an `AgentProvider`
 ## Architectural Conventions Standard
-**Per-project architectural standard (default-off).** Beyond the `make`-style gates (which check syntax/types/tests, not *where code lives*), each project can carry a repo-canonical `.roboco/conventions.yml` — an architecture map (which definition *kinds* belong in which modules), a toggleable rule set, custom regex rules, and waivers — so an agent cannot land a Pydantic model defined inside a router, a helper in a route file, or a `# noqa` / `# type: ignore`. Gated by `ROBOCO_CONVENTIONS_ENABLED`; fully inert when off.
+**Per-project architectural standard (default-off).** Beyond the `make`-style gates (which check syntax/types/tests, not *where code lives*), each project can carry a repo-canonical `.roboco/conventions.yml` — an architecture map (which definition *kinds* belong in which modules), a toggleable rule set, custom regex rules, and waivers — so an agent cannot land a Pydantic model defined inside a router or a `# noqa` / `# type: ignore`. Placement of a *helper* (any top-level function) only **warns** — too blunt to hard-block; `thin_routes` doesn't count an explicit `db.commit()`; and a small allowlist of unavoidable framework suppressions (ruff `TC001`–`TC003`, pydantic `prop-decorator`) is exempt. Gated by `ROBOCO_CONVENTIONS_ENABLED`; fully inert when off. RoboCo itself ships a canonical `.roboco/conventions.yml`.
-**Effective map.** Consumers read the *effective* map — auto-derived defaults (from a repo scan + `BUILTIN_RULES`) overlaid by the committed file — so behaviour is identical whether the file is present, absent, or partial. `ConventionsService` (`roboco/services/conventions.py`) builds it, caches it per `(project, HEAD sha)` in `project_conventions_cache` (migration `043`), renders the per-task baseline constraints + the ambient prompt block, and scaffolds/restores the file via a PR (`GitService.open_conventions_pr`). The schema lives in `roboco/foundation/policy/conventions/` (pure).
+**Effective map.** Consumers read the *effective* map — auto-derived defaults (from a repo scan + `BUILTIN_RULES`, excluding `tests/`/`docs/` trees) overlaid by the committed file — so behaviour is identical whether the file is present, absent, or partial. `ConventionsService` (`roboco/services/conventions.py`) builds it, caches it per `(project, HEAD sha)` in `project_conventions_cache` (migration `043`), renders the per-task baseline constraints + the ambient prompt block, and scaffolds/restores the file via a PR (`GitService.open_conventions_pr`). The committed file + scan are read from a dedicated project-level **read clone** the service ensures on demand (`WorkspaceService.ensure_read_clone`, pinned to the default branch's HEAD) — the backfill that makes the standard resolve even for a project created before it existed, with no manual `workspace_path`. The schema lives in `roboco/foundation/policy/conventions/` (pure).
 **Validator.** A single Python CLI, `python -m roboco.conventions check --root <repo> --files <a> <b> ...` (`roboco/conventions/`), uses tree-sitter (Python + TypeScript grammars, shipped in the agent image) to classify each changed definition and flag forbidden placements + hygiene + custom-rule matches as JSONL findings, after waiver filtering. Precision over recall (it abstains when uncertain so a `block` gate can't false-positive-strand a task) and fail-loud (a validator that cannot run exits 3 so the gate blocks, never silently passes).
@@ -102,7 +102,7 @@ The gateway enforces some of these; the rest are convention but failing one of t
 5. ✅ `note(scope='reflect', task_id=...)` walks through every criterion (gateway-enforced as `journal:reflect`).
 6. ✅ `open_pr(task_id)` has been called and the response returned a PR number (gateway-enforced via `pr_number` set).
 7. ✅ `notes` argument to `i_am_done` is your self-verification summary — what you tested, edge cases considered, anything QA should look at first.
-8. ✅ Each definition lives in the module the project's architectural map (`.roboco/conventions.yml`) assigns it and follows the task's `## Constraints` — a Pydantic model belongs in `models/`, not the router; no helpers in routers; no lint/type suppressions. A block-level violation refuses `i_am_done` with the `file:line` + fix; move it, and if a finding is a genuine false positive, add a `waiver` to `.roboco/conventions.yml` in your branch for the PR to review.
+8. ✅ Each definition lives in the module the project's architectural map (`.roboco/conventions.yml`) assigns it and follows the task's `## Constraints` — a Pydantic model belongs in `models/`, not the router; keep helpers out of routers (advisory — a misplaced helper only *warns*); no lint/type suppressions. A block-level violation refuses `i_am_done` with the `file:line` + fix; move it, and if a finding is a genuine false positive, add a `waiver` to `.roboco/conventions.yml` in your branch for the PR to review.
 If any item fails, do not retry `i_am_done`; fix the missing piece first.
@@ -111,7 +111,7 @@ If any item fails, do not retry `i_am_done`; fix the missing piece first.
 Beyond placement and hygiene, the Architectural Conventions Standard now enforces MODULARIZATION via a "modularity" AST check family that inspects a definition's body and a file's composition. Write to it from the start — a block-level modularity finding refuses `i_am_done` (and the PR reviewer's `pr_pass`) with the offending `file:line` + a fix hint, and surfaces in QA's `claim_review` evidence as `convention_findings`. The checks are language-aware: a Python/API project carries `thin_routes`; a TypeScript/React project carries `thin_components`; `modular_cohesion` and `god_class` apply to both.
 - **One architectural concern per file (`modular_cohesion`).** A file must own a single concern. Do not define a Pydantic model inside a router, or a schema inside a component — split each concern into its own module (`models/`, `schemas/`, the hook, …).
- **Keep route handlers thin (`thin_routes`, Python/API).** A route delegates data access and business logic to a service. It must NOT run its own database access in the route body — no `session.execute`/`query`/`commit`/`add`, no `select()`/`insert()`/`update()`/`delete()`. Move that into the service the route calls.
+- **Keep route handlers thin (`thin_routes`, Python/API).** A route delegates data access and business logic to a service. It must NOT run its own database access in the route body — no `session.execute`/`query`/`scalars`/`add`, no `select()`/`insert()`/`update()`/`delete()`. Move that into the service the route calls. (An explicit `await db.commit()` to close the unit of work after delegating is fine — transaction-lifecycle calls don't count.)
 - **Keep components presentational (`thin_components`, TypeScript/React).** Data fetching (`fetch`/`axios`) and logic belong in a hook, not the component body. The component renders; the hook fetches.
 - **No god classes (`god_class`).** A class past the method-count threshold is doing too much — decompose it along its responsibilities.
@@ -43,7 +43,7 @@ The PR is from an outside contributor: its code is **untrusted**. Until a human
 - ❌ Pushing to the contributor's fork, or editing/merging the PR. You review; you never write or merge.
 - ❌ A trickle of vague comments. Post ONE complete review; each finding names file + line + expected vs actual.
 - ❌ Approving without reading the full diff.
- ❌ Being lax on the architectural standard. Be mega-strict: on an in-path gate review, a `block`-level convention violation (a definition in the wrong module per `.roboco/conventions.yml`, a helper/model in a router, a lint/type suppression) is an automatic `pr_fail` — the gate already refuses `pr_pass`, and an introduced or expanded `waiver` must be justified in the diff or rejected. Hold placement and house-style to the same bar as correctness.
+- ❌ Being lax on the architectural standard. Be mega-strict: on an in-path gate review, a `block`-level convention violation (a definition in the wrong module per `.roboco/conventions.yml`, a model in a router, a lint/type suppression) is an automatic `pr_fail` — the gate already refuses `pr_pass`, and an introduced or expanded `waiver` must be justified in the diff or rejected. Hold placement and house-style to the same bar as correctness.
 - ❌ Letting a non-modular assembled change through. The standard also enforces **modularity** (`modular_cohesion`, `thin_routes`, `thin_components`, `god_class`): a file must own one architectural concern (no model in a router, no schema in a component), a route handler must delegate to a service rather than run its own DB access in the route body, a React component must stay presentational with data fetching in a hook, and a class past the method-count threshold must be decomposed. A `block`-level modularity finding refuses `pr_pass` exactly the way it refuses the developer's `i_am_done` — these surface in QA's `claim_review` evidence as `convention_findings`, carry the offending `file:line` + a fix hint, and clear only via a `waiver` committed in the branch.
 ## When the gateway returns an error
@@ -21,15 +21,15 @@ The rules live in a per-project `.roboco/conventions.yml` with four curated part
 The validator runs four check families over each changed file:
- **Placement** — a definition whose *kind* is forbidden in its module (a model in a router).
+- **Placement** — a definition whose *kind* is forbidden in its module (a model in a router). A misplaced **model**, **route**, or **component** is `block` by default; a misplaced **helper** only `warn`s — `helper` matches *any* top-level function, too blunt a signal to hard-block a route file's small private glue (the body-level `thin_routes` check is the real fat-handler guard).
- **Hygiene** — the universal, stack-agnostic house-style rules seeded into every project: `no_lint_suppressions` (`block` by default — no `# noqa`, `# type: ignore`, `eslint-disable`) and `no_inline_comments` (`warn`).
+- **Hygiene** — the universal, stack-agnostic house-style rules seeded into every project: `no_lint_suppressions` (`block` by default — no `# noqa`, `# type: ignore`, `eslint-disable`) and `no_inline_comments` (`warn`). A small allowlist of *structurally unavoidable* framework codes is exempt from `no_lint_suppressions` — ruff's flake8-type-checking codes (`TC001`–`TC003`, for an import a framework needs at runtime) and pydantic's `prop-decorator` — so the rule keeps its teeth on genuine error-silencing without footgunning every pydantic/FastAPI project. A bare `# noqa` / `# type: ignore` or any other code is still flagged.
 - **Custom** — your project-specific regex rules.
 - **Modularity** — separation-of-concerns judgements the linters are blind to, inspecting a file's *composition* and a definition's *body*, not just its top-level kind:
 | Rule | Fires when | Default level |
 |------|-----------|---------------|
 | `modular_cohesion` | One file mixes architectural concerns (e.g. a model *and* a route *and* a component) | `block` |
-| `thin_routes` | A route handler does its own data access (SQLAlchemy `execute`/`commit`/`select`…) instead of delegating to a service | `block` |
+| `thin_routes` | A route handler does its own data access (SQLAlchemy `execute`/`scalars`/`add`/`select`…) instead of delegating to a service. Transaction-lifecycle calls (`commit`/`flush`/`refresh`) do **not** count — an explicit `await db.commit()` after delegating is a valid pattern | `block` |
 | `thin_components` | A React component fetches data in its body instead of through a hook | `block` |
 | `god_class` | A class grows past 15 methods (single-responsibility smell) | `warn` |
@@ -38,10 +38,13 @@ The validator runs four check families over each changed file:
 ## The effective map: defaults, present, absent, or partial
-Consumers never read the raw committed file — they read the **effective map**, so behaviour is identical whether `.roboco/conventions.yml` is present, absent, or partial. `ConventionsService` (`roboco/services/conventions.py`) builds it by auto-deriving a baseline from a repo scan (it infers modules from directory names like `routers/`, `models/`, `services/`, `components/`, `hooks/`; detects languages by file extension; seeds the universal hygiene rules) and then overlaying the committed file on top. The result is cached per `(project, HEAD sha)`.
+Consumers never read the raw committed file — they read the **effective map**, so behaviour is identical whether `.roboco/conventions.yml` is present, absent, or partial. `ConventionsService` (`roboco/services/conventions.py`) builds it by auto-deriving a baseline from a repo scan (it infers modules from directory names like `routers/`, `models/`, `services/`, `components/`, `hooks/`; **excludes test and documentation trees** — `tests/`, `docs/` — since those legitimately define fixtures and aren't enforced code; detects languages by file extension; seeds the universal hygiene rules) and then overlaying the committed file on top. The result is cached per `(project, HEAD sha)`.
 That is the load-bearing property: **the standard is enforced even before any file is committed.** A project with no `.roboco/conventions.yml` still gets sensible auto-derived rules and is gated by them. Resilience is built in — a missing file degrades to the auto-derived defaults, and an unparseable file falls back to the last-good cached map (status `degraded`) so the standard is never silently switched off by a typo.
 !!! info "Reads come from a dedicated clone — no setup needed for old projects"
    The committed file and the repo scan are read from a project-level **read clone** that the service ensures on demand (pinned to the default branch's HEAD), not from any agent's working clone. This is the backfill: a project created long before the standard existed — with no manually-configured workspace path — still resolves its committed `.roboco/conventions.yml` the first time the panel, a spawn, or a task asks for it. There is nothing to wire up.
 ## The per-project Conventions editor
 Each project carries a **Conventions** tab in its edit dialog (panel component `panel/src/components/conventions/conventions-tab.tsx`). From there you manage the whole standard without hand-editing YAML:
@@ -1,6 +1,6 @@
 # Architectural Conventions Standard
-A per-project, repo-canonical standard for *where code lives*, how a definition is *built*, and basic house-style hygiene — the layer above the `make`-style gates (which check syntax, types, and tests, not placement or structure). It exists so an agent cannot land a model defined inside a router, a route handler that runs its own database access, a helper in a route file, or a lint suppression, even when the code compiles and the tests pass. It enforces the separation of concerns a senior would demand in review, not just linting.
+A per-project, repo-canonical standard for *where code lives*, how a definition is *built*, and basic house-style hygiene — the layer above the `make`-style gates (which check syntax, types, and tests, not placement or structure). It exists so an agent cannot land a model defined inside a router, a route handler that runs its own database access, or a lint suppression, even when the code compiles and the tests pass (a misplaced *helper* — any top-level function — only warns, since that signal is too blunt to hard-block). It enforces the separation of concerns a senior would demand in review, not just linting.
 The standard is gated by `ROBOCO_CONVENTIONS_ENABLED` (default off) and is fully inert when off.
@@ -8,7 +8,9 @@ The standard is gated by `ROBOCO_CONVENTIONS_ENABLED` (default off) and is fully
 Each project carries a repo-canonical `.roboco/conventions.yml`. It is auto-scaffolded into a project's clone the first time the project is worked on, editable from the per-project **Conventions** tab in the panel, and committed like any other repo file.
-Consumers always read the *effective* map: auto-derived defaults (from a repo scan plus the built-in rules) overlaid by the committed file. Behaviour is identical whether the file is present, absent, or partial — a missing file just means "defaults only".
+Consumers always read the *effective* map: auto-derived defaults (from a repo scan plus the built-in rules) overlaid by the committed file. Behaviour is identical whether the file is present, absent, or partial — a missing file just means "defaults only". The scan excludes test and documentation trees (`tests/`, `docs/`) — those legitimately define fixtures and aren't enforced code.
 The committed file and the scan are read from a project-level **read clone** the service ensures on demand (pinned to the default branch's HEAD), not from any agent's working clone — so the standard resolves even for a project created long before it existed, with no manual workspace configuration.
 ```yaml
 # .roboco/conventions.yml
@@ -25,9 +27,9 @@ modules:
  - path: app/services
    purpose: business logic + side effects
-# Toggle or re-level the built-in rules.
+# Toggle or re-level the built-in rules (each fires at `warn` or `block`).
 rules:
-  no_models_in_routers: { level: block }   # block | warn | off
+  no_models_in_routers: { level: block }
  no_inline_comments: { level: warn }
 # Project-specific regex rules.
@@ -59,7 +61,7 @@ It favours precision over recall — it abstains when it cannot classify a defin
 Beyond placement and hygiene, the standard enforces modularization with a **modularity** check family. Where placement asks *which module a definition belongs in*, modularity inspects a definition's **body** and a file's **composition** — the structural questions a senior asks in code review:
 - **`modular_cohesion`** (any stack) — a file must own one architectural concern. A file that mixes them (a Pydantic model defined in a router, a schema defined in a component) is a monolith to split.
- **`thin_routes`** (Python / API) — a route handler must delegate to a service. It may not run its own database access (no `session.execute` / `query` / `commit` / `add`, no `select()` / `insert()` / `update()` / `delete()`) in the route body.
+- **`thin_routes`** (Python / API) — a route handler must delegate to a service. It may not run its own database access (no `session.execute` / `query` / `scalars` / `add`, no `select()` / `insert()` / `update()` / `delete()`) in the route body. Transaction-lifecycle calls — `commit` / `flush` / `refresh` — do not count: an explicit `await db.commit()` after delegating to a service is a valid pattern.
 - **`thin_components`** (TypeScript / React) — a component must stay presentational. Data fetching (`fetch` / `axios`) belongs in a hook, not in the component body.
 - **`god_class`** (any stack) — a class past a method-count threshold is doing too much; decompose it to keep a single responsibility.
@@ -81,6 +83,8 @@ A `warn`-level finding is reported but never blocks.
 A false positive is relieved by a **waiver** the developer commits in their branch — so the escape is accountable and reviewed in the PR, not a silent in-code suppression (`# noqa` / `# type: ignore` are themselves hygiene violations the standard flags). Add the waiver to `.roboco/conventions.yml`, commit it, and the finding is filtered on the next check.
 The one exception to "suppressions are violations" is a small allowlist of *structurally unavoidable* framework codes that the validator does not flag: ruff's flake8-type-checking codes (`TC001`–`TC003`, for an import a framework needs at runtime) and pydantic's `prop-decorator`. A bare `# noqa` / `# type: ignore` or any other code is still a finding.
 ## Panel
 The per-project **Conventions** tab (in the edit-project dialog) shows the effective architecture map and its health, and offers **Save** (commit an edited map back to the repo via a PR) and **Restore** (re-scaffold the canonical file).
@@ -276,7 +276,10 @@ async def conventions_ambient_layer(
    service = get_conventions_service(session)
    blocks: list[str] = []
    for project in projects:
-        block = await service.render_ambient_block(project)
+        # Ensure the read clone so the standard resolves even when the project
        # has no manually-configured workspace_path (the backfill path).
        workspace = await service.resolve_workspace(project)
        block = await service.render_ambient_block(project, workspace=workspace)
        if not block:
            continue
        blocks.append(
@@ -473,8 +473,12 @@ async def get_conventions(
    """Return the project's effective conventions map + its current health."""
    project = await _get_project_or_404(get_project_service(db), project_id)
    conv = get_conventions_service(db)
-    standard = await conv.get_map(project)
+    # Ensure a default-branch read clone once, then read the map + health from
-    health = await conv.health(project)
+    # it. This is the backfill: a project created before the standard existed
    # (no manual workspace_path) still resolves its committed conventions file.
    workspace = await conv.resolve_workspace(project)
    standard = await conv.get_map(project, workspace=workspace)
    health = await conv.health(project, workspace=workspace)
    await db.commit()
    return ConventionsResponse(
        standard=standard.model_dump(mode="json"),
@@ -17,11 +17,10 @@ Phase 5.
 from __future__ import annotations
 import json
-from typing import TYPE_CHECKING, Any, Literal
+from typing import TYPE_CHECKING, Any
 from uuid import UUID, uuid4
 from fastapi import APIRouter, HTTPException, Request, status
 from pydantic import BaseModel, Field, model_validator
 from sse_starlette import EventSourceResponse
 from roboco.api.deps import (
@@ -30,6 +29,13 @@ from roboco.api.deps import (
    get_orchestrator,
    require_pm_or_above,
 )
 from roboco.api.schemas.prompter_live import (
    AgentEvent,
    LiveConfirmRequest,
    LiveMessageRequest,
    StartLiveRequest,
    StartLiveResponse,
 )
 from roboco.services.base import NotFoundError, ServiceError, ValidationError
 from roboco.services.prompter import get_prompter_service
 from roboco.services.prompter_live import get_live_registry
@@ -62,41 +68,6 @@ def _translate_service_error(e: ServiceError) -> HTTPException:
    )
 class StartLiveRequest(BaseModel):
    """Open a live intake chat scoped to a project XOR a product."""
    project_id: UUID | None = None
    product_id: UUID | None = None
    initial_message: str | None = Field(default=None, min_length=1)
    @model_validator(mode="after")
    def _exactly_one_scope(self) -> StartLiveRequest:
        if bool(self.project_id) == bool(self.product_id):
            raise ValueError("provide exactly one of project_id / product_id")
        return self
 class StartLiveResponse(BaseModel):
    """The new session's id — the panel opens its stream and posts messages to it."""
    session_id: str
 class LiveMessageRequest(BaseModel):
    """The human's message in an active intake chat."""
    text: str = Field(..., min_length=1)
 class AgentEvent(BaseModel):
    """One normalized event the container relays (mirrors driver.StreamChunk)."""
    kind: str
    text: str = ""
    tool: str = ""
    data: dict[str, Any] = Field(default_factory=dict)
@router.post(
    "/live/start",
    response_model=StartLiveResponse,
@@ -188,32 +159,6 @@ async def stop_live(session_id: str) -> dict[str, bool]:
    return {"stopped": True}
 class LiveConfirmRequest(BaseModel):
    """Confirm the agent's draft → a task, scoped to exactly one target.
    ``route`` is which start button the human pressed: ``"board"`` (Board review
    & Start → PO + HoM review first) or ``"main_pm"`` (Approve & Start → straight
    to the Main PM).
    """
    project_id: UUID | None = None
    product_id: UUID | None = None
    draft: dict[str, Any]
    route: Literal["board", "main_pm"] = "board"
    # Set on a board-informed re-draft: confirm updates this existing task in
    # place instead of creating a new one. When present, project/product scope
    # is taken from the task, so neither is required here.
    task_id: UUID | None = None
    @model_validator(mode="after")
    def _exactly_one_target(self) -> LiveConfirmRequest:
        if self.task_id is not None:
            return self
        if bool(self.project_id) == bool(self.product_id):
            raise ValueError("provide exactly one of project_id / product_id")
        return self
@router.post("/live/{session_id}/confirm", status_code=status.HTTP_201_CREATED)
 async def confirm_live(
    session_id: str,
@@ -20,10 +20,15 @@ from typing import TYPE_CHECKING, Any
 from uuid import uuid4
 from fastapi import APIRouter, HTTPException, Request, status
 from pydantic import BaseModel, Field
 from sse_starlette import EventSourceResponse
 from roboco.api.deps import get_orchestrator
 from roboco.api.schemas.secretary_live import (
    AgentEvent,
    LiveMessageRequest,
    StartSecretaryRequest,
    StartSecretaryResponse,
 )
 from roboco.services.prompter_live import get_live_registry
 if TYPE_CHECKING:
@@ -32,29 +37,6 @@ if TYPE_CHECKING:
 router = APIRouter()
 class StartSecretaryRequest(BaseModel):
    """Open a live Secretary chat (optionally with an opening message)."""
    initial_message: str | None = Field(default=None, min_length=1)
 class StartSecretaryResponse(BaseModel):
    session_id: str
 class LiveMessageRequest(BaseModel):
    text: str = Field(..., min_length=1)
 class AgentEvent(BaseModel):
    """One event relayed from the container onto the session stream."""
    kind: str
    text: str = ""
    tool: str = ""
    data: dict[str, Any] = Field(default_factory=dict)
@router.post(
    "/live/start",
    response_model=StartSecretaryResponse,
@@ -16,36 +16,13 @@ from __future__ import annotations
 from datetime import datetime, timedelta
 from fastapi import APIRouter
 from pydantic import BaseModel, ConfigDict
 from pydantic.alias_generators import to_camel
 from roboco.api.schemas.system import RateLimitEntry, RateLimitListResponse
 from roboco.services.gateway.rate_limit_tracker import RateLimitStateTracker
 router = APIRouter()
 class _CamelModel(BaseModel):
    """Serialize with camelCase aliases so the panel consumes fields directly."""
    model_config = ConfigDict(alias_generator=to_camel, populate_by_name=True)
 class RateLimitEntry(_CamelModel):
    """A single provider's active rate-limit state, in the panel's shape."""
    provider: str
    affected_agents: list[str]
    hit_at: str | None
    resume_at: str | None
    retry_after_seconds: float | None
 class RateLimitListResponse(_CamelModel):
    """The envelope the panel's rate-limit store expects: ``{ "entries": [...] }``."""
    entries: list[RateLimitEntry]
 def _resume_at(hit_at: str | None, retry_after: float | None) -> str | None:
    """Estimated lift time = hit_at + retry_after, ISO; falls back to hit_at."""
    if not hit_at or retry_after is None:
@@ -0,0 +1,73 @@
 """Request / response schemas for the live intake (Prompter) chat bridge.
 Moved out of the route module so the HTTP layer stays handler-only and these
 models live with the other API schemas (architectural-conventions placement).
 """
 from __future__ import annotations
 from typing import Any, Literal
 from uuid import UUID  # noqa: TC003 — pydantic resolves these annotations at runtime
 from pydantic import BaseModel, Field, model_validator
 class StartLiveRequest(BaseModel):
    """Open a live intake chat scoped to a project XOR a product."""
    project_id: UUID | None = None
    product_id: UUID | None = None
    initial_message: str | None = Field(default=None, min_length=1)
    @model_validator(mode="after")
    def _exactly_one_scope(self) -> StartLiveRequest:
        if bool(self.project_id) == bool(self.product_id):
            raise ValueError("provide exactly one of project_id / product_id")
        return self
 class StartLiveResponse(BaseModel):
    """The new session's id — the panel opens its stream and posts messages to it."""
    session_id: str
 class LiveMessageRequest(BaseModel):
    """The human's message in an active intake chat."""
    text: str = Field(..., min_length=1)
 class AgentEvent(BaseModel):
    """One normalized event the container relays (mirrors driver.StreamChunk)."""
    kind: str
    text: str = ""
    tool: str = ""
    data: dict[str, Any] = Field(default_factory=dict)
 class LiveConfirmRequest(BaseModel):
    """Confirm the agent's draft → a task, scoped to exactly one target.
    ``route`` is which start button the human pressed: ``"board"`` (Board review
    & Start → PO + HoM review first) or ``"main_pm"`` (Approve & Start → straight
    to the Main PM).
    """
    project_id: UUID | None = None
    product_id: UUID | None = None
    draft: dict[str, Any]
    route: Literal["board", "main_pm"] = "board"
    # Set on a board-informed re-draft: confirm updates this existing task in
    # place instead of creating a new one. When present, project/product scope
    # is taken from the task, so neither is required here.
    task_id: UUID | None = None
    @model_validator(mode="after")
    def _exactly_one_target(self) -> LiveConfirmRequest:
        if self.task_id is not None:
            return self
        if bool(self.project_id) == bool(self.product_id):
            raise ValueError("provide exactly one of project_id / product_id")
        return self
@@ -0,0 +1,34 @@
 """Request / response schemas for the live Secretary chat bridge.
 Moved out of the route module so the HTTP layer stays handler-only and these
 models live with the other API schemas (architectural-conventions placement).
 """
 from __future__ import annotations
 from typing import Any
 from pydantic import BaseModel, Field
 class StartSecretaryRequest(BaseModel):
    """Open a live Secretary chat (optionally with an opening message)."""
    initial_message: str | None = Field(default=None, min_length=1)
 class StartSecretaryResponse(BaseModel):
    session_id: str
 class LiveMessageRequest(BaseModel):
    text: str = Field(..., min_length=1)
 class AgentEvent(BaseModel):
    """One event relayed from the container onto the session stream."""
    kind: str
    text: str = ""
    tool: str = ""
    data: dict[str, Any] = Field(default_factory=dict)
@@ -0,0 +1,33 @@
 """Schemas for the system monitoring endpoints.
 Serialized with camelCase aliases so the control panel consumes the fields
 directly. Moved out of the route module so the HTTP layer stays handler-only
 (architectural-conventions placement).
 """
 from __future__ import annotations
 from pydantic import BaseModel, ConfigDict
 from pydantic.alias_generators import to_camel
 class _CamelModel(BaseModel):
    """Serialize with camelCase aliases so the panel consumes fields directly."""
    model_config = ConfigDict(alias_generator=to_camel, populate_by_name=True)
 class RateLimitEntry(_CamelModel):
    """A single provider's active rate-limit state, in the panel's shape."""
    provider: str
    affected_agents: list[str]
    hit_at: str | None
    resume_at: str | None
    retry_after_seconds: float | None
 class RateLimitListResponse(_CamelModel):
    """The envelope the panel's rate-limit store expects: ``{ "entries": [...] }``."""
    entries: list[RateLimitEntry]
@@ -7,6 +7,7 @@ not) is allowed. Suppression markers are language-scoped.
 from __future__ import annotations
 import re
 from typing import TYPE_CHECKING
 from roboco.foundation.policy.conventions.models import (
@@ -27,6 +28,21 @@ _SUPPRESSIONS: dict[str, tuple[str, ...]] = {
    "typescript": ("eslint-disable", "ts-ignore", "ts-expect-error"),
    "tsx": ("eslint-disable", "ts-ignore", "ts-expect-error"),
 }
 # Suppression codes that are the SANCTIONED escape hatch rather than a silenced
 # error: ruff's flake8-type-checking codes (TC001/2/3 — an import a framework
 # needs at runtime, e.g. pydantic / SQLAlchemy / FastAPI, cannot move into a
 # TYPE_CHECKING block) and pydantic's computed_field ``prop-decorator``. A
 # suppression is allowed only when it carries codes AND every one is in this set;
 # a bare ``noqa`` / ``type: ignore`` (blanket, no code) or any other code stays a
 # finding. Keeps the rule's teeth on genuine error-silencing without footgunning
 # the framework-mandated annotations every pydantic project needs.
 _ALLOWED_SUPPRESSION_CODES = frozenset({"TC001", "TC002", "TC003", "prop-decorator"})
 _NOQA_CODES = re.compile(r"noqa(?::\s*(?P<codes>[A-Za-z0-9, ]+))?")
 _TYPE_IGNORE_CODES = re.compile(r"type:\s*ignore(?:\[(?P<codes>[^\]]*)\])?")
 _SUPPRESSION_CODE_PATTERNS: dict[str, tuple[re.Pattern[str], ...]] = {
    "python": (_NOQA_CODES, _TYPE_IGNORE_CODES),
 }
 _HYGIENE_TEXT: dict[str, tuple[str, str]] = {
    "no_inline_comments": (
        "inline comment trailing code — keep narration out of the code",
@@ -63,11 +79,29 @@ def _comment_findings(
    if _is_inline(lines, row, col):
        out.append(_finding(rel_path, row + 1, "no_inline_comments", standard))
    text = comment.text.decode(errors="replace") if comment.text else ""
-    if any(marker in text for marker in _SUPPRESSIONS.get(language, ())):
+    if any(
        marker in text for marker in _SUPPRESSIONS.get(language, ())
    ) and not _suppression_allowed(text, language):
        out.append(_finding(rel_path, row + 1, "no_lint_suppressions", standard))
    return out
 def _suppression_allowed(text: str, language: str) -> bool:
    """True iff this suppression lists only sanctioned framework-escape codes.
    A bare marker (no code) suppresses everything and is never allowed; a
    language with no code grammar here (e.g. TypeScript) is never allowed.
    """
    codes: list[str] = []
    for pattern in _SUPPRESSION_CODE_PATTERNS.get(language, ()):
        for match in pattern.finditer(text):
            group = match.group("codes")
            if group is None:
                return False
            codes += [c.strip() for c in re.split(r"[,\s]+", group) if c.strip()]
    return bool(codes) and all(c in _ALLOWED_SUPPRESSION_CODES for c in codes)
 def _is_inline(lines: list[bytes], row: int, col: int) -> bool:
    if row >= len(lines):
        return False
@@ -37,16 +37,18 @@ _MAX_CONCERNS_PER_FILE = 1
 # SQLAlchemy session methods + 2.0 constructs that signal data access. A route
 # whose body calls one of these is doing a repository's / service's job.
 # Transaction-lifecycle calls (commit / flush / refresh) are deliberately NOT
 # here: a thin route legitimately commits the unit of work after delegating to a
 # service — an explicit `await db.commit()` in the handler is a common pattern
 # (e.g. when middleware-driven auto-commit is unreliable) and must not, on its
 # own, count as the route doing data access.
 _DB_METHODS = frozenset(
    {
        "execute",
        "scalar",
        "scalars",
        "commit",
        "add",
        "add_all",
        "flush",
        "refresh",
        "merge",
        "query",
    }
@@ -41,6 +41,13 @@ _IGNORE_DIRS = frozenset(
    }
 )
 # Directory trees that are never placement-enforced. Test trees legitimately
 # define fixtures, fakes, and helper functions of every kind, and documentation
 # trees (e.g. a docs site) are not code at all. A candidate module sitting under
 # any of these path segments is dropped from the map so a test or docs task is
 # never stranded — this holds for any project, not just RoboCo.
 _NON_SOURCE_SEGMENTS = frozenset({"tests", "test", "docs", "__tests__"})
 # Directory-name keywords -> (purpose, forbidden definition kinds). Only kinds
 # the classifiers actually emit can ever fire, so extra entries are harmless.
 _MODULE_PATTERNS: tuple[tuple[frozenset[str], str, tuple[DefinitionKind, ...]], ...] = (
@@ -85,6 +92,9 @@ _LANGUAGE_BY_SUFFIX = {".py": "python", ".ts": "typescript", ".tsx": "typescript
 _IMPERATIVE = re.compile(r"\b(never|don't|do not|avoid|no)\b", re.IGNORECASE)
 _CODE_SPAN = re.compile(r"`([^`]+)`")
 _MAX_LIFTED_RULES = 25
 # A separator-free, all-lowercase token shorter than this is treated as a common
 # word and not lifted into a custom rule (it would match everywhere).
 _MIN_SPECIFIC_TOKEN_LEN = 12
 def derive_from_scan(root: Path | str) -> ConventionsStandard:
@@ -105,6 +115,15 @@ def derive_from_scan(root: Path | str) -> ConventionsStandard:
 # rule to warn per-rule via the panel editor or the committed file.
 _PLACEMENT_DEFAULT: RuleLevel = "block"
 # 'helper' is the catch-all kind — ANY top-level function classifies as one — so
 # a misplaced helper is too weak a signal to hard-block: a route file's small
 # private glue would otherwise strand a task. Helper placement therefore seeds at
 # warn, while the precise kinds (model / route / component) stay block. A project
 # can still promote no_helpers_in_<leaf> to block in .roboco/conventions.yml. The
 # fat-handler signal that does have teeth is the body-level thin_routes check.
 _SOFT_PLACEMENT_KINDS = frozenset({"helper"})
 _SOFT_PLACEMENT_DEFAULT: RuleLevel = "warn"
 # Modularity rules — the separation-of-concerns checks that go beyond linting.
 # Cohesion + god-class apply to any classifiable project; the body checks are
 # stack-specific (thin routes for Python APIs, thin components for TS/React), so
@@ -141,7 +160,12 @@ def _seed_rules(modules: list[Module], languages: list[str]) -> dict[str, Rule]:
        leaf = module.path.rstrip("/").rsplit("/", 1)[-1]
        for kind in module.forbidden:
            name = f"no_{kind}s_in_{leaf}"
-            rules.setdefault(name, Rule(name=name, level=_PLACEMENT_DEFAULT))
+            level = (
                _SOFT_PLACEMENT_DEFAULT
                if kind in _SOFT_PLACEMENT_KINDS
                else _PLACEMENT_DEFAULT
            )
            rules.setdefault(name, Rule(name=name, level=level))
    if languages:
        for name, any_level in _MODULARITY_ANY.items():
            rules.setdefault(name, Rule(name=name, level=any_level))
@@ -168,9 +192,13 @@ def _scan_modules(root: Path) -> list[Module]:
            spec = _match_module(name)
            if spec is None:
                continue
-            rel = (Path(dirpath) / name).relative_to(root).as_posix()
+            rel = (Path(dirpath) / name).relative_to(root)
            if _NON_SOURCE_SEGMENTS.intersection(rel.parts):
                continue
            purpose, forbidden = spec
-            modules.append(Module(path=rel, purpose=purpose, forbidden=list(forbidden)))
+            modules.append(
                Module(path=rel.as_posix(), purpose=purpose, forbidden=list(forbidden))
            )
    return modules
@@ -207,6 +235,21 @@ def _lift_claude_md(root: Path) -> list[CustomRule]:
    return rules
 def _is_specific_token(token: str) -> bool:
    """Only lift identifiers specific enough not to match everywhere.
    A bare common-word token (``commit``, ``triage``) lifted into a regex would
    flag every legitimate use of the word — pure noise. Keep tokens with a
    separator (``.`` / ``_`` / ``/`` / ``-``), any uppercase letter or digit, or
    a length that makes an incidental match unlikely.
    """
    if any(sep in token for sep in "._/-"):
        return True
    if any(ch.isupper() or ch.isdigit() for ch in token):
        return True
    return len(token) >= _MIN_SPECIFIC_TOKEN_LEN
 def _rule_from_line(line: str, seen: set[str]) -> CustomRule | None:
    if not _IMPERATIVE.search(line):
        return None
@@ -214,6 +257,8 @@ def _rule_from_line(line: str, seen: set[str]) -> CustomRule | None:
    if span is None:
        return None
    token = span.group(1).strip()
    if not _is_specific_token(token):
        return None
    rule_id = _slug(token)
    if not token or not rule_id or rule_id in seen:
        return None
@@ -13,6 +13,7 @@ on are pure.
 from __future__ import annotations
 import subprocess
 from dataclasses import dataclass
 from pathlib import Path
 from typing import TYPE_CHECKING, Any
@@ -39,7 +40,7 @@ if TYPE_CHECKING:
    from roboco.db.tables import ProjectTable
 _SCAFFOLD_BRANCH = CONVENTIONS_SCAFFOLD_BRANCH
-_AMBIENT_CHAR_CAP = 1200
+_AMBIENT_CHAR_CAP = 2000
@dataclass(frozen=True)
@@ -63,28 +64,32 @@ class ConventionsHealth:
 class ConventionsService(BaseService):
    """Cache, render, scaffold, and restore a project's conventions standard."""
-    async def get_map(self, project: ProjectTable) -> ConventionsStandard:
+    async def get_map(
        self, project: ProjectTable, *, workspace: Path | None = None
    ) -> ConventionsStandard:
        """Return the effective standard for ``project`` at its current HEAD."""
        pid = self._pid(project)
-        head = self._head_sha(project)
+        root, head = self._resolve(project, workspace)
        cached = await self._cache_get(pid, head)
        if cached is not None:
            return ConventionsStandard.model_validate(cached.effective_map)
-        file_standard, status = self._read_committed_standard(project)
+        file_standard, status = self._read_committed_standard(root)
        if status == "degraded":
            last_good = await self._latest_ok_map(pid)
            if last_good is not None:
                await self._cache_put(pid, head, last_good, status)
                return last_good
-        mapping = effective_map(self._derive(project), file_standard)
+        mapping = effective_map(self._derive(root), file_standard)
        await self._cache_put(pid, head, mapping, status)
        return mapping
-    async def baseline_constraints(self, project: ProjectTable) -> list[str]:
+    async def baseline_constraints(
        self, project: ProjectTable, *, workspace: Path | None = None
    ) -> list[str]:
        """Render the project's block rules + module boundaries as constraints."""
-        mapping = await self.get_map(project)
+        mapping = await self.get_map(project, workspace=workspace)
        constraints = [
            f"Convention (block): {name.replace('_', ' ')}"
            for name, rule in mapping.rules.items()
@@ -98,33 +103,53 @@ class ConventionsService(BaseService):
        ]
        return constraints
-    async def render_ambient_block(self, project: ProjectTable) -> str:
+    async def render_ambient_block(
-        """Render a compact, bounded 'Architectural Standard' prompt block."""
+        self, project: ProjectTable, *, workspace: Path | None = None
-        mapping = await self.get_map(project)
+    ) -> str:
-        lines = [
+        """Render a compact, bounded 'Architectural Standard' prompt block.
        Only modules that actually forbid a kind are listed — an unconstrained
        module adds no signal. If the module list would exceed the budget it is
        truncated at a line boundary with a ``+N more`` pointer (never cut
        mid-line), and the block-level rule summary is always kept.
        """
        mapping = await self.get_map(project, workspace=workspace)
        header = [
            "## Architectural Standard",
            "Place each definition in the module that owns its kind:",
        ]
-        for module in mapping.modules:
+        constrained = [m for m in mapping.modules if m.forbidden]
            suffix = (
                f" — forbidden: {', '.join(module.forbidden)}"
                if module.forbidden
                else ""
            )
            lines.append(f"- `{module.path}`: {module.purpose}{suffix}")
        block = sorted(n for n, r in mapping.rules.items() if r.level == "block")
-        if block:
+        footer = ["Block-level rules: " + ", ".join(block) + "."] if block else []
-            lines.append("Block-level rules: " + ", ".join(block) + ".")
+
-        text = "\n".join(lines)
+        # Reserve room for the fixed header/footer plus a possible '+N more' line
-        if len(text) > _AMBIENT_CHAR_CAP:
+        # so the budget is spent on whole module lines.
-            return text[: _AMBIENT_CHAR_CAP - 1].rstrip() + "…"
+        reserve = len("\n".join(header + footer)) + 80
-        return text
+        budget = max(0, _AMBIENT_CHAR_CAP - reserve)
        kept: list[str] = []
        used = 0
        for module in constrained:
            line = (
                f"- `{module.path}`: {module.purpose} "
                f"— forbidden: {', '.join(module.forbidden)}"
            )
            if used + len(line) + 1 > budget:
                break
            kept.append(line)
            used += len(line) + 1
        if len(kept) < len(constrained):
            extra = len(constrained) - len(kept)
            plural = "s" if extra != 1 else ""
            kept.append(
                f"- (+{extra} more module{plural} — see .roboco/conventions.yml)"
            )
        return "\n".join(header + kept + footer)
    async def scaffold(
        self, project: ProjectTable, *, workspace: Path | None = None
    ) -> ScaffoldResult:
        """Open a PR adding the auto-scaffolded ``.roboco/conventions.yml``."""
-        mapping = await self.get_map(project)
+        mapping = await self.get_map(project, workspace=workspace)
        return await self._publish(
            project, render_yaml(mapping), restore=False, workspace=workspace
        )
@@ -134,7 +159,11 @@ class ConventionsService(BaseService):
    ) -> ScaffoldResult:
        """Open a PR re-committing the file from the last-good map (or a scan)."""
        last_good = await self._latest_ok_map(self._pid(project))
-        mapping = last_good if last_good is not None else self._derive(project)
+        if last_good is not None:
            mapping = last_good
        else:
            root, _ = self._resolve(project, workspace)
            mapping = self._derive(root)
        return await self._publish(
            project, render_yaml(mapping), restore=True, workspace=workspace
        )
@@ -151,10 +180,12 @@ class ConventionsService(BaseService):
            project, render_yaml(standard), restore=False, workspace=workspace
        )
-    async def health(self, project: ProjectTable) -> ConventionsHealth:
+    async def health(
        self, project: ProjectTable, *, workspace: Path | None = None
    ) -> ConventionsHealth:
        """Report the standard's status at HEAD + the last-good commit SHA."""
        pid = self._pid(project)
-        head = self._head_sha(project)
+        _root, head = self._resolve(project, workspace)
        current = await self._cache_get(pid, head)
        last_ok = await self._latest_ok_row(pid)
        return ConventionsHealth(
@@ -233,14 +264,71 @@ class ConventionsService(BaseService):
        path = Path(project.workspace_path)
        return path if path.exists() else None
-    def _derive(self, project: ProjectTable) -> ConventionsStandard:
+    async def resolve_workspace(self, project: ProjectTable) -> Path | None:
-        root = self._workspace_root(project)
+        """Ensure (clone / refresh) the project's read clone; None if unavailable."""
        from roboco.services.workspace import get_workspace_service
        try:
            return await get_workspace_service(self.session).ensure_read_clone(
                project.slug
            )
        except Exception as exc:
            self.log.warning(
                "conventions: read-clone unavailable; falling back to workspace_path",
                project=getattr(project, "slug", None),
                error=str(exc),
            )
            return None
    def _resolve(
        self, project: ProjectTable, workspace: Path | None
    ) -> tuple[Path | None, str]:
        """Resolve the repo root to read the standard from, plus its HEAD sha.
        Uses an explicit ``workspace`` (the read clone the caller resolved via
        :meth:`resolve_workspace`) when given, else the legacy persisted
        ``workspace_path``. When a clone is resolved, its path + real HEAD are
        persisted back onto the project — the backfill — so the next read (and
        the cache key) reflect the committed standard, even for a project
        created before the standard existed.
        """
        root: Path | None = None
        if workspace is not None and Path(workspace).exists():
            root = Path(workspace)
        else:
            root = self._workspace_root(project)
        if root is None:
            return None, self._head_sha(project)
        sha = self._head_sha_at(root)
        project.workspace_path = str(root)
        if sha is not None:
            # Only a real rev-parse (an actual git clone) updates the cache key;
            # a non-git legacy path keeps the persisted head_commit / "HEAD".
            project.head_commit = sha
        return root, sha or self._head_sha(project)
    @staticmethod
    def _head_sha_at(root: Path) -> str | None:
        """The clone's HEAD sha, or None if ``root`` is not a readable git repo."""
        try:
            result = subprocess.run(
                ["git", "rev-parse", "HEAD"],
                cwd=str(root),
                capture_output=True,
                text=True,
                check=False,
                timeout=10,
            )
        except (OSError, subprocess.SubprocessError):
            return None
        return result.stdout.strip() or None
    def _derive(self, root: Path | None) -> ConventionsStandard:
        return derive_from_scan(root) if root is not None else ConventionsStandard()
    def _read_committed_standard(
-        self, project: ProjectTable
+        self, root: Path | None
    ) -> tuple[ConventionsStandard | None, str]:
        root = self._workspace_root(project)
        if root is None:
            return None, "missing"
        path = root / ".roboco" / "conventions.yml"
@@ -703,9 +703,9 @@ class TaskService(BaseService):
        if project is None:
            return []
        try:
-            return await get_conventions_service(self.session).baseline_constraints(
+            conv = get_conventions_service(self.session)
-                project
+            workspace = await conv.resolve_workspace(project)
-            )
+            return await conv.baseline_constraints(project, workspace=workspace)
        except Exception as exc:
            self.log.warning(
                "Baseline-constraints attach failed (non-fatal)",
@@ -192,6 +192,12 @@ _ENSURE_WORKSPACE_LOCKS: dict[tuple[str, str], asyncio.Lock] = {}
 # clone) is attempted at most once per project per process, even across agents.
 _SCAFFOLD_ATTEMPTED: set[str] = set()
 # Project-level read clones (the conventions standard) refresh at most this often
 # on a healthy clone, so a burst of spawns / task creations doesn't fetch on
 # every call. Keyed by workspace path; process-wide.
 _READ_CLONE_FETCH_TTL_SECONDS = 30.0
 _read_clone_synced: dict[str, float] = {}
 def _ensure_lock_for(project_slug: str, agent_slug: str) -> asyncio.Lock:
    """Return the asyncio.Lock for a (project, agent) pair, creating lazily."""
@@ -770,6 +776,68 @@ class WorkspaceService:
                error=str(exc),
            )
    async def ensure_read_clone(self, project_slug: str) -> Path:
        """Ensure a project-level read clone pinned to the default branch's HEAD.
        The architectural-conventions standard is read from the committed
        ``.roboco/conventions.yml`` plus a scan of the repo tree. Per-agent
        working clones are the wrong source — one may sit on a feature branch or
        be stale — so metadata reads use this dedicated clone instead. It is
        never mounted into an agent container and is always hard-reset to
        ``origin/<default_branch>``, which makes destructive refresh safe. This
        is what lets the standard work for a project created before the standard
        existed: no manually-configured ``workspace_path`` is required.
        """
        from roboco.services.project import get_project_service
        project_service = get_project_service(self.session)
        project = await project_service.get_by_slug(project_slug)
        if not project:
            raise WorkspaceError(f"Project not found: {project_slug}")
        default_branch = project.default_branch or "master"
        git_url = project.git_url
        workspace = self.root / project_slug / "_meta" / "conventions"
        lock = _ensure_lock_for(project_slug, "_meta-conventions")
        async with lock:
            if self._is_workspace_healthy(workspace):
                now = _monotonic()
                last = _read_clone_synced.get(str(workspace), -math.inf)
                if (now - last) >= _READ_CLONE_FETCH_TTL_SECONDS:
                    await asyncio.to_thread(self._prune_broken_refs, workspace)
                    await self._fetch_origin_best_effort(workspace, project_slug)
                    await asyncio.to_thread(
                        self._reset_to_default, workspace, default_branch
                    )
                    _read_clone_synced[str(workspace)] = _monotonic()
                return workspace
            if workspace.exists():
                shutil.rmtree(workspace)
            git_token = await self._resolve_git_token(
                project_service, project_slug, git_url
            )
            await self._clone_repo(
                workspace, git_url, default_branch, git_token, agent=None
            )
            _read_clone_synced[str(workspace)] = _monotonic()
            return workspace
    @staticmethod
    def _reset_to_default(workspace: Path, default_branch: str) -> None:
        """Hard-reset the read clone to ``origin/<default_branch>``. Best-effort."""
        def _git(*args: str) -> subprocess.CompletedProcess[str]:
            return subprocess.run(
                ["git", *args],
                cwd=str(workspace),
                capture_output=True,
                text=True,
                check=False,
            )
        _git("checkout", default_branch)
        _git("reset", "--hard", f"origin/{default_branch}")
    async def _clone_repo(
        self,
        workspace: Path,
@@ -8,7 +8,7 @@ from unittest.mock import AsyncMock, MagicMock
 import pytest
 from fastapi import HTTPException
 from roboco.api.routes import secretary_live as sl
-from roboco.api.routes.secretary_live import (
+from roboco.api.schemas.secretary_live import (
    AgentEvent,
    LiveMessageRequest,
    StartSecretaryRequest,
@@ -59,6 +59,41 @@ def test_python_marker_not_applied_to_typescript() -> None:
    assert _rules(findings, "no_lint_suppressions") == []
 def _src(line: str) -> bytes:
    # Build via a variable so a literal suppression marker never sits on this
    # test file's own source line (ruff would parse it as a real directive).
    return (line + "\n").encode()
 def test_runtime_typing_noqa_is_allowed() -> None:
    # A runtime-needed typing import (pydantic / SQLAlchemy) — the sanctioned
    # escape, not error-silencing.
    findings = check_hygiene(
        "a.py", _src("from uuid import UUID  # noqa: TC003"), "python", _STD
    )
    assert _rules(findings, "no_lint_suppressions") == []
 def test_pydantic_prop_decorator_ignore_is_allowed() -> None:
    findings = check_hygiene(
        "a.py", _src("y = f()  # type: ignore[prop-decorator]"), "python", _STD
    )
    assert _rules(findings, "no_lint_suppressions") == []
 def test_other_ignore_code_is_still_flagged() -> None:
    findings = check_hygiene(
        "a.py", _src("x = bad()  # type: ignore[arg-type]"), "python", _STD
    )
    assert _rules(findings, "no_lint_suppressions")
 def test_mixed_allowed_and_disallowed_codes_is_flagged() -> None:
    # One allowed code does not launder a disallowed one alongside it.
    findings = check_hygiene("a.py", _src("x = 1  # noqa: TC003, E501"), "python", _STD)
    assert _rules(findings, "no_lint_suppressions")
 def test_rule_level_override_from_standard() -> None:
    std = ConventionsStandard(
        rules={"no_inline_comments": Rule(name="no_inline_comments", level="block")}
@@ -77,6 +77,21 @@ def test_thin_routes_clean_when_delegating_to_a_service() -> None:
    assert "thin_routes" not in rules
 def test_thin_routes_allows_explicit_commit_in_route() -> None:
    # Committing the unit of work after delegating is a common, valid pattern —
    # a bare `db.commit()` must not count as the route doing data access.
    rules = _py(
        "from fastapi import APIRouter\n"
        "router = APIRouter()\n"
        "@router.post('/users')\n"
        "async def create_user(svc, db):\n"
        "    user = await svc.create()\n"
        "    await db.commit()\n"
        "    return user\n"
    )
    assert "thin_routes" not in rules
 # --- Thin components: data fetching belongs in a hook ----------------------- #
@@ -48,11 +48,35 @@ def test_scan_ignores_vendored_directories(tmp_path: Path) -> None:
 def test_scan_lifts_claude_md_imperative_into_custom_rule(tmp_path: Path) -> None:
    _sample_repo(tmp_path)
-    (tmp_path / "CLAUDE.md").write_text("- Never use `print()`; use the logger.\n")
+    (tmp_path / "CLAUDE.md").write_text("- Never call `os.system()`; use subprocess.\n")
    custom = derive_from_scan(tmp_path).custom
    assert custom
    assert custom[0].level == "warn"
-    assert "print" in custom[0].pattern
+    assert custom[0].id == "os-system"
 def test_scan_skips_bare_common_word_in_claude_md(tmp_path: Path) -> None:
    # A bare word like `commit` would match everywhere; it must not be lifted.
    _sample_repo(tmp_path)
    (tmp_path / "CLAUDE.md").write_text("- Never `commit` straight to master.\n")
    assert derive_from_scan(tmp_path).custom == []
 def test_scan_excludes_test_and_docs_trees(tmp_path: Path) -> None:
    (tmp_path / "tests" / "unit" / "services").mkdir(parents=True)
    (tmp_path / "docs" / "api").mkdir(parents=True)
    (tmp_path / "app" / "services").mkdir(parents=True)
    paths = {m.path for m in derive_from_scan(tmp_path).modules}
    assert "app/services" in paths
    assert not any(p.startswith(("tests/", "docs/")) for p in paths)
 def test_scan_seeds_helper_placement_as_warn(tmp_path: Path) -> None:
    _sample_repo(tmp_path)
    std = derive_from_scan(tmp_path)
    # A misplaced model is a hard error; a misplaced helper only warns.
    assert std.rules["no_models_in_routers"].level == "block"
    assert std.rules["no_helpers_in_routers"].level == "warn"
 def test_render_yaml_round_trips_through_parse(tmp_path: Path) -> None:
@@ -0,0 +1,66 @@
 """ConventionsService root/HEAD resolution + backfill persistence (no DB)."""
 from __future__ import annotations
 import subprocess
 from types import SimpleNamespace
 from typing import TYPE_CHECKING
 from roboco.services.conventions import ConventionsService
 if TYPE_CHECKING:
    from pathlib import Path
 def _git_repo(root: Path) -> str:
    (root / "roboco" / "services").mkdir(parents=True)
    (root / "roboco" / "services" / "x.py").write_text("def f():\n    return 1\n")
    for cmd in (
        ["git", "init", "-q"],
        ["git", "add", "-A"],
        ["git", "-c", "user.email=t@t", "-c", "user.name=t", "commit", "-qm", "i"],
    ):
        subprocess.run(cmd, cwd=root, check=True, capture_output=True)
    return subprocess.run(
        ["git", "rev-parse", "HEAD"],
        cwd=root,
        capture_output=True,
        text=True,
        check=False,
    ).stdout.strip()
 def _svc() -> ConventionsService:
    return ConventionsService(session=None)  # type: ignore[arg-type]
 def test_resolve_reads_clone_head_and_backfills(tmp_path: Path) -> None:
    sha = _git_repo(tmp_path)
    project = SimpleNamespace(workspace_path=None, head_commit=None, slug="p")
    root, head = _svc()._resolve(project, tmp_path)
    assert root == tmp_path
    assert head == sha
    # The backfill: the resolved path + real HEAD are persisted on the project.
    assert project.workspace_path == str(tmp_path)
    assert project.head_commit == sha
 def test_resolve_non_git_path_keeps_persisted_head(tmp_path: Path) -> None:
    project = SimpleNamespace(
        workspace_path=str(tmp_path), head_commit="deadbeef", slug="p"
    )
    _root, head = _svc()._resolve(project, None)
    # A non-git legacy path must not clobber the persisted head_commit.
    assert head == "deadbeef"
    assert project.head_commit == "deadbeef"
 def test_resolve_no_workspace_returns_none_root() -> None:
    project = SimpleNamespace(workspace_path=None, head_commit=None, slug="p")
    root, head = _svc()._resolve(project, None)
    assert root is None
    assert head == "HEAD"
 def test_head_sha_at_non_git_returns_none(tmp_path: Path) -> None:
    assert ConventionsService._head_sha_at(tmp_path) is None