mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
10f039c36f
* fix(notifications): exponential backoff + CAS claim for expired-unacked re-escalation The sweep re-escalated every expired unacked ack-required notification on every ~60s tick, forever — the live incident: 3 fresh blocker escalations + Telegram DMs per minute from a static stale pile. Now each notification carries reescalation_count / last_reescalated_at / reescalation_delivered_count (migration 079): first fire at expiry, then doubling intervals from 1h capped at 24h, hard stop after ROBOCO_NOTIFICATION_MAX_REESCALATIONS (default 5) with one permanent log carrying attempts-vs-delivered so 'seen and ignored' is distinguishable from 'route never worked'. The due/wait/capped decision is a pure function in foundation/policy/communications.py. Per adversarial review, the attempt slot is claimed by compare-and-set (UPDATE ... WHERE reescalation_count = :n) BEFORE delivery — the previous draft leaned on the 60s dedup window, which never engages for BLOCKER_ESCALATION (_LOOP_PRONE_TYPES excludes it), so concurrent sweeps would have double-delivered. A lost claim skips delivery outright. Legacy rows read as count=0 and keep today's first-fire semantics. 61 tests incl. a two-session CAS race and a real alembic upgrade/downgrade round trip. * feat(budgets): per-task and per-project cost budgets (flag-gated) tasks.budget_usd + projects.monthly_budget_usd (migration 080, chained on 079; adds ix_agent_spawn_sessions_task_id since both enforcement seams filter on bare task_id). Behind ROBOCO_TASK_BUDGETS_ENABLED (default off, feature-flags card) — verifiably inert when off. Claim-time: a project-month-spend guard applies to WORK-STARTING claims only (i_will_work_on / i_will_plan) — per adversarial review, review/ doc/gate/inbound-PR claims are exempt so in-flight work can always finish reviewing and merging at cap. Spend counts closed sessions' estimated_cost_usd PLUS open sessions priced live from token snapshots (the original closed-only sum read parallel long sessions as $0). Sweep-side: the existing budget sweep also prices the active task's spend vs budget_usd (TaskType defaults when null); on breach the task is BLOCKED (HUMAN resolver, budget marker) BEFORE the graceful stop so the unclaim no-ops and the dispatcher never respawns onto it, and the CEO notification names both recovery steps. unblock on a budget-blocked task re-checks live spend and refuses while still over — no silent re-breach loop. Panel: budget inputs in both dialogs (0 rejected — a zero budget silently blocks everything), spend logic consolidated in TaskService.task_spend_usd. 42 new tests incl. a real-DB spend-query suite and a two-tick non-refire sweep test. * feat(eval): golden-task eval harness + doctrine cohort stamp roboco/eval: 6 BenchTaskSpec fixtures run through the real lifecycle in a disposable environment (the e2e_smoke harness's fake GitHub + local git origin + throwaway DB catalog — real isolation, not convention), scored deterministically (terminal status, revision_count, cycle time, tokens/cost via the agent_spawn_sessions task_id join) plus a local- model judge whose output is nested under a non_deterministic-marked object so cohort diffs don't read judge noise as regression. CLI: python -m roboco.eval run --role <slug> --cohort <name>. Source- checkout-only by declared posture (deptry-scoped ignore + a hard ImportError guard naming why; tests/ never ships in images or wheels). agent_spawn_sessions.doctrine_version (migration 081, chained on 080) is stamped at spawn-session finalize from the composed prompt layers — with the session's model column it identifies a cohort durably. Per adversarial review: bench runs patch the vault flags off (they were writing real markdown into the operator's vault), and the real-spawn OrchestratorStageSpawner is deliberately cut to NotImplementedError — spawned containers' MCP wiring resolves to the production orchestrator under real agent UUIDs, so real spawns wait for a dedicated follow-up; the injectable scripted spawner is the working path. Full suite 13852 passed / 94% coverage in the source worktree; deptry/mypy/xenon clean. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com>
58 lines
2.6 KiB
Python
58 lines
2.6 KiB
Python
"""Doctrine-version stamp on agent_spawn_sessions, for the eval harness.
|
|
|
|
The eval harness (roboco/eval/) scores a (role, model/provider config) cohort
|
|
by replaying golden tasks through a real agent spawn. To attribute a quality
|
|
delta to a prompt/doctrine change (fable-mode, ponytail, a team-prompt edit,
|
|
...) the resulting spawn session needs to carry a fingerprint of exactly what
|
|
system prompt it ran with — otherwise two cohort runs are only comparable if
|
|
the operator remembers to keep everything else byte-for-byte identical.
|
|
|
|
``doctrine_version`` is a short hash of the composed system prompt (base +
|
|
role + team + identity + doctrine layers) for that spawn, stamped at
|
|
``_finalize_spawn_session`` in roboco/runtime/orchestrator.py — NOT at
|
|
``_record_spawn_session`` (spawn creation). The composed prompt string itself
|
|
is not passed through the AgentConfig the finalize call site holds, but the
|
|
file it was written to (``config.blueprint_path``, from
|
|
``_generate_composed_prompt``) is still on disk and unchanged at finalize
|
|
time (nothing in the spawn/stop path deletes it), so the finalize call reads
|
|
it back and hashes it there. Every provider gets one — ``_prepare_agent_spawn``
|
|
composes and writes the blueprint unconditionally, before provider/route
|
|
resolution, so GROK agents carry a real blueprint file too, same as Claude.
|
|
Nullable + additive: every existing row, and any row where the read
|
|
genuinely fails (a provider-parked stub instance that never actually
|
|
spawned — ``blueprint_path=Path()`` — an evicted temp dir, ...), simply gets
|
|
NULL — a pure quality-of-life addition to the sessions the eval harness
|
|
scores, never a hard requirement of the spawn/stop path.
|
|
|
|
Revision ID: 081_doctrine_version
|
|
Revises: 080_task_project_budgets
|
|
Create Date: 2026-07-22
|
|
|
|
Note: re-chained onto 080_task_project_budgets (sibling PRs #652/#654 own
|
|
079/080 at this branch's base commit, da4d9b33, where 078 was the head);
|
|
080 does not exist in this worktree, so the local migration-graph/enum-parity
|
|
tests are expected to fail here until this branch integrates alongside its
|
|
siblings — the same expected-failure posture the budgets sibling reported.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import sqlalchemy as sa
|
|
from alembic import op
|
|
|
|
revision = "081_doctrine_version"
|
|
down_revision = "080_task_project_budgets"
|
|
branch_labels: dict[str, str] | None = None
|
|
depends_on: dict[str, str] | None = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.add_column(
|
|
"agent_spawn_sessions",
|
|
sa.Column("doctrine_version", sa.String(length=32), nullable=True),
|
|
)
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.drop_column("agent_spawn_sessions", "doctrine_version")
|