Feat/provider overload break (#242)

* feat(conventions): standard schema models + effective-map merge

* feat(orchestrator): park provider on persistent server overload (529/500)

A 429 rate limit already parks a provider — queue its spawns, probe until it
recovers — but a persistent 529/500/503 overload had no such break: the run
died and the orchestrator crash-retried straight back into the overload,
burning tokens in a respawn loop.

Generalize the park to provider-unavailability. On a non-graceful Anthropic
agent exit, match the API's overload markers (overloaded_error /
internal_server_error / "API Error: 5xx") against the dead container's own
output and park the provider with kind="overloaded"; the existing spawn gate
already queues any parked provider, and the probe-resume loop revives the task
when it recovers. Grok keeps its exit-75 path; both now route through one
_park_provider_unavailable helper. Markers are kept specific so an agent that
merely writes about HTTP 500/529 can't trip the break.

Fix the recovery probe to require a 2xx: it treated any non-429 as recovered,
so a probe that itself got a 529 would have resumed agents straight back into
the overload — wrong for the new path and for a 429 that lifts into a 5xx.

Gated by ROBOCO_OVERLOAD_BREAK_ENABLED (default on; off => crash-retry).

---------

Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
Renzo F
2026-06-22 03:49:45 +02:00
committed by GitHub
co-authored by Renn F
parent ba8b877a05
commit 01e10ad693
14 changed files with 826 additions and 29 deletions
+2
View File
@@ -44,6 +44,7 @@ dependencies = [
# Direct imports (promoted from transitive)
"cryptography", # utils/crypto.py — Fernet-encrypted project git tokens
"packaging", # services/toolchain.py — PEP 440 requires-python resolution
"pyyaml", # foundation/policy/conventions — .roboco/conventions.yml parse
"claude-agent-sdk>=0.2.105",
]
@@ -72,6 +73,7 @@ dev = [
# Type Stubs
"types-passlib",
"types-python-jose",
"types-PyYAML",
# Development
"ipython",