mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
d87e2d9b4e4c7e263b169f55c80aa87e51f7b545
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
93739a9dca |
fix(notifications): stop tick-wide row locks + poisoned-session swallows behind the PendingRollbackError 500s (#743)
* fix(notifications): release re-escalation row locks per row; stop swallowing DB errors in notify_get The re-escalation sweep ran one tick-wide transaction, so each CAS claim's row lock was held across every remaining delivery until the single commit — a concurrent mark-read UPDATE on a claimed row starved into the 60s lock_timeout. The sweep now commits per row (claim commit releases the lock before delivery and makes the burned slot durable), re-fetches each row by snapshotted id so one row's rollback can't expire the rest of the tick, and savepoints each recipient's delivery. notify_get's bare except swallowed the resulting LockNotAvailableError into a false "notification not found" and returned a poisoned session to the commit-at-send middleware, which blew up with PendingRollbackError; it now catches only the two domain outcomes. defer_after_commit's listeners fire on SAVEPOINT release too, which would have drained deferred telegram/bus work before real durability — they now skip savepoint boundaries via get_nested_transaction() (the root get_transaction() is non-None inside the listener even at a real commit). acknowledge_for_recipient's Redis dedup-clear moved before the flush so the row lock never spans a Redis round-trip. The five best-effort CEO-notify swallows that persist notification rows are savepointed. * fix(services): contain swallowed best-effort DB write failures instead of poisoning the session Sweep of the same class as the notify_get incident: broad except-Exception handlers that swallow a failure whose try-body writes through the shared session leave the session rollback-pending, and the verb/request then dies later with PendingRollbackError at commit-at-send. Confirmed-dangerous sites now run the write inside a savepoint (safe since defer_after_commit skips savepoint boundaries): ceo_approve's verified-stamp, completion/pitch/postmortem-style CEO notifies, _inherit_upstream_base, _link_commit_to_task (covers every commit route), board-program LEARN records, the QA/PR-gate/PM-merge verified-stamps, and the documenter->PM handoff. _ack_pending_wake_notifications gets the same treatment so a wake-ack failure can't fail the A2A read. telegram_inbound's per-update loop and intake confirm roll back explicitly instead (their success paths commit mid-flow, so a savepoint doesn't fit). A swallowed savepoint rollback fully expires any ORM object mutated inside the block, and the next attribute read raises MissingGreenlet — strictly worse than the original bug. The two paths that keep using the object after the swallow (doc handoff's envelope build, base inheritance's claim continuation) refresh it in the except path; regression tests run against a real session and were verified to fail with the refresh reverted. * test: shape mocked session.execute results so sync accessors stop leaking unawaited coroutines An AsyncMock's auto-created children are themselves AsyncMock, so production code that correctly awaits session.execute() and then calls sync accessors (.scalars().all(), .scalar_one_or_none()) on the result was silently collecting unawaited coroutines in 22 test files — 80 RuntimeWarnings per unit run, and in test_flow_soup_guard one mock raised a real TypeError that a coincidentally-matching invalid_state envelope masked. Each affected fixture now returns a plain MagicMock shaped like a real Result. Zero AsyncMock warnings remain. * docs: document per-row sweep commits and the savepoint/refresh containment pattern --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
879afc14a4 |
Board Program LEARN context, ruff 0.16, and verb-rejection observability (#700)
* fix(board): LEARN decisions name the item, not its per-cycle index A cycle's reject reasons are rendered into the NEXT cycle's exploration prompt, but the ref recorded alongside each reason was the item's stored id (item-0/item-1) — a per-cycle index that means something different every cycle and appears nowhere the explorer can resolve. The reason survived the loop; what it was about did not. Record the item's title instead, via a shared learn_ref() helper (falls back to the id when title-less, and reads target_task_title for Scales, whose items name the live task they mutate). * chore(lint): satisfy ruff 0.16 — keyword-only signatures and markdown formatting The dev toolchain resolved ruff 0.16.0, which stabilises PLR0917 (too many positional arguments) and formats python code blocks inside markdown. Both fired repo-wide and neither had anything to do with the code they flagged. - 36 signatures gain a `*` so their tail arguments are keyword-only, and the 104 call sites that passed them positionally are converted. mypy was the safety net for the static ones; the full suite caught nine more that only bind at runtime (the MCP tool functions, whose real callers already pass named JSON arguments). - 28 markdown files reformatted by 0.16's code-block formatter. - One RUF036 (`None` mid-union) autofixed in the GitLab provider. * fix(gateway): log the reason when a verb rejects A rejected envelope rides an HTTP 200, its body is never logged, and there is no trace table — so in the access log a verb an agent could not satisfy looks identical to one that worked. On 2026-07-25 four Board Programs (Periscope, Sentinel, Scales, Barfly) each POSTed their propose verb three or four times, persisted nothing, and left their exploration tasks PENDING; the reason was unrecoverable afterwards, from the logs or from the agents' own transcripts. Log error/message/remediate/missing plus the calling agent at envelope_to_response — the one chokepoint every v1 flow and do route returns through. Success envelopes stay silent. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
|
312ec990dd |
fix: prod triage 2026-07-08 — MCP auth residue, gateway envelopes, verb-loop cap, A2A interjection, manual spawn UX (#334)
* fix(auth): pass agent UUID to CLI-arg MCP servers (optimal/docs/search) The container token is HMAC-signed over the agent UUID (#314), but the optimal/docs/search MCP servers received the slug as their CLI arg and sent X-Agent-ID=<slug>, so every research/RAG/docs call 401ed with signature mismatch under enforced auth. Pass the already-computed agent_uuid in the three args lists instead. * fix(gateway): include remediate in gateway.rejected audit details Conventions-gate rejections carry the offending file:line listing only in the envelope's remediate field, which the audit row dropped -- ops logs showed just the violation count with no way to see what blocked. * fix(gateway): return envelope on do/commit git failure A GitError from the commit verb propagated to the generic middleware handler, so agents got a raw error blob with no remediate/next. Catch it and return an error envelope; 'no changes added to commit' with an explicit files list now names the mismatch and the omit-files fallback. * fix(agent-sdk): absolute rejection cap breaks slow-drip verb loops The verb circuit breaker only counted rejections inside a 60s sliding window, so an agent retrying i_am_done every 3-4 minutes looped for 30+ minutes without tripping it. Add a session-scoped cumulative per-(verb, task) cap at 3x the windowed limit that trips regardless of pacing. * feat(a2a): CEO chime-in interjects into the viewed conversation Previously reply_as_ceo re-homed the message into a canonical CEO<->target conversation with no panel surface, so a chime-in reported success but was invisible and only opportunistically delivered. interject_as_ceo now inserts the message into the conversation being viewed (from_agent=ceo, directed via an @target content prefix), bumps that conversation's counters with the unread ping keyed to the addressed participant, and both participants see it in transcript and read_a2a. * feat(panel): manual spawn carries task + message, surfaces refusals The agent detail page spawned with no request body (task/message impossible), the spawn button could double-fire (2.5ms double-POST seen live), and refusal reasons never reached the UI: readiness refusals were generic 500s and the already-running no-op looked like success. Detail page now uses SpawnAgentDialog, a synchronous ref guard blocks re-entry, AgentReadinessError maps to 409 with its reason shown, already_running is signalled and toasted, and a task_id builds a task-aware prompt instructing the claim (task_id alone never did), with the CEO's message appended as a note. * test(panel): align a2a page test with the interjection footer copy The chime-in rebuild changed the composer footer; the page-level test asserting the old copy was outside the rebuild's scoped vitest run. * fix(api): commit the request DB session before the response is sent FastAPI unwinds yield-dependencies after the response bytes go out, so get_db's post-yield commit raced the client's next request -- a verb could return ok while its claim/status write was still uncommitted (the e2e ok-without-effect flake family), and a failed commit was silently lost behind an already-sent 200. DbCommitMiddleware (innermost, pure ASGI) commits the session stashed by get_db_committed before forwarding http.response.start; commit failure now surfaces as a 5xx. get_db is untouched for its direct non-request callers. * fix(db): invalidate, not rollback, the session on request cancellation With the commit moved into the send path, the flow-verb timeout can cancel mid-commit; rolling back then issues another command over an asyncpg connection stranded mid-wire-protocol, and the poisoned connection segfaults uvloop/asyncpg when a later checkout recycles it (3/3 identical CI faulthandler dumps). On CancelledError discard the connection via session.invalidate() -- SQLAlchemy's documented handling for a timeout during commit -- and keep rollback for plain exceptions. --------- Co-authored-by: Renn F <rennf93@users.noreply.github.com> |
||
|
Renn F
|
41ef7f6b4e |
feat(gateway): C8 PM-decision gate windowed satisfaction
_check_pm_decision_required now requires the latest journal:decision within pm_decision_window_seconds (default 300). Older decisions no longer satisfy the gate. Adds JournalService.latest_decision_at. Future-tighten (out of scope): per-verb-group consumption tracking would need persistent state — Choreographer is per-request today. |
||
|
Renn F
|
4829f93a68 |
fix(gateway): unblock task claim; full Phase 0/1/2 remediation
Resolves the 100% claim-failure rate introduced by the gateway rewrite
(commit
|
||
|
Renn F
|
38246050d6 |
feat(gateway): audit-log every gate rejection
Choreographer takes an audit dep but never used it. Now every rejection Envelope (invalid_state, not_authorized, tracing_gap, not_found) writes a gateway.rejected row with verb + reason + missing fields. Forensic signal for stuck flows; no behaviour change for happy path. Adds AuditService.log_event() — generic free-form event_type write so the gateway doesn't have to extend AuditEventType for every new surface. Audit writes are best-effort: a failure logs a warning but never propagates, since the agent's response Envelope is the contract. |