[77719d3f] A2A team telemetry: coordination event notifications for 5 event types (#477)

* [13d03d5c] Add 5 coordination-event notification producers + wire at chokepoints (#472) (#474)

* [13d03d5c] Add 5 coordination-event notification producer methods

* [13d03d5c] Wire reassignment/collision/unblock/dependency-revival notifications

* [13d03d5c] Wire stale-claim-reaped notification into orchestrator reaper

* [13d03d5c] fix(runtime): guard reaper's UUID annotation + defensive attr access

The stale-claim-reaped notification hook added a runtime-unquoted
`UUID` type annotation (only imported under TYPE_CHECKING, so the
module raised NameError on import) and a direct `t.assigned_to`
attribute access that crashes against the minimal test doubles the
existing reaper test suite uses. Quote the annotation and switch to
getattr-defensive access, matching `_assignee_is_provider_parked`'s
existing convention in the same file.

* [13d03d5c] test(notification): unit coverage for 5 coordination-event producers

One test per new send_* method (reassignment, collision-sequencing,
unblock, dependency-revival, stale-claim-reaped) following the
existing _FakeDb/_patch_db_context pattern, asserting subject/body/
related_task_id/priority/recipient-count, plus a no-recipients no-op
case for reassignment.

* [13d03d5c] test(task): prove reassign + unblock don't double-fire notifications

Two chokepoint-level tests mocking NotificationService at its defining
module: a repeated reassign() to the same already-current target skips
the notification (guarded by comparing against the pre-mutation
assignee), and a repeated unblock() on the same task only notifies
once since the second call short-circuits on the status!=BLOCKED
guard.

* [13d03d5c] style(task): ruff format the collision-sequencing wiring block

No behavior change — reflows the newly-added _notify_collision_sequencing
call site to satisfy ruff format's line-length rules.

* [13d03d5c] docs(backend): add coordination-event notification producers guide

Documented the 5 new NotificationService producers (reassignment, collision-sequencing,
unblock, dependency-revival, stale-claim-reaped) with fire conditions, double-fire
prevention mechanisms, and implementation patterns. Updated backend README to link the
new services guide for developers integrating new coordination events.

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>
Co-authored-by: Backend Documenter <be-doc@roboco.tech>

* [3ee8150b] Frontend: render coordination-event notifications + e2e smoke coverage (#475)

* [69777c3a] test(e2e-smoke): add coverage for soft-block + unblock coordination notifications (#471)

Co-authored-by: Frontend Developer 2 <fe-dev-2@roboco.tech>

* [8eb82639] Render 5 coordination-event notification types with task deep-links (#470)

* [8eb82639] feat(notifications): add APPROVAL type icon and deep-link component test

Add missing APPROVAL member to the frontend NotificationType enum to
match backend roboco/models/base.py, wire its icon into the existing
typeIcons Record in the notifications page, and add a component test
covering type rendering and the task deep-link.

* [8eb82639] docs(notifications): document 5 coordination-event types and APPROVAL enum addition

Added comprehensive reference guide explaining the 5 notification types
(TASK_ASSIGNMENT, BLOCKER_ESCALATION, REVIEW_REQUEST, DOCUMENTATION_REQUEST,
APPROVAL), their visual identities (icon + color), use cases, and
deep-linking behavior to related tasks. Updated panel README with quick
reference table. TypeScript Record pattern ensures exhaustive type coverage
at build time.

---------

Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech>
Co-authored-by: Frontend Documenter <fe-doc@roboco.tech>

---------

Co-authored-by: Frontend Developer 2 <fe-dev-2@roboco.tech>
Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech>
Co-authored-by: Frontend Documenter <fe-doc@roboco.tech>

* [a27de2a8] fix(docs): reflow hard-wrapped notification-types.md to pass markdown gate (#479) (#481)

The Python quality gate on assembled PR #477 was red because the newly
added docs/frontend/components/notification-types.md (introduced by the
frontend coordination-event rendering commit) had manually wrapped prose
paragraphs, which scripts/reflow_md.py --check rejects as part of make
quality. Reflowed the file with scripts/reflow_md.py --apply (whitespace
only, no content change) so the check passes. ruff format/check, mypy,
xenon, vulture, bandit, and the full pytest suite (10284 passed) all
confirmed green on this commit; notification.py, task.py, and
orchestrator.py are untouched.

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>

* [705419d5] Remove duplicate unblock notification and fix its dependent tests (#485) (#488)

* [705419d5] fix(notifications): remove duplicate unblock notification, fix its tests

The /unblock route was still calling delivery.notify_assignee_of_unblock()
(TASK_ASSIGNMENT) after TaskService.unblock() already sent the
send_unblock_notification() ALERT wired in by an earlier task — a real
duplicate notification on every unblock. Delete the route-layer call and
the now-dead NotificationDeliveryService.notify_assignee_of_unblock
method, fix the integration test that mocked it, and fix/extend the e2e
notification-coordination-events test to assert the persisted ALERT rows
(exact subjects) for both the direct-unblock and dependency-revival
producers instead of the old TASK_ASSIGNMENT assertion.

* [705419d5] docs(backend): update coordination-events doc for unblock duplicate removal

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>
Co-authored-by: Backend Documenter <be-doc@roboco.tech>

* [6c142a73] docs(changelog): document restored coordination-event notification producers and add collision-sequencing double-fire test (#489) (#490)

Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech>

* [77719d3f] Seed system agent in e2e harness to fix unblock/dependency-revival notifications

The e2e harness's seed_company omitted the system sentinel agent that
production seeds via initial_data.py. The unblock and dependency-revival
notification producers default to from_agent="system", which
_resolve_agent_uuid looks up by slug in the DB. With no system row the
resolver returns None and _create_notification silently skips the
notification, so the two ALERT assertions got 0 rows instead of 1.

The soft-block test passed because it uses NotificationDeliveryService
which creates the notification directly with a real agent UUID as
from_agent, bypassing the slug resolution path entirely.

* [77719d3f] Use foundation UUID for system agent to avoid slug collision

The first attempt seeded the system agent with a random UUID. Other
tests (_seed_system_and_secretary, _seed_video_agents) check by the
fixed foundation UUID via session.get(AgentTable, uuid); not finding
it they INSERT their own system row, hitting ix_agents_slug. Using the
foundation UUID makes their check find the seed_company row and skip.

* [77719d3f] Fix dependency-revival notification event loop mismatch

The dependency-revival test calls _unblock_dependents directly via
stack.run_db, which creates a new asyncio event loop. Inside,
_notify_dependency_revival -> NotificationService._create_notification
opened its own session via get_db_context(), which reuses the singleton
_DbHolder engine — bound to the FastAPI server's event loop. The
asyncpg connection raised 'Future attached to a different loop' and the
exception was silently caught + logged as a warning, so the notification
never persisted and the test saw 0 rows.

Fix: add an optional db_session parameter to _create_notification and
the two send methods. When provided, use the caller's session directly
and skip the internal commit (the caller owns the transaction). The
TaskService's _notify_unblock and _notify_dependency_revival now pass
self.session, keeping the notification in the same event loop + session
as the task transition.

* [77719d3f] Scope system-agent seeding to notification tests only

Seeding the system sentinel in seed_company (commits 3bba7b32/617b7890)
fixed the 0-notification bug but caused 3 i_documented gateway_timeout
failures: every e2e test now paid notification-creation latency for
system-origin notifications that were previously silently skipped,
pushing the already-slow i_documented verb past its 120s timeout.

Move system-agent seeding out of seed_company and into a scoped
_seed_system_agent helper called only by the two coordination-event
tests that exercise send_unblock_notification /
send_dependency_revival_notification (both resolve from_agent='system'
via DB lookup). dev_lifecycle and state_machine tests revert to the
pre-fix behavior (system-origin notifications silently skipped, no extra
latency).

The event-loop fix (commit 7b95d77d: pass db_session=self.session to
_create_notification) is unchanged — dependency_revival still needs it
because stack.run_db creates a new event loop while _DbHolder.engine is
bound to the FastAPI server loop.

* [77719d3f] Fix reassignment notification deadlock + suppressed-notification commit regression

Two fixes in notification.py / task.py:

1. Cross-session self-deadlock in send_reassignment_notification:
   TaskService.reassign() flushes an uncommitted row lock on the task,
   then calls _notify_reassignment -> send_reassignment_notification ->
   _create_notification(db_session=None) which opens a SEPARATE session
   via get_db_context() and INSERTs a notification with related_task_id
   FK -> tasks.id. The FK key-share lock blocks on the request session's
   uncommitted exclusive lock, but the request can't commit until the
   notify returns -> 120s verb hard-cut. Fix: pass db_session=self.session
   so the notification joins the verb's own transaction, same pattern as
   the unblock/dependency-revival fix in 7b95d77d.

2. Suppressed-notification commit regression: the 7b95d77d refactor moved
   await db.commit() out of _create_notification_with_session into
   _create_notification's db_session=None branch, where it ran
   unconditionally — even when _create_notification_with_session returned
   early (suppressed: unresolvable from_agent / no recipients /
   refire-guard / dedup-hit). Fix: _create_notification_with_session now
   returns bool (False at each early return, True after delivery);
   _create_notification commits only when created is True.

---------

Co-authored-by: Backend Developer 1 <be-dev-1@roboco.tech>
Co-authored-by: Backend Documenter <be-doc@roboco.tech>
Co-authored-by: Frontend Developer 2 <fe-dev-2@roboco.tech>
Co-authored-by: Frontend Developer 1 <fe-dev-1@roboco.tech>
Co-authored-by: Frontend Documenter <fe-doc@roboco.tech>
Co-authored-by: Backend Developer 2 <be-dev-2@roboco.tech>
Co-authored-by: Renn F <rennf93@users.noreply.github.com>

tests/e2e_smoke/test_notification_coordination_events.py

@@ -0,0 +1,301 @@
+"""Scenario: coordination-event notification producers land real DB rows.
+
+Drives three of the coordination-event notification producers wired at
+``TaskService``'s transition chokepoints (``roboco/services/task.py``) —
+soft-block (BLOCKER_ESCALATION to the cell PM), unblock (ALERT to the
+restored owner + CEO, ``send_unblock_notification``), and dependency-revival
+(ALERT to the revived owner + CEO, ``send_dependency_revival_notification``)
+— through the real REST task surface mounted at ``/api/tasks`` in this
+harness (the same surface scenario 3's CEO approve-and-merge call uses), plus
+one direct ``TaskService`` chokepoint call for the dependency-revival
+producer (mirrors ``arcs.wire_dependency``'s pattern of driving
+``TaskService`` directly for setup that has no bespoke REST endpoint). Real
+in-process API, real ``NotificationService``/``NotificationDeliveryService``,
+real ephemeral Postgres — no mocks. Each assertion reads the persisted
+``NotificationTable`` row back out of the DB via ``E2EStack.run_db``, exactly
+as the harness's other DB-truth checks do.
+
+The unblock route (``POST /api/tasks/{id}/unblock``) used to ALSO send a
+second, duplicate TASK_ASSIGNMENT notification from the route handler
+itself (``notify_assignee_of_unblock``) on top of the ALERT
+``TaskService.unblock()`` already sends. That duplicate route-layer call has
+been removed — unblock fires exactly one notification now, the ALERT below.
+"""
+
+from __future__ import annotations
+
+from http import HTTPStatus
+from typing import TYPE_CHECKING, Any
+
+import httpx
+from tests.e2e_smoke.arcs import seed_company, seed_project, seed_task
+
+if TYPE_CHECKING:
+    from sqlalchemy.ext.asyncio import AsyncSession
+    from tests.e2e_smoke.harness import E2EStack
+
+
+def _agent_headers(agent_id: Any, role: str) -> dict[str, str]:
+    return {"X-Agent-ID": str(agent_id), "X-Agent-Role": role}
+
+
+def _seed_system_agent(stack: E2EStack) -> None:
+    """Seed the ``system`` sentinel at its fixed foundation UUID.
+
+    Production seeds it via ``initial_data.py``; the e2e harness's
+    ``seed_company`` deliberately does NOT (seeding it globally adds
+    notification-creation latency to every test, pushing ``i_documented``
+    past its 120 s verb timeout). Only the coordination-event tests that
+    exercise ``send_unblock_notification`` / ``send_dependency_revival_notification``
+    need it — both resolve ``from_agent="system"`` to a UUID via DB lookup.
+    """
+    from roboco.db.tables import AgentTable
+    from roboco.foundation import identity as _foundation
+    from roboco.models import AgentRole, AgentStatus
+
+    async def _run(session: AsyncSession) -> None:
+        session.add(
+            AgentTable(
+                id=_foundation.AGENTS["system"].uuid,
+                name="system",
+                slug="system",
+                role=AgentRole.SYSTEM,
+                team=None,
+                status=AgentStatus.ACTIVE,
+                model_config={},
+                system_prompt="system",
+                capabilities=[],
+                permissions={},
+                metrics={},
+            )
+        )
+        await session.flush()
+
+    stack.run_db(_run)
+
+
+def _notifications_for_task(
+    stack: E2EStack, task_id: Any, notification_type: Any
+) -> list[dict[str, Any]]:
+    from roboco.db.tables import NotificationTable
+    from sqlalchemy import select
+
+    async def _run(session: AsyncSession) -> list[dict[str, Any]]:
+        rows = (
+            (
+                await session.execute(
+                    select(NotificationTable).where(
+                        NotificationTable.related_task_id == task_id,
+                        NotificationTable.type == notification_type,
+                    )
+                )
+            )
+            .scalars()
+            .all()
+        )
+        return [
+            {
+                "type": str(r.type),
+                "related_task_id": r.related_task_id,
+                "subject": r.subject,
+                "priority": str(r.priority),
+                "to_agents": list(r.to_agents),
+            }
+            for r in rows
+        ]
+
+    rows: list[dict[str, Any]] = stack.run_db(_run)
+    return rows
+
+
+def test_soft_block_persists_blocker_escalation_notification(
+    e2e_stack: E2EStack,
+) -> None:
+    """soft-block chokepoint: notify_pm_of_block -> BLOCKER_ESCALATION."""
+    stack = e2e_stack
+    company = seed_company(stack)
+    project_id, _project_slug = seed_project(stack, company)
+
+    from roboco.models.base import TaskStatus
+
+    task_id = seed_task(
+        stack,
+        title="Investigate flaky upstream API",
+        description=(
+            "Dev is mid-work and hits an external dependency outage; "
+            "soft-blocking so the cell PM is paged for resolution."
+        ),
+        acceptance_criteria=["the upstream API responds reliably again"],
+        project_id=project_id,
+        created_by=company.cell_pm_id,
+        assigned_to=company.dev_id,
+        claimed_by=company.dev_id,
+        active_claimant_id=company.dev_id,
+        status=TaskStatus.IN_PROGRESS,
+        branch_name="feature/backend/e2e-soft-block-notify",
+    )
+
+    resp = httpx.post(
+        f"{stack.base_url}/api/tasks/{task_id}/soft-block",
+        json={
+            "reason": "The upstream payments API is returning 503s.",
+            "blocker_type": "external",
+            "what_needed": "Wait for the upstream provider to recover.",
+            "resolver_type": "agent",
+        },
+        headers=_agent_headers(company.dev_id, "developer"),
+        timeout=30,
+    )
+    assert resp.status_code == HTTPStatus.OK, (
+        f"soft-block: {resp.status_code} {resp.text[:1500]}"
+    )
+
+    from roboco.models import NotificationType
+
+    notifications = _notifications_for_task(
+        stack, task_id, NotificationType.BLOCKER_ESCALATION
+    )
+    assert len(notifications) == 1, notifications
+    note = notifications[0]
+    assert "blocker_escalation" in note["type"].lower(), note
+    assert note["related_task_id"] == task_id, note
+    assert note["subject"], "subject must be populated"
+    assert note["priority"], "priority must be populated"
+    assert company.cell_pm_id in note["to_agents"], note
+
+
+def test_unblock_persists_alert_notification(e2e_stack: E2EStack) -> None:
+    """unblock chokepoint: TaskService.unblock -> send_unblock_notification -> ALERT.
+
+    Exactly one notification fires for unblock (the route-layer
+    ``notify_assignee_of_unblock`` TASK_ASSIGNMENT duplicate was removed).
+    """
+    stack = e2e_stack
+    company = seed_company(stack)
+    project_id, _project_slug = seed_project(stack, company)
+    _seed_system_agent(stack)
+
+    from roboco.models.base import TaskStatus
+
+    task_id = seed_task(
+        stack,
+        title="Rotate the expired staging credential",
+        description=(
+            "Dev soft-blocked waiting on a credential rotation; the cell "
+            "PM resolves it and unblocks the task so the dev resumes."
+        ),
+        acceptance_criteria=["the staging credential is valid again"],
+        project_id=project_id,
+        created_by=company.cell_pm_id,
+        assigned_to=company.dev_id,
+        claimed_by=company.dev_id,
+        active_claimant_id=company.dev_id,
+        status=TaskStatus.IN_PROGRESS,
+        branch_name="feature/backend/e2e-unblock-notify",
+    )
+
+    block_resp = httpx.post(
+        f"{stack.base_url}/api/tasks/{task_id}/soft-block",
+        json={
+            "reason": "The staging DB credential expired overnight.",
+            "blocker_type": "external",
+            "what_needed": "A rotated staging credential from the cell PM.",
+            "resolver_type": "agent",
+        },
+        headers=_agent_headers(company.dev_id, "developer"),
+        timeout=30,
+    )
+    assert block_resp.status_code == HTTPStatus.OK, (
+        f"soft-block: {block_resp.status_code} {block_resp.text[:1500]}"
+    )
+
+    unblock_resp = httpx.post(
+        f"{stack.base_url}/api/tasks/{task_id}/unblock",
+        headers=_agent_headers(company.cell_pm_id, "cell_pm"),
+        timeout=30,
+    )
+    assert unblock_resp.status_code == HTTPStatus.OK, (
+        f"unblock: {unblock_resp.status_code} {unblock_resp.text[:1500]}"
+    )
+
+    from roboco.models import NotificationType
+
+    notifications = _notifications_for_task(stack, task_id, NotificationType.ALERT)
+    assert len(notifications) == 1, notifications
+    note = notifications[0]
+    assert "alert" in note["type"].lower(), note
+    assert note["related_task_id"] == task_id, note
+    assert note["subject"] == f"Task {task_id} unblocked", note
+    assert note["priority"], "priority must be populated"
+    assert company.dev_id in note["to_agents"], note
+
+    # The deleted route-layer TASK_ASSIGNMENT duplicate must not reappear.
+    stale = _notifications_for_task(stack, task_id, NotificationType.TASK_ASSIGNMENT)
+    assert stale == [], stale
+
+
+def test_dependency_revival_persists_alert_notification(e2e_stack: E2EStack) -> None:
+    """dependency-revival chokepoint: _unblock_dependents ->
+    send_dependency_revival_notification -> ALERT.
+
+    No resolver calls unblock here — a dependent task blocked on another
+    task auto-resumes the moment that dependency's completion clears the
+    last outstanding dependency, at the same ``_unblock_dependents``
+    chokepoint ``TaskService.complete``/``ceo_approve`` call in production.
+    Driven directly against ``TaskService`` (mirrors
+    ``arcs.wire_dependency``'s pattern) since there is no bespoke REST
+    endpoint for "a dependency just completed".
+    """
+    stack = e2e_stack
+    company = seed_company(stack)
+    project_id, _project_slug = seed_project(stack, company)
+    _seed_system_agent(stack)
+
+    from roboco.models.base import TaskStatus
+
+    dependency_id = seed_task(
+        stack,
+        title="Ship the shared auth helper",
+        description="Upstream task the dependent below is blocked on.",
+        acceptance_criteria=["the shared auth helper is merged"],
+        project_id=project_id,
+        created_by=company.cell_pm_id,
+        assigned_to=company.dev_id,
+        status=TaskStatus.IN_PROGRESS,
+    )
+    dependent_id = seed_task(
+        stack,
+        title="Wire the new endpoint to the shared auth helper",
+        description=(
+            "Blocked on the shared auth helper landing; should auto-resume "
+            "the moment that dependency completes, with no resolver acting."
+        ),
+        acceptance_criteria=["the endpoint uses the shared auth helper"],
+        project_id=project_id,
+        created_by=company.cell_pm_id,
+        assigned_to=company.dev_id,
+        claimed_by=company.dev_id,
+        status=TaskStatus.BLOCKED,
+        dependency_ids=[dependency_id],
+        branch_name="feature/backend/e2e-dependency-revival",
+    )
+
+    from roboco.services.task import get_task_service
+
+    async def _complete_dependency(session: AsyncSession) -> None:
+        await get_task_service(session)._unblock_dependents(dependency_id)
+
+    stack.run_db(_complete_dependency)
+
+    from roboco.models import NotificationType
+
+    notifications = _notifications_for_task(stack, dependent_id, NotificationType.ALERT)
+    assert len(notifications) == 1, notifications
+    note = notifications[0]
+    assert "alert" in note["type"].lower(), note
+    assert note["related_task_id"] == dependent_id, note
+    assert note["subject"] == f"Task {dependent_id} revived by dependency completion", (
+        note
+    )
+    assert note["priority"], "priority must be populated"
+    assert company.dev_id in note["to_agents"], note