mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
* [684dace4] Self-hosted LLM provider: API layer, hooks, UI section, routing mode button, and Mix mode grouping (#124) (#126) * [684dace4] feat(providers): add self-hosted LLM API types, endpoints, and React Query hooks - Add ModelProvider.SELF_HOSTED enum value to types/index.ts - Extend RoutingMode to include 'self_hosted' in lib/api/providers.ts - Add SelfHostedConfig, SelfHostedTestResult, SelfHostedModel interfaces - Add SelfHostedConfigPayload for PUT requests - Add 5 providersApi methods: getSelfHostedConfig, saveSelfHostedConfig, testSelfHosted, getSelfHostedModels, refreshSelfHostedModels - Add 5 React Query hooks: useSelfHostedConfig, useSetSelfHostedConfig, useTestSelfHosted, useSelfHostedModels, useRefreshSelfHostedModels - Cache keys follow existing providerKeys pattern with proper invalidation * [684dace4] feat(settings): create SelfHostedSection component with full self-hosted LLM UI - Base URL text input with placeholder showing saved URL when set - Optional auth token field (type='password') with Eye/EyeOff toggle button - Save button that calls useSetSelfHostedConfig mutation - Test Connection button disabled until a URL is saved; shows inline green 'Connected — N models' badge on success or red error badge on fail - Three empty states: no URL configured (CTA), error state (last-checked + Retry), connected with 0 models (pull-guidance) - Model list with auto-discovered chip, Refresh Models button, and Last refreshed relative timestamp when test_status === 'connected' - Token field shows masked placeholder when has_auth_token is true (consistent with Ollama Cloud key field pattern) * [684dace4] feat(settings): add Self-Hosted mode button, model picker, and Mix mode provider grouping - Wire SelfHostedSection into AIRoutingCard with testResult state tracking - Expand routing mode grid from 3 to 4 buttons (2×2 on mobile, 4-col on md+) - 4th 'Self-Hosted' mode button disabled until test_status === 'connected' - Self-hosted model picker appears below mode grid when mode === 'self_hosted' - flipToSelfHosted handler sends mode='self_hosted' with optional default_model - Mix mode per-agent dropdown now groups entries under SelectGroup/SelectLabel headings: Anthropic, Ollama Cloud, Self-Hosted with colored ProviderBadge pill - saveMix validates self-hosted model selection requires a successful test - ProviderBadge helper renders blue/violet/purple pills for each provider type - pnpm typecheck and pnpm lint pass with zero errors --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> * [2897ce90] Implement self-hosted LLM provider API, routing, and discovery (#125) (#127) * [2897ce90] feat(provider): add self-hosted LLM provider API, routing, and discovery - Add migration 027 to seed Self-Hosted (Ollama) LOCAL provider row - Add probe_ollama_tags() helper for Ollama /api/tags connectivity checks - Extend ModelRoutingService: derive_mode returns 'self_hosted' for LOCAL GLOBAL assignments; apply_mode handles 'self_hosted' mode; upsert_assignment routes non-catalog model names to LOCAL provider; resolve_for_agent falls back to Anthropic when self-hosted server is unreachable - Add PUT /api/providers/self-hosted, POST /api/providers/self-hosted/test, GET /api/providers/self-hosted/models endpoints - Extend ApplyModeRequest and ModeResponse literals with 'self_hosted' - Add SelfHostedConfigRequest, SelfHostedConfigResponse, SelfHostedTestResponse schemas * [2897ce90] test(provider): add integration tests for self-hosted routing and route endpoints - Add llm_setup_with_local fixture that seeds LOCAL provider row - Test derive_mode returns 'self_hosted' for single GLOBAL LOCAL assignment - Test apply_mode('self_hosted') clears prior assignments, enables LOCAL, inserts GLOBAL - Test apply_mode('self_hosted') requires default_model argument - Test upsert_assignment routes non-catalog model names to LOCAL provider - Test mix mode accepts self-hosted model names without ValueError - Test resolve_for_agent returns base_url when LOCAL server is reachable - Test resolve_for_agent falls back to Anthropic when LOCAL server is unreachable - Test upsert_assignment raises ValueError when model unknown and no LOCAL provider - Add app_client_with_local fixture for route tests - Test PUT /self-hosted saves base_url and enables provider - Test PUT /self-hosted stores encrypted token when auth_token provided - Test PUT /self-hosted returns 404 when LOCAL provider not seeded - Test POST /self-hosted/test returns {ok:true,model_count:N} when reachable - Test POST /self-hosted/test returns {ok:false,error} (never 500) when unreachable - Test GET /self-hosted/models returns model name list - Test GET /self-hosted/models returns 404 when not configured - Test GET /self-hosted/models returns 503 when server unreachable - Rename migration from 027 to 028 to rebase on 027_system_settings * [2897ce90] chore(migration): remove superseded 027 migration, fix formatter changes to provider schemas --------- Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * [042462df] feat(providers): align self-hosted types, hooks, and UI to backend contract (#129) (#131) - SelfHostedConfig now has {base_url: string, has_token: boolean, enabled: boolean} - SelfHostedTestResult now has {ok: boolean, model_count: number | null, error: string | null} - Remove SelfHostedTestStatus type and refreshSelfHostedModels POST API function - Remove SELF_HOSTED from ModelProvider enum (LOCAL covers self-hosted semantics) - useRefreshSelfHostedModels now invalidates GET cache instead of calling POST - isSelfHostedConnected derived from testResult?.ok === true - Self-hosted model picker uses value='__clear__' sentinel (no empty-string SelectItem) - self-hosted-section.tsx reads result.ok/result.error and config?.has_token - pnpm typecheck passes with zero errors Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> * [f66d6d4d] Fix self-hosted API S1-S4/L1-L5: routes, schemas, services, migration 028, and tests (#130) (#132) * [f66d6d4d] fix(provider): self-hosted API S1-S4/L1-L5 - routes, schemas, services, migration 028, and tests AC1: Add GET /providers/self-hosted returning {base_url, has_token, enabled} AC2: GET /self-hosted/models now returns list[SelfHostedModelEntry] with model_name and display_name AC3: probe_ollama_tags generic except logs exception server-side and returns hardcoded generic string AC4: upsert_assignment calls ProviderService.update_provider(enabled=True) when routing to LOCAL AC5: derive_mode return annotation is Literal[...] — type:ignore comments removed AC6: All migration refs in routes/services say 028 (not 027) AC7: Migration 028 downgrade() deletes model_assignments before provider_configs AC8: PUT /self-hosted only passes enabled=True when data.base_url is non-empty AC9: ModelProvider.LOCAL docstring updated to describe self-hosted Ollama provider AC10: Direct unit tests for probe_ollama_tags (5 cases) in tests/unit/llm/ AC11: Contract tests added/updated for GET /providers/self-hosted, models, and test endpoints AC12: test_migration_028_seed_self_hosted.py with upgrade and FK-safe downgrade tests AC13: test_apply_mode_ollama_without_provider_returns_404 asserts exactly HTTPStatus.NOT_FOUND AC14: ruff and mypy pass with zero errors * [f66d6d4d] fix(tests): add AC4 test proving LOCAL.enabled transitions False->True in upsert_assignment The existing tests (test_upsert_assignment_routes_unknown_model_to_local and test_mix_mode_with_self_hosted_models) both use llm_setup_with_local which seeds LOCAL with enabled=True, making the AC4 assertion vacuous. New test test_upsert_assignment_enables_local_when_disabled: - Creates LOCAL ProviderConfigTable row with enabled=False - Asserts pre-condition: local.enabled is False - Calls upsert_assignment with a non-catalog model name ('non-catalog-model:7b') - Refreshes LOCAL row via db_session.refresh(local) - Asserts row.provider.type == ModelProvider.LOCAL and local.enabled is True This proves the state transition from False->True, not merely that the already-enabled state is preserved. ruff and mypy still pass with zero errors. --------- Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * [7cd6ae6e] fix(providers): type SelfHostedConfig.base_url as string | null to match backend contract (#133) (#136) Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> * [46ee9104] test(migration_028): replace upgrade test with self-seeding contract test (#134) (#135) Remove test_migration_028_upgrade_local_row_inserted which relied on alembic upgrade head having run (and thus the Self-Hosted Ollama row being present). Replace it with test_migration_028_upgrade_insert_contract that: - Executes the exact INSERT SQL from migration 028 upgrade() directly - Asserts name='Self-Hosted (Ollama)', type='local', enabled=False - Runs the INSERT a second time and asserts exactly one row (ON CONFLICT DO NOTHING idempotency) The downgrade test is left byte-for-byte unchanged. Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * [f0d19f30] test(provider): add DELETE-before-seed isolation and app_client_with_ollama fixture (#137) (#138) - Add ModelAssignmentTable import to test_provider_routes.py - Fix app_client_with_local: execute DELETE on ModelAssignmentTable then DELETE on ProviderConfigTable (FK-safe order) and flush before seeding - Add new app_client_with_ollama fixture with same isolation pattern, seeding only ANTHROPIC + OLLAMA_CLOUD rows - Update 7 tests to use app_client_with_ollama instead of app_client: test_get_catalog, test_get_ollama_key_status, test_set_ollama_key, test_get_current_mode, test_apply_mode_anthropic_clears_assignments, test_apply_mode_unknown_returns_4xx, test_apply_mode_mix_without_per_agent_returns_400 Fixes order-dependent failures in test_get_self_hosted_models_not_configured_returns_404: routes call db.commit() which persists rows across test sessions; without DELETE-before-seed, stale LOCAL provider rows with base_url set from prior runs cause the test to see 503 instead of 404. Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * refactor(llm): split resolve_for_agent and apply_mode to clear xenon rank C resolve_for_agent and apply_mode were cyclomatic rank C, failing the xenon gate (--max-absolute B). Extract behavior-preserving helpers: - resolve_for_agent -> _resolve_assignment (precedence ladder), _route_from_resolved / _local_route_or_none / _decrypt_route_or_none (None signals fall-through to legacy), _legacy_route. - apply_mode -> _apply_anthropic / _apply_ollama / _apply_self_hosted / _apply_mix dispatched from a thin if/elif. No behavior change. Also correct the stale 'default: Kimi K2.6' docstring (OLLAMA_DEFAULT_MODEL is minimax-m3:cloud). --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> Co-authored-by: Renn F <rennf93@users.noreply.github.com>
169 lines
5.6 KiB
Python
169 lines
5.6 KiB
Python
"""Migration 028 tests — seed_self_hosted_provider.
|
|
|
|
Verifies the post-upgrade state and exercises the downgrade SQL ordering
|
|
to prove the FK-safe delete sequence works.
|
|
|
|
NOT a real alembic round-trip — the suite builds the test DB via
|
|
Base.metadata.create_all (see conftest). Migration 028's upgrade()/downgrade()
|
|
bodies are reviewed here; the tests guard the resulting DB-level contract.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import TYPE_CHECKING
|
|
from uuid import uuid4
|
|
|
|
import pytest
|
|
from roboco.db.tables import ModelAssignmentTable, ProviderConfigTable
|
|
from roboco.models.base import AssignmentScope, ModelProvider
|
|
from sqlalchemy import text
|
|
|
|
if TYPE_CHECKING:
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_migration_028_upgrade_insert_contract(
|
|
db_session: AsyncSession,
|
|
) -> None:
|
|
"""The upgrade INSERT SQL seeds the correct LOCAL provider row and is idempotent.
|
|
|
|
Executes the INSERT ... ON CONFLICT DO NOTHING SQL from migration 028's
|
|
upgrade() directly in the test session, verifying:
|
|
- name='Self-Hosted (Ollama)', type='local', enabled=False on the row.
|
|
- Running the same INSERT a second time leaves exactly one row (idempotency).
|
|
"""
|
|
_insert_sql = text(
|
|
"""
|
|
INSERT INTO provider_configs
|
|
(id, name, type, base_url, auth_token_encrypted, enabled, created_at)
|
|
VALUES
|
|
(
|
|
gen_random_uuid(),
|
|
'Self-Hosted (Ollama)',
|
|
'local',
|
|
NULL,
|
|
NULL,
|
|
false,
|
|
now()
|
|
)
|
|
ON CONFLICT (name) DO NOTHING
|
|
"""
|
|
)
|
|
|
|
# --- First run: the row should be inserted.
|
|
await db_session.execute(_insert_sql)
|
|
await db_session.flush()
|
|
|
|
# Verify the field contract on the newly-inserted row.
|
|
result = await db_session.execute(
|
|
text(
|
|
"SELECT name, type, enabled "
|
|
"FROM provider_configs "
|
|
"WHERE name = 'Self-Hosted (Ollama)'"
|
|
)
|
|
)
|
|
rows = list(result)
|
|
assert len(rows) == 1
|
|
name, ptype, enabled = rows[0]
|
|
assert name == "Self-Hosted (Ollama)"
|
|
assert ptype == "local"
|
|
assert enabled is False # starts disabled; user configures via PUT /self-hosted
|
|
|
|
# --- Second run: ON CONFLICT DO NOTHING must not create a duplicate.
|
|
await db_session.execute(_insert_sql)
|
|
await db_session.flush()
|
|
|
|
result = await db_session.execute(
|
|
text("SELECT id FROM provider_configs WHERE name = 'Self-Hosted (Ollama)'")
|
|
)
|
|
assert len(list(result)) == 1, (
|
|
"Expected exactly one 'Self-Hosted (Ollama)' row after two INSERT "
|
|
"executions; ON CONFLICT DO NOTHING must prevent duplicates."
|
|
)
|
|
|
|
|
|
@pytest.mark.asyncio
|
|
async def test_migration_028_downgrade_deletes_assignments_before_config(
|
|
db_session: AsyncSession,
|
|
) -> None:
|
|
"""Downgrade SQL deletes model_assignments before provider_configs.
|
|
|
|
Simulates the downgrade() logic from migration 028:
|
|
1. DELETE FROM model_assignments WHERE provider_config_id IN (SELECT id ...)
|
|
2. DELETE FROM provider_configs WHERE name = 'Self-Hosted (Ollama)'
|
|
|
|
A FK RESTRICT constraint on model_assignments.provider_config_id means that
|
|
executing step 2 before step 1 would raise an IntegrityError. This test
|
|
proves that doing them in the correct order succeeds without violation.
|
|
"""
|
|
# --- Arrange: insert a fresh LOCAL provider row and a referencing assignment.
|
|
suffix = uuid4().hex[:8]
|
|
local = ProviderConfigTable(
|
|
name=f"Self-Hosted (Ollama)-test-{suffix}",
|
|
type=ModelProvider.LOCAL,
|
|
enabled=False,
|
|
)
|
|
db_session.add(local)
|
|
await db_session.flush()
|
|
|
|
assignment = ModelAssignmentTable(
|
|
scope=AssignmentScope.AGENT_SLUG,
|
|
scope_value=f"test-agent-{suffix}",
|
|
provider_config_id=local.id,
|
|
model_name="llama3.1:8b",
|
|
)
|
|
db_session.add(assignment)
|
|
await db_session.flush()
|
|
|
|
# Verify both rows exist before we run the downgrade SQL.
|
|
result = await db_session.execute(
|
|
text("SELECT id FROM provider_configs WHERE name = :name").bindparams(
|
|
name=local.name
|
|
)
|
|
)
|
|
assert result.scalar_one_or_none() is not None
|
|
|
|
result = await db_session.execute(
|
|
text("SELECT id FROM model_assignments WHERE scope_value = :sv").bindparams(
|
|
sv=assignment.scope_value
|
|
)
|
|
)
|
|
assert result.scalar_one_or_none() is not None
|
|
|
|
# --- Act: execute downgrade SQL in the correct FK-safe order.
|
|
# Step 1: delete referencing model_assignments first.
|
|
await db_session.execute(
|
|
text(
|
|
"DELETE FROM model_assignments "
|
|
"WHERE provider_config_id IN ("
|
|
" SELECT id FROM provider_configs WHERE name = :name"
|
|
")"
|
|
).bindparams(name=local.name)
|
|
)
|
|
# Step 2: now safe to delete the provider row.
|
|
await db_session.execute(
|
|
text("DELETE FROM provider_configs WHERE name = :name").bindparams(
|
|
name=local.name
|
|
)
|
|
)
|
|
|
|
# --- Assert: both rows are gone, no IntegrityError was raised.
|
|
result = await db_session.execute(
|
|
text("SELECT id FROM provider_configs WHERE name = :name").bindparams(
|
|
name=local.name
|
|
)
|
|
)
|
|
assert result.scalar_one_or_none() is None, (
|
|
"provider_configs row should be deleted by downgrade"
|
|
)
|
|
|
|
result = await db_session.execute(
|
|
text("SELECT id FROM model_assignments WHERE scope_value = :sv").bindparams(
|
|
sv=assignment.scope_value
|
|
)
|
|
)
|
|
assert result.scalar_one_or_none() is None, (
|
|
"model_assignments row should be deleted before provider_configs"
|
|
)
|