mirror of
https://github.com/rennf93/roboco.git
synced 2026-08-03 07:23:24 +02:00
[3cc1729c] Add self-hosted LLM provider with dynamic model discovery (#128)
* [684dace4] Self-hosted LLM provider: API layer, hooks, UI section, routing mode button, and Mix mode grouping (#124) (#126) * [684dace4] feat(providers): add self-hosted LLM API types, endpoints, and React Query hooks - Add ModelProvider.SELF_HOSTED enum value to types/index.ts - Extend RoutingMode to include 'self_hosted' in lib/api/providers.ts - Add SelfHostedConfig, SelfHostedTestResult, SelfHostedModel interfaces - Add SelfHostedConfigPayload for PUT requests - Add 5 providersApi methods: getSelfHostedConfig, saveSelfHostedConfig, testSelfHosted, getSelfHostedModels, refreshSelfHostedModels - Add 5 React Query hooks: useSelfHostedConfig, useSetSelfHostedConfig, useTestSelfHosted, useSelfHostedModels, useRefreshSelfHostedModels - Cache keys follow existing providerKeys pattern with proper invalidation * [684dace4] feat(settings): create SelfHostedSection component with full self-hosted LLM UI - Base URL text input with placeholder showing saved URL when set - Optional auth token field (type='password') with Eye/EyeOff toggle button - Save button that calls useSetSelfHostedConfig mutation - Test Connection button disabled until a URL is saved; shows inline green 'Connected — N models' badge on success or red error badge on fail - Three empty states: no URL configured (CTA), error state (last-checked + Retry), connected with 0 models (pull-guidance) - Model list with auto-discovered chip, Refresh Models button, and Last refreshed relative timestamp when test_status === 'connected' - Token field shows masked placeholder when has_auth_token is true (consistent with Ollama Cloud key field pattern) * [684dace4] feat(settings): add Self-Hosted mode button, model picker, and Mix mode provider grouping - Wire SelfHostedSection into AIRoutingCard with testResult state tracking - Expand routing mode grid from 3 to 4 buttons (2×2 on mobile, 4-col on md+) - 4th 'Self-Hosted' mode button disabled until test_status === 'connected' - Self-hosted model picker appears below mode grid when mode === 'self_hosted' - flipToSelfHosted handler sends mode='self_hosted' with optional default_model - Mix mode per-agent dropdown now groups entries under SelectGroup/SelectLabel headings: Anthropic, Ollama Cloud, Self-Hosted with colored ProviderBadge pill - saveMix validates self-hosted model selection requires a successful test - ProviderBadge helper renders blue/violet/purple pills for each provider type - pnpm typecheck and pnpm lint pass with zero errors --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> * [2897ce90] Implement self-hosted LLM provider API, routing, and discovery (#125) (#127) * [2897ce90] feat(provider): add self-hosted LLM provider API, routing, and discovery - Add migration 027 to seed Self-Hosted (Ollama) LOCAL provider row - Add probe_ollama_tags() helper for Ollama /api/tags connectivity checks - Extend ModelRoutingService: derive_mode returns 'self_hosted' for LOCAL GLOBAL assignments; apply_mode handles 'self_hosted' mode; upsert_assignment routes non-catalog model names to LOCAL provider; resolve_for_agent falls back to Anthropic when self-hosted server is unreachable - Add PUT /api/providers/self-hosted, POST /api/providers/self-hosted/test, GET /api/providers/self-hosted/models endpoints - Extend ApplyModeRequest and ModeResponse literals with 'self_hosted' - Add SelfHostedConfigRequest, SelfHostedConfigResponse, SelfHostedTestResponse schemas * [2897ce90] test(provider): add integration tests for self-hosted routing and route endpoints - Add llm_setup_with_local fixture that seeds LOCAL provider row - Test derive_mode returns 'self_hosted' for single GLOBAL LOCAL assignment - Test apply_mode('self_hosted') clears prior assignments, enables LOCAL, inserts GLOBAL - Test apply_mode('self_hosted') requires default_model argument - Test upsert_assignment routes non-catalog model names to LOCAL provider - Test mix mode accepts self-hosted model names without ValueError - Test resolve_for_agent returns base_url when LOCAL server is reachable - Test resolve_for_agent falls back to Anthropic when LOCAL server is unreachable - Test upsert_assignment raises ValueError when model unknown and no LOCAL provider - Add app_client_with_local fixture for route tests - Test PUT /self-hosted saves base_url and enables provider - Test PUT /self-hosted stores encrypted token when auth_token provided - Test PUT /self-hosted returns 404 when LOCAL provider not seeded - Test POST /self-hosted/test returns {ok:true,model_count:N} when reachable - Test POST /self-hosted/test returns {ok:false,error} (never 500) when unreachable - Test GET /self-hosted/models returns model name list - Test GET /self-hosted/models returns 404 when not configured - Test GET /self-hosted/models returns 503 when server unreachable - Rename migration from 027 to 028 to rebase on 027_system_settings * [2897ce90] chore(migration): remove superseded 027 migration, fix formatter changes to provider schemas --------- Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * [042462df] feat(providers): align self-hosted types, hooks, and UI to backend contract (#129) (#131) - SelfHostedConfig now has {base_url: string, has_token: boolean, enabled: boolean} - SelfHostedTestResult now has {ok: boolean, model_count: number | null, error: string | null} - Remove SelfHostedTestStatus type and refreshSelfHostedModels POST API function - Remove SELF_HOSTED from ModelProvider enum (LOCAL covers self-hosted semantics) - useRefreshSelfHostedModels now invalidates GET cache instead of calling POST - isSelfHostedConnected derived from testResult?.ok === true - Self-hosted model picker uses value='__clear__' sentinel (no empty-string SelectItem) - self-hosted-section.tsx reads result.ok/result.error and config?.has_token - pnpm typecheck passes with zero errors Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> * [f66d6d4d] Fix self-hosted API S1-S4/L1-L5: routes, schemas, services, migration 028, and tests (#130) (#132) * [f66d6d4d] fix(provider): self-hosted API S1-S4/L1-L5 - routes, schemas, services, migration 028, and tests AC1: Add GET /providers/self-hosted returning {base_url, has_token, enabled} AC2: GET /self-hosted/models now returns list[SelfHostedModelEntry] with model_name and display_name AC3: probe_ollama_tags generic except logs exception server-side and returns hardcoded generic string AC4: upsert_assignment calls ProviderService.update_provider(enabled=True) when routing to LOCAL AC5: derive_mode return annotation is Literal[...] — type:ignore comments removed AC6: All migration refs in routes/services say 028 (not 027) AC7: Migration 028 downgrade() deletes model_assignments before provider_configs AC8: PUT /self-hosted only passes enabled=True when data.base_url is non-empty AC9: ModelProvider.LOCAL docstring updated to describe self-hosted Ollama provider AC10: Direct unit tests for probe_ollama_tags (5 cases) in tests/unit/llm/ AC11: Contract tests added/updated for GET /providers/self-hosted, models, and test endpoints AC12: test_migration_028_seed_self_hosted.py with upgrade and FK-safe downgrade tests AC13: test_apply_mode_ollama_without_provider_returns_404 asserts exactly HTTPStatus.NOT_FOUND AC14: ruff and mypy pass with zero errors * [f66d6d4d] fix(tests): add AC4 test proving LOCAL.enabled transitions False->True in upsert_assignment The existing tests (test_upsert_assignment_routes_unknown_model_to_local and test_mix_mode_with_self_hosted_models) both use llm_setup_with_local which seeds LOCAL with enabled=True, making the AC4 assertion vacuous. New test test_upsert_assignment_enables_local_when_disabled: - Creates LOCAL ProviderConfigTable row with enabled=False - Asserts pre-condition: local.enabled is False - Calls upsert_assignment with a non-catalog model name ('non-catalog-model:7b') - Refreshes LOCAL row via db_session.refresh(local) - Asserts row.provider.type == ModelProvider.LOCAL and local.enabled is True This proves the state transition from False->True, not merely that the already-enabled state is preserved. ruff and mypy still pass with zero errors. --------- Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * [7cd6ae6e] fix(providers): type SelfHostedConfig.base_url as string | null to match backend contract (#133) (#136) Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> * [46ee9104] test(migration_028): replace upgrade test with self-seeding contract test (#134) (#135) Remove test_migration_028_upgrade_local_row_inserted which relied on alembic upgrade head having run (and thus the Self-Hosted Ollama row being present). Replace it with test_migration_028_upgrade_insert_contract that: - Executes the exact INSERT SQL from migration 028 upgrade() directly - Asserts name='Self-Hosted (Ollama)', type='local', enabled=False - Runs the INSERT a second time and asserts exactly one row (ON CONFLICT DO NOTHING idempotency) The downgrade test is left byte-for-byte unchanged. Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * [f0d19f30] test(provider): add DELETE-before-seed isolation and app_client_with_ollama fixture (#137) (#138) - Add ModelAssignmentTable import to test_provider_routes.py - Fix app_client_with_local: execute DELETE on ModelAssignmentTable then DELETE on ProviderConfigTable (FK-safe order) and flush before seeding - Add new app_client_with_ollama fixture with same isolation pattern, seeding only ANTHROPIC + OLLAMA_CLOUD rows - Update 7 tests to use app_client_with_ollama instead of app_client: test_get_catalog, test_get_ollama_key_status, test_set_ollama_key, test_get_current_mode, test_apply_mode_anthropic_clears_assignments, test_apply_mode_unknown_returns_4xx, test_apply_mode_mix_without_per_agent_returns_400 Fixes order-dependent failures in test_get_self_hosted_models_not_configured_returns_404: routes call db.commit() which persists rows across test sessions; without DELETE-before-seed, stale LOCAL provider rows with base_url set from prior runs cause the test to see 503 instead of 404. Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> * refactor(llm): split resolve_for_agent and apply_mode to clear xenon rank C resolve_for_agent and apply_mode were cyclomatic rank C, failing the xenon gate (--max-absolute B). Extract behavior-preserving helpers: - resolve_for_agent -> _resolve_assignment (precedence ladder), _route_from_resolved / _local_route_or_none / _decrypt_route_or_none (None signals fall-through to legacy), _legacy_route. - apply_mode -> _apply_anthropic / _apply_ollama / _apply_self_hosted / _apply_mix dispatched from a thin if/elif. No behavior change. Also correct the stale 'default: Kimi K2.6' docstring (OLLAMA_DEFAULT_MODEL is minimax-m3:cloud). --------- Co-authored-by: Frontend Developer 1 <fe-dev-1@agents.roboco.dev> Co-authored-by: Backend Developer 1 <be-dev-1@agents.roboco.dev> Co-authored-by: Renn F <rennf93@users.noreply.github.com>
This commit is contained in:
Renzo F
co-authored by
Frontend Developer 1
Backend Developer 1
Renn F
Frontend Developer 1
Backend Developer 1
Renn F
parent
0daef044d2
commit
73b7c16211
@@ -0,0 +1,68 @@
|
||||
"""Idempotently seed the Self-Hosted (Ollama) LOCAL provider row.
|
||||
|
||||
Revision ID: 028_seed_self_hosted_provider
|
||||
Revises: 027_system_settings
|
||||
Create Date: 2026-06-12
|
||||
|
||||
The `provider_configs` table already has `type='local'` in the
|
||||
`modelprovider` enum (seeded in 004_provider_routing). This migration
|
||||
seeds the corresponding row so the Settings UI can configure a
|
||||
self-hosted Ollama server without requiring an additional API call.
|
||||
|
||||
The row starts disabled with no base_url — the user configures those
|
||||
via PUT /api/providers/self-hosted. ON CONFLICT DO NOTHING makes this
|
||||
migration safe to re-run on a DB that already has the row (e.g. from
|
||||
a prior manual insert or a future fresh-schema bootstrap).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision = "028_seed_self_hosted_provider"
|
||||
down_revision = "027_system_settings"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute(
|
||||
sa.text(
|
||||
"""
|
||||
INSERT INTO provider_configs
|
||||
(id, name, type, base_url, auth_token_encrypted, enabled, created_at)
|
||||
VALUES
|
||||
(
|
||||
gen_random_uuid(),
|
||||
'Self-Hosted (Ollama)',
|
||||
'local',
|
||||
NULL,
|
||||
NULL,
|
||||
false,
|
||||
now()
|
||||
)
|
||||
ON CONFLICT (name) DO NOTHING
|
||||
"""
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Delete any model_assignments that point to the LOCAL provider row first
|
||||
# to avoid a FK RESTRICT violation on provider_configs.id.
|
||||
op.execute(
|
||||
sa.text(
|
||||
"DELETE FROM model_assignments "
|
||||
"WHERE provider_config_id IN ("
|
||||
" SELECT id FROM provider_configs WHERE name = 'Self-Hosted (Ollama)'"
|
||||
")"
|
||||
)
|
||||
)
|
||||
op.execute(
|
||||
sa.text(
|
||||
"DELETE FROM provider_configs "
|
||||
"WHERE name = 'Self-Hosted (Ollama)'"
|
||||
)
|
||||
)
|
||||
@@ -1,12 +1,13 @@
|
||||
"use client";
|
||||
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import { useEffect, useMemo, useState, useCallback } from "react";
|
||||
import {
|
||||
useApplyMode,
|
||||
useCatalog,
|
||||
useOllamaKey,
|
||||
useRoutingMode,
|
||||
useSetOllamaKey,
|
||||
useSelfHostedModels,
|
||||
} from "@/hooks/use-providers";
|
||||
import {
|
||||
Card,
|
||||
@@ -21,7 +22,9 @@ import { Label } from "@/components/ui/label";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectGroup,
|
||||
SelectItem,
|
||||
SelectLabel,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select";
|
||||
@@ -31,12 +34,14 @@ import {
|
||||
Cpu,
|
||||
Key,
|
||||
KeyRound,
|
||||
Server,
|
||||
ShieldCheck,
|
||||
Sparkles,
|
||||
} from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
import { AssignmentScope, ModelProvider } from "@/types";
|
||||
import type { RoutingMode } from "@/lib/api/providers";
|
||||
import type { RoutingMode, SelfHostedTestResult } from "@/lib/api/providers";
|
||||
import { SelfHostedSection } from "@/components/settings/self-hosted-section";
|
||||
|
||||
// Matches the roboco agents_config AGENT_ROLE_MAP / AGENT_TEAM_MAP.
|
||||
// Hard-coded so Mix mode shows a stable 18-row picker without an extra
|
||||
@@ -70,6 +75,7 @@ export function AIRoutingCard() {
|
||||
const { data: catalog = [] } = useCatalog();
|
||||
const { data: keyStatus } = useOllamaKey();
|
||||
const { data: snapshot } = useRoutingMode();
|
||||
const { data: selfHostedModels = [] } = useSelfHostedModels();
|
||||
|
||||
const setKey = useSetOllamaKey();
|
||||
const applyMode = useApplyMode();
|
||||
@@ -77,6 +83,21 @@ export function AIRoutingCard() {
|
||||
const hasOllamaKey = !!keyStatus?.has_key;
|
||||
const currentMode: RoutingMode = snapshot?.mode ?? "anthropic";
|
||||
|
||||
// Track the latest self-hosted test result so ModeButton can gate access.
|
||||
const [selfHostedTestResult, setSelfHostedTestResult] =
|
||||
useState<SelfHostedTestResult | null>(null);
|
||||
const isSelfHostedConnected = selfHostedTestResult?.ok === true;
|
||||
|
||||
const handleSelfHostedTestResult = useCallback(
|
||||
(result: SelfHostedTestResult) => {
|
||||
setSelfHostedTestResult(result);
|
||||
},
|
||||
[],
|
||||
);
|
||||
|
||||
// Selected self-hosted model (used when mode === 'self_hosted').
|
||||
const [selfHostedModel, setSelfHostedModel] = useState<string>("");
|
||||
|
||||
// --- API key input ---
|
||||
const [apiKey, setApiKey] = useState("");
|
||||
const [clearKey, setClearKey] = useState(false);
|
||||
@@ -123,6 +144,9 @@ export function AIRoutingCard() {
|
||||
const catalogOllamaOnly = catalog.filter(
|
||||
(c: { provider_type: ModelProvider }) => c.provider_type === ModelProvider.OLLAMA_CLOUD,
|
||||
);
|
||||
const catalogAnthropicOnly = catalog.filter(
|
||||
(c: { provider_type: ModelProvider }) => c.provider_type === ModelProvider.ANTHROPIC,
|
||||
);
|
||||
|
||||
// --- Mode toggle handlers ---
|
||||
const flipToAnthropic = async () => {
|
||||
@@ -149,6 +173,28 @@ export function AIRoutingCard() {
|
||||
}
|
||||
};
|
||||
|
||||
const flipToSelfHosted = async () => {
|
||||
if (!isSelfHostedConnected) {
|
||||
toast.error("Test the self-hosted connection first");
|
||||
return;
|
||||
}
|
||||
if (
|
||||
!confirm(
|
||||
"Switch every agent to the self-hosted LLM? Clears any overrides.",
|
||||
)
|
||||
)
|
||||
return;
|
||||
try {
|
||||
await applyMode.mutateAsync({
|
||||
mode: "self_hosted",
|
||||
...(selfHostedModel ? { default_model: selfHostedModel } : {}),
|
||||
});
|
||||
toast.success("All agents now on Self-Hosted LLM");
|
||||
} catch (e) {
|
||||
toast.error("Switch failed: " + errMsg(e));
|
||||
}
|
||||
};
|
||||
|
||||
const saveMix = async () => {
|
||||
// Filter out empty picks (nothing selected = inherit global).
|
||||
const per_agent: Record<string, string> = {};
|
||||
@@ -172,6 +218,15 @@ export function AIRoutingCard() {
|
||||
);
|
||||
return;
|
||||
}
|
||||
const needsSelfHosted = Object.values(per_agent).some((m) =>
|
||||
selfHostedModels.find((sh) => sh.model_name === m),
|
||||
);
|
||||
if (needsSelfHosted && !isSelfHostedConnected) {
|
||||
toast.error(
|
||||
"At least one agent is routed to a self-hosted model but the connection has not been tested",
|
||||
);
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await applyMode.mutateAsync({ mode: "mix", per_agent });
|
||||
toast.success("Per-agent routing saved");
|
||||
@@ -189,7 +244,8 @@ export function AIRoutingCard() {
|
||||
<CardDescription>
|
||||
Decide which model backs each agent. Anthropic uses the mounted
|
||||
<code className="px-1"> ~/.claude </code> auth; Ollama Cloud uses
|
||||
the API key you save below.
|
||||
the API key you save below; Self-Hosted connects to any OpenAI-compatible
|
||||
endpoint you run locally.
|
||||
</CardDescription>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-6">
|
||||
@@ -242,10 +298,19 @@ export function AIRoutingCard() {
|
||||
|
||||
<Separator />
|
||||
|
||||
{/* -------- Self-Hosted LLM -------- */}
|
||||
<SelfHostedSection
|
||||
testResult={selfHostedTestResult}
|
||||
onTestResult={handleSelfHostedTestResult}
|
||||
onTestSuccess={() => undefined}
|
||||
/>
|
||||
|
||||
<Separator />
|
||||
|
||||
{/* -------- Mode toggle -------- */}
|
||||
<section className="space-y-3">
|
||||
<Label className="text-sm font-medium">Routing mode</Label>
|
||||
<div className="grid grid-cols-1 md:grid-cols-3 gap-2">
|
||||
<div className="grid grid-cols-2 md:grid-cols-4 gap-2">
|
||||
<ModeButton
|
||||
icon={<ShieldCheck className="h-4 w-4" />}
|
||||
label="Anthropic"
|
||||
@@ -266,6 +331,18 @@ export function AIRoutingCard() {
|
||||
onClick={flipToOllama}
|
||||
disabled={applyMode.isPending || !hasOllamaKey}
|
||||
/>
|
||||
<ModeButton
|
||||
icon={<Server className="h-4 w-4" />}
|
||||
label="Self-Hosted"
|
||||
description={
|
||||
isSelfHostedConnected
|
||||
? "Every agent uses your self-hosted LLM endpoint."
|
||||
: "Test the connection above first."
|
||||
}
|
||||
active={currentMode === "self_hosted"}
|
||||
onClick={flipToSelfHosted}
|
||||
disabled={applyMode.isPending || !isSelfHostedConnected}
|
||||
/>
|
||||
<ModeButton
|
||||
icon={<Cpu className="h-4 w-4" />}
|
||||
label="Mix"
|
||||
@@ -287,6 +364,43 @@ export function AIRoutingCard() {
|
||||
) : null}
|
||||
</section>
|
||||
|
||||
{/* -------- Self-Hosted model picker (when self_hosted mode active) -------- */}
|
||||
{currentMode === "self_hosted" && (
|
||||
<>
|
||||
<Separator />
|
||||
<section className="space-y-2">
|
||||
<Label className="text-sm font-medium">
|
||||
Self-Hosted default model
|
||||
</Label>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Choose which discovered model all agents should use in
|
||||
self-hosted mode.
|
||||
</p>
|
||||
<Select
|
||||
value={selfHostedModel || "__clear__"}
|
||||
onValueChange={(v: string) =>
|
||||
setSelfHostedModel(v === "__clear__" ? "" : v)
|
||||
}
|
||||
>
|
||||
<SelectTrigger className="w-full max-w-sm">
|
||||
<SelectValue placeholder="(use server default)" />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value="__clear__">(use server default)</SelectItem>
|
||||
{selfHostedModels.map((m) => (
|
||||
<SelectItem key={m.model_name} value={m.model_name}>
|
||||
{m.display_name}
|
||||
{m.display_name !== m.model_name
|
||||
? ` — ${m.model_name}`
|
||||
: ""}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</section>
|
||||
</>
|
||||
)}
|
||||
|
||||
{/* -------- Mix-mode per-agent picker -------- */}
|
||||
<Separator />
|
||||
<section className="space-y-3">
|
||||
@@ -327,16 +441,72 @@ export function AIRoutingCard() {
|
||||
}))
|
||||
}
|
||||
>
|
||||
<SelectTrigger>
|
||||
<SelectTrigger className="w-full">
|
||||
<SelectValue placeholder="(inherit)" />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value="__clear__">(inherit global)</SelectItem>
|
||||
{catalogForMix.map((c: { model_name: string; display_name: string }) => (
|
||||
<SelectItem key={c.model_name} value={c.model_name}>
|
||||
{c.display_name} — {c.model_name}
|
||||
</SelectItem>
|
||||
))}
|
||||
|
||||
{/* Anthropic models */}
|
||||
{catalogAnthropicOnly.length > 0 && (
|
||||
<SelectGroup>
|
||||
<SelectLabel>
|
||||
<ProviderBadge variant="anthropic" />
|
||||
Anthropic
|
||||
</SelectLabel>
|
||||
{catalogAnthropicOnly.map(
|
||||
(c: { model_name: string; display_name: string }) => (
|
||||
<SelectItem key={c.model_name} value={c.model_name}>
|
||||
{c.display_name}
|
||||
</SelectItem>
|
||||
),
|
||||
)}
|
||||
</SelectGroup>
|
||||
)}
|
||||
|
||||
{/* Ollama Cloud models */}
|
||||
{catalogOllamaOnly.length > 0 && (
|
||||
<SelectGroup>
|
||||
<SelectLabel>
|
||||
<ProviderBadge variant="ollama" />
|
||||
Ollama Cloud
|
||||
</SelectLabel>
|
||||
{catalogOllamaOnly.map(
|
||||
(c: { model_name: string; display_name: string }) => (
|
||||
<SelectItem key={c.model_name} value={c.model_name}>
|
||||
{c.display_name}
|
||||
</SelectItem>
|
||||
),
|
||||
)}
|
||||
</SelectGroup>
|
||||
)}
|
||||
|
||||
{/* Self-Hosted models */}
|
||||
{selfHostedModels.length > 0 && (
|
||||
<SelectGroup>
|
||||
<SelectLabel>
|
||||
<ProviderBadge variant="self-hosted" />
|
||||
Self-Hosted
|
||||
</SelectLabel>
|
||||
{selfHostedModels.map((m) => (
|
||||
<SelectItem key={m.model_name} value={m.model_name}>
|
||||
{m.display_name}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectGroup>
|
||||
)}
|
||||
|
||||
{/* Fallback: un-grouped catalog when no grouping is possible */}
|
||||
{catalogAnthropicOnly.length === 0 &&
|
||||
catalogOllamaOnly.length === 0 &&
|
||||
selfHostedModels.length === 0 &&
|
||||
catalogForMix.map(
|
||||
(c: { model_name: string; display_name: string }) => (
|
||||
<SelectItem key={c.model_name} value={c.model_name}>
|
||||
{c.display_name} — {c.model_name}
|
||||
</SelectItem>
|
||||
),
|
||||
)}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
@@ -402,3 +572,34 @@ function ModeButton({
|
||||
function errMsg(e: unknown): string {
|
||||
return e instanceof Error ? e.message : "Unknown error";
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Small colored pill to distinguish providers in the Mix mode dropdown.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function ProviderBadge({
|
||||
variant,
|
||||
}: {
|
||||
variant: "anthropic" | "ollama" | "self-hosted";
|
||||
}) {
|
||||
const styles: Record<string, string> = {
|
||||
anthropic: "bg-blue-500/20 text-blue-700 dark:text-blue-400",
|
||||
ollama: "bg-violet-500/20 text-violet-700 dark:text-violet-400",
|
||||
"self-hosted": "bg-purple-500/20 text-purple-700 dark:text-purple-400",
|
||||
};
|
||||
const labels: Record<string, string> = {
|
||||
anthropic: "A",
|
||||
ollama: "O",
|
||||
"self-hosted": "S",
|
||||
};
|
||||
return (
|
||||
<span
|
||||
className={
|
||||
"mr-1 inline-flex h-4 w-4 items-center justify-center rounded-full text-[10px] font-bold " +
|
||||
(styles[variant] ?? "")
|
||||
}
|
||||
>
|
||||
{labels[variant]}
|
||||
</span>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,363 @@
|
||||
"use client";
|
||||
|
||||
import { useState, useCallback } from "react";
|
||||
import {
|
||||
useSelfHostedConfig,
|
||||
useSetSelfHostedConfig,
|
||||
useTestSelfHosted,
|
||||
useSelfHostedModels,
|
||||
useRefreshSelfHostedModels,
|
||||
} from "@/hooks/use-providers";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { Label } from "@/components/ui/label";
|
||||
import { Badge } from "@/components/ui/badge";
|
||||
import {
|
||||
AlertTriangle,
|
||||
CheckCircle2,
|
||||
Eye,
|
||||
EyeOff,
|
||||
RefreshCw,
|
||||
Server,
|
||||
XCircle,
|
||||
} from "lucide-react";
|
||||
import { toast } from "sonner";
|
||||
import type { SelfHostedTestResult } from "@/lib/api/providers";
|
||||
|
||||
// Relative-time helper (no date-fns dependency).
|
||||
function relativeTime(isoDate: string | null): string {
|
||||
if (!isoDate) return "never";
|
||||
const diff = Date.now() - new Date(isoDate).getTime();
|
||||
const s = Math.floor(diff / 1000);
|
||||
if (s < 60) return `${s}s ago`;
|
||||
const m = Math.floor(s / 60);
|
||||
if (m < 60) return `${m}m ago`;
|
||||
const h = Math.floor(m / 60);
|
||||
if (h < 24) return `${h}h ago`;
|
||||
return `${Math.floor(h / 24)}d ago`;
|
||||
}
|
||||
|
||||
function errMsg(e: unknown): string {
|
||||
return e instanceof Error ? e.message : "Unknown error";
|
||||
}
|
||||
|
||||
/** Props exposed to the parent (ai-routing-card) so it can read test status. */
|
||||
export interface SelfHostedSectionProps {
|
||||
/** Called whenever a successful connection test completes. */
|
||||
onTestSuccess?: (modelCount: number) => void;
|
||||
/** The last known test result — driven by parent if the parent stores it. */
|
||||
testResult?: SelfHostedTestResult | null;
|
||||
/** Called when the user clicks "Test Connection" and we get any result. */
|
||||
onTestResult?: (result: SelfHostedTestResult) => void;
|
||||
}
|
||||
|
||||
export function SelfHostedSection({
|
||||
onTestSuccess,
|
||||
testResult,
|
||||
onTestResult,
|
||||
}: SelfHostedSectionProps) {
|
||||
const { data: config } = useSelfHostedConfig();
|
||||
const { data: models = [] } = useSelfHostedModels();
|
||||
|
||||
const saveConfig = useSetSelfHostedConfig();
|
||||
const testConnection = useTestSelfHosted();
|
||||
const refreshModels = useRefreshSelfHostedModels();
|
||||
|
||||
// Local form state
|
||||
const [baseUrl, setBaseUrl] = useState("");
|
||||
const [authToken, setAuthToken] = useState("");
|
||||
const [showToken, setShowToken] = useState(false);
|
||||
|
||||
// Timestamps for "Last refreshed" label
|
||||
const [lastRefreshed, setLastRefreshed] = useState<string | null>(null);
|
||||
|
||||
const hasSavedUrl = !!(config?.base_url);
|
||||
|
||||
// ---- Save handler --------------------------------------------------------
|
||||
const handleSave = async () => {
|
||||
const url = baseUrl.trim();
|
||||
if (!url) {
|
||||
toast.error("Enter a base URL first");
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await saveConfig.mutateAsync({
|
||||
base_url: url,
|
||||
...(authToken ? { auth_token: authToken } : {}),
|
||||
});
|
||||
toast.success("Self-hosted config saved");
|
||||
setBaseUrl("");
|
||||
setAuthToken("");
|
||||
} catch (e) {
|
||||
toast.error("Save failed: " + errMsg(e));
|
||||
}
|
||||
};
|
||||
|
||||
// ---- Test Connection handler ---------------------------------------------
|
||||
const handleTest = async () => {
|
||||
try {
|
||||
const result = await testConnection.mutateAsync();
|
||||
onTestResult?.(result);
|
||||
if (result.ok) {
|
||||
onTestSuccess?.(result.model_count ?? 0);
|
||||
setLastRefreshed(new Date().toISOString());
|
||||
const count = result.model_count ?? 0;
|
||||
toast.success(
|
||||
`Connected — ${count} model${count === 1 ? "" : "s"} available`,
|
||||
);
|
||||
} else {
|
||||
toast.error(`Connection failed: ${result.error ?? "unknown error"}`);
|
||||
}
|
||||
} catch (e) {
|
||||
toast.error("Test failed: " + errMsg(e));
|
||||
}
|
||||
};
|
||||
|
||||
// ---- Refresh Models handler ----------------------------------------------
|
||||
const handleRefresh = useCallback(async () => {
|
||||
try {
|
||||
await refreshModels.mutateAsync();
|
||||
setLastRefreshed(new Date().toISOString());
|
||||
toast.success("Model list refreshed");
|
||||
} catch (e) {
|
||||
toast.error("Refresh failed: " + errMsg(e));
|
||||
}
|
||||
}, [refreshModels]);
|
||||
|
||||
// ---- Retry handler (from error empty state) ------------------------------
|
||||
const handleRetry = () => handleTest();
|
||||
|
||||
// Determine which empty state to show, if any.
|
||||
const showNoUrlState = !hasSavedUrl;
|
||||
const showErrorState =
|
||||
hasSavedUrl && testResult?.ok === false && !showNoUrlState;
|
||||
const showNoModelsState =
|
||||
hasSavedUrl &&
|
||||
testResult?.ok === true &&
|
||||
models.length === 0 &&
|
||||
!showNoUrlState;
|
||||
const showModelList =
|
||||
hasSavedUrl && testResult?.ok === true && models.length > 0;
|
||||
|
||||
return (
|
||||
<section className="space-y-4">
|
||||
{/* Header */}
|
||||
<div className="flex items-center gap-2">
|
||||
<Server className="h-4 w-4 text-muted-foreground" />
|
||||
<Label className="text-sm font-medium">Self-Hosted LLM</Label>
|
||||
{testResult?.ok === true && (
|
||||
<span className="inline-flex items-center gap-1 rounded-full bg-emerald-500/10 px-2 py-0.5 text-xs font-medium text-emerald-600">
|
||||
<CheckCircle2 className="h-3 w-3" /> connected
|
||||
</span>
|
||||
)}
|
||||
{testResult?.ok === false && (
|
||||
<span className="inline-flex items-center gap-1 rounded-full bg-red-500/10 px-2 py-0.5 text-xs font-medium text-red-600">
|
||||
<XCircle className="h-3 w-3" /> error
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Base URL input */}
|
||||
<div className="space-y-1">
|
||||
<Label className="text-xs text-muted-foreground">Base URL</Label>
|
||||
<div className="flex gap-2">
|
||||
<Input
|
||||
type="text"
|
||||
value={baseUrl}
|
||||
onChange={(e) => setBaseUrl(e.target.value)}
|
||||
placeholder={
|
||||
hasSavedUrl
|
||||
? config.base_url ?? "http://localhost:11434"
|
||||
: "http://localhost:11434"
|
||||
}
|
||||
className="font-mono text-sm"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Auth token input with Eye toggle */}
|
||||
<div className="space-y-1">
|
||||
<Label className="text-xs text-muted-foreground">
|
||||
Auth token{" "}
|
||||
<span className="text-muted-foreground/60">(optional)</span>
|
||||
</Label>
|
||||
<div className="flex gap-2">
|
||||
<div className="relative flex-1">
|
||||
<Input
|
||||
type={showToken ? "text" : "password"}
|
||||
value={authToken}
|
||||
onChange={(e) => setAuthToken(e.target.value)}
|
||||
placeholder={
|
||||
config?.has_token
|
||||
? "•••••••••••• (leave blank to keep)"
|
||||
: "sk-… or Bearer token"
|
||||
}
|
||||
className="pr-10"
|
||||
/>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setShowToken((v) => !v)}
|
||||
className="absolute right-2 top-1/2 -translate-y-1/2 text-muted-foreground hover:text-foreground"
|
||||
aria-label={showToken ? "Hide token" : "Show token"}
|
||||
>
|
||||
{showToken ? (
|
||||
<EyeOff className="h-4 w-4" />
|
||||
) : (
|
||||
<Eye className="h-4 w-4" />
|
||||
)}
|
||||
</button>
|
||||
</div>
|
||||
<Button onClick={handleSave} disabled={saveConfig.isPending}>
|
||||
{saveConfig.isPending ? "Saving…" : "Save"}
|
||||
</Button>
|
||||
</div>
|
||||
{config?.has_token && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
A token is stored. Type a new value to update it.
|
||||
</p>
|
||||
)}
|
||||
{!config?.has_token && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Stored Fernet-encrypted server-side; never returned by the API.
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Test Connection button + inline result badge */}
|
||||
<div className="flex items-center gap-3">
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={handleTest}
|
||||
disabled={!hasSavedUrl || testConnection.isPending}
|
||||
>
|
||||
{testConnection.isPending ? (
|
||||
<>
|
||||
<RefreshCw className="mr-1.5 h-3.5 w-3.5 animate-spin" />
|
||||
Testing…
|
||||
</>
|
||||
) : (
|
||||
"Test Connection"
|
||||
)}
|
||||
</Button>
|
||||
|
||||
{/* Inline result badge */}
|
||||
{testResult?.ok === true && (
|
||||
<span className="inline-flex items-center gap-1 rounded-full bg-emerald-500/10 px-3 py-1 text-xs font-medium text-emerald-700 dark:text-emerald-400">
|
||||
<CheckCircle2 className="h-3.5 w-3.5" />
|
||||
Connected — {testResult.model_count ?? 0} model
|
||||
{(testResult.model_count ?? 0) === 1 ? "" : "s"} available
|
||||
</span>
|
||||
)}
|
||||
{testResult?.ok === false && (
|
||||
<span className="inline-flex items-center gap-1 rounded-full bg-red-500/10 px-3 py-1 text-xs font-medium text-red-700 dark:text-red-400">
|
||||
<XCircle className="h-3.5 w-3.5" />
|
||||
{testResult.error ?? "Connection failed"}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* ── Empty state 1: no base URL configured ── */}
|
||||
{showNoUrlState && (
|
||||
<div className="rounded-md border border-dashed p-4 text-sm text-muted-foreground">
|
||||
<p className="font-medium">No base URL configured</p>
|
||||
<p className="mt-1 text-xs">
|
||||
Enter the URL of your self-hosted LLM endpoint (e.g.{" "}
|
||||
<code>http://localhost:11434</code> for Ollama) and click{" "}
|
||||
<strong>Save</strong>. Then click <strong>Test Connection</strong>{" "}
|
||||
to verify and discover available models.
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* ── Empty state 2: error state ── */}
|
||||
{showErrorState && (
|
||||
<div className="rounded-md border border-red-200 bg-red-50 p-4 dark:border-red-900/40 dark:bg-red-950/20">
|
||||
<div className="flex items-start gap-2">
|
||||
<AlertTriangle className="mt-0.5 h-4 w-4 shrink-0 text-red-600" />
|
||||
<div className="flex-1 space-y-1 text-sm">
|
||||
<p className="font-medium text-red-700 dark:text-red-400">
|
||||
Last test failed
|
||||
</p>
|
||||
<p className="text-xs text-red-600 dark:text-red-500">
|
||||
{testResult?.error ?? "Unknown error"}
|
||||
</p>
|
||||
</div>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={handleRetry}
|
||||
disabled={testConnection.isPending}
|
||||
>
|
||||
Retry
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* ── Empty state 3: connected but 0 models ── */}
|
||||
{showNoModelsState && (
|
||||
<div className="rounded-md border border-amber-200 bg-amber-50 p-4 dark:border-amber-900/40 dark:bg-amber-950/20">
|
||||
<div className="flex items-start gap-2">
|
||||
<AlertTriangle className="mt-0.5 h-4 w-4 shrink-0 text-amber-600" />
|
||||
<p className="text-sm text-amber-700 dark:text-amber-400">
|
||||
Connected but no models found. Pull a model first (e.g.{" "}
|
||||
<code className="font-mono">ollama pull llama3.2</code>) then
|
||||
click <strong>Refresh Models</strong>.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* ── Model list ── */}
|
||||
{showModelList && (
|
||||
<div className="space-y-2">
|
||||
<div className="flex items-center justify-between">
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Last refreshed:{" "}
|
||||
<span className="font-medium">
|
||||
{relativeTime(lastRefreshed)}
|
||||
</span>
|
||||
</p>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={handleRefresh}
|
||||
disabled={refreshModels.isPending}
|
||||
>
|
||||
{refreshModels.isPending ? (
|
||||
<>
|
||||
<RefreshCw className="mr-1.5 h-3 w-3 animate-spin" />
|
||||
Refreshing…
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<RefreshCw className="mr-1.5 h-3 w-3" />
|
||||
Refresh Models
|
||||
</>
|
||||
)}
|
||||
</Button>
|
||||
</div>
|
||||
<div className="divide-y rounded-md border">
|
||||
{models.map((m) => (
|
||||
<div
|
||||
key={m.model_name}
|
||||
className="flex items-center justify-between px-3 py-2"
|
||||
>
|
||||
<div>
|
||||
<span className="text-sm font-medium">{m.display_name}</span>
|
||||
<span className="ml-2 font-mono text-xs text-muted-foreground">
|
||||
{m.model_name}
|
||||
</span>
|
||||
</div>
|
||||
<Badge variant="secondary" className="text-xs">
|
||||
auto-discovered
|
||||
</Badge>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
);
|
||||
}
|
||||
@@ -2,6 +2,7 @@ import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import {
|
||||
providersApi,
|
||||
type ApplyModePayload,
|
||||
type SelfHostedConfigPayload,
|
||||
} from "@/lib/api/providers";
|
||||
|
||||
export const providerKeys = {
|
||||
@@ -9,6 +10,9 @@ export const providerKeys = {
|
||||
catalog: () => [...providerKeys.all, "catalog"] as const,
|
||||
ollamaKey: () => [...providerKeys.all, "ollama-key"] as const,
|
||||
mode: () => [...providerKeys.all, "mode"] as const,
|
||||
selfHostedConfig: () => [...providerKeys.all, "self-hosted-config"] as const,
|
||||
selfHostedModels: () => [...providerKeys.all, "self-hosted-models"] as const,
|
||||
selfHostedTest: () => [...providerKeys.all, "self-hosted-test"] as const,
|
||||
};
|
||||
|
||||
export function useCatalog() {
|
||||
@@ -56,3 +60,59 @@ export function useApplyMode() {
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Self-hosted LLM hooks
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Query: fetch saved self-hosted config (base_url + has_token flag). */
|
||||
export function useSelfHostedConfig() {
|
||||
return useQuery({
|
||||
queryKey: providerKeys.selfHostedConfig(),
|
||||
queryFn: () => providersApi.getSelfHostedConfig(),
|
||||
staleTime: 30_000,
|
||||
});
|
||||
}
|
||||
|
||||
/** Mutation: save self-hosted base URL + optional auth token. */
|
||||
export function useSetSelfHostedConfig() {
|
||||
const qc = useQueryClient();
|
||||
return useMutation({
|
||||
mutationFn: (payload: SelfHostedConfigPayload) =>
|
||||
providersApi.saveSelfHostedConfig(payload),
|
||||
onSuccess: () => {
|
||||
qc.invalidateQueries({ queryKey: providerKeys.selfHostedConfig() });
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/** Mutation: test the self-hosted connection and return status + model count. */
|
||||
export function useTestSelfHosted() {
|
||||
const qc = useQueryClient();
|
||||
return useMutation({
|
||||
mutationFn: () => providersApi.testSelfHosted(),
|
||||
onSuccess: () => {
|
||||
// After a successful test, also refresh the model list.
|
||||
qc.invalidateQueries({ queryKey: providerKeys.selfHostedModels() });
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/** Query: list models discovered from the self-hosted endpoint. */
|
||||
export function useSelfHostedModels() {
|
||||
return useQuery({
|
||||
queryKey: providerKeys.selfHostedModels(),
|
||||
queryFn: () => providersApi.getSelfHostedModels(),
|
||||
staleTime: 2 * 60_000, // 2 minutes
|
||||
});
|
||||
}
|
||||
|
||||
/** Mutation: invalidate the cached model list so it is re-fetched from GET /self-hosted/models. */
|
||||
export function useRefreshSelfHostedModels() {
|
||||
const qc = useQueryClient();
|
||||
return useMutation({
|
||||
mutationFn: async () => {
|
||||
await qc.invalidateQueries({ queryKey: providerKeys.selfHostedModels() });
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
@@ -24,7 +24,7 @@ export interface ModelAssignment {
|
||||
model_name: string;
|
||||
}
|
||||
|
||||
export type RoutingMode = "anthropic" | "ollama" | "mix";
|
||||
export type RoutingMode = "anthropic" | "ollama" | "self_hosted" | "mix";
|
||||
|
||||
export interface ModeSnapshot {
|
||||
mode: RoutingMode;
|
||||
@@ -38,6 +38,36 @@ export interface ApplyModePayload {
|
||||
per_agent?: Record<string, string>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Self-hosted LLM types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Configuration stored server-side for the self-hosted provider. */
|
||||
export interface SelfHostedConfig {
|
||||
base_url: string | null;
|
||||
has_token: boolean;
|
||||
enabled: boolean;
|
||||
}
|
||||
|
||||
/** Result of a test-connection call. */
|
||||
export interface SelfHostedTestResult {
|
||||
ok: boolean;
|
||||
model_count: number | null;
|
||||
error: string | null;
|
||||
}
|
||||
|
||||
/** One model entry returned by the discovery endpoint. */
|
||||
export interface SelfHostedModel {
|
||||
model_name: string;
|
||||
display_name: string;
|
||||
}
|
||||
|
||||
/** Payload for saving the self-hosted config (base URL + optional token). */
|
||||
export interface SelfHostedConfigPayload {
|
||||
base_url: string;
|
||||
auth_token?: string; // omit to leave token unchanged; "" to clear
|
||||
}
|
||||
|
||||
export const providersApi = {
|
||||
catalog: async (): Promise<CatalogEntry[]> => {
|
||||
const { data } = await api.get<CatalogEntry[]>("/providers/catalog");
|
||||
@@ -65,4 +95,37 @@ export const providersApi = {
|
||||
const { data } = await api.post<ModeSnapshot>("/providers", payload);
|
||||
return data;
|
||||
},
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Self-hosted provider
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
getSelfHostedConfig: async (): Promise<SelfHostedConfig> => {
|
||||
const { data } = await api.get<SelfHostedConfig>("/providers/self-hosted");
|
||||
return data;
|
||||
},
|
||||
|
||||
saveSelfHostedConfig: async (
|
||||
payload: SelfHostedConfigPayload,
|
||||
): Promise<SelfHostedConfig> => {
|
||||
const { data } = await api.put<SelfHostedConfig>(
|
||||
"/providers/self-hosted",
|
||||
payload,
|
||||
);
|
||||
return data;
|
||||
},
|
||||
|
||||
testSelfHosted: async (): Promise<SelfHostedTestResult> => {
|
||||
const { data } = await api.post<SelfHostedTestResult>(
|
||||
"/providers/self-hosted/test",
|
||||
);
|
||||
return data;
|
||||
},
|
||||
|
||||
getSelfHostedModels: async (): Promise<SelfHostedModel[]> => {
|
||||
const { data } = await api.get<SelfHostedModel[]>(
|
||||
"/providers/self-hosted/models",
|
||||
);
|
||||
return data;
|
||||
},
|
||||
};
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
"""
|
||||
Provider Routes
|
||||
|
||||
Thin HTTP plumbing for the Settings UI's AI-routing panel. Four endpoints
|
||||
cover the whole UX: fetch the catalog, get / set the Ollama key, fetch the
|
||||
current mode + assignments, apply a mode change. No provider CRUD — the
|
||||
two providers (Anthropic + Ollama Cloud) are pre-seeded by migration 004.
|
||||
Thin HTTP plumbing for the Settings UI's AI-routing panel. Endpoints
|
||||
cover the whole UX: fetch the catalog, get / set the Ollama key,
|
||||
configure / test / discover the self-hosted server, fetch the current
|
||||
mode + assignments, apply a mode change. No provider CRUD — the
|
||||
providers (Anthropic, Ollama Cloud, Self-Hosted) are pre-seeded by
|
||||
migrations 004 and 028.
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter, HTTPException, status
|
||||
@@ -15,14 +17,19 @@ from roboco.api.schemas.provider import (
|
||||
CatalogEntryResponse,
|
||||
ModeResponse,
|
||||
OllamaKeyStatus,
|
||||
SelfHostedConfigRequest,
|
||||
SelfHostedConfigResponse,
|
||||
SelfHostedModelEntry,
|
||||
SelfHostedTestResponse,
|
||||
SetOllamaKeyRequest,
|
||||
assignment_to_response,
|
||||
)
|
||||
from roboco.models.base import ModelProvider
|
||||
from roboco.models.llm_catalog import MODEL_CATALOG
|
||||
from roboco.services.base import NotFoundError
|
||||
from roboco.services.llm import get_model_routing_service
|
||||
from roboco.services.provider import get_provider_service
|
||||
from roboco.services.llm import get_model_routing_service, probe_ollama_tags
|
||||
from roboco.services.provider import ProviderUpdate, get_provider_service
|
||||
from roboco.utils.converters import require_uuid
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -106,6 +113,172 @@ async def set_ollama_key(
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# SELF-HOSTED (LOCAL) OLLAMA SERVER
|
||||
# =============================================================================
|
||||
|
||||
|
||||
@router.get("/self-hosted", response_model=SelfHostedConfigResponse)
|
||||
async def get_self_hosted_config(
|
||||
db: DbSession,
|
||||
agent: CurrentAgentContext,
|
||||
) -> SelfHostedConfigResponse:
|
||||
"""Return the current configuration of the LOCAL (self-hosted) provider.
|
||||
|
||||
The LOCAL provider row must be seeded (migration 028). Returns
|
||||
``{base_url, has_token, enabled}`` so the Settings UI can display
|
||||
the current state without exposing the encrypted token.
|
||||
"""
|
||||
require_pm_or_above(agent.role, "view the self-hosted provider config")
|
||||
provider_svc = get_provider_service(db)
|
||||
providers = await provider_svc.list_providers(include_disabled=True)
|
||||
local = next(
|
||||
(p for p in providers if p.type == ModelProvider.LOCAL),
|
||||
None,
|
||||
)
|
||||
if local is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=(
|
||||
"Self-Hosted provider not seeded. "
|
||||
"Run alembic upgrade head (migration 028)."
|
||||
),
|
||||
)
|
||||
return SelfHostedConfigResponse(
|
||||
base_url=local.base_url,
|
||||
has_token=bool(local.auth_token_encrypted),
|
||||
enabled=local.enabled,
|
||||
)
|
||||
|
||||
|
||||
@router.put("/self-hosted", response_model=SelfHostedConfigResponse)
|
||||
async def set_self_hosted_config(
|
||||
data: SelfHostedConfigRequest,
|
||||
db: DbSession,
|
||||
agent: CurrentAgentContext,
|
||||
) -> SelfHostedConfigResponse:
|
||||
"""Save the base URL (and optionally an auth token) for the LOCAL provider.
|
||||
|
||||
The LOCAL provider row must be seeded (migration 028). The token, when
|
||||
provided and non-empty, is Fernet-encrypted before storing. An empty
|
||||
string for `auth_token` clears the stored token. The provider is
|
||||
automatically enabled only when a non-empty base_url is provided.
|
||||
"""
|
||||
require_pm_or_above(agent.role, "configure the self-hosted provider")
|
||||
provider_svc = get_provider_service(db)
|
||||
providers = await provider_svc.list_providers(include_disabled=True)
|
||||
local = next(
|
||||
(p for p in providers if p.type == ModelProvider.LOCAL),
|
||||
None,
|
||||
)
|
||||
if local is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=(
|
||||
"Self-Hosted provider not seeded. "
|
||||
"Run alembic upgrade head (migration 028)."
|
||||
),
|
||||
)
|
||||
|
||||
# Determine token update intent:
|
||||
# None → leave unchanged, "" → clear, non-empty str → re-encrypt.
|
||||
clear_token = data.auth_token is not None and data.auth_token == ""
|
||||
new_token = data.auth_token if (data.auth_token and data.auth_token != "") else None
|
||||
|
||||
await provider_svc.update_provider(
|
||||
require_uuid(local.id),
|
||||
ProviderUpdate(
|
||||
base_url=data.base_url,
|
||||
auth_token=new_token,
|
||||
clear_auth_token=clear_token,
|
||||
# Only enable the provider when a non-empty base_url is provided.
|
||||
enabled=bool(data.base_url),
|
||||
),
|
||||
)
|
||||
await db.commit()
|
||||
|
||||
# Re-fetch after commit to get the persisted state.
|
||||
updated = await provider_svc.list_providers(include_disabled=True)
|
||||
local_updated = next(p for p in updated if p.type == ModelProvider.LOCAL)
|
||||
return SelfHostedConfigResponse(
|
||||
base_url=local_updated.base_url,
|
||||
has_token=bool(local_updated.auth_token_encrypted),
|
||||
enabled=local_updated.enabled,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/self-hosted/test", response_model=SelfHostedTestResponse)
|
||||
async def test_self_hosted_connection(
|
||||
db: DbSession,
|
||||
agent: CurrentAgentContext,
|
||||
) -> SelfHostedTestResponse:
|
||||
"""Probe the configured self-hosted Ollama server.
|
||||
|
||||
Returns ``{ok: true, model_count: N}`` when the server is reachable
|
||||
and returns a valid model list from ``{base_url}/api/tags``.
|
||||
Returns ``{ok: false, error: '<message>'}`` on any failure — never
|
||||
raises a 500, so the Settings UI can display a friendly error.
|
||||
"""
|
||||
require_pm_or_above(agent.role, "test the self-hosted connection")
|
||||
provider_svc = get_provider_service(db)
|
||||
providers = await provider_svc.list_providers(include_disabled=True)
|
||||
local = next(
|
||||
(p for p in providers if p.type == ModelProvider.LOCAL),
|
||||
None,
|
||||
)
|
||||
if local is None or not local.base_url:
|
||||
return SelfHostedTestResponse(
|
||||
ok=False,
|
||||
error=(
|
||||
"Self-hosted server is not configured."
|
||||
" Set base_url first via PUT /self-hosted."
|
||||
),
|
||||
)
|
||||
|
||||
models, error = await probe_ollama_tags(local.base_url)
|
||||
if error is not None:
|
||||
return SelfHostedTestResponse(ok=False, error=error)
|
||||
return SelfHostedTestResponse(ok=True, model_count=len(models))
|
||||
|
||||
|
||||
@router.get("/self-hosted/models", response_model=list[SelfHostedModelEntry])
|
||||
async def get_self_hosted_models(
|
||||
db: DbSession,
|
||||
agent: CurrentAgentContext,
|
||||
) -> list[SelfHostedModelEntry]:
|
||||
"""Return the list of models available on the self-hosted Ollama server.
|
||||
|
||||
Queries ``{base_url}/api/tags`` and returns ``[{model_name, display_name}]``
|
||||
for each model entry. Raises 503 if the server is unreachable, 404 if
|
||||
the LOCAL provider is not configured.
|
||||
"""
|
||||
require_pm_or_above(agent.role, "list self-hosted models")
|
||||
provider_svc = get_provider_service(db)
|
||||
providers = await provider_svc.list_providers(include_disabled=True)
|
||||
local = next(
|
||||
(p for p in providers if p.type == ModelProvider.LOCAL),
|
||||
None,
|
||||
)
|
||||
if local is None or not local.base_url:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=(
|
||||
"Self-hosted provider is not configured. "
|
||||
"Set base_url via PUT /self-hosted first."
|
||||
),
|
||||
)
|
||||
|
||||
model_names, error = await probe_ollama_tags(local.base_url)
|
||||
if error is not None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail=f"Self-hosted server unreachable: {error}",
|
||||
)
|
||||
return [
|
||||
SelfHostedModelEntry(model_name=name, display_name=name) for name in model_names
|
||||
]
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# MODE (the three-way toggle)
|
||||
# =============================================================================
|
||||
@@ -122,7 +295,7 @@ async def get_current_mode(
|
||||
mode = await routing.derive_mode()
|
||||
assignments = await routing.list_assignments()
|
||||
return ModeResponse(
|
||||
mode=mode, # type: ignore[arg-type]
|
||||
mode=mode,
|
||||
assignments=[assignment_to_response(a) for a in assignments],
|
||||
)
|
||||
|
||||
@@ -157,6 +330,6 @@ async def apply_mode(
|
||||
mode = await routing.derive_mode()
|
||||
assignments = await routing.list_assignments()
|
||||
return ModeResponse(
|
||||
mode=mode, # type: ignore[arg-type]
|
||||
mode=mode,
|
||||
assignments=[assignment_to_response(a) for a in assignments],
|
||||
)
|
||||
|
||||
@@ -4,8 +4,9 @@ Providers API Schemas
|
||||
Minimal surface that backs the Settings UI:
|
||||
- fetch the preset catalog of selectable models
|
||||
- set / clear / check the single Ollama Cloud API key
|
||||
- configure / test / discover the self-hosted (LOCAL) Ollama server
|
||||
- read current routing assignments (so the UI renders Mix mode)
|
||||
- apply a routing mode (anthropic | ollama | mix)
|
||||
- apply a routing mode (anthropic | ollama | mix | self_hosted)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -57,6 +58,66 @@ class SetOllamaKeyRequest(BaseModel):
|
||||
api_key: str = Field(default="")
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# SELF-HOSTED (LOCAL) OLLAMA SERVER
|
||||
# =============================================================================
|
||||
|
||||
|
||||
class SelfHostedConfigRequest(BaseModel):
|
||||
"""Save the base URL (and optionally an auth token) for the self-hosted server.
|
||||
|
||||
`base_url` is the root URL of the Ollama instance, e.g.
|
||||
``http://192.168.1.50:11434``. The Settings UI sends this on every
|
||||
save; the service stores it on the LOCAL provider row.
|
||||
|
||||
`auth_token`, when present and non-empty, is Fernet-encrypted before
|
||||
storing. Pass ``None`` or omit to leave any existing token unchanged.
|
||||
Pass an empty string to clear the stored token.
|
||||
"""
|
||||
|
||||
base_url: str = Field(..., description="Root URL of the self-hosted Ollama server")
|
||||
auth_token: str | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Optional bearer token for the Ollama server; omit to leave unchanged"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class SelfHostedConfigResponse(BaseModel):
|
||||
"""Current configuration state of the LOCAL provider row."""
|
||||
|
||||
base_url: str | None
|
||||
has_token: bool
|
||||
enabled: bool
|
||||
|
||||
|
||||
class SelfHostedTestResponse(BaseModel):
|
||||
"""Result of a connectivity probe to the self-hosted server.
|
||||
|
||||
The endpoint always returns HTTP 200; reachability is indicated by
|
||||
the `ok` field so the UI can display a human-readable error without
|
||||
triggering its generic error handler.
|
||||
"""
|
||||
|
||||
ok: bool
|
||||
model_count: int | None = None
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class SelfHostedModelEntry(BaseModel):
|
||||
"""One model available on the self-hosted Ollama server.
|
||||
|
||||
`model_name` is the raw Ollama tag identifier (e.g. ``llama3.1:8b``).
|
||||
`display_name` is a human-readable label for the Settings UI dropdown;
|
||||
for self-hosted models it mirrors `model_name` since Ollama's ``/api/tags``
|
||||
does not return a separate display label.
|
||||
"""
|
||||
|
||||
model_name: str
|
||||
display_name: str
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# MODEL ASSIGNMENTS (read-only for the UI)
|
||||
# =============================================================================
|
||||
@@ -99,10 +160,13 @@ class ApplyModeRequest(BaseModel):
|
||||
`default_model` (if omitted, the service picks a sensible default).
|
||||
- mode="mix": clear existing per-agent pins; upsert the `per_agent`
|
||||
map verbatim. Role + GLOBAL rows are left untouched so the user can
|
||||
layer with an existing partial setup.
|
||||
layer with an existing partial setup. Self-hosted model names in
|
||||
`per_agent` are routed to the LOCAL provider automatically.
|
||||
- mode="self_hosted": clear every assignment; enable LOCAL provider;
|
||||
set GLOBAL default to `default_model` (a self-hosted model name).
|
||||
"""
|
||||
|
||||
mode: Literal["anthropic", "ollama", "mix"]
|
||||
mode: Literal["anthropic", "ollama", "mix", "self_hosted"]
|
||||
default_model: str | None = None
|
||||
per_agent: dict[str, str] | None = None
|
||||
|
||||
@@ -110,5 +174,5 @@ class ApplyModeRequest(BaseModel):
|
||||
class ModeResponse(BaseModel):
|
||||
"""Server-side view of the current mode + a snapshot of active rules."""
|
||||
|
||||
mode: Literal["anthropic", "ollama", "mix"]
|
||||
mode: Literal["anthropic", "ollama", "mix", "self_hosted"]
|
||||
assignments: list[AssignmentResponse]
|
||||
|
||||
@@ -191,7 +191,10 @@ class ModelProvider(StrEnum):
|
||||
`ANTHROPIC` is the built-in default — routed via the mounted `~/.claude/`
|
||||
credentials inside each agent container. `OLLAMA_CLOUD` routes via
|
||||
`ANTHROPIC_BASE_URL` + `ANTHROPIC_AUTH_TOKEN` env injection at spawn.
|
||||
OPENAI / LOCAL are historical placeholders (unused today).
|
||||
`LOCAL` is the self-hosted Ollama provider: the operator configures its
|
||||
base URL via PUT /api/providers/self-hosted (seeded by migration 028).
|
||||
Agents assigned to LOCAL are routed to that server at spawn time.
|
||||
`OPENAI` is reserved for future use.
|
||||
"""
|
||||
|
||||
ANTHROPIC = "anthropic"
|
||||
|
||||
+263
-80
@@ -10,13 +10,26 @@ If none apply, falls back to the legacy `ROLE_MODEL_MAP` + implicit
|
||||
Anthropic provider so deployments with zero rows behave exactly as
|
||||
before. Decryption failures are contained: the service logs the error
|
||||
and downgrades to the legacy path rather than failing the spawn.
|
||||
|
||||
Self-hosted (LOCAL) provider support:
|
||||
- derive_mode() returns 'self_hosted' when there is exactly one
|
||||
GLOBAL assignment pointing to a LOCAL provider.
|
||||
- apply_mode('self_hosted', ...) enables the LOCAL provider and
|
||||
sets a GLOBAL assignment to the given model name.
|
||||
- upsert_assignment() accepts model names not in the MODEL_CATALOG
|
||||
when the target provider is LOCAL (self-hosted models are dynamic;
|
||||
they bypass catalog validation).
|
||||
- resolve_for_agent() checks reachability of the LOCAL base_url
|
||||
and falls back to Anthropic if the server is unreachable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING, Any, ClassVar, cast
|
||||
from typing import TYPE_CHECKING, Any, ClassVar, Literal, cast
|
||||
|
||||
import httpx
|
||||
import structlog
|
||||
from sqlalchemy import delete as sa_delete
|
||||
from sqlalchemy import select
|
||||
|
||||
@@ -39,6 +52,46 @@ if TYPE_CHECKING:
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module-level HTTP helper — decoupled from the service so tests can patch it.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_OLLAMA_TAGS_TIMEOUT = 5.0 # seconds
|
||||
_log = structlog.get_logger(__name__)
|
||||
|
||||
|
||||
async def probe_ollama_tags(base_url: str) -> tuple[list[str], str | None]:
|
||||
"""Fetch the model list from a running Ollama server.
|
||||
|
||||
Hits ``{base_url}/api/tags`` and returns ``(model_names, None)`` on
|
||||
success or ``([], error_message)`` on any failure. Never raises.
|
||||
|
||||
Returns:
|
||||
A tuple of (list_of_model_name_strings, error_string_or_None).
|
||||
"""
|
||||
url = base_url.rstrip("/") + "/api/tags"
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=_OLLAMA_TAGS_TIMEOUT) as client:
|
||||
resp = await client.get(url)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
models: list[str] = [m["name"] for m in data.get("models", [])]
|
||||
return models, None
|
||||
except httpx.TimeoutException:
|
||||
return [], f"Connection to {base_url} timed out after {_OLLAMA_TAGS_TIMEOUT}s"
|
||||
except httpx.ConnectError:
|
||||
return [], f"Could not connect to {base_url} — server may be offline"
|
||||
except httpx.HTTPStatusError as exc:
|
||||
return [], f"Server at {base_url} returned HTTP {exc.response.status_code}"
|
||||
except Exception as exc:
|
||||
_log.error(
|
||||
"Unexpected error probing Ollama server",
|
||||
base_url=base_url,
|
||||
error=str(exc),
|
||||
)
|
||||
return [], "An unexpected error occurred while probing the self-hosted server."
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentRoute:
|
||||
"""Resolved routing for a single agent spawn.
|
||||
@@ -71,38 +124,88 @@ class ModelRoutingService(BaseService):
|
||||
async def resolve_for_agent(self, agent_slug: str) -> AgentRoute:
|
||||
"""Resolve routing for `agent_slug` using the precedence ladder.
|
||||
|
||||
Never raises for a normal agent — decrypt failures and missing
|
||||
agents both downgrade to the legacy Anthropic path, because a
|
||||
stalled spawn is worse than a routing miss.
|
||||
Never raises for a normal agent — decrypt failures, unreachable
|
||||
self-hosted servers, and missing agents all downgrade to the
|
||||
legacy Anthropic path, because a stalled spawn is worse than a
|
||||
routing miss.
|
||||
"""
|
||||
role = get_agent_role(agent_slug) or ""
|
||||
resolved = await self._resolve_assignment(agent_slug, role)
|
||||
if resolved is not None and resolved.provider.enabled:
|
||||
route = await self._route_from_resolved(resolved, agent_slug)
|
||||
if route is not None:
|
||||
return route
|
||||
return self._legacy_route(role)
|
||||
|
||||
# 1) agent override
|
||||
async def _resolve_assignment(
|
||||
self, agent_slug: str, role: str
|
||||
) -> _ResolvedAssignment | None:
|
||||
"""Walk the precedence ladder: agent override > role override > global."""
|
||||
resolved = await self._find_assignment(
|
||||
scope=AssignmentScope.AGENT_SLUG, scope_value=agent_slug
|
||||
)
|
||||
# 2) role override
|
||||
if resolved is None and role:
|
||||
resolved = await self._find_assignment(
|
||||
scope=AssignmentScope.ROLE, scope_value=role
|
||||
)
|
||||
# 3) global default
|
||||
if resolved is None:
|
||||
resolved = await self._find_assignment(
|
||||
scope=AssignmentScope.GLOBAL, scope_value=None
|
||||
)
|
||||
return resolved
|
||||
|
||||
if resolved is not None and resolved.provider.enabled:
|
||||
try:
|
||||
return await self._route_from_assignment(resolved)
|
||||
except EncryptionError:
|
||||
self.log.error(
|
||||
"Provider token decrypt failed; falling back to legacy path",
|
||||
provider_id=str(resolved.provider.id),
|
||||
agent_slug=agent_slug,
|
||||
)
|
||||
async def _route_from_resolved(
|
||||
self, resolved: _ResolvedAssignment, agent_slug: str
|
||||
) -> AgentRoute | None:
|
||||
"""Build a route from a resolved+enabled assignment.
|
||||
|
||||
# 4) legacy fallback: role-default short name through MODEL_MAP.
|
||||
Returns ``None`` to signal the caller should fall through to the
|
||||
legacy Anthropic path (unreachable self-hosted server, empty
|
||||
base_url, or a token-decrypt failure).
|
||||
"""
|
||||
if resolved.provider.type == ModelProvider.LOCAL:
|
||||
return await self._local_route_or_none(resolved, agent_slug)
|
||||
return await self._decrypt_route_or_none(resolved, agent_slug)
|
||||
|
||||
async def _local_route_or_none(
|
||||
self, resolved: _ResolvedAssignment, agent_slug: str
|
||||
) -> AgentRoute | None:
|
||||
"""Route to a LOCAL provider only if it is configured and reachable.
|
||||
|
||||
Probes ``{base_url}/api/tags`` first; if the server is down (or no
|
||||
base_url is configured) returns ``None`` so the spawn falls back to
|
||||
Anthropic — better a wrong provider than no spawn.
|
||||
"""
|
||||
base_url = resolved.provider.base_url or ""
|
||||
if not base_url:
|
||||
return None # unconfigured → fall through
|
||||
_, error = await probe_ollama_tags(base_url)
|
||||
if error is not None:
|
||||
self.log.warning(
|
||||
"Self-hosted server unreachable; falling back to Anthropic",
|
||||
base_url=base_url,
|
||||
error=error,
|
||||
agent_slug=agent_slug,
|
||||
)
|
||||
return None
|
||||
return await self._decrypt_route_or_none(resolved, agent_slug)
|
||||
|
||||
async def _decrypt_route_or_none(
|
||||
self, resolved: _ResolvedAssignment, agent_slug: str
|
||||
) -> AgentRoute | None:
|
||||
"""Build the route, downgrading to ``None`` on a token-decrypt failure."""
|
||||
try:
|
||||
return await self._route_from_assignment(resolved)
|
||||
except EncryptionError:
|
||||
self.log.error(
|
||||
"Provider token decrypt failed; falling back to legacy path",
|
||||
provider_id=str(resolved.provider.id),
|
||||
agent_slug=agent_slug,
|
||||
)
|
||||
return None
|
||||
|
||||
def _legacy_route(self, role: str) -> AgentRoute:
|
||||
"""Legacy fallback: role-default short name through MODEL_MAP."""
|
||||
short = ROLE_MODEL_MAP.get(role, "sonnet")
|
||||
return AgentRoute(
|
||||
provider_id=None,
|
||||
@@ -141,21 +244,49 @@ class ModelRoutingService(BaseService):
|
||||
scope: AssignmentScope,
|
||||
scope_value: str | None,
|
||||
model_name: str,
|
||||
provider_type_override: ModelProvider | None = None,
|
||||
) -> ModelAssignmentTable:
|
||||
"""Insert-or-update (by unique (scope, scope_value)).
|
||||
|
||||
Provider is derived from `MODEL_CATALOG` — the UI never picks a
|
||||
provider separately, so the service looks up the pre-seeded
|
||||
Provider is normally derived from `MODEL_CATALOG` — the UI never
|
||||
picks a provider separately, so the service looks up the pre-seeded
|
||||
provider row for the catalog entry's type.
|
||||
|
||||
When `provider_type_override` is supplied (used internally by
|
||||
`apply_mode('self_hosted', ...)` and mix mode for LOCAL models),
|
||||
the catalog look-up is skipped and the named provider type is used
|
||||
directly. This allows self-hosted model names (which are not in the
|
||||
static catalog) to be assigned to the LOCAL provider.
|
||||
"""
|
||||
self._validate_scope(scope, scope_value)
|
||||
entry = MODEL_CATALOG_BY_NAME.get(model_name)
|
||||
if entry is None:
|
||||
raise ValueError(
|
||||
f"Unknown model '{model_name}'. Use one from "
|
||||
"GET /api/providers/catalog."
|
||||
|
||||
if provider_type_override is not None:
|
||||
provider = await self._get_seeded_provider(provider_type_override)
|
||||
provider_type_for_log = provider_type_override
|
||||
else:
|
||||
entry = MODEL_CATALOG_BY_NAME.get(model_name)
|
||||
if entry is None:
|
||||
# Try to route to LOCAL if a LOCAL provider is seeded — this
|
||||
# allows self-hosted model names in mix mode without an error.
|
||||
local_provider = await self._find_local_provider()
|
||||
if local_provider is None:
|
||||
raise ValueError(
|
||||
f"Unknown model '{model_name}'. Use one from "
|
||||
"GET /api/providers/catalog."
|
||||
)
|
||||
provider = local_provider
|
||||
provider_type_for_log = ModelProvider.LOCAL
|
||||
else:
|
||||
provider = await self._get_seeded_provider(entry.provider_type)
|
||||
provider_type_for_log = entry.provider_type
|
||||
|
||||
# Whenever an assignment resolves to LOCAL, ensure the LOCAL provider
|
||||
# row is enabled so resolve_for_agent() will actually use it.
|
||||
if provider_type_for_log == ModelProvider.LOCAL:
|
||||
provider_svc = ProviderService(self.session)
|
||||
await provider_svc.update_provider(
|
||||
require_uuid(provider.id), ProviderUpdate(enabled=True)
|
||||
)
|
||||
provider = await self._get_seeded_provider(entry.provider_type)
|
||||
|
||||
row = await self.get_assignment(scope=scope, scope_value=scope_value)
|
||||
if row is None:
|
||||
@@ -175,18 +306,19 @@ class ModelRoutingService(BaseService):
|
||||
"Assignment upserted",
|
||||
scope=scope.value,
|
||||
scope_value=scope_value,
|
||||
provider_type=entry.provider_type.value,
|
||||
provider_type=provider_type_for_log.value,
|
||||
model_name=model_name,
|
||||
)
|
||||
return row
|
||||
|
||||
async def derive_mode(self) -> str:
|
||||
async def derive_mode(self) -> Literal["anthropic", "ollama", "mix", "self_hosted"]:
|
||||
"""Return the current "mode" label for the Settings UI.
|
||||
|
||||
Decision tree matches what `apply_mode` writes:
|
||||
- no assignments at all → "anthropic"
|
||||
- only a global row, Ollama → "ollama"
|
||||
- anything else → "mix"
|
||||
- no assignments at all → "anthropic"
|
||||
- only a global row, Ollama Cloud → "ollama"
|
||||
- only a global row, LOCAL → "self_hosted"
|
||||
- anything else → "mix"
|
||||
"""
|
||||
assignments = await self.list_assignments()
|
||||
if not assignments:
|
||||
@@ -194,11 +326,11 @@ class ModelRoutingService(BaseService):
|
||||
only_global = (
|
||||
len(assignments) == 1 and assignments[0].scope == AssignmentScope.GLOBAL
|
||||
)
|
||||
is_ollama = (
|
||||
only_global and assignments[0].provider.type == ModelProvider.OLLAMA_CLOUD
|
||||
)
|
||||
if is_ollama:
|
||||
return "ollama"
|
||||
if only_global:
|
||||
if assignments[0].provider.type == ModelProvider.OLLAMA_CLOUD:
|
||||
return "ollama"
|
||||
if assignments[0].provider.type == ModelProvider.LOCAL:
|
||||
return "self_hosted"
|
||||
return "mix"
|
||||
|
||||
async def set_ollama_api_key(self, api_key: str) -> ProviderConfigTable:
|
||||
@@ -268,58 +400,100 @@ class ModelRoutingService(BaseService):
|
||||
"""Apply a routing "mode" in a single transactional call.
|
||||
|
||||
Modes:
|
||||
- "anthropic": wipe all assignments so every spawn falls through
|
||||
- "anthropic": wipe all assignments so every spawn falls through
|
||||
to the legacy ROLE_MODEL_MAP + mounted ~/.claude path.
|
||||
- "ollama": wipe role/agent overrides, set GLOBAL to the given
|
||||
Ollama model (default: Kimi K2.6). CEO-type pins can be layered
|
||||
back manually if the user wants them.
|
||||
- "mix": apply per-agent map verbatim. Any agent not in the
|
||||
- "ollama": wipe role/agent overrides, set GLOBAL to the given
|
||||
Ollama model (default: OLLAMA_DEFAULT_MODEL). CEO-type pins can be
|
||||
layered back manually if the user wants them.
|
||||
- "self_hosted": wipe all assignments, enable the LOCAL provider,
|
||||
and set the GLOBAL default to `default_model` (a self-hosted
|
||||
model name — not validated against the static catalog).
|
||||
- "mix": apply per-agent map verbatim. Any agent not in the
|
||||
map falls through to the GLOBAL default — which is whatever it
|
||||
was (preserves prior state).
|
||||
was (preserves prior state). Self-hosted model names (not in the
|
||||
catalog) are automatically routed to the LOCAL provider.
|
||||
"""
|
||||
if mode == "anthropic":
|
||||
await self.session.execute(sa_delete(ModelAssignmentTable))
|
||||
await self.session.flush()
|
||||
self.log.info("Mode applied: anthropic (all assignments cleared)")
|
||||
return
|
||||
await self._apply_anthropic()
|
||||
elif mode == "ollama":
|
||||
await self._apply_ollama(default_model)
|
||||
elif mode == "self_hosted":
|
||||
await self._apply_self_hosted(default_model)
|
||||
elif mode == "mix":
|
||||
await self._apply_mix(per_agent)
|
||||
else:
|
||||
raise ValueError(
|
||||
f"Unknown mode '{mode}'."
|
||||
" Use 'anthropic', 'ollama', 'self_hosted', or 'mix'."
|
||||
)
|
||||
|
||||
if mode == "ollama":
|
||||
await self.session.execute(sa_delete(ModelAssignmentTable))
|
||||
await self.session.flush()
|
||||
async def _apply_anthropic(self) -> None:
|
||||
"""Wipe all assignments so every spawn uses the legacy Anthropic path."""
|
||||
await self.session.execute(sa_delete(ModelAssignmentTable))
|
||||
await self.session.flush()
|
||||
self.log.info("Mode applied: anthropic (all assignments cleared)")
|
||||
|
||||
async def _apply_ollama(self, default_model: str | None) -> None:
|
||||
"""Wipe assignments, set the GLOBAL default to an Ollama Cloud model."""
|
||||
await self.session.execute(sa_delete(ModelAssignmentTable))
|
||||
await self.session.flush()
|
||||
model_name = default_model or OLLAMA_DEFAULT_MODEL
|
||||
await self.upsert_assignment(
|
||||
scope=AssignmentScope.GLOBAL,
|
||||
scope_value=None,
|
||||
model_name=model_name,
|
||||
)
|
||||
self.log.info("Mode applied: ollama", default_model=model_name)
|
||||
|
||||
async def _apply_self_hosted(self, default_model: str | None) -> None:
|
||||
"""Wipe assignments, enable the LOCAL provider, point GLOBAL at it."""
|
||||
if not default_model:
|
||||
raise ValueError(
|
||||
"self_hosted mode requires a default_model (self-hosted model name)"
|
||||
)
|
||||
await self.session.execute(sa_delete(ModelAssignmentTable))
|
||||
await self.session.flush()
|
||||
# Enable the LOCAL provider row so resolve_for_agent() will use it.
|
||||
local = await self._find_local_provider()
|
||||
if local is None:
|
||||
raise NotFoundError(
|
||||
resource_type="Provider",
|
||||
resource_id=f"type={ModelProvider.LOCAL.value}",
|
||||
)
|
||||
provider_svc = ProviderService(self.session)
|
||||
await provider_svc.update_provider(
|
||||
require_uuid(local.id),
|
||||
ProviderUpdate(enabled=True),
|
||||
)
|
||||
await self.upsert_assignment(
|
||||
scope=AssignmentScope.GLOBAL,
|
||||
scope_value=None,
|
||||
model_name=default_model,
|
||||
provider_type_override=ModelProvider.LOCAL,
|
||||
)
|
||||
self.log.info("Mode applied: self_hosted", default_model=default_model)
|
||||
|
||||
async def _apply_mix(self, per_agent: dict[str, str] | None) -> None:
|
||||
"""Apply a per-agent override map; leave role + global rows untouched."""
|
||||
if not per_agent:
|
||||
raise ValueError("mix mode requires a per_agent map")
|
||||
# Clear existing agent-slug overrides so the new map is authoritative.
|
||||
await self.session.execute(
|
||||
sa_delete(ModelAssignmentTable).where(
|
||||
ModelAssignmentTable.scope == AssignmentScope.AGENT_SLUG
|
||||
)
|
||||
)
|
||||
await self.session.flush()
|
||||
for agent_slug, model_name in per_agent.items():
|
||||
if not model_name:
|
||||
continue
|
||||
# upsert_assignment will route to LOCAL for non-catalog names.
|
||||
await self.upsert_assignment(
|
||||
scope=AssignmentScope.GLOBAL,
|
||||
scope_value=None,
|
||||
model_name=default_model or OLLAMA_DEFAULT_MODEL,
|
||||
scope=AssignmentScope.AGENT_SLUG,
|
||||
scope_value=agent_slug,
|
||||
model_name=model_name,
|
||||
)
|
||||
self.log.info(
|
||||
"Mode applied: ollama",
|
||||
default_model=default_model or OLLAMA_DEFAULT_MODEL,
|
||||
)
|
||||
return
|
||||
|
||||
if mode == "mix":
|
||||
if not per_agent:
|
||||
raise ValueError("mix mode requires a per_agent map")
|
||||
# Clear existing agent-slug overrides so the new map is
|
||||
# authoritative; leave role + global alone.
|
||||
await self.session.execute(
|
||||
sa_delete(ModelAssignmentTable).where(
|
||||
ModelAssignmentTable.scope == AssignmentScope.AGENT_SLUG
|
||||
)
|
||||
)
|
||||
await self.session.flush()
|
||||
for agent_slug, model_name in per_agent.items():
|
||||
if not model_name:
|
||||
continue
|
||||
await self.upsert_assignment(
|
||||
scope=AssignmentScope.AGENT_SLUG,
|
||||
scope_value=agent_slug,
|
||||
model_name=model_name,
|
||||
)
|
||||
self.log.info("Mode applied: mix", agents=len(per_agent))
|
||||
return
|
||||
|
||||
raise ValueError(f"Unknown mode '{mode}'. Use 'anthropic', 'ollama', or 'mix'.")
|
||||
self.log.info("Mode applied: mix", agents=len(per_agent))
|
||||
|
||||
# =========================================================================
|
||||
# INTERNAL
|
||||
@@ -334,6 +508,15 @@ class ModelRoutingService(BaseService):
|
||||
# Relationship is lazy="joined" in the ORM so `.provider` is loaded.
|
||||
return _ResolvedAssignment(provider=row.provider, model_name=row.model_name)
|
||||
|
||||
async def _find_local_provider(self) -> ProviderConfigTable | None:
|
||||
"""Return the LOCAL provider row, or None if not seeded."""
|
||||
result = await self.session.execute(
|
||||
select(ProviderConfigTable).where(
|
||||
ProviderConfigTable.type == ModelProvider.LOCAL
|
||||
)
|
||||
)
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
async def _route_from_assignment(self, resolved: _ResolvedAssignment) -> AgentRoute:
|
||||
provider = resolved.provider
|
||||
# Decrypt only when the provider has a stored token (ollama_cloud).
|
||||
|
||||
@@ -347,3 +347,246 @@ async def test_resolve_for_agent_falls_back_on_decrypt_error(
|
||||
route = await svc.resolve_for_agent("be-dev-1")
|
||||
# Falls back to legacy ANTHROPIC route.
|
||||
assert route.provider_type == ModelProvider.ANTHROPIC
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Self-hosted (LOCAL) provider
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def llm_setup_with_local(
|
||||
db_session: AsyncSession,
|
||||
) -> AsyncIterator[dict]:
|
||||
"""Seed Anthropic, Ollama Cloud, and LOCAL provider rows."""
|
||||
anthropic = ProviderConfigTable(
|
||||
name="anthropic-test-local",
|
||||
type=ModelProvider.ANTHROPIC,
|
||||
enabled=True,
|
||||
)
|
||||
ollama = ProviderConfigTable(
|
||||
name="ollama-test-local",
|
||||
type=ModelProvider.OLLAMA_CLOUD,
|
||||
enabled=True,
|
||||
base_url="https://ollama.example.com",
|
||||
)
|
||||
local = ProviderConfigTable(
|
||||
name="self-hosted-test",
|
||||
type=ModelProvider.LOCAL,
|
||||
enabled=True,
|
||||
base_url="http://localhost:11434",
|
||||
)
|
||||
db_session.add_all([anthropic, ollama, local])
|
||||
await db_session.flush()
|
||||
yield {"svc": ModelRoutingService(db_session), "local": local}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_derive_mode_self_hosted_when_only_local_global(
|
||||
llm_setup_with_local: dict,
|
||||
) -> None:
|
||||
"""Single GLOBAL assignment pointing to LOCAL → 'self_hosted' mode."""
|
||||
svc = llm_setup_with_local["svc"]
|
||||
await svc.upsert_assignment(
|
||||
scope=AssignmentScope.GLOBAL,
|
||||
scope_value=None,
|
||||
model_name="llama3.1:8b",
|
||||
provider_type_override=ModelProvider.LOCAL,
|
||||
)
|
||||
assert await svc.derive_mode() == "self_hosted"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mode_self_hosted_sets_global_local(
|
||||
llm_setup_with_local: dict,
|
||||
) -> None:
|
||||
"""apply_mode('self_hosted') clears assignments, enables LOCAL, inserts GLOBAL."""
|
||||
svc = llm_setup_with_local["svc"]
|
||||
await svc.apply_mode(mode="self_hosted", default_model="llama3.1:8b")
|
||||
assignments = await svc.list_assignments()
|
||||
assert len(assignments) == 1
|
||||
assert assignments[0].scope == AssignmentScope.GLOBAL
|
||||
assert assignments[0].provider.type == ModelProvider.LOCAL
|
||||
assert assignments[0].model_name == "llama3.1:8b"
|
||||
# Verify derive_mode reflects the new state.
|
||||
assert await svc.derive_mode() == "self_hosted"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mode_self_hosted_requires_default_model(
|
||||
llm_setup_with_local: dict,
|
||||
) -> None:
|
||||
"""apply_mode('self_hosted') without default_model raises ValueError."""
|
||||
svc = llm_setup_with_local["svc"]
|
||||
with pytest.raises(ValueError, match="requires a default_model"):
|
||||
await svc.apply_mode(mode="self_hosted")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mode_self_hosted_clears_prior_assignments(
|
||||
llm_setup_with_local: dict,
|
||||
) -> None:
|
||||
"""apply_mode('self_hosted') clears ALL prior assignments."""
|
||||
svc = llm_setup_with_local["svc"]
|
||||
anthropic_model = _first_model_for_type(ModelProvider.ANTHROPIC)
|
||||
await svc.upsert_assignment(
|
||||
scope=AssignmentScope.AGENT_SLUG,
|
||||
scope_value="be-dev-1",
|
||||
model_name=anthropic_model,
|
||||
)
|
||||
await svc.upsert_assignment(
|
||||
scope=AssignmentScope.GLOBAL,
|
||||
scope_value=None,
|
||||
model_name=anthropic_model,
|
||||
)
|
||||
assert len(await svc.list_assignments()) == 2 # noqa: PLR2004
|
||||
await svc.apply_mode(mode="self_hosted", default_model="gemma2:9b")
|
||||
assignments = await svc.list_assignments()
|
||||
assert len(assignments) == 1 # Only the new GLOBAL row.
|
||||
assert assignments[0].provider.type == ModelProvider.LOCAL
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_upsert_assignment_routes_unknown_model_to_local(
|
||||
llm_setup_with_local: dict,
|
||||
) -> None:
|
||||
"""Non-catalog model names are silently routed to LOCAL if seeded."""
|
||||
svc = llm_setup_with_local["svc"]
|
||||
row = await svc.upsert_assignment(
|
||||
scope=AssignmentScope.AGENT_SLUG,
|
||||
scope_value="be-dev-1",
|
||||
model_name="my-custom-model:latest",
|
||||
)
|
||||
assert row.provider.type == ModelProvider.LOCAL
|
||||
assert row.model_name == "my-custom-model:latest"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mix_mode_with_self_hosted_models(
|
||||
llm_setup_with_local: dict,
|
||||
) -> None:
|
||||
"""mix mode accepts self-hosted model names without raising ValueError."""
|
||||
svc = llm_setup_with_local["svc"]
|
||||
anthropic_model = _first_model_for_type(ModelProvider.ANTHROPIC)
|
||||
await svc.apply_mode(
|
||||
mode="mix",
|
||||
per_agent={
|
||||
"be-dev-1": anthropic_model, # catalog model
|
||||
"be-dev-2": "self-hosted-model:7b", # non-catalog → routed to LOCAL
|
||||
},
|
||||
)
|
||||
rows = await svc.list_assignments()
|
||||
by_slug = {r.scope_value: r for r in rows if r.scope == AssignmentScope.AGENT_SLUG}
|
||||
assert by_slug["be-dev-1"].provider.type == ModelProvider.ANTHROPIC
|
||||
assert by_slug["be-dev-2"].provider.type == ModelProvider.LOCAL
|
||||
assert by_slug["be-dev-2"].model_name == "self-hosted-model:7b"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_for_agent_self_hosted_returns_base_url(
|
||||
llm_setup_with_local: dict,
|
||||
) -> None:
|
||||
"""When LOCAL assignment is reachable, route has base_url from provider."""
|
||||
svc = llm_setup_with_local["svc"]
|
||||
await svc.upsert_assignment(
|
||||
scope=AssignmentScope.GLOBAL,
|
||||
scope_value=None,
|
||||
model_name="llama3.1:8b",
|
||||
provider_type_override=ModelProvider.LOCAL,
|
||||
)
|
||||
with patch(
|
||||
"roboco.services.llm.probe_ollama_tags",
|
||||
new_callable=AsyncMock,
|
||||
return_value=(["llama3.1:8b"], None),
|
||||
):
|
||||
route = await svc.resolve_for_agent("be-dev-1")
|
||||
assert route.provider_type == ModelProvider.LOCAL
|
||||
assert route.base_url == "http://localhost:11434"
|
||||
assert route.base_url is not None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_for_agent_falls_back_when_self_hosted_unreachable(
|
||||
llm_setup_with_local: dict,
|
||||
) -> None:
|
||||
"""When LOCAL provider is unreachable, falls back to Anthropic default."""
|
||||
svc = llm_setup_with_local["svc"]
|
||||
await svc.upsert_assignment(
|
||||
scope=AssignmentScope.GLOBAL,
|
||||
scope_value=None,
|
||||
model_name="llama3.1:8b",
|
||||
provider_type_override=ModelProvider.LOCAL,
|
||||
)
|
||||
with patch(
|
||||
"roboco.services.llm.probe_ollama_tags",
|
||||
new_callable=AsyncMock,
|
||||
return_value=([], "Could not connect to http://localhost:11434"),
|
||||
):
|
||||
route = await svc.resolve_for_agent("be-dev-1")
|
||||
assert route.provider_type == ModelProvider.ANTHROPIC
|
||||
assert route.base_url is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_upsert_assignment_unknown_model_without_local_raises(
|
||||
llm_setup: dict,
|
||||
) -> None:
|
||||
"""Without LOCAL provider seeded, non-catalog models raise ValueError."""
|
||||
svc = llm_setup["svc"]
|
||||
with pytest.raises(ValueError, match="Unknown model"):
|
||||
await svc.upsert_assignment(
|
||||
scope=AssignmentScope.AGENT_SLUG,
|
||||
scope_value="be-dev-1",
|
||||
model_name="ghost-model:latest",
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_upsert_assignment_enables_local_when_disabled(
|
||||
db_session: AsyncSession,
|
||||
) -> None:
|
||||
"""upsert_assignment transitions LOCAL provider from enabled=False to enabled=True.
|
||||
|
||||
AC4: proves that mix-mode assignment of a non-catalog model name resolves
|
||||
to provider_type LOCAL and LOCAL.enabled is True afterward — even when the
|
||||
LOCAL provider starts with enabled=False (the seeded state from migration 028
|
||||
before the operator configures a base_url via PUT /providers/self-hosted).
|
||||
"""
|
||||
# Arrange: Anthropic provider (required by ModelRoutingService internals)
|
||||
# and LOCAL provider starting with enabled=False.
|
||||
anthropic = ProviderConfigTable(
|
||||
name="anthropic-ac4-test",
|
||||
type=ModelProvider.ANTHROPIC,
|
||||
enabled=True,
|
||||
)
|
||||
local = ProviderConfigTable(
|
||||
name="self-hosted-ac4-test",
|
||||
type=ModelProvider.LOCAL,
|
||||
enabled=False, # Starts DISABLED — this is the state to be transitioned.
|
||||
base_url="http://localhost:11434",
|
||||
)
|
||||
db_session.add_all([anthropic, local])
|
||||
await db_session.flush()
|
||||
|
||||
# Pre-condition: LOCAL is disabled before the call.
|
||||
assert local.enabled is False
|
||||
|
||||
# Act: upsert a non-catalog model name → resolves to LOCAL →
|
||||
# calls ProviderService.update_provider(enabled=True) on the LOCAL row.
|
||||
svc = ModelRoutingService(db_session)
|
||||
row = await svc.upsert_assignment(
|
||||
scope=AssignmentScope.AGENT_SLUG,
|
||||
scope_value="test-agent-ac4",
|
||||
model_name="non-catalog-model:7b",
|
||||
)
|
||||
|
||||
# Refresh local from DB so the in-memory object reflects the DB write.
|
||||
await db_session.refresh(local)
|
||||
|
||||
# Assert: assignment resolved to LOCAL and LOCAL provider is now enabled.
|
||||
assert row.provider.type == ModelProvider.LOCAL
|
||||
assert row.model_name == "non-catalog-model:7b"
|
||||
assert local.enabled is True, (
|
||||
"upsert_assignment must call update_provider(enabled=True) on LOCAL "
|
||||
"whenever it routes a non-catalog model to the LOCAL provider"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
"""Migration 028 tests — seed_self_hosted_provider.
|
||||
|
||||
Verifies the post-upgrade state and exercises the downgrade SQL ordering
|
||||
to prove the FK-safe delete sequence works.
|
||||
|
||||
NOT a real alembic round-trip — the suite builds the test DB via
|
||||
Base.metadata.create_all (see conftest). Migration 028's upgrade()/downgrade()
|
||||
bodies are reviewed here; the tests guard the resulting DB-level contract.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from roboco.db.tables import ModelAssignmentTable, ProviderConfigTable
|
||||
from roboco.models.base import AssignmentScope, ModelProvider
|
||||
from sqlalchemy import text
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_migration_028_upgrade_insert_contract(
|
||||
db_session: AsyncSession,
|
||||
) -> None:
|
||||
"""The upgrade INSERT SQL seeds the correct LOCAL provider row and is idempotent.
|
||||
|
||||
Executes the INSERT ... ON CONFLICT DO NOTHING SQL from migration 028's
|
||||
upgrade() directly in the test session, verifying:
|
||||
- name='Self-Hosted (Ollama)', type='local', enabled=False on the row.
|
||||
- Running the same INSERT a second time leaves exactly one row (idempotency).
|
||||
"""
|
||||
_insert_sql = text(
|
||||
"""
|
||||
INSERT INTO provider_configs
|
||||
(id, name, type, base_url, auth_token_encrypted, enabled, created_at)
|
||||
VALUES
|
||||
(
|
||||
gen_random_uuid(),
|
||||
'Self-Hosted (Ollama)',
|
||||
'local',
|
||||
NULL,
|
||||
NULL,
|
||||
false,
|
||||
now()
|
||||
)
|
||||
ON CONFLICT (name) DO NOTHING
|
||||
"""
|
||||
)
|
||||
|
||||
# --- First run: the row should be inserted.
|
||||
await db_session.execute(_insert_sql)
|
||||
await db_session.flush()
|
||||
|
||||
# Verify the field contract on the newly-inserted row.
|
||||
result = await db_session.execute(
|
||||
text(
|
||||
"SELECT name, type, enabled "
|
||||
"FROM provider_configs "
|
||||
"WHERE name = 'Self-Hosted (Ollama)'"
|
||||
)
|
||||
)
|
||||
rows = list(result)
|
||||
assert len(rows) == 1
|
||||
name, ptype, enabled = rows[0]
|
||||
assert name == "Self-Hosted (Ollama)"
|
||||
assert ptype == "local"
|
||||
assert enabled is False # starts disabled; user configures via PUT /self-hosted
|
||||
|
||||
# --- Second run: ON CONFLICT DO NOTHING must not create a duplicate.
|
||||
await db_session.execute(_insert_sql)
|
||||
await db_session.flush()
|
||||
|
||||
result = await db_session.execute(
|
||||
text("SELECT id FROM provider_configs WHERE name = 'Self-Hosted (Ollama)'")
|
||||
)
|
||||
assert len(list(result)) == 1, (
|
||||
"Expected exactly one 'Self-Hosted (Ollama)' row after two INSERT "
|
||||
"executions; ON CONFLICT DO NOTHING must prevent duplicates."
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_migration_028_downgrade_deletes_assignments_before_config(
|
||||
db_session: AsyncSession,
|
||||
) -> None:
|
||||
"""Downgrade SQL deletes model_assignments before provider_configs.
|
||||
|
||||
Simulates the downgrade() logic from migration 028:
|
||||
1. DELETE FROM model_assignments WHERE provider_config_id IN (SELECT id ...)
|
||||
2. DELETE FROM provider_configs WHERE name = 'Self-Hosted (Ollama)'
|
||||
|
||||
A FK RESTRICT constraint on model_assignments.provider_config_id means that
|
||||
executing step 2 before step 1 would raise an IntegrityError. This test
|
||||
proves that doing them in the correct order succeeds without violation.
|
||||
"""
|
||||
# --- Arrange: insert a fresh LOCAL provider row and a referencing assignment.
|
||||
suffix = uuid4().hex[:8]
|
||||
local = ProviderConfigTable(
|
||||
name=f"Self-Hosted (Ollama)-test-{suffix}",
|
||||
type=ModelProvider.LOCAL,
|
||||
enabled=False,
|
||||
)
|
||||
db_session.add(local)
|
||||
await db_session.flush()
|
||||
|
||||
assignment = ModelAssignmentTable(
|
||||
scope=AssignmentScope.AGENT_SLUG,
|
||||
scope_value=f"test-agent-{suffix}",
|
||||
provider_config_id=local.id,
|
||||
model_name="llama3.1:8b",
|
||||
)
|
||||
db_session.add(assignment)
|
||||
await db_session.flush()
|
||||
|
||||
# Verify both rows exist before we run the downgrade SQL.
|
||||
result = await db_session.execute(
|
||||
text("SELECT id FROM provider_configs WHERE name = :name").bindparams(
|
||||
name=local.name
|
||||
)
|
||||
)
|
||||
assert result.scalar_one_or_none() is not None
|
||||
|
||||
result = await db_session.execute(
|
||||
text("SELECT id FROM model_assignments WHERE scope_value = :sv").bindparams(
|
||||
sv=assignment.scope_value
|
||||
)
|
||||
)
|
||||
assert result.scalar_one_or_none() is not None
|
||||
|
||||
# --- Act: execute downgrade SQL in the correct FK-safe order.
|
||||
# Step 1: delete referencing model_assignments first.
|
||||
await db_session.execute(
|
||||
text(
|
||||
"DELETE FROM model_assignments "
|
||||
"WHERE provider_config_id IN ("
|
||||
" SELECT id FROM provider_configs WHERE name = :name"
|
||||
")"
|
||||
).bindparams(name=local.name)
|
||||
)
|
||||
# Step 2: now safe to delete the provider row.
|
||||
await db_session.execute(
|
||||
text("DELETE FROM provider_configs WHERE name = :name").bindparams(
|
||||
name=local.name
|
||||
)
|
||||
)
|
||||
|
||||
# --- Assert: both rows are gone, no IntegrityError was raised.
|
||||
result = await db_session.execute(
|
||||
text("SELECT id FROM provider_configs WHERE name = :name").bindparams(
|
||||
name=local.name
|
||||
)
|
||||
)
|
||||
assert result.scalar_one_or_none() is None, (
|
||||
"provider_configs row should be deleted by downgrade"
|
||||
)
|
||||
|
||||
result = await db_session.execute(
|
||||
text("SELECT id FROM model_assignments WHERE scope_value = :sv").bindparams(
|
||||
sv=assignment.scope_value
|
||||
)
|
||||
)
|
||||
assert result.scalar_one_or_none() is None, (
|
||||
"model_assignments row should be deleted before provider_configs"
|
||||
)
|
||||
@@ -4,6 +4,7 @@ from __future__ import annotations
|
||||
|
||||
from http import HTTPStatus
|
||||
from typing import TYPE_CHECKING
|
||||
from unittest.mock import AsyncMock, patch
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
@@ -12,7 +13,7 @@ from fastapi import FastAPI
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from roboco.api.deps import get_agent_context, get_db
|
||||
from roboco.api.routes.provider import router as provider_router
|
||||
from roboco.db.tables import ProviderConfigTable
|
||||
from roboco.db.tables import ModelAssignmentTable, ProviderConfigTable
|
||||
from roboco.models import AgentRole, Team
|
||||
from roboco.models.base import ModelProvider
|
||||
from roboco.models.permissions import AgentContext
|
||||
@@ -77,12 +78,54 @@ async def app_client(
|
||||
app.dependency_overrides.clear()
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def app_client_with_ollama(
|
||||
db_session: AsyncSession,
|
||||
) -> AsyncIterator[AsyncClient]:
|
||||
"""App client pre-seeded with Anthropic and Ollama Cloud providers.
|
||||
|
||||
Begins with a DELETE-before-seed isolation step: deletes all rows from
|
||||
ModelAssignmentTable (FK-safe) then ProviderConfigTable before adding
|
||||
fresh ANTHROPIC + OLLAMA_CLOUD rows. This ensures tests are
|
||||
order-independent regardless of what prior tests committed.
|
||||
"""
|
||||
app = _make_app(db_session)
|
||||
suffix = uuid4().hex[:8]
|
||||
# FK-safe cleanup: model_assignments.provider_config_id references
|
||||
# provider_configs.id, so assignments must be deleted first.
|
||||
await db_session.execute(delete(ModelAssignmentTable))
|
||||
await db_session.execute(delete(ProviderConfigTable))
|
||||
await db_session.flush()
|
||||
db_session.add(
|
||||
ProviderConfigTable(
|
||||
name=f"anthropic-test-{suffix}",
|
||||
type=ModelProvider.ANTHROPIC,
|
||||
enabled=True,
|
||||
)
|
||||
)
|
||||
db_session.add(
|
||||
ProviderConfigTable(
|
||||
name=f"ollama-test-{suffix}",
|
||||
type=ModelProvider.OLLAMA_CLOUD,
|
||||
enabled=False,
|
||||
base_url="https://ollama.example.com",
|
||||
)
|
||||
)
|
||||
await db_session.flush()
|
||||
transport = ASGITransport(app=app)
|
||||
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
yield client
|
||||
app.dependency_overrides.clear()
|
||||
|
||||
|
||||
_HDR_PM = {"X-Agent-ID": str(uuid4()), "X-Agent-Role": "main_pm"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_catalog(app_client: AsyncClient) -> None:
|
||||
response = await app_client.get("/api/providers/catalog", headers=_HDR_PM)
|
||||
async def test_get_catalog(app_client_with_ollama: AsyncClient) -> None:
|
||||
response = await app_client_with_ollama.get(
|
||||
"/api/providers/catalog", headers=_HDR_PM
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
assert isinstance(response.json(), list)
|
||||
|
||||
@@ -103,8 +146,10 @@ async def test_get_catalog_forbidden_for_developer(
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_ollama_key_status(app_client: AsyncClient) -> None:
|
||||
response = await app_client.get("/api/providers/ollama-key", headers=_HDR_PM)
|
||||
async def test_get_ollama_key_status(app_client_with_ollama: AsyncClient) -> None:
|
||||
response = await app_client_with_ollama.get(
|
||||
"/api/providers/ollama-key", headers=_HDR_PM
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
body = response.json()
|
||||
assert "has_key" in body
|
||||
@@ -112,8 +157,8 @@ async def test_get_ollama_key_status(app_client: AsyncClient) -> None:
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_ollama_key(app_client: AsyncClient) -> None:
|
||||
response = await app_client.put(
|
||||
async def test_set_ollama_key(app_client_with_ollama: AsyncClient) -> None:
|
||||
response = await app_client_with_ollama.put(
|
||||
"/api/providers/ollama-key",
|
||||
json={"api_key": "secret-key-123"},
|
||||
headers=_HDR_PM,
|
||||
@@ -124,8 +169,8 @@ async def test_set_ollama_key(app_client: AsyncClient) -> None:
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_current_mode(app_client: AsyncClient) -> None:
|
||||
response = await app_client.get("/api/providers", headers=_HDR_PM)
|
||||
async def test_get_current_mode(app_client_with_ollama: AsyncClient) -> None:
|
||||
response = await app_client_with_ollama.get("/api/providers", headers=_HDR_PM)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
body = response.json()
|
||||
assert body["mode"] in {"anthropic", "ollama", "mix"}
|
||||
@@ -133,9 +178,9 @@ async def test_get_current_mode(app_client: AsyncClient) -> None:
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mode_anthropic_clears_assignments(
|
||||
app_client: AsyncClient,
|
||||
app_client_with_ollama: AsyncClient,
|
||||
) -> None:
|
||||
response = await app_client.post(
|
||||
response = await app_client_with_ollama.post(
|
||||
"/api/providers", json={"mode": "anthropic"}, headers=_HDR_PM
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
@@ -144,9 +189,11 @@ async def test_apply_mode_anthropic_clears_assignments(
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mode_unknown_returns_4xx(app_client: AsyncClient) -> None:
|
||||
async def test_apply_mode_unknown_returns_4xx(
|
||||
app_client_with_ollama: AsyncClient,
|
||||
) -> None:
|
||||
"""Unknown mode is rejected — Pydantic 422 at schema layer or 400 at service."""
|
||||
response = await app_client.post(
|
||||
response = await app_client_with_ollama.post(
|
||||
"/api/providers", json={"mode": "quantum"}, headers=_HDR_PM
|
||||
)
|
||||
assert response.status_code in (
|
||||
@@ -229,10 +276,10 @@ async def test_get_mode_developer_forbidden(
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mode_mix_without_per_agent_returns_400(
|
||||
app_client: AsyncClient,
|
||||
app_client_with_ollama: AsyncClient,
|
||||
) -> None:
|
||||
"""Apply 'mix' mode without per_agent triggers ValueError → 400 (lines 149-152)."""
|
||||
response = await app_client.post(
|
||||
response = await app_client_with_ollama.post(
|
||||
"/api/providers",
|
||||
json={"mode": "mix"},
|
||||
headers=_HDR_PM,
|
||||
@@ -259,4 +306,318 @@ async def test_apply_mode_ollama_without_provider_returns_404(
|
||||
"/api/providers", json={"mode": "ollama"}, headers=_HDR_PM
|
||||
)
|
||||
app.dependency_overrides.clear()
|
||||
assert response.status_code in (HTTPStatus.NOT_FOUND, HTTPStatus.OK)
|
||||
assert response.status_code == HTTPStatus.NOT_FOUND
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Self-hosted endpoints
|
||||
# =============================================================================
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def app_client_with_local(
|
||||
db_session: AsyncSession,
|
||||
) -> AsyncIterator[AsyncClient]:
|
||||
"""App client pre-seeded with Anthropic, Ollama Cloud, and LOCAL providers.
|
||||
|
||||
Begins with a DELETE-before-seed isolation step: deletes all rows from
|
||||
ModelAssignmentTable (FK-safe) then ProviderConfigTable before adding
|
||||
fresh rows. This ensures tests are order-independent regardless of what
|
||||
prior tests committed.
|
||||
"""
|
||||
app = _make_app(db_session)
|
||||
suffix = uuid4().hex[:8]
|
||||
# FK-safe cleanup: model_assignments.provider_config_id references
|
||||
# provider_configs.id, so assignments must be deleted first.
|
||||
await db_session.execute(delete(ModelAssignmentTable))
|
||||
await db_session.execute(delete(ProviderConfigTable))
|
||||
await db_session.flush()
|
||||
db_session.add(
|
||||
ProviderConfigTable(
|
||||
name=f"anthropic-local-{suffix}",
|
||||
type=ModelProvider.ANTHROPIC,
|
||||
enabled=True,
|
||||
)
|
||||
)
|
||||
db_session.add(
|
||||
ProviderConfigTable(
|
||||
name=f"ollama-local-{suffix}",
|
||||
type=ModelProvider.OLLAMA_CLOUD,
|
||||
enabled=False,
|
||||
base_url="https://ollama.example.com",
|
||||
)
|
||||
)
|
||||
db_session.add(
|
||||
ProviderConfigTable(
|
||||
name=f"self-hosted-local-{suffix}",
|
||||
type=ModelProvider.LOCAL,
|
||||
enabled=False,
|
||||
)
|
||||
)
|
||||
await db_session.flush()
|
||||
transport = ASGITransport(app=app)
|
||||
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
yield client
|
||||
app.dependency_overrides.clear()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_put_self_hosted_saves_base_url(
|
||||
app_client_with_local: AsyncClient,
|
||||
) -> None:
|
||||
"""PUT /self-hosted saves base_url and enables the LOCAL provider."""
|
||||
response = await app_client_with_local.put(
|
||||
"/api/providers/self-hosted",
|
||||
json={"base_url": "http://192.168.1.10:11434"},
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
body = response.json()
|
||||
assert body["base_url"] == "http://192.168.1.10:11434"
|
||||
assert body["enabled"] is True
|
||||
assert body["has_token"] is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_put_self_hosted_with_token_stores_encrypted(
|
||||
app_client_with_local: AsyncClient,
|
||||
) -> None:
|
||||
"""PUT /self-hosted with auth_token stores Fernet-encrypted token."""
|
||||
response = await app_client_with_local.put(
|
||||
"/api/providers/self-hosted",
|
||||
json={
|
||||
"base_url": "http://192.168.1.10:11434",
|
||||
"auth_token": "secret-ollama-key",
|
||||
},
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
body = response.json()
|
||||
assert body["has_token"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_put_self_hosted_not_seeded_returns_404(
|
||||
db_session: AsyncSession,
|
||||
) -> None:
|
||||
"""PUT /self-hosted when LOCAL provider not seeded returns 404."""
|
||||
await db_session.execute(
|
||||
delete(ProviderConfigTable).where(
|
||||
ProviderConfigTable.type == ModelProvider.LOCAL
|
||||
)
|
||||
)
|
||||
await db_session.flush()
|
||||
|
||||
app = _make_app(db_session)
|
||||
transport = ASGITransport(app=app)
|
||||
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
response = await client.put(
|
||||
"/api/providers/self-hosted",
|
||||
json={"base_url": "http://localhost:11434"},
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
app.dependency_overrides.clear()
|
||||
assert response.status_code == HTTPStatus.NOT_FOUND
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_put_self_hosted_developer_forbidden(
|
||||
db_session: AsyncSession,
|
||||
) -> None:
|
||||
"""PUT /self-hosted is forbidden for developer role."""
|
||||
app = _make_app(db_session, role=AgentRole.DEVELOPER, team=Team.BACKEND)
|
||||
transport = ASGITransport(app=app)
|
||||
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
response = await client.put(
|
||||
"/api/providers/self-hosted",
|
||||
json={"base_url": "http://localhost:11434"},
|
||||
headers={"X-Agent-ID": str(uuid4()), "X-Agent-Role": "developer"},
|
||||
)
|
||||
app.dependency_overrides.clear()
|
||||
assert response.status_code == HTTPStatus.FORBIDDEN
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_post_test_self_hosted_when_reachable(
|
||||
app_client_with_local: AsyncClient,
|
||||
) -> None:
|
||||
"""POST /self-hosted/test returns {ok: true, model_count: N} when reachable."""
|
||||
# First configure the base_url.
|
||||
await app_client_with_local.put(
|
||||
"/api/providers/self-hosted",
|
||||
json={"base_url": "http://192.168.1.10:11434"},
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
with patch(
|
||||
"roboco.api.routes.provider.probe_ollama_tags",
|
||||
new_callable=AsyncMock,
|
||||
return_value=(["llama3.1:8b", "gemma2:9b"], None),
|
||||
):
|
||||
response = await app_client_with_local.post(
|
||||
"/api/providers/self-hosted/test",
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
body = response.json()
|
||||
# Contract: field names and types for the test response schema.
|
||||
assert "ok" in body
|
||||
assert "model_count" in body
|
||||
assert "error" in body
|
||||
assert isinstance(body["ok"], bool)
|
||||
assert body["ok"] is True
|
||||
assert body["model_count"] == 2 # noqa: PLR2004
|
||||
assert body["error"] is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_post_test_self_hosted_when_unreachable(
|
||||
app_client_with_local: AsyncClient,
|
||||
) -> None:
|
||||
"""POST /self-hosted/test returns {ok: false, error: '...'} when unreachable."""
|
||||
await app_client_with_local.put(
|
||||
"/api/providers/self-hosted",
|
||||
json={"base_url": "http://192.168.1.10:11434"},
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
with patch(
|
||||
"roboco.api.routes.provider.probe_ollama_tags",
|
||||
new_callable=AsyncMock,
|
||||
return_value=([], "Could not connect to http://192.168.1.10:11434"),
|
||||
):
|
||||
response = await app_client_with_local.post(
|
||||
"/api/providers/self-hosted/test",
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
# Must be 200 with ok=false, NOT 500.
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
body = response.json()
|
||||
assert body["ok"] is False
|
||||
assert body["error"] is not None
|
||||
assert body["model_count"] is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_post_test_self_hosted_not_configured(
|
||||
app_client_with_local: AsyncClient,
|
||||
) -> None:
|
||||
"""POST /self-hosted/test when no base_url returns {ok: false} without 500."""
|
||||
# LOCAL provider seeded but no base_url configured.
|
||||
response = await app_client_with_local.post(
|
||||
"/api/providers/self-hosted/test",
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
body = response.json()
|
||||
assert body["ok"] is False
|
||||
assert body["error"] is not None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_self_hosted_config_returns_200(
|
||||
app_client_with_local: AsyncClient,
|
||||
) -> None:
|
||||
"""GET /self-hosted returns {base_url, has_token, enabled} when LOCAL is seeded."""
|
||||
response = await app_client_with_local.get(
|
||||
"/api/providers/self-hosted",
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
body = response.json()
|
||||
# Contract: field names and types must match the schema.
|
||||
assert "base_url" in body
|
||||
assert "has_token" in body
|
||||
assert "enabled" in body
|
||||
assert isinstance(body["has_token"], bool)
|
||||
assert isinstance(body["enabled"], bool)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_self_hosted_config_not_seeded_returns_404(
|
||||
db_session: AsyncSession,
|
||||
) -> None:
|
||||
"""GET /self-hosted when LOCAL provider not seeded returns 404."""
|
||||
await db_session.execute(
|
||||
delete(ProviderConfigTable).where(
|
||||
ProviderConfigTable.type == ModelProvider.LOCAL
|
||||
)
|
||||
)
|
||||
await db_session.flush()
|
||||
|
||||
app = _make_app(db_session)
|
||||
transport = ASGITransport(app=app)
|
||||
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
response = await client.get(
|
||||
"/api/providers/self-hosted",
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
app.dependency_overrides.clear()
|
||||
assert response.status_code == HTTPStatus.NOT_FOUND
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_self_hosted_models_returns_list(
|
||||
app_client_with_local: AsyncClient,
|
||||
) -> None:
|
||||
"""GET /self-hosted/models returns [{model_name, display_name}] objects."""
|
||||
await app_client_with_local.put(
|
||||
"/api/providers/self-hosted",
|
||||
json={"base_url": "http://192.168.1.10:11434"},
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
with patch(
|
||||
"roboco.api.routes.provider.probe_ollama_tags",
|
||||
new_callable=AsyncMock,
|
||||
return_value=(["llama3.1:8b", "gemma2:9b", "qwen2.5:14b"], None),
|
||||
):
|
||||
response = await app_client_with_local.get(
|
||||
"/api/providers/self-hosted/models",
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
assert response.status_code == HTTPStatus.OK
|
||||
models = response.json()
|
||||
assert isinstance(models, list)
|
||||
assert len(models) == 3 # noqa: PLR2004
|
||||
# Contract: each entry must be an object with model_name and display_name.
|
||||
first = models[0]
|
||||
assert isinstance(first, dict)
|
||||
assert "model_name" in first
|
||||
assert "display_name" in first
|
||||
assert isinstance(first["model_name"], str)
|
||||
assert isinstance(first["display_name"], str)
|
||||
# Verify specific entry present.
|
||||
names = [m["model_name"] for m in models]
|
||||
assert "llama3.1:8b" in names
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_self_hosted_models_not_configured_returns_404(
|
||||
app_client_with_local: AsyncClient,
|
||||
) -> None:
|
||||
"""GET /self-hosted/models when no base_url configured returns 404."""
|
||||
response = await app_client_with_local.get(
|
||||
"/api/providers/self-hosted/models",
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
assert response.status_code == HTTPStatus.NOT_FOUND
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_self_hosted_models_unreachable_returns_503(
|
||||
app_client_with_local: AsyncClient,
|
||||
) -> None:
|
||||
"""GET /self-hosted/models when server unreachable returns 503."""
|
||||
await app_client_with_local.put(
|
||||
"/api/providers/self-hosted",
|
||||
json={"base_url": "http://192.168.1.10:11434"},
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
with patch(
|
||||
"roboco.api.routes.provider.probe_ollama_tags",
|
||||
new_callable=AsyncMock,
|
||||
return_value=([], "Could not connect"),
|
||||
):
|
||||
response = await app_client_with_local.get(
|
||||
"/api/providers/self-hosted/models",
|
||||
headers=_HDR_PM,
|
||||
)
|
||||
assert response.status_code == HTTPStatus.SERVICE_UNAVAILABLE
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
"""Unit tests for roboco.services.llm.probe_ollama_tags.
|
||||
|
||||
Covers all five branches of the function:
|
||||
1. Successful JSON parse → returns model names list
|
||||
2. httpx.TimeoutException → returns ([], timeout message)
|
||||
3. httpx.ConnectError → returns ([], connect-error message)
|
||||
4. httpx.HTTPStatusError → returns ([], http-status message)
|
||||
5. Generic Exception → logs server-side and returns ([], generic error string)
|
||||
|
||||
All tests mock `httpx.AsyncClient` so they require no network or DB.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from roboco.services.llm import probe_ollama_tags
|
||||
|
||||
_BASE_URL = "http://localhost:11434"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_probe_ollama_tags_success() -> None:
|
||||
"""Successful /api/tags response returns list of model name strings."""
|
||||
mock_resp = MagicMock()
|
||||
mock_resp.raise_for_status = MagicMock()
|
||||
mock_resp.json.return_value = {
|
||||
"models": [
|
||||
{"name": "llama3.1:8b"},
|
||||
{"name": "gemma2:9b"},
|
||||
{"name": "qwen2.5:14b"},
|
||||
]
|
||||
}
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.get = AsyncMock(return_value=mock_resp)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=False)
|
||||
|
||||
with patch("roboco.services.llm.httpx.AsyncClient", return_value=mock_client):
|
||||
models, error = await probe_ollama_tags(_BASE_URL)
|
||||
|
||||
assert error is None
|
||||
assert models == ["llama3.1:8b", "gemma2:9b", "qwen2.5:14b"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_probe_ollama_tags_timeout() -> None:
|
||||
"""httpx.TimeoutException returns ([], message) mentioning the timeout."""
|
||||
mock_client = AsyncMock()
|
||||
mock_client.get = AsyncMock(side_effect=httpx.TimeoutException("timed out"))
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=False)
|
||||
|
||||
with patch("roboco.services.llm.httpx.AsyncClient", return_value=mock_client):
|
||||
models, error = await probe_ollama_tags(_BASE_URL)
|
||||
|
||||
assert models == []
|
||||
assert error is not None
|
||||
assert "timed out" in error.lower() or "timeout" in error.lower()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_probe_ollama_tags_connect_error() -> None:
|
||||
"""httpx.ConnectError returns ([], message) mentioning connection failure."""
|
||||
mock_client = AsyncMock()
|
||||
mock_client.get = AsyncMock(side_effect=httpx.ConnectError("connection refused"))
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=False)
|
||||
|
||||
with patch("roboco.services.llm.httpx.AsyncClient", return_value=mock_client):
|
||||
models, error = await probe_ollama_tags(_BASE_URL)
|
||||
|
||||
assert models == []
|
||||
assert error is not None
|
||||
assert "connect" in error.lower() or "offline" in error.lower()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_probe_ollama_tags_http_status_error() -> None:
|
||||
"""httpx.HTTPStatusError returns ([], message) containing the HTTP status code."""
|
||||
mock_response = MagicMock()
|
||||
mock_response.status_code = 503
|
||||
|
||||
http_err = httpx.HTTPStatusError(
|
||||
"service unavailable",
|
||||
request=MagicMock(),
|
||||
response=mock_response,
|
||||
)
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.get = AsyncMock(side_effect=http_err)
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=False)
|
||||
|
||||
with patch("roboco.services.llm.httpx.AsyncClient", return_value=mock_client):
|
||||
models, error = await probe_ollama_tags(_BASE_URL)
|
||||
|
||||
assert models == []
|
||||
assert error is not None
|
||||
assert "503" in error
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_probe_ollama_tags_generic_exception_logs_and_returns_generic() -> None:
|
||||
"""Generic Exception logs the error server-side and returns a generic string.
|
||||
|
||||
The raw exception message must NOT appear in the returned error string
|
||||
(to avoid leaking internal server details in HTTP responses).
|
||||
"""
|
||||
secret_detail = "internal secret detail that must not leak"
|
||||
|
||||
mock_client = AsyncMock()
|
||||
mock_client.get = AsyncMock(side_effect=RuntimeError(secret_detail))
|
||||
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
|
||||
mock_client.__aexit__ = AsyncMock(return_value=False)
|
||||
|
||||
with (
|
||||
patch("roboco.services.llm.httpx.AsyncClient", return_value=mock_client),
|
||||
patch("roboco.services.llm._log") as mock_log,
|
||||
):
|
||||
models, error = await probe_ollama_tags(_BASE_URL)
|
||||
|
||||
assert models == []
|
||||
assert error is not None
|
||||
# The returned error string must NOT contain the raw exception text.
|
||||
assert secret_detail not in error
|
||||
# The logger must have been called to record the exception server-side.
|
||||
mock_log.error.assert_called_once()
|
||||
Reference in New Issue
Block a user