mirror of
https://github.com/nexu-io/open-design.git
synced 2026-10-27 09:51:12 +01:00
* fix(security): reject loopback for BYOK asset download URLs (#5478) The assertExternalAssetUrl guard previously routed through validateBaseUrlResolved without forbidLoopback, so asset URLs pointing at 127.0.0.1, localhost, or ::1 passed validation and were fetched by the daemon. This allowed a compromised or misconfigured upstream gateway to exfiltrate data from internal services via SSRF. Changes: - Add forbidLoopback option to ValidateBaseUrlOptions (packages/contracts) - validateBaseUrl rejects loopback hosts when forbidLoopback is set - validateBaseUrlResolved skips the loopback early-return when forbidLoopback - validateBaseUrlResolved: when forbidLoopback is true, DNS-resolved loopback addresses are also rejected (not just literal loopback hosts) - assertExternalAssetUrl now passes forbidLoopback: true - User-configured provider endpoints (validateUserProviderBaseUrl) are unaffected — they still allow loopback for local gateways - Adds regression tests covering literal loopback rejection, DNS-resolved loopback rejection (IPv4, IPv6, mixed results), and provider endpoint exemption Note: PR #5529 fixed the library-ingest SSRF path. This PR closes the separate BYOK asset-download path (assertExternalAssetUrl) that retains the loopback carve-out. Closes #5478 * fix(security): pin DNS resolution to prevent rebinding in asset fetch (#5478) The validation step (validateBaseUrlResolved) and the fetch step (assertAndFetchExternalAsset) previously performed independent DNS lookups. A DNS-rebinding domain could return a public IP for validation and then 127.0.0.1 / ::1 at fetch time, bypassing the loopback guard. Fix: - validateBaseUrlResolved now attaches the DNS-resolved addresses that passed validation to its return value (BaseUrlValidationResult.resolvedAddresses) - assertExternalAssetUrl passes these addresses through on the ok branch - assertAndFetchExternalAsset creates an Undici Agent with a custom connect.lookup that returns only the validated addresses, pinning the TCP connection regardless of any DNS rebind between validation and fetch - 3 call sites in byok-tools.ts that used validate-then-fetch separately now route through assertAndFetchExternalAsset for a single unified path - 4 regression tests covering: single validated address, DNS-rebind mock (public→loopback), round-robin DNS, and IP-literal short-circuit * fix(security): fail-closed on DNS error + connection-time validating lookup (#5478) Addresses mrcfps third review: the fallback to unpinned fetch when validation did not attach resolved addresses left a DNS-rebinding hole where an attacker could make the validation lookup throw (ENOTFOUND / SERVFAIL) and then answer loopback for the fetch lookup. Changes: 1. validateBaseUrlResolved: when forbidLoopback is true, DNS lookup failures now return a forbidden result instead of the sync success. This prevents the fail-then-rebind vector. 2. assertAndFetchExternalAsset: removed the unpinned-fetch fallback. Non-IP-literal hostnames without resolvedAddresses now throw — the fetch never happens. IP literals are safe because they were validated synchronously and have no hostname to rebind. 3. Replaced the per-request pinned Agent with a long-lived Undici Agent (assetDispatcher) whose connection-time lookup (createAssetValidatingLookup) rejects any non-public address. This is defense-in-depth on top of the pre-validation, following the same pattern as brands/safe-fetch.ts and plugins/plugin-asset-cache.ts. A shared dispatcher also avoids the keep-alive socket leak of per-request Agents. 4. createAssetValidatingLookup is exported for unit testing. 5. Expanded test coverage: - DNS failure → fail-closed (forbidLoopback true and false) - createAssetValidatingLookup: rejects loopback, RFC1918, metadata IP; allows public - assertAndFetchExternalAsset: throws on blocked URLs and internal IPs - Updated existing public-CDN tests to use IP literals (fail-closed behavior now correctly rejects hostnames when DNS is unavailable) * fix(security): injectable lookup + testable fetch for asset SSRF guard (#5478) Addresses mrcfps 4th review: daemon CI was red because fail-closed behavior caused hostname-based test fixtures to NXDOMAIN, and undiciFetch bypassed vi.stubGlobal('fetch') stubs. Changes: 1. assertExternalAssetUrl and assertAndFetchExternalAsset now accept an optional injectable `lookup` (DnsLookupFn) and `fetchImpl` parameter. Production callers pass neither, so default DNS and globalThis.fetch are used. Tests can inject mock lookups/fetch. 2. assertAndFetchExternalAsset uses globalThis.fetch (not undiciFetch) so vi.stubGlobal('fetch', ...) stubs in existing test suites (byok-tools, aihubmix, proxy-routes, senseaudio) still intercept. 3. Existing test fixtures: replaced cdn.example.test with 93.184.216.34 (public IP literal) in 6 test files (41 substitutions) so IP literals skip DNS resolution and don't trigger fail-closed on NXDOMAIN .test hostnames. 4. Added 3 TOCTOU regression tests driving assertAndFetchExternalAsset with injected lookup + fetch: - Validation lookup public → fetch called with redirect:error - Validation lookup throws → assertAndFetchExternalAsset rejects without invoking fetch (fail-then-rebind vector closed) - IP literal → skips DNS, calls fetch directly 5. assetDispatcher + createAssetValidatingLookup kept for production connection-time defense-in-depth (exported, available for future wiring to a production fetch dispatcher). Total: 31 tests in asset-ssrf-loopback.test.ts, all passing. * fix(security): attach validating dispatcher to asset fetch (#5478) mrcfps 5th review: assetDispatcher was built but never attached to the fetch call. The validating lookup only runs at connect time through the dispatcher, so without it the fetch re-resolves the attacker-controlled hostname and a public-then-loopback rebind still reaches the daemon. Fix: attach assetDispatcher to the RequestInit via the dispatcher key (same pattern as plugins/plugin-asset-cache.ts safeExternalFetch). Production globalThis.fetch (Node/undici) uses it to refuse connect-time non-public addresses. Test stubs still see redirect:'error' and can ignore dispatcher. Also fixed the misnamed TOCTOU test to assert dispatcher is present on the captured RequestInit, and removed unused undiciFetch import. * fix(tests): update dispatcher assertions for assertAndFetchExternalAsset assertAndFetchExternalAsset now attaches a validating assetDispatcher to the fetch RequestInit (issue #5478). Tests that blanket-checked init?.dispatcher === testMock fail because asset-download calls correctly carry the validating dispatcher instead. Fix: conditional assertion — when init.redirect === 'error' (asset download path), check dispatcher exists rather than matching the test mock. API calls still verify the caller's dispatcher is forwarded unchanged. * fix(tests): URL-keyed dispatcher assertions + document asset dispatcher override Address mrcfps's non-blocking review items: 1. Replace the redirect==='error' heuristic with URL-keyed assertions. The heuristic silently dropped the submit dispatcher identity check on the AIHubMix video path (its submit hop also sets redirect:'error'). Submit/poll hops now assert toBe(callerDispatcher); the asset hop asserts toBe(getAssetValidatingDispatcher()) — exported from connectionTest.ts so the exact validating instance is compared. 2. Document the intentional dispatcher override on the BYOKToolContext.requestInit JSDoc: asset downloads through assertAndFetchExternalAsset intentionally never ride the turn proxy dispatcher; submit/poll hops keep it. --------- Co-authored-by: CVE-Hunter-Leo <cve-hunter-leo@users.noreply.github.com>
377 lines
11 KiB
TypeScript
377 lines
11 KiB
TypeScript
import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
|
|
import { tmpdir } from 'node:os';
|
|
import path from 'node:path';
|
|
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
|
|
|
import { generateMedia } from '../../src/media/index.js';
|
|
|
|
const TEST_SENSEAUDIO_BASE_URL = 'https://senseaudio-gateway.example.test';
|
|
const WIRE_MODEL = 'senseaudio-tts-1.5-260319';
|
|
|
|
function buildOkResponse(audioBytes: Buffer, opts?: { audioLength?: number }) {
|
|
return new Response(
|
|
JSON.stringify({
|
|
data: { audio: audioBytes.toString('hex'), status: 2 },
|
|
extra_info: {
|
|
audio_length: opts?.audioLength ?? 12340,
|
|
audio_sample_rate: 32000,
|
|
audio_size: audioBytes.length,
|
|
bitrate: 128000,
|
|
audio_format: 'mp3',
|
|
audio_channel: 2,
|
|
word_count: 8,
|
|
usage_characters: 8,
|
|
},
|
|
trace_id: 'trace-test',
|
|
base_resp: { status_code: 0, status_msg: 'success' },
|
|
}),
|
|
{ status: 200, headers: { 'content-type': 'application/json' } },
|
|
);
|
|
}
|
|
|
|
describe('senseaudio media generation', () => {
|
|
let root: string;
|
|
let projectRoot: string;
|
|
let projectsRoot: string;
|
|
const realFetch = globalThis.fetch;
|
|
const originalMediaConfigDir = process.env.OD_MEDIA_CONFIG_DIR;
|
|
const originalDataDir = process.env.OD_DATA_DIR;
|
|
|
|
beforeEach(async () => {
|
|
root = await mkdtemp(path.join(tmpdir(), 'od-senseaudio-'));
|
|
projectRoot = path.join(root, 'project-root');
|
|
projectsRoot = path.join(projectRoot, '.od', 'projects');
|
|
await mkdir(projectsRoot, { recursive: true });
|
|
delete process.env.OD_MEDIA_CONFIG_DIR;
|
|
delete process.env.OD_DATA_DIR;
|
|
delete process.env.OD_SENSEAUDIO_API_KEY;
|
|
delete process.env.SENSEAUDIO_API_KEY;
|
|
});
|
|
|
|
afterEach(async () => {
|
|
globalThis.fetch = realFetch;
|
|
if (originalMediaConfigDir == null) {
|
|
delete process.env.OD_MEDIA_CONFIG_DIR;
|
|
} else {
|
|
process.env.OD_MEDIA_CONFIG_DIR = originalMediaConfigDir;
|
|
}
|
|
if (originalDataDir == null) {
|
|
delete process.env.OD_DATA_DIR;
|
|
} else {
|
|
process.env.OD_DATA_DIR = originalDataDir;
|
|
}
|
|
delete process.env.OD_SENSEAUDIO_API_KEY;
|
|
delete process.env.SENSEAUDIO_API_KEY;
|
|
await rm(root, { recursive: true, force: true });
|
|
});
|
|
|
|
async function writeConfig(data: unknown) {
|
|
const file = path.join(projectRoot, '.od', 'media-config.json');
|
|
await mkdir(path.dirname(file), { recursive: true });
|
|
await writeFile(file, JSON.stringify(data), 'utf8');
|
|
}
|
|
|
|
it('renders SenseAudio speech with the documented defaults', async () => {
|
|
await writeConfig({
|
|
providers: {
|
|
senseaudio: {
|
|
apiKey: 'sense-test-key',
|
|
baseUrl: TEST_SENSEAUDIO_BASE_URL,
|
|
},
|
|
},
|
|
});
|
|
|
|
const mp3Bytes = Buffer.from([0x49, 0x44, 0x33, 0x04, 0x00, 0x00, 0x73, 0x65, 0x6e]);
|
|
const fetchMock = vi.fn(async (input: unknown, init?: RequestInit) => {
|
|
expect(String(input)).toBe(`${TEST_SENSEAUDIO_BASE_URL}/v1/t2a_v2`);
|
|
expect(init?.method).toBe('POST');
|
|
expect(init?.headers).toMatchObject({
|
|
authorization: 'Bearer sense-test-key',
|
|
'content-type': 'application/json',
|
|
});
|
|
expect(JSON.parse(String(init?.body))).toEqual({
|
|
model: WIRE_MODEL,
|
|
text: '你好,欢迎使用 SenseAudio 语音合成服务。',
|
|
stream: false,
|
|
voice_setting: {
|
|
voice_id: 'female_0033_b',
|
|
speed: 1,
|
|
vol: 1,
|
|
pitch: 0,
|
|
},
|
|
audio_setting: {
|
|
format: 'mp3',
|
|
sample_rate: 32000,
|
|
bitrate: 128000,
|
|
channel: 2,
|
|
},
|
|
});
|
|
return buildOkResponse(mp3Bytes);
|
|
});
|
|
vi.stubGlobal('fetch', fetchMock);
|
|
|
|
const result = await generateMedia({
|
|
projectRoot,
|
|
projectsRoot,
|
|
projectId: 'project-1',
|
|
surface: 'audio',
|
|
model: 'senseaudio-tts',
|
|
audioKind: 'speech',
|
|
prompt: '你好,欢迎使用 SenseAudio 语音合成服务。',
|
|
output: 'senseaudio-speech.mp3',
|
|
});
|
|
|
|
expect(fetchMock).toHaveBeenCalledTimes(1);
|
|
expect(result.providerId).toBe('senseaudio');
|
|
expect(result.providerNote).toContain(`senseaudio/${WIRE_MODEL}`);
|
|
expect(result.providerNote).toContain('female_0033_b');
|
|
|
|
const bytes = await readFile(path.join(projectsRoot, 'project-1', 'senseaudio-speech.mp3'));
|
|
expect(bytes.equals(mp3Bytes)).toBe(true);
|
|
});
|
|
|
|
it('passes custom voice id through to the request body', async () => {
|
|
await writeConfig({
|
|
providers: {
|
|
senseaudio: {
|
|
apiKey: 'sense-test-key',
|
|
baseUrl: TEST_SENSEAUDIO_BASE_URL,
|
|
},
|
|
},
|
|
});
|
|
|
|
const mp3Bytes = Buffer.from([0xff, 0xfb, 0x90, 0x00]);
|
|
const fetchMock = vi.fn(async (_input: unknown, init?: RequestInit) => {
|
|
const body = JSON.parse(String(init?.body));
|
|
expect(body.voice_setting.voice_id).toBe('male_0001_a');
|
|
return buildOkResponse(mp3Bytes);
|
|
});
|
|
vi.stubGlobal('fetch', fetchMock);
|
|
|
|
const result = await generateMedia({
|
|
projectRoot,
|
|
projectsRoot,
|
|
projectId: 'project-1',
|
|
surface: 'audio',
|
|
model: 'senseaudio-tts',
|
|
audioKind: 'speech',
|
|
voice: 'male_0001_a',
|
|
prompt: 'Custom voice line.',
|
|
output: 'senseaudio-custom.mp3',
|
|
});
|
|
|
|
expect(result.providerNote).toContain('male_0001_a');
|
|
});
|
|
|
|
it('falls back to the canonical base URL when none is configured', async () => {
|
|
await writeConfig({
|
|
providers: {
|
|
senseaudio: { apiKey: 'sense-test-key' },
|
|
},
|
|
});
|
|
|
|
const fetchMock = vi.fn(async (input: unknown) => {
|
|
expect(String(input)).toBe('https://api.senseaudio.cn/v1/t2a_v2');
|
|
return buildOkResponse(Buffer.from([0x01, 0x02, 0x03]));
|
|
});
|
|
vi.stubGlobal('fetch', fetchMock);
|
|
|
|
await generateMedia({
|
|
projectRoot,
|
|
projectsRoot,
|
|
projectId: 'project-1',
|
|
surface: 'audio',
|
|
model: 'senseaudio-tts',
|
|
audioKind: 'speech',
|
|
prompt: 'Default base url.',
|
|
output: 'senseaudio-default-base.mp3',
|
|
});
|
|
|
|
expect(fetchMock).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it('reads the API key from OD_SENSEAUDIO_API_KEY when storage is empty', async () => {
|
|
process.env.OD_SENSEAUDIO_API_KEY = 'env-sense-key';
|
|
const fetchMock = vi.fn(async (_input: unknown, init?: RequestInit) => {
|
|
expect(init?.headers).toMatchObject({
|
|
authorization: 'Bearer env-sense-key',
|
|
});
|
|
return buildOkResponse(Buffer.from([0x10, 0x20]));
|
|
});
|
|
vi.stubGlobal('fetch', fetchMock);
|
|
|
|
await generateMedia({
|
|
projectRoot,
|
|
projectsRoot,
|
|
projectId: 'project-1',
|
|
surface: 'audio',
|
|
model: 'senseaudio-tts',
|
|
audioKind: 'speech',
|
|
prompt: 'Env-only key.',
|
|
output: 'senseaudio-env.mp3',
|
|
});
|
|
|
|
expect(fetchMock).toHaveBeenCalledTimes(1);
|
|
});
|
|
|
|
it('forwards requestInit.dispatcher through SenseAudio image submit and download fetches', async () => {
|
|
await writeConfig({
|
|
providers: {
|
|
senseaudio: {
|
|
apiKey: 'sense-test-key',
|
|
baseUrl: TEST_SENSEAUDIO_BASE_URL,
|
|
},
|
|
},
|
|
});
|
|
|
|
const dispatcher = {} as NonNullable<RequestInit['dispatcher']>;
|
|
const fetchMock = vi.fn(async (input: unknown, init?: RequestInit) => {
|
|
if (String(input) === `${TEST_SENSEAUDIO_BASE_URL}/v1/image/sync`) {
|
|
expect(init?.dispatcher).toBe(dispatcher);
|
|
return new Response(JSON.stringify({
|
|
url: 'https://93.184.216.34/senseaudio-image.png',
|
|
base_resp: { status_code: 0, status_msg: 'success' },
|
|
}), {
|
|
status: 200,
|
|
headers: { 'content-type': 'application/json' },
|
|
});
|
|
}
|
|
expect(String(input)).toBe('https://93.184.216.34/senseaudio-image.png');
|
|
// Asset download goes through assertAndFetchExternalAsset which replaces
|
|
// the caller's dispatcher with a validating one (issue #5478).
|
|
expect(init?.dispatcher).toBeDefined();
|
|
expect(init?.redirect).toBe('error');
|
|
return new Response(Buffer.from([0x89, 0x50, 0x4e, 0x47]));
|
|
});
|
|
vi.stubGlobal('fetch', fetchMock);
|
|
|
|
const result = await generateMedia({
|
|
projectRoot,
|
|
projectsRoot,
|
|
projectId: 'project-1',
|
|
surface: 'image',
|
|
model: 'senseaudio-image-2.0-260319',
|
|
prompt: 'A reference render.',
|
|
output: 'senseaudio-image.png',
|
|
requestInit: { dispatcher },
|
|
});
|
|
|
|
expect(result.providerId).toBe('senseaudio');
|
|
expect(fetchMock).toHaveBeenCalledTimes(2);
|
|
});
|
|
|
|
it('errors when no API key is configured', async () => {
|
|
const fetchMock = vi.fn();
|
|
vi.stubGlobal('fetch', fetchMock);
|
|
|
|
await expect(
|
|
generateMedia({
|
|
projectRoot,
|
|
projectsRoot,
|
|
projectId: 'project-1',
|
|
surface: 'audio',
|
|
model: 'senseaudio-tts',
|
|
audioKind: 'speech',
|
|
prompt: 'Should fail.',
|
|
output: 'senseaudio-no-key.mp3',
|
|
}),
|
|
).rejects.toThrow(/no SenseAudio API key/);
|
|
|
|
expect(fetchMock).not.toHaveBeenCalled();
|
|
});
|
|
|
|
it('surfaces upstream base_resp failures verbatim', async () => {
|
|
await writeConfig({
|
|
providers: {
|
|
senseaudio: { apiKey: 'sense-test-key', baseUrl: TEST_SENSEAUDIO_BASE_URL },
|
|
},
|
|
});
|
|
|
|
const fetchMock = vi.fn(async () =>
|
|
new Response(
|
|
JSON.stringify({
|
|
data: null,
|
|
base_resp: { status_code: 1004, status_msg: 'voice_id not found' },
|
|
}),
|
|
{ status: 200, headers: { 'content-type': 'application/json' } },
|
|
),
|
|
);
|
|
vi.stubGlobal('fetch', fetchMock);
|
|
|
|
await expect(
|
|
generateMedia({
|
|
projectRoot,
|
|
projectsRoot,
|
|
projectId: 'project-1',
|
|
surface: 'audio',
|
|
model: 'senseaudio-tts',
|
|
audioKind: 'speech',
|
|
voice: 'does-not-exist',
|
|
prompt: 'Bad voice.',
|
|
output: 'senseaudio-bad-voice.mp3',
|
|
}),
|
|
).rejects.toThrow('senseaudio tts api error 1004: voice_id not found');
|
|
});
|
|
|
|
it('errors when the audio payload is missing', async () => {
|
|
await writeConfig({
|
|
providers: {
|
|
senseaudio: { apiKey: 'sense-test-key', baseUrl: TEST_SENSEAUDIO_BASE_URL },
|
|
},
|
|
});
|
|
|
|
const fetchMock = vi.fn(async () =>
|
|
new Response(
|
|
JSON.stringify({
|
|
data: {},
|
|
base_resp: { status_code: 0, status_msg: 'success' },
|
|
}),
|
|
{ status: 200, headers: { 'content-type': 'application/json' } },
|
|
),
|
|
);
|
|
vi.stubGlobal('fetch', fetchMock);
|
|
|
|
await expect(
|
|
generateMedia({
|
|
projectRoot,
|
|
projectsRoot,
|
|
projectId: 'project-1',
|
|
surface: 'audio',
|
|
model: 'senseaudio-tts',
|
|
audioKind: 'speech',
|
|
prompt: 'Missing audio.',
|
|
output: 'senseaudio-missing-audio.mp3',
|
|
}),
|
|
).rejects.toThrow('senseaudio tts response missing data.audio');
|
|
});
|
|
|
|
it('surfaces HTTP-level failures with the status code and truncated body', async () => {
|
|
await writeConfig({
|
|
providers: {
|
|
senseaudio: { apiKey: 'sense-test-key', baseUrl: TEST_SENSEAUDIO_BASE_URL },
|
|
},
|
|
});
|
|
|
|
const fetchMock = vi.fn(async () =>
|
|
new Response('unauthorized', {
|
|
status: 401,
|
|
headers: { 'content-type': 'text/plain' },
|
|
}),
|
|
);
|
|
vi.stubGlobal('fetch', fetchMock);
|
|
|
|
await expect(
|
|
generateMedia({
|
|
projectRoot,
|
|
projectsRoot,
|
|
projectId: 'project-1',
|
|
surface: 'audio',
|
|
model: 'senseaudio-tts',
|
|
audioKind: 'speech',
|
|
prompt: 'Bad auth.',
|
|
output: 'senseaudio-401.mp3',
|
|
}),
|
|
).rejects.toThrow('senseaudio tts 401: unauthorized');
|
|
});
|
|
});
|