fix(dictation): resume suspended AudioContext; chunk audio under IPC cap

Two P2 review findings:

1. Resume the AudioContext before wiring the worklet. The WebView can create
   the context in the 'suspended' state after the async start path (mic prompt
   etc.); a suspended context never pulls the worklet's process(), so no PCM
   reaches push_dictation_audio while the UI shows an active recording. Mirror
   the huddle capture path and resume() when suspended.

2. Chunk merged PCM under the native 100 KB IPC cap. push_dictation_audio
   rejects any raw batch over MAX_AUDIO_BATCH_BYTES (100 KB = ~0.53s at 48kHz
   f32 mono). If the batch timer is delayed (main-thread stall) the merged
   buffer can exceed the cap, native rejects it, and the .catch silently drops
   the whole chunk. Split into <=24,000-sample (96 KB) chunks sent in order.
This commit is contained in:
npub13fn4ahfnvaa2qwylvegdgeajqs0mph6v4qsw4jcqnw4mjh3hzh2quuucm5
2026-07-11 16:19:22 +01:00
parent 3fff53a9c2
commit 8bbdeec000
@@ -41,6 +41,16 @@ const MODEL_POLL_INTERVAL_MS = 5_000;
*/
const AUDIO_BATCH_MS = 100;
/**
* Max samples per `push_dictation_audio` IPC call. The native command rejects
* any raw batch over 100 KB (`MAX_AUDIO_BATCH_BYTES` in `dictation.rs`); at
* 48 kHz f32 mono that is 25,600 samples (~0.53s). We chunk under that cap
* (24,000 samples / 96 KB, leaving headroom) so a stalled main thread that
* lets the batch grow past ~0.5s can't produce a single oversized buffer that
* native rejects and we silently drop. Chunks are sent in order.
*/
const MAX_IPC_SAMPLES = 24_000;
/**
* Local STT dictation hook using the Parakeet model via Tauri native commands.
*
@@ -137,14 +147,28 @@ export function useLocalDictation({
}
audioBatchRef.current = [];
const bytes = new Uint8Array(
merged.buffer,
merged.byteOffset,
merged.byteLength,
);
return invokeRawBinary("push_dictation_audio", bytes)
.then(() => {})
.catch(() => {});
// Split into chunks under the native IPC cap and send them in order.
// A single unbounded buffer can exceed the 100 KB native limit if the
// batch timer was delayed (e.g. main-thread stall), in which case native
// rejects it and the whole chunk is silently lost.
const sendChunks = async (): Promise<void> => {
for (let start = 0; start < merged.length; start += MAX_IPC_SAMPLES) {
const slice = merged.subarray(
start,
Math.min(start + MAX_IPC_SAMPLES, merged.length),
);
// Copy into a fresh, tightly-bound buffer so the raw IPC payload is
// exactly this chunk (a subarray view shares the parent's ArrayBuffer).
const chunk = new Uint8Array(
slice.buffer.slice(
slice.byteOffset,
slice.byteOffset + slice.byteLength,
),
);
await invokeRawBinary("push_dictation_audio", chunk).catch(() => {});
}
};
return sendChunks();
}, []);
const cleanup = useCallback(() => {
@@ -307,6 +331,14 @@ export function useLocalDictation({
const audioContext = new AudioContext({ sampleRate: 48000 });
audioContextRef.current = audioContext;
// Resume if the WebView created the context suspended (autoplay policy).
// A suspended context never pulls the worklet's `process()`, so no PCM
// would reach `push_dictation_audio` while the UI shows an active
// recording. Mirrors the huddle capture path (`huddle/lib/audioWorklet.ts`).
if (audioContext.state === "suspended") {
await audioContext.resume();
}
// Create a processor that accumulates audio frames and posts them
// to the main thread. Batching happens on the main thread side via
// a timer to reduce IPC overhead.