Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions javascript/src/voice/__tests__/interrupt-truncation.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,30 @@ describe("defaultVoiceCall publishes the agent speaking event", () => {
await expect(adapter.agentSpeakingEvent!.wait()).resolves.toBeUndefined();
});

it("does not set the speaking event when the first chunk is empty", async () => {
const adapter = new SpeakingAdapter([
new AudioChunk({ data: new Uint8Array(0) }),
]);

await defaultVoiceCall(adapter, bareInput);

expect(adapter.agentSpeakingEvent).toBeDefined();
expect(adapter.agentSpeakingEvent!.isSet()).toBe(false);
});

it("sets the speaking event when audio follows a leading empty chunk", async () => {
const adapter = new SpeakingAdapter([
new AudioChunk({ data: new Uint8Array(0) }),
tone(0.1, "after empty"),
]);

await defaultVoiceCall(adapter, bareInput);

expect(adapter.agentSpeakingEvent).toBeDefined();
expect(adapter.agentSpeakingEvent!.isSet()).toBe(true);
await expect(adapter.agentSpeakingEvent!.wait()).resolves.toBeUndefined();
});

it("clears the event at the start of the next turn (re-armed)", async () => {
// Turn-2's first chunk is delayed so the cleared-but-not-yet-set window is
// observable across a macrotask yield. Without it the immediate chunk would
Expand Down
13 changes: 10 additions & 3 deletions javascript/src/voice/adapter.runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -719,10 +719,16 @@ async function drainInner(
"voice.audio.first_chunk_latency_ms",
Math.round(performance.now() - receiveStart),
);
if (first.data.length > 0) {
// The initial receive can be an empty prefix while the drain still yields
// audio. Publish speech on the first playable chunk wherever it appears.
let sawPlayableChunk = false;
const markFirstPlayableChunk = (chunk: AudioChunk): void => {
if (sawPlayableChunk || chunk.data.length === 0) return;
sawPlayableChunk = true;
onFirstChunk();
}
speakingEvent.set();
speakingEvent.set();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 — A leading empty chunk does not mean the response contains no playable audio. drainInner continues reading after this first chunk, so an input sequence such as [empty, non-empty, end] emits audio but never calls onFirstChunk() or sets speakingEvent. That leaves the speaking/interrupt state inconsistent for a real response. Keep discarding zero-length prefixes until the first playable chunk (or terminal end), then set the event at that point; add a regression test for an empty chunk followed by audio.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 3380f97.

  • drainInner now marks the first playable chunk wherever it appears, so a leading empty chunk no longer suppresses onFirstChunk() or speakingEvent.set().
  • Added the requested regression for [empty, non-empty, end], while retaining the empty-only assertion.
  • Verified interrupt-truncation.test.ts (12 passed), the Voice suites (436 passed, 1 skipped), and the full JavaScript suite (1036 passed, 4 skipped). Build, lint, typecheck, and CJS/ESM smoke checks also pass.

Could you please re-review?

};
markFirstPlayableChunk(first);

const chunks: AudioChunk[] = [first];
let accumulated = first.durationSeconds;
Expand Down Expand Up @@ -761,6 +767,7 @@ async function drainInner(
terminatedReason = "terminal_chunk";
break;
}
markFirstPlayableChunk(next);
chunks.push(next);
accumulated += next.durationSeconds;
maybeWarnSoftCap();
Expand Down