Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions packages/client/src/internal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,15 @@ export {
} from "./utils/location.js";
export { sourceInfo, setSourceInfo } from "./sourceInfo.js";
export type { SourceInfo } from "./sourceInfo.js";
export { setSetupStrategy } from "./platform/VoiceSessionSetup.js";
export { webSessionSetup } from "./platform/web/VoiceSessionSetup.js";
export {
setSetupStrategy,
setupWebRTCSession,
} from "./platform/VoiceSessionSetup.js";
export type {
VoiceSessionSetupStrategy,
VoiceSessionSetupResult,
} from "./platform/VoiceSessionSetup.js";
export { createConnection } from "./utils/ConnectionFactory.js";
export {
MIN_VOICE_FREQUENCY,
MAX_VOICE_FREQUENCY,
Expand Down
72 changes: 72 additions & 0 deletions packages/client/src/platform/VoiceSessionSetup.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
import { describe, it, expect, vi } from "vitest";

vi.mock("livekit-client", () => ({
Room: vi.fn(),
RoomEvent: {},
Track: { Kind: { Audio: "audio" }, Source: { Microphone: "microphone" } },
ConnectionState: {},
createLocalAudioTrack: vi.fn(),
}));

import { setupWebRTCSession } from "./VoiceSessionSetup.js";
import { WebRTCConnection } from "../utils/WebRTCConnection.js";
import { WebSocketConnection } from "../utils/WebSocketConnection.js";

describe("setupWebRTCSession", () => {
it("returns input/output from a WebRTCConnection", () => {
const mockInput = { close: vi.fn(), setMuted: vi.fn() };
const mockOutput = { close: vi.fn(), setVolume: vi.fn() };

// Create a minimal object that passes the instanceof check
const connection = Object.create(WebRTCConnection.prototype, {
input: { value: mockInput },
output: { value: mockOutput },
});

const result = setupWebRTCSession(connection);

expect(result.connection).toBe(connection);
expect(result.input).toBe(mockInput);
expect(result.output).toBe(mockOutput);
expect(result.playbackEventTarget).toBeNull();
expect(result.detach).toBeTypeOf("function");
});

it("detach is a no-op", () => {
const connection = Object.create(WebRTCConnection.prototype, {
input: { value: {} },
output: { value: {} },
});

const result = setupWebRTCSession(connection);
expect(() => result.detach()).not.toThrow();
});

it("throws when given a WebSocketConnection", () => {
const connection = Object.create(WebSocketConnection.prototype);

expect(() => setupWebRTCSession(connection)).toThrow(
"setupWebRTCSession requires a WebRTCConnection"
);
});

it("throws when given a plain object", () => {
const connection = { input: {}, output: {} } as any;

expect(() => setupWebRTCSession(connection)).toThrow(
"setupWebRTCSession requires a WebRTCConnection"
);
});

it("throws with a descriptive message when given null", () => {
expect(() => setupWebRTCSession(null as any)).toThrow(
"Received: object"
);
});

it("throws with a descriptive message when given undefined", () => {
expect(() => setupWebRTCSession(undefined as any)).toThrow(
"Received: undefined"
);
});
});
24 changes: 24 additions & 0 deletions packages/client/src/platform/VoiceSessionSetup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import type { BaseConnection } from "../utils/BaseConnection.js";
import type { InputController } from "../InputController.js";
import type { OutputController } from "../OutputController.js";
import type { PlaybackEventTarget } from "../OutputController.js";
import { WebRTCConnection } from "../utils/WebRTCConnection.js";

export type VoiceSessionSetupResult = {
connection: BaseConnection;
Expand Down Expand Up @@ -30,3 +31,26 @@ export let setupStrategy: VoiceSessionSetupStrategy | undefined;
export function setSetupStrategy(strategy: VoiceSessionSetupStrategy) {
setupStrategy = strategy;
}

/**
* Sets up a voice session for a WebRTC connection.
* Platform-agnostic: extracts the input/output controllers that
* WebRTCConnection already provides via LiveKit.
*/
export function setupWebRTCSession(
connection: BaseConnection
): VoiceSessionSetupResult {
if (!(connection instanceof WebRTCConnection)) {
throw new Error(
"setupWebRTCSession requires a WebRTCConnection. " +
`Received: ${connection?.constructor?.name ?? typeof connection}`
);
}
return {
connection,
input: connection.input,
output: connection.output,
playbackEventTarget: null,
detach: () => {},
};
}
82 changes: 38 additions & 44 deletions packages/client/src/platform/web/VoiceSessionSetup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import type { Options } from "../../BaseConversation.js";
import type { BaseConnection } from "../../utils/BaseConnection.js";

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in Phase 4 (b4f451288eaf14): the web setup file was rewritten and BaseConnection is no longer imported. WebRTCConnection is still used for the instanceof check in the WebRTC branch.

import {
setSetupStrategy,
setupWebRTCSession,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused WebRTCConnection import after refactoring

Low Severity

WebRTCConnection is imported as a value on line 11 but is never referenced in the file after the instanceof WebRTCConnection check was moved into setupWebRTCSession. The project has @typescript-eslint/no-unused-vars disabled and noUnusedLocals is not set, so no tooling catches this. The type-only BaseConnection import on line 2 is also unused.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit d51ad5f. Configure here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in Phase 4 (88eaf14): the web setup file was rewritten — WebRTCConnection is used again (for the instanceof check in the WebRTC branch) and BaseConnection is no longer imported.

type VoiceSessionSetupResult,
} from "../VoiceSessionSetup.js";
import { MediaDeviceOutput } from "./output.js";
Expand All @@ -13,53 +14,40 @@ import { attachConnectionToOutput } from "../../utils/attachConnectionToOutput.j
import { createConnection } from "../../utils/ConnectionFactory.js";

/**
* Sets up input and output controllers for an existing connection.
* Shared helper used by platform-specific setup strategies.
* Sets up WebSocket-specific input and output controllers using
* web MediaDevice APIs (AudioContext, AudioWorklet, etc.).
*/
export async function setupInputOutput(
async function setupWebSocketIO(
options: Options,
connection: BaseConnection
connection: WebSocketConnection
): Promise<Omit<VoiceSessionSetupResult, "connection">> {
if (connection instanceof WebRTCConnection) {
return {
input: connection.input,
output: connection.output,
playbackEventTarget: null,
detach: () => {},
};
} else if (connection instanceof WebSocketConnection) {
const [input, output] = await Promise.all([
MediaDeviceInput.create({
...connection.inputFormat,
preferHeadphonesForIosDevices: options.preferHeadphonesForIosDevices,
inputDeviceId: options.inputDeviceId,
workletPaths: options.workletPaths,
libsampleratePath: options.libsampleratePath,
}),
MediaDeviceOutput.create({
...connection.outputFormat,
outputDeviceId: options.outputDeviceId,
workletPaths: options.workletPaths,
}),
]);
const [input, output] = await Promise.all([
MediaDeviceInput.create({
...connection.inputFormat,
preferHeadphonesForIosDevices: options.preferHeadphonesForIosDevices,
inputDeviceId: options.inputDeviceId,
workletPaths: options.workletPaths,
libsampleratePath: options.libsampleratePath,
}),
MediaDeviceOutput.create({
...connection.outputFormat,
outputDeviceId: options.outputDeviceId,
workletPaths: options.workletPaths,
}),
]);

const detachInput = attachInputToConnection(input, connection);
const detachOutput = attachConnectionToOutput(connection, output);
const detachInput = attachInputToConnection(input, connection);
const detachOutput = attachConnectionToOutput(connection, output);

return {
input,
output,
playbackEventTarget: output,
detach: () => {
detachInput();
detachOutput();
},
};
} else {
throw new Error(
`Unsupported connection type: ${connection.constructor.name}`
);
}
return {
input,
output,
playbackEventTarget: output,
detach: () => {
detachInput();
detachOutput();
},
};
}

/**
Expand All @@ -70,8 +58,14 @@ export async function webSessionSetup(
options: Options
): Promise<VoiceSessionSetupResult> {
const connection = await createConnection(options);
const io = await setupInputOutput(options, connection);
return { connection, ...io };

if (connection instanceof WebSocketConnection) {
const io = await setupWebSocketIO(options, connection);
return { connection, ...io };
}

// WebRTC — platform-agnostic setup
return setupWebRTCSession(connection);
}

// Register the web strategy as the default
Expand Down
20 changes: 17 additions & 3 deletions packages/react-native/src/index.react-native.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ import {
import type { Options } from "@elevenlabs/client";
import {
setSetupStrategy,
webSessionSetup,
createConnection,
setupWebRTCSession,
type VoiceSessionSetupResult,
} from "@elevenlabs/client/internal";
import { attachNativeVolume } from "./nativeVolume.js";
Expand All @@ -18,13 +19,25 @@ registerGlobals();
* React Native voice session setup strategy.
*
* 1. Configures and starts the native AudioSession
* 2. Delegates connection + input/output setup to the web strategy
* 2. Creates a WebRTC connection and extracts its I/O controllers
* 3. Wraps input/output controllers with native volume processors
* 4. Wraps detach to stop the native AudioSession on cleanup
*
* Only WebRTC connections are supported on React Native.
* WebSocket connections require Web Audio APIs (AudioContext,
* AudioWorkletNode) that are not available in React Native.
*/
async function reactNativeSessionSetup(
options: Options
): Promise<VoiceSessionSetupResult> {
if (options.connectionType === "websocket" || options.signedUrl) {
throw new Error(
"WebSocket connections are not supported on React Native. " +
"Only WebRTC connections are available. " +
"Remove the connectionType/signedUrl option or use connectionType: 'webrtc'."
);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

React Native guard misses textOnly WebSocket path

Medium Severity

The early guard intended to catch unsupported WebSocket connections on React Native only checks for options.connectionType === "websocket" and options.signedUrl, but createConnection also infers "websocket" when textOnly: true is set (via determineConnectionType in ConnectionFactory.ts). Passing { agentId: "...", textOnly: true } bypasses the guard, starts the AudioSession, then fails inside setupWebRTCSession with a confusing "requires a WebRTCConnection" error instead of the friendly "WebSocket connections are not supported on React Native" message.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit b4f4512. Configure here.


await AudioSession.configureAudio({
android: {
preferredOutputList: ["speaker"],
Expand All @@ -36,7 +49,8 @@ async function reactNativeSessionSetup(
});
await AudioSession.startAudioSession();

const result = attachNativeVolume(await webSessionSetup(options));
const connection = await createConnection(options);
const result = attachNativeVolume(setupWebRTCSession(connection));

Comment on lines +52 to 54

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b4f4512: added an early guard that checks for connectionType: "websocket" or signedUrl and throws a clear RN-specific error: "WebSocket connections are not supported on React Native." Also updated the doc comment to document the limitation.

const originalDetach = result.detach;
return {
Expand Down
Loading