Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,17 @@ python/outputs/recordings/background_handoff/segments/
!python/outputs/recordings/stt_swap/
python/outputs/recordings/stt_swap/segments/

# TypeScript SDK artifacts — same shape as python/outputs/ above. Generated
# per run and NOT committed, except the whitelisted evidence directories
# below, which are the canonical "prove it" artifacts for reviewers.
# See javascript/outputs/README.md.
javascript/outputs/*
!javascript/outputs/README.md
!javascript/outputs/recordings/
javascript/outputs/recordings/*
!javascript/outputs/recordings/README.md
!javascript/outputs/recordings/issue533_voicestyle/

# TypeScript voice demo outputs (recordings + manifests) — same policy as
# python/outputs/recordings/ above. Regenerated per local run by the @e2e demo tests
# under javascript/examples/vitest/tests/voice/; the committed subset below
Expand Down
13 changes: 9 additions & 4 deletions docs/docs/pages/voice/recipes/effects.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -95,10 +95,15 @@ scenario.user("I'm really upset!", voice_style="angry")
```

<Callout type="info">
When `voice_style` is set, the SDK emits a warning if the active TTS backend
does not yet wire prosody — the audio is still synthesized, but the style hint
is not applied. `audio_effects` post-processing is always applied regardless of
the backend.
In **TypeScript**, `voiceStyle` reaches the TTS provider: ElevenLabs prepends an
inline `[angry]` marker (honoured by the pinned `eleven_v3` model) and OpenAI
passes it as `instructions` on `gpt-4o-mini-tts`. A provider that has not
declared style support warns once and synthesizes unstyled.

In **Python**, `voice_style` is accepted but not yet routed to a provider — the
SDK emits a warning and the audio is synthesized without the style hint.

`audio_effects` post-processing is always applied regardless of the backend.
</Callout>

## Worked example
Expand Down
24 changes: 24 additions & 0 deletions javascript/outputs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# TypeScript SDK artifacts

Sibling of `python/outputs/` — the parent directory for checked-in "prove it"
artifacts produced by the TypeScript SDK. Room for `traces/`, `logs/`,
`screenshots/` later; `recordings/` is the audio subdir and the only one
whitelisted in `.gitignore` today.

Everything under here is generated. Only the explicitly whitelisted
subdirectories are committed, and only because a reviewer needs to hear or see
what a change actually produced against a live provider — a passing test cannot
carry that.

## `recordings/`

| Directory | Recorded | What it proves |
|---|---|---|
| `issue533_voicestyle/` | 2026-07-31 | A `voiceStyle` threaded through the user simulator reaches the real ElevenLabs API and is honoured as a delivery directive rather than spoken aloud (issue #533). Play `bare.wav` against `angry.wav`; `manifest.json` carries the requests, hashes, and Scribe transcripts. |

Regenerate with the harness named in each `manifest.json` — for
`issue533_voicestyle/` that is:

```bash
cd javascript && ELEVENLABS_API_KEY=... npx tsx scripts/voice-style-live-proof.ts
```
44 changes: 44 additions & 0 deletions javascript/outputs/recordings/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# TypeScript SDK recordings

Audio evidence produced by the TypeScript SDK against **live** providers, kept
because a reviewer needs to hear what a change actually produced. Mirrors
`python/outputs/recordings/` — see that directory's README for the original
statement of the policy.

Everything here is generated. Only the directories whitelisted in `.gitignore`
are committed, and each one exists to answer a specific reviewer question that a
passing test cannot.

| Directory | Recorded | What it proves |
|---|---|---|
| `issue533_voicestyle/` | 2026-07-31 | A `voiceStyle` threaded through the user simulator reaches the real ElevenLabs API and is honoured as a *delivery directive* rather than spoken aloud (issue #533). |

## What each directory contains

- The clips themselves, named for the condition they capture (here: `bare.wav`
vs `angry.wav` — same sentence, same voice, same model, one variable).
- `manifest.json` — the exact request text put on the wire, byte counts and
hashes, independent STT transcripts, the checks that ran, and a **`not_proven`
field** naming what the evidence does *not* establish.
- Screenshots of the harness output, so the PR body can embed proof that lives
in-tree rather than on a link that rots.

## Reading the evidence

**Byte-difference is not proof.** ElevenLabs `eleven_v3` is non-deterministic:
two identical requests already return different bytes. Any recording here that
claims "styled differs from unstyled" must discriminate on something stronger —
for `issue533_voicestyle/` that is the STT transcript, because `eleven_v3` is the
only EL model that *consumes* an inline `[angry]` marker as a directive instead
of reading it aloud.

## Regenerating

Each `manifest.json` names its `harness`. For `issue533_voicestyle/`:

```bash
cd javascript && ELEVENLABS_API_KEY=... npx tsx scripts/voice-style-live-proof.ts
```

These harnesses spend real provider credits, so they are hand-run and never part
of `vitest run`.
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
58 changes: 58 additions & 0 deletions javascript/outputs/recordings/issue533_voicestyle/manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
{
"issue": 533,
"recorded": "2026-07-31",
"what_it_proves": "A voiceStyle threaded through the TypeScript simulator reaches the real ElevenLabs API and is honoured as a delivery directive, not spoken as text.",
"provider": "elevenlabs",
"tts_model": "eleven_v3",
"voice_id": "EXAVITQu4vr4xnSDxMaL",
"source_text": "I have been on hold for forty minutes and nobody has helped me.",
"harness": "javascript/scripts/voice-style-live-proof.ts",
"why_bytes_are_not_the_proof": "eleven_v3 is non-deterministic: two identical unstyled requests already return different bytes (168960 vs 161280 in this run). A byte-difference between styled and unstyled therefore proves nothing on its own. The discriminator is the transcript: eleven_v3 is the only ElevenLabs model that CONSUMES an inline [angry] marker as a delivery instruction \u2014 every other model reads it aloud. The styled clip never says 'angry', so the marker was consumed, not voiced.",
"clips": [
{
"file": "bare.wav",
"request_text": "I have been on hold for forty minutes and nobody has helped me.",
"voice_style": null,
"pcm_bytes": 168960,
"pcm_sha256_16": "10bf6a62071bcee5",
"transcript_scribe_v1": "I have been on hold for 40 minutes and nobody has helped me."
},
{
"file": "angry.wav",
"request_text": "[angry] I have been on hold for forty minutes and nobody has helped me.",
"voice_style": "angry",
"pcm_bytes": 195840,
"pcm_sha256_16": "e97d57ff6c8ac594",
"transcript_scribe_v1": "I have been on hold for 40 minutes and nobody has helped me"
}
],
"checks": [
{
"name": "marker not voiced",
"result": "PASS",
"detail": "the styled clip never says \"angry\" or \"[\" \u2014 the marker was consumed as a delivery directive"
},
{
"name": "line still intact",
"result": "PASS",
"detail": "the styled clip still speaks the source line, so nothing was swallowed"
},
{
"name": "cache does not collide",
"result": "PASS",
"detail": "router bare sha=d448e4345a1e1170 vs angry sha=5eb13fc16bbbd2c7"
},
{
"name": "unstyled entry survives the styled call",
"result": "PASS",
"detail": "re-read of the unstyled key returned the cached bare bytes, not the styled ones"
}
],
"format": {
"container": "wav",
"encoding": "pcm_s16le",
"sample_rate_hz": 24000,
"channels": 1
},
"not_proven": "Whether the styled clip SOUNDS angry to a human ear. No automated check can assert timbre \u2014 play both clips to judge that."
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
163 changes: 163 additions & 0 deletions javascript/scripts/voice-style-live-proof.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
/**
* Live ElevenLabs proof for issue #533 — `voiceStyle` on the wire.
*
* Not a test: it costs real ElevenLabs credits, so it is a hand-run harness,
* never part of `vitest run`. Regenerates the checked-in evidence under
* `javascript/outputs/recordings/issue533_voicestyle/`.
*
* ELEVENLABS_API_KEY=... npx tsx scripts/voice-style-live-proof.ts
*
* Same text, same voice, twice: bare vs voiceStyle="angry", against the REAL
* ElevenLabs API.
*
* Byte-difference alone proves nothing: `eleven_v3` is non-deterministic, so
* two identical requests already return different bytes. The real
* discriminator is what the audio SAYS. `eleven_v3` is the only EL model that
* CONSUMES an inline `[angry]` marker as a delivery instruction; every other
* model READS IT ALOUD as text. So:
*
* - if the styled clip's transcript contains "angry" -> the marker was
* spoken, i.e. treated as text, i.e. the mapping is wrong;
* - if it does not, and the transcript still matches the source line, the
* marker was consumed as a directive and never voiced -> the style
* reached the wire and was honoured as a style.
*
* Transcription is done by ElevenLabs Scribe (the SDK's own STT leaf), so the
* check is independent of the TTS request we are asserting about.
*/
import { createHash } from "node:crypto";
import { mkdirSync, writeFileSync } from "node:fs";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";

import { AudioChunk } from "../src/voice/audio-chunk";
import { ElevenLabsSTTProvider } from "../src/voice/stt";
import { synthesize, clearTtsCache } from "../src/voice/tts";
import { elevenLabsSynthesizeBytes } from "../src/voice/tts/elevenlabs-tts";

const OUT_DIR = resolve(
dirname(fileURLToPath(import.meta.url)),
"..",
"outputs",
"recordings",
"issue533_voicestyle",
);
const BARE_OUT = resolve(OUT_DIR, "bare.wav");
const ANGRY_OUT = resolve(OUT_DIR, "angry.wav");

const TEXT = "I have been on hold for forty minutes and nobody has helped me.";
const VOICE_ID = process.env.ELEVENLABS_VOICE_ID ?? "EXAVITQu4vr4xnSDxMaL";

function sha(bytes: Uint8Array): string {
return createHash("sha256").update(bytes).digest("hex").slice(0, 16);
}

// PCM16/24kHz mono -> minimal WAV so GitHub renders an inline player.
function toWav(pcm: Uint8Array, sampleRate = 24000): Buffer {
const header = Buffer.alloc(44);
header.write("RIFF", 0);
header.writeUInt32LE(36 + pcm.length, 4);
header.write("WAVE", 8);
header.write("fmt ", 12);
header.writeUInt32LE(16, 16);
header.writeUInt16LE(1, 20); // PCM
header.writeUInt16LE(1, 22); // mono
header.writeUInt32LE(sampleRate, 24);
header.writeUInt32LE(sampleRate * 2, 28);
header.writeUInt16LE(2, 32);
header.writeUInt16LE(16, 34);
header.write("data", 36);
header.writeUInt32LE(pcm.length, 40);
return Buffer.concat([header, Buffer.from(pcm)]);
}

let failures = 0;
function check(label: string, ok: boolean, detail: string) {
console.log(`${ok ? "PASS" : "FAIL"} — ${label}: ${detail}`);
if (!ok) failures += 1;
}

async function main() {
if (!process.env.ELEVENLABS_API_KEY) {
console.error("ELEVENLABS_API_KEY is not set — cannot run the live proof");
process.exit(2);
}
console.log(`text : ${JSON.stringify(TEXT)}`);
console.log(`voice : elevenlabs/${VOICE_ID} (model eleven_v3)`);
console.log("");

// --- Control: is the API even deterministic? -----------------------------
const bareA = await elevenLabsSynthesizeBytes(TEXT, VOICE_ID, {});
const bareB = await elevenLabsSynthesizeBytes(TEXT, VOICE_ID, {});
console.log(
`control bare#1 bytes=${bareA.length} sha=${sha(bareA)} | ` +
`bare#2 bytes=${bareB.length} sha=${sha(bareB)}`,
);
console.log(
sha(bareA) === sha(bareB)
? " (deterministic — byte-diff would be meaningful)"
: " (NON-deterministic — byte-diff proves nothing; see transcripts below)",
);
console.log("");

// --- The styled call -----------------------------------------------------
const angry = await elevenLabsSynthesizeBytes(TEXT, VOICE_ID, {
voiceStyle: "angry",
});
mkdirSync(OUT_DIR, { recursive: true });
writeFileSync(BARE_OUT, toWav(bareA));
writeFileSync(ANGRY_OUT, toWav(angry));
console.log(`bare bytes=${bareA.length} sha=${sha(bareA)} -> ${BARE_OUT}`);
console.log(`angry bytes=${angry.length} sha=${sha(angry)} -> ${ANGRY_OUT}`);
console.log("");

// --- The real discriminator: what does the audio SAY? --------------------
const stt = new ElevenLabsSTTProvider();
const bareText = await stt.transcribe(new AudioChunk({ data: bareA }));
const angryText = await stt.transcribe(new AudioChunk({ data: angry }));
console.log(`transcript(bare) = ${JSON.stringify(bareText)}`);
console.log(`transcript(angry) = ${JSON.stringify(angryText)}`);
console.log("");

const spokeMarker = /angry|\[/i.test(angryText);
check(
"marker not voiced",
!spokeMarker,
spokeMarker
? "the styled clip SPEAKS the marker — eleven_v3 read it as text, not as a directive"
: 'the styled clip never says "angry" or "[" — the marker was consumed as a delivery directive',
);
// Scribe normalizes number words to digits ("forty" -> "40"), so accept both.
const carriesLine = /hold for (forty|40) minutes/i.test(angryText);
check(
"line still intact",
carriesLine,
carriesLine
? "the styled clip still speaks the source line, so nothing was swallowed"
: "the styled clip lost the source line",
);

// --- Router level: does the cache keep styled and unstyled apart? --------
clearTtsCache();
const rBare = await synthesize(TEXT, `elevenlabs/${VOICE_ID}`);
const rAngry = await synthesize(TEXT, `elevenlabs/${VOICE_ID}`, undefined, {
voiceStyle: "angry",
});
const rBareAgain = await synthesize(TEXT, `elevenlabs/${VOICE_ID}`);
check(
"cache does not collide",
sha(rBare.data) !== sha(rAngry.data),
`router bare sha=${sha(rBare.data)} vs angry sha=${sha(rAngry.data)}`,
);
check(
"unstyled entry survives the styled call",
sha(rBareAgain.data) === sha(rBare.data),
`re-read of the unstyled key sha=${sha(rBareAgain.data)} (cache hit, not overwritten by the styled call)`,
);

console.log("");
console.log(failures === 0 ? "ALL CHECKS PASSED" : `${failures} CHECK(S) FAILED`);
process.exit(failures === 0 ? 0 : 1);
}

void main();
Loading
Loading