feat(voice): wire voiceStyle through _synthesize to the TTS providers by drewdrewthis · Pull Request #863 · langwatch/scenario

drewdrewthis · 2026-07-31T13:23:58Z

Scariest thing — the TTS cache key. voiceStyle changes the synthesized audio, so if it were left out of the key an angry turn and a neutral turn with the same (text, voice) would share one entry and whichever ran first would silently serve its bytes to the other. That is a wrong-audio-in-production bug that no type error catches. It is in the key (tts.ts cacheKey), and mutation M4 below proves the test that guards it is load-bearing.
Start here — javascript/src/voice/tts/elevenlabs-tts.ts → applyVoiceStyle. That function is where this PR departs from the issue's literal wording, and the doc comment on it is the argument for why.

Why

UserSimulatorAgent.setOneShotOverride({ voiceStyle }) set _voiceStyleOverride, emitted a one-shot console.warn, and then dropped the value — no TTS provider ever saw it. A scenario author writing user("I'm really upset!", { voiceStyle: "angry" }) got cheerful audio and a warning. This threads the style all the way to the wire and re-enables the spec scenario that was retagged @todo in #528.

Closes #533

What changed

voiceStyle is a per-call TtsSynthesisOptions field, not part of the voice string → alternative was folding it into the litellm provider/name identity (elevenlabs/rachel:angry). Rejected: the style varies per utterance while the voice does not, so baking it into the identity would fragment the provider registry and make a one-turn override a different backend. Consequence: TTSCallable grew an optional third parameter — source-compatible, and synthesizeRaw only widens the call when an option is actually set, so a pre-existing toHaveBeenCalledWith(text, voice) assertion and any third-party 2-arg provider keep their exact call shape.
ElevenLabs maps the style to an inline [angry] marker, not to voiceSettings.style → the issue said "ElevenLabs style field". That field is a numeric 0–1 exaggeration knob, so the string "angry" has nowhere to go, and mapping a name onto a number would be inventing a meaning the API never promised. eleven_v3 — already the pinned model, for exactly this reason — consumes inline paralinguistic markers. Consequence: this is the deviation most worth a second opinion; if you want the numeric knob exposed too, that is an additive follow-up, not a rework.
OpenAI maps the style to instructions, rather than "ignoring it with a warning" → the issue said OpenAI TTS has no style support and "needs design". That is out of date: gpt-4o-mini-tts, the pinned model, exposes instructions for precisely this. Consequence: the feature works on the default provider instead of only on ElevenLabs. The warn-and-ignore path the issue asked for still exists — it now fires for any provider that has not declared supportsVoiceStyle, which is the honest home for it.
A style the backend cannot honour is loud, not silent → alternative was passing it through and letting the provider ignore it. Rejected: silent unstyled audio is the exact failure this issue was filed about. Consequence: one console.warn per provider per process, then the style is stripped (only the style — any other option still reaches the backend).

How it works

script/voice-steps.ts  withUserStepOverride()      installs the one-shot override for one turn
  └── agents/user-simulator-agent.ts
        ├── effectiveVoiceStyle()                  precedence: one-shot > cfg.voiceStyle > voice.tts.voiceStyle
        └── _synthesize(text, voice, voiceStyle)    ← the gap #533 was filed about
              └── voice/tts/tts.ts  synthesize(text, voice, effectFn?, { voiceStyle })
                    ├── cacheKey(sha256(text), voice, voiceStyle)   styled ≠ unstyled entry
                    └── synthesizeRaw → registry
                          ├── supportsVoiceStyle: true  → provider gets the option
                          │     ├── elevenlabs-tts.ts  applyVoiceStyle → "[angry] <text>"  (idempotent)
                          │     └── openai-tts.ts      instructions: "Speak in a angry tone."
                          └── not declared → warn once per provider, strip the style, synthesize unstyled

_voiceStyleOverride === null keeps its existing meaning of "no override installed" — not "force unstyled" — so between steps the configured defaults still apply.

Test plan

cd javascript && npx tsc --noEmit && npx vitest run

Test Files 97 passed | 1 skipped (98) · Tests 1101 passed | 4 skipped (1105) · tsc exit 0.

New tests (35 across 6 files, all #533-tagged):

File	Covers
`src/agents/__tests__/user-simulator-voice-style.test.ts`	override → `_synthesize` arg 3 and revert; config default; override-beats-config; per-run `voice.tts.voiceStyle`; sim-beats-per-run; `voiceifyText` scripted path
`src/voice/tts/__tests__/synthesize-voice-style.test.ts`	option reaches the provider; cache does not collide across styles; repeat of the same triple is a cache hit
`src/voice/tts/__tests__/elevenlabs-voice-style.test.ts`	`[angry]` prepend, bare text without a style, no double-prepend
`src/voice/tts/__tests__/openai-voice-style.test.ts`	`instructions` present with a style, key absent without one
`src/voice/tts/__tests__/voice-style-unsupported-warning.test.ts`	warns once per provider, names provider + style, per-provider not global, never for an opted-in provider
`src/agents/__tests__/user-simulator-voice.test.ts`	the re-bound spec scenario (AC-3 + AC-4)

Lint: every file this branch touches is eslint-clean (exit 0). main carries ~122 pre-existing errors in files this PR does not touch, so the claim is a zero delta, not absolute green — two pre-existing import/order errors in a touched test file were fixed on the way past.

Human verification

git fetch && git checkout issue533/voicestyle-override && cd javascript && pnpm install
npx vitest run src/voice/tts/__tests__ src/agents/__tests__/user-simulator-voice-style.test.ts → 6 files, 35 tests green.
Confirm the tests are load-bearing, don't take my word for it. Break the production line and watch them go red:
sed -i 's/this._synthesize(text, voice, voiceStyle)/this._synthesize(text, voice)/' src/agents/user-simulator-agent.ts → re-run → 7 failures. git checkout -- src/agents/user-simulator-agent.ts to restore.
Hear it. Play both clips below. bare.wav is 3.52 s, angry.wav is 4.08 s — same sentence, same voice, same model.
Reproduce the live run yourself: ELEVENLABS_API_KEY=... npx tsx scripts/voice-style-live-proof.ts → regenerates the two clips and re-runs all four checks.

How I can prove I was successful

1. The style reaches the real ElevenLabs API and is honoured — proven

Run this session against the live API via javascript/scripts/voice-style-live-proof.ts.

Byte-difference is deliberately NOT the proof. The control line shows eleven_v3 is non-deterministic — two identical unstyled requests returned 168960 vs 161280 bytes. So "styled bytes ≠ unstyled bytes" would have been a fake proof, and this is the trap the harness is built to avoid.

The real discriminator is what the audio says. eleven_v3 is the only ElevenLabs model that consumes an inline marker as a delivery instruction; every other model reads it aloud as text. Transcribing both clips with ElevenLabs Scribe (an independent path from the TTS request being asserted about):

clip	request text on the wire	Scribe transcript
`bare.wav`	`I have been on hold for forty minutes…`	`"I have been on hold for 40 minutes and nobody has helped me."`
`angry.wav`	`[angry] I have been on hold for forty minutes…`	`"I have been on hold for 40 minutes and nobody has helped me"`

The styled clip never says "angry" and still speaks the full line → the marker was consumed as a directive, not voiced. Plus, through the router: the cache did not collide, and re-reading the unstyled key returned the cached bare bytes rather than the styled ones.

Play them — this is the part no assertion can make:

bare.wav
angry.wav
manifest.json — requests, hashes, transcripts, and an explicit not_proven field.

2. The tests actually fail without the code — proven

Green tests are not evidence, so I mutated production five ways and re-ran. Every mutant was killed, and the tree was restored and re-verified green afterwards:

Mutation	Tests killed
M1 — simulator drops `voiceStyle` before `_synthesize` (the original #533 bug)	7
M2 — `applyVoiceStyle` becomes identity	1
M3 — OpenAI drops the `instructions` mapping	1
M4 — cache key drops `voiceStyle`	5
M5 — unsupported-provider warning removed	2

M1 restores the exact pre-PR behaviour and turns the re-bound spec scenario red — which is the direct evidence that AC-3 and AC-4 are wired to real behaviour and not to a tautology.

3. Acceptance criteria — all four proven

#	Issue AC	Commit	Evidence
1	Thread `_voiceStyleOverride` + base `voiceStyle` into `_synthesize` args	`f3a8148`	mutation M1 kills 7 tests; `effectiveVoiceStyle` precedence covered 6 ways
2	Provider-specific mapping; others ignore with a warning	`70d282e`	mutations M2/M3/M5; live EL transcript; OpenAI `instructions` present/absent
3	Re-tag the spec scenario `@unit @ts-simulator`	`b308c9a`	`specs/voice-agents.feature` — Given/When/Then left byte-identical so the binding matches
4	Re-add the binding, asserting the stubbed `_synthesize` and the revert	`3665db6`	the scenario now runs; M1 turns it red

4. No user-observable surface

This is a backend-only SDK change with no UI surface — the observable surface is the synthesized audio, which is what §1 exercises against the live provider.

Anything surprising?

Bisect hazard. Commit b308c9a re-tags the spec scenario one commit before 3665db6 adds its binding, so that single commit has a matched-but-unbound scenario and a red suite. This is inherent to the requested commit-per-criterion order — HEAD and every other commit are green.
Python is untouched and still warns-and-drops. Issue feat(simulator): wire voiceStyle override through _synthesize #533 scoped TypeScript only. docs/docs/pages/voice/recipes/effects.mdx was updated to state the split explicitly rather than imply parity, and the Python half is tracked in feat(python): wire voice_style through _synthesize (parity with TS #533) #862.
Not proven: that angry.wav sounds angry to a human. No automated check can assert timbre. Everything above proves the marker reached ElevenLabs and was consumed as a directive; whether the delivery is convincing is a judgement for your ears, which is why both clips are checked in.
javascript/outputs/ is new, mirroring python/outputs/'s gitignore-whitelist shape. Only issue533_voicestyle/ is whitelisted; everything else under it stays ignored.

`setOneShotOverride({ voiceStyle })` installed a style, the simulator logged "no TTS provider currently honours it", and then called `_synthesize(text, voice)` — the style was dropped on the floor (#533). Wire it end to end: - `TtsSynthesisOptions { voiceStyle? }` — per-call synthesis options that are NOT part of the voice identity. `TTSCallable` widens to a third optional argument, so existing two-argument providers stay assignable, and `synthesize()` forwards it to the provider. - The LRU cache key becomes `(sha256(text), voice, voiceStyle)`. Without the style in the key, an angry turn and a neutral turn with the same text+voice share one entry and whichever ran first silently serves its bytes to the other. Effects still stay OUT of the key. - `UserSimulatorVoiceConfig.voiceStyle` and `TtsConfig.voiceStyle` give a simulator-level and a per-run default; `effectiveVoiceStyle()` resolves them with the same precedence shape as `effectiveVoice`/`effectiveAudioEffects` — one-shot override, then simulator config, then per-run config. A null override still means "none installed", not "force unstyled". - Delete `warnVoiceStyleOnce`: the warning it emitted is now false. The "provider ignores the style" case moves to the TTS router in the next commit. TS leads Python here — `python/scenario/voice/tts.py` still has no style channel, so this is explicitly NOT parity. The router forwards `options` only when one is actually set, keeping the unstyled path's historical two-argument call shape for providers (and spies) that observe arity. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWeYLrzADoJe5EJS87Lg4M

… instructions) The style now reaches `_synthesize`, but no backend did anything with it. Give each provider a real style channel and make the gap visible where one does not exist: - `TtsProvider.supportsVoiceStyle` — providers opt in. The registry keeps the flag alongside the callable, and `synthesizeRaw` warns ONCE per provider (naming the provider and the style) before stripping the style for one that has not opted in. Silence would be worse than the old wrong warning: a scripted `user("…", { voiceStyle: "angry" })` would return cheerful audio with nothing saying why. - ElevenLabs: prepend an inline `[angry]` marker to the text. EL's `voiceSettings.style` is a NUMERIC 0-1 exaggeration knob, not a named style, so `"angry"` has nowhere to go there; `eleven_v3` — already the pinned model for exactly this reason — DOES honour inline paralinguistic markers, so that is the mechanism that makes a named style audible. Idempotent, so a caller who hand-wrote `[angry] …` is not double-prepended. - OpenAI: `gpt-4o-mini-tts` takes an `instructions` parameter for delivery control. The key is omitted entirely when no style is set, so the unstyled request stays byte-for-byte the one this callable always sent. This is the one documented exception to the file's "swap the whole callable rather than parameterise it" rule — voiceStyle is per-utterance, not per-backend. `__resetVoiceStyleWarnings()` is an @internal test seam, kept separate from `clearTtsCache()` because a suite that clears the cache between cases must not silently re-arm the warning it is asserting fires exactly once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWeYLrzADoJe5EJS87Lg4M

@todo

The scenario was @todo because "no TTS backend changes timbre by style yet". With the ElevenLabs and OpenAI style channels wired, that is no longer true — re-tag it @ts-simulator and point the comment at the binding. Given/When/Then text is byte-identical; the test binds on those literal strings. Also corrects the two other places that repeated the stale claim: the user-simulator-voice test docstring, and the effects recipe's callout, which told every reader the style hint is never applied. That callout now splits the two SDKs explicitly rather than claiming a parity that does not exist — TypeScript routes the style to the provider, Python still warns and drops it. The binding itself lands in the next commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWeYLrzADoJe5EJS87Lg4M

Binds `Per-step voice override applies to only that step`, closing the @ts-simulator set at 5. The `_synthesize` stub records `(text, voice, voiceStyle)` per turn, so the assertion is on what the TTS layer was actually asked for — a scenario that only checked the audio came back would have passed against the old code that dropped the style. Widens the shared `stubSynth` helper to the three-argument `_synthesize` signature. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWeYLrzADoJe5EJS87Lg4M

…dence A passing unit test proves the `[angry]` marker is put on the request; it cannot prove ElevenLabs honours it. `scripts/voice-style-live-proof.ts` runs the real API and discriminates on the transcript, not on bytes — `eleven_v3` is non-deterministic, so two identical unstyled requests already differ. It is the only EL model that CONSUMES an inline marker as a delivery instruction; every other model reads it aloud. The styled clip never says "angry", so the marker was consumed, not voiced. Checked-in evidence under `javascript/outputs/recordings/issue533_voicestyle/` (bare.wav, angry.wav, manifest.json, screenshots) so a reviewer can hear the difference — the one thing no assertion can make. `javascript/outputs/` gets the same gitignore whitelist shape as `python/outputs/`. Also fixes the import/order lint errors in the two touched test files (two of them pre-existing), keeping a zero lint delta on everything this branch edits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWeYLrzADoJe5EJS87Lg4M

coderabbitai · 2026-07-31T13:24:09Z

Warning

Review limit reached

@drewdrewthis, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 36 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d6081b16-a3fb-4ec0-ad42-921e20588541

📥 Commits

Reviewing files that changed from the base of the PR and between 88aec40 and dbbbb88.

⛔ Files ignored due to path filters (4)

javascript/outputs/recordings/issue533_voicestyle/angry.wav is excluded by !**/*.wav
javascript/outputs/recordings/issue533_voicestyle/bare.wav is excluded by !**/*.wav
javascript/outputs/recordings/issue533_voicestyle/live-el-proof.png is excluded by !**/*.png
javascript/outputs/recordings/issue533_voicestyle/mutation-proof.png is excluded by !**/*.png

📒 Files selected for processing (20)

.gitignore
docs/docs/pages/voice/recipes/effects.mdx
javascript/outputs/README.md
javascript/outputs/recordings/README.md
javascript/outputs/recordings/issue533_voicestyle/manifest.json
javascript/scripts/voice-style-live-proof.ts
javascript/src/agents/__tests__/user-simulator-voice-style.test.ts
javascript/src/agents/__tests__/user-simulator-voice.test.ts
javascript/src/agents/user-simulator-agent.ts
javascript/src/voice/config.ts
javascript/src/voice/index.ts
javascript/src/voice/tts/__tests__/elevenlabs-voice-style.test.ts
javascript/src/voice/tts/__tests__/openai-voice-style.test.ts
javascript/src/voice/tts/__tests__/synthesize-voice-style.test.ts
javascript/src/voice/tts/__tests__/voice-style-unsupported-warning.test.ts
javascript/src/voice/tts/elevenlabs-tts.ts
javascript/src/voice/tts/index.ts
javascript/src/voice/tts/openai-tts.ts
javascript/src/voice/tts/tts.ts
specs/voice-agents.feature

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

…olicy `python/outputs/` carries two READMEs — a parent index and an audio policy at `recordings/`. The javascript side had only the parent. The audio-policy one is where the load-bearing warning belongs: byte-difference is NOT proof, because eleven_v3 is non-deterministic, so any recording claiming "styled differs from unstyled" has to discriminate on something stronger. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWeYLrzADoJe5EJS87Lg4M

rogeriochaves

there are a bunch of screenshots for the pr that shouldnt be commited here, we should push pr screenshots to langwatch/pr-screenshots instead

github-actions · 2026-08-12T06:42:32Z

Automated low-risk assessment

This PR was evaluated against the repository's Low-Risk Pull Requests procedure and does not qualify as low risk.

The diff changes the TTS integration surface and runtime behavior: it threads a per-call voiceStyle through the synth pipeline, alters provider call signatures/mappings (ElevenLabs inline markers, OpenAI instructions), and changes the TTS cache key to include voiceStyle. These are changes to integrations with third‑party systems and to production audio behaviour (including adding a live proof harness and checked-in recordings), which the low‑risk policy explicitly disallows.

This PR requires a manual review before merging.

Ubuntu and others added 5 commits July 31, 2026 12:56

drewdrewthis self-assigned this Jul 31, 2026

drewdrewthis requested a review from rogeriochaves July 31, 2026 13:24

drewdrewthis marked this pull request as ready for review July 31, 2026 13:24

langwatch-agent added review: targeted PR Hound review mode hound-checked Triaged by the pr-hound agent at the current head SHA labels Aug 3, 2026

prajvalrasik mentioned this pull request Aug 9, 2026

feat(python): wire voice_style through _synthesize (parity with TS #533) #862

Open

4 tasks

langwatch-agent added the ci-green Latest run of every check is passing (checks API, not the legacy commit-status index) label Aug 11, 2026

rogeriochaves requested changes Aug 11, 2026

View reviewed changes

langwatch-agent added the P3 - low Low priority, nice to have label Aug 12, 2026

Navigation Menu

Uh oh!

feat(voice): wire voiceStyle through _synthesize to the TTS providers - #863

feat(voice): wire voiceStyle through _synthesize to the TTS providers#863
drewdrewthis wants to merge 6 commits into
mainfrom
issue533/voicestyle-override

drewdrewthis commented Jul 31, 2026

Uh oh!

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading

Review limit reached

Uh oh!

rogeriochaves left a comment

Uh oh!

github-actions Bot commented Aug 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

drewdrewthis commented Jul 31, 2026

Why

What changed

How it works

Test plan

Human verification

How I can prove I was successful

1. The style reaches the real ElevenLabs API and is honoured — proven

2. The tests actually fail without the code — proven

3. Acceptance criteria — all four proven

4. No user-observable surface

Anything surprising?

Uh oh!

coderabbitai Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

rogeriochaves left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading