Skip to content

feat: add voice providers and fix research provider mappings - #5

Merged
dittops merged 1 commit into
mainfrom
feat/add-voice-providers
Sep 21, 2026
Merged

dittops merged 1 commit into
mainfrom
feat/add-voice-providers

Conversation

@dittops

@dittops dittops commented Sep 21, 2026

Copy link
Copy Markdown
Member

What

Adds speech-to-text and text-to-speech models to the catalog, and fixes two mapping bugs found while validating the merge against live upstream data.

1. Voice providers

Adds deepgram, elevenlabs, assemblyai, aws_polly and groq to LITELLM_TO_TENSORZERO / TENSORZERO_PROVIDERS, with matching STRIP_PREFIXES entries (all five key their LiteLLM entries as {provider}/{model}, so the prefix is stripped rather than doubled). deepgram, elevenlabs and groq also map into LITELLM_TO_RESEARCH, where truefoundry/models carries cost data for them.

63 voice models now in the catalog:

Provider Models Modes
deepgram 42 audio_transcription
groq 10 chat 6, audio_speech 2, audio_transcription 2
elevenlabs 5 audio_transcription 3, audio_speech 2
aws_polly 4 audio_speech
assemblyai 2 audio_transcription

e.g. deepgram/nova-3 → input_cost_per_second: 7.167e-05; elevenlabs/eleven_multilingual_v2 → input_cost_per_character: 0.0001.

2. Fix: azure and xai research mappings

LITELLM_TO_RESEARCH pointed at azure-openai and x-ai; the real truefoundry/models directories are azure-open-ai and xai. The lookup silently never matched, so both providers kept LiteLLM's costs and skipped deprecation filtering entirely.

Fixing it matches 117 azure and 55 xai models.

3. Fix: coerce string costs from research YAML

truefoundry writes small costs as 5e-7. YAML 1.1 requires a decimal point in scientific notation, so PyYAML parses those as strings — and the merge was replacing LiteLLM's numeric cost with a string, breaking any downstream arithmetic.

This already affected 166 values on main (openai, anthropic, gemini, mistral, together_ai). The azure/xai fix above would have pushed it to 259. Now 0.

Why

The voice providers were the ask; the two fixes fell out of verifying it. The string-cost bug is pre-existing on main and independent of the voice work — it just gets worse without the fix, so it ships here.

Reviewer notes

Two output changes that will look like regressions in a naive diff but are intended:

  • 41 azure models disappear from the catalog. They're deprecated and were never being filtered, because the typo meant the lookup never matched. Catalog total still rises: 1438 → 1460.
  • cost_fields_updated drops 366 → 133. The string-vs-float mismatch was counting '5e-7' != 5e-7 as an update on every merge. The stat now reflects real changes.

Deliberately not included:

  • sarvam and tencent — on the voice provider list and present in LiteLLM, but all four entries are mode: chat, not audio. Adding them would put chat models under providers that bud-connect routes to WaaV.
  • moonshotai → moonshot-ai is left as-is. Not a typo — upstream has no moonshot directory at all.
  • Cartesia, LMNT, Murf, PlayHT, Speechmatics and the rest of the voice list have no entries in either upstream source.

groq is added whole, which brings 15 chat models alongside its 5 audio ones. They're real Groq models with real pricing; filtering a provider to audio-only seemed the stranger choice. Easy to gate if you'd rather.

Downstream

This affects bud-connect's NO_MODEL_PROVIDERS (constants.py). Those five providers now have catalog models, so the seeder's model walk reaches them and the explicit entries become redundant — harmless, but the comment's premise no longer holds for them, and tests/test_voice_providers.py likely needs updating. Separate repo, separate change.

Testing

Verified end-to-end against live LiteLLM and truefoundry/models data, before/after diffed.

  • 70 tests pass (9 new), merger.py at 100% coverage
  • ruff check and mypy clean

Note: the dev machine only has Python 3.9 and the project requires ≥3.10, so the suite was run against a shimmed copy outside the repo. CI covers 3.10–3.13 properly.

🤖 Generated with Claude Code

Adds speech-to-text and text-to-speech models to the catalog, and fixes
two mapping bugs found while validating the merge against live upstream
data.

Voice providers
---------------
Adds deepgram, elevenlabs, assemblyai, aws_polly and groq to
LITELLM_TO_TENSORZERO / TENSORZERO_PROVIDERS, with matching STRIP_PREFIXES
entries (all five key their LiteLLM entries as `{provider}/{model}`).
deepgram, elevenlabs and groq also map into LITELLM_TO_RESEARCH, where
truefoundry/models carries cost data for them.

This brings 63 voice models into the catalog: deepgram 42, groq 10,
elevenlabs 5, aws_polly 4, assemblyai 2, priced per second or per
character.

sarvam and tencent are deliberately left out — they appear in LiteLLM but
every entry is `mode: chat`, not audio. Cartesia and the other vendors on
the voice list have no entries in either upstream source.

Fix: azure and xai research mappings
------------------------------------
LITELLM_TO_RESEARCH pointed at `azure-openai` and `x-ai`; the real
truefoundry/models directories are `azure-open-ai` and `xai`. The lookup
silently never matched, so both providers kept LiteLLM's costs and skipped
deprecation filtering entirely. Fixing it matches 117 azure and 55 xai
models, and correctly drops 41 stale azure entries.

`moonshotai` -> `moonshot-ai` is left as-is: not a typo, upstream has no
moonshot directory at all.

Fix: coerce string costs from research YAML
-------------------------------------------
truefoundry writes small costs as `5e-7`. YAML 1.1 requires a decimal
point in scientific notation, so PyYAML parses those as strings and the
merge replaced LiteLLM's numeric cost with a string, breaking downstream
arithmetic. This already affected 166 values at HEAD (openai, anthropic,
gemini, mistral, together_ai) and the azure/xai fix above would have
pushed it to 259. Now 0.

Verified end-to-end against live LiteLLM and truefoundry data.
Catalog total 1438 -> 1460. 70 tests pass (9 new), merger.py at 100%
coverage, ruff check and mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dittops
dittops merged commit bef44ac into main Sep 21, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant