Self-hosted, real-time caption translation for BigBlueButton. Speaker audio is transcribed by Voxtral (speech-to-text on vLLM); this service translates each finalized caption into several languages and the BBB bridge shows them live on per-language caption tracks.
Fully on-premise — no cloud translation APIs, no transcript data leaves your servers.
Non-commercial deployment. The default translation model (NLLB-200) is licensed CC-BY-NC. The code is AGPL-3.0; using it with NLLB also makes the deployment non-commercial. See Licensing.
BigBlueButton ──▶ LiveKit bridge ──▶ Voxtral (vLLM) ← speech-to-text
│
└──────────▶ translation-service ← this project (NMT)
NLLB-200 · CTranslate2 · GPU
The bridge calls translation-service once per finalized caption and emits the
translated captions itself. This service is not in the transcript path: if it is
slow or down, original captions are completely unaffected.
- NLLB-200 translation via CTranslate2 (int8, GPU) — one model, all language pairs.
- Shares one GPU with Voxtral vLLM (e.g. a single RTX 3090) — no second card needed.
- Multi-target: translate each caption into several languages in one call.
- Resilient: per-language failure isolation, timeouts + retry, backpressure — a failure in one language never affects the caption or the others.
- Private by default: bearer-token auth; transcript/translation text is never logged or persisted.
- Observable: Prometheus
/metrics+ a ready-made Grafana dashboard.
- An NVIDIA GPU with ~24 GB (e.g. RTX 3090/4090), shared with Voxtral.
- NVIDIA driver ≥ 560 (the image is CUDA 12.6) and the NVIDIA Container Toolkit.
- Docker (or Podman) with Compose.
- A running Voxtral vLLM and the BBB
LiveKit bridge (
bbb-livekit-stt).
Everything runs from prebuilt images — no source checkout on the server.
# 1. Get the compose file and env template.
curl -O https://raw.githubusercontent.com/virtUOS/bbb-live-translations/main/docker-compose.prod.yml
curl -o .env https://raw.githubusercontent.com/virtUOS/bbb-live-translations/main/.env.prod.example
$EDITOR .env # set TRANSLATION_API_TOKENS (openssl rand -hex 32) and BIND_IP
# 2. Download + convert the model once (writes ./models, ~5 min).
docker run --rm -v "$PWD/models:/models" \
ghcr.io/virtuos/bbb-live-translations/model-tools:latest
# add --size 600M for the faster, lower-quality model
# 3. Start Voxtral + translation-service (they share the GPU).
docker compose -f docker-compose.prod.yml up -dThen point the BBB bridge at the service (on its own server):
VOXTRAL_TRANSLATION_SERVICE_URL=http://<gpu-host>:8001 # or https:// via your proxy
VOXTRAL_TRANSLATION_SERVICE_TOKEN=<same as TRANSLATION_API_TOKENS>
VOXTRAL_TRANSLATION_TARGET_LANGUAGES=de,en,fr,es,pt,ukCheck it's up: curl -H "Authorization: Bearer <token>" http://<gpu-host>:8001/healthz
→ "model_loaded": true.
Full operations guide (GPU memory tuning, TLS, model swaps, load testing,
metrics/Grafana): docs/runbook.md.
Set in .env (see .env.prod.example for all options):
| Variable | Default | Purpose |
|---|---|---|
TRANSLATION_API_TOKENS |
— (required) | Bearer token(s), shared with the bridge. |
TRANSLATION_TARGET_LANGUAGES |
de,en,fr,es,pt,uk |
Languages to translate into. |
NLLB_SIZE |
1.3B |
1.3B (quality) or 600M (latency). |
BIND_IP |
0.0.0.0 |
Interface to expose ports on — use a private/VPN address. |
VLLM_GPU_MEMORY_UTILIZATION |
0.75 |
vLLM's VRAM share; the rest is for translation. |
HF_TOKEN |
— | Only if the Voxtral model is gated for your account. |
Translates into: German, English, French, Spanish, Portuguese, Ukrainian
(configurable). Accepts source captions in any of: de, en, es, fr, hi, it, ja, pt, ru, zh, uk. Codes are ISO-639-1.
translation-service is an HTTP service (the bridge is its only client):
| Endpoint | Purpose |
|---|---|
POST /translate_batch |
Translate one caption into multiple languages. |
POST /translate |
Translate a single string. |
GET /languages |
Supported source/target languages. |
GET /healthz |
Model loaded, device, queue depth. |
GET /metrics |
Prometheus metrics. |
The request/response format is documented in
docs/translation-http-contract.md (useful if
you integrate a different bridge or STT frontend).
- Auth: a bearer token is required on every endpoint except
/healthz//metrics. - Network: the service is meant to sit on a private/internal network; publish it only to the bridge host, and terminate TLS with your own reverse proxy for anything crossing hosts.
- Privacy: captions and translations are never written to logs, metrics, or disk — only metadata (ids, languages, latencies).
- Code: GNU AGPL-3.0-or-later. Copyright © 2026 virtUOS, Universität Osnabrück. Because this is a network service, AGPL §13 applies: if you modify it and offer the modified version to users over a network, you must make your modified source available to those users.
- Models: licensed separately. The default NLLB-200 models are CC-BY-NC-4.0
(non-commercial only) — deploying with them makes your deployment non-commercial.
Every downloaded model's license is recorded in
models/LICENSES.md.