Skip to content

About

Prototype to do live translations of live transcriptions in BigBlueButton

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

BBB Live Translations

Self-hosted, real-time caption translation for BigBlueButton. Speaker audio is transcribed by Voxtral (speech-to-text on vLLM); this service translates each finalized caption into several languages and the BBB bridge shows them live on per-language caption tracks.

Fully on-premise — no cloud translation APIs, no transcript data leaves your servers.

Non-commercial deployment. The default translation model (NLLB-200) is licensed CC-BY-NC. The code is AGPL-3.0; using it with NLLB also makes the deployment non-commercial. See Licensing.

How it fits together

 BigBlueButton ──▶ LiveKit bridge ──▶ Voxtral (vLLM)   ← speech-to-text
                        │
                        └──────────▶ translation-service  ← this project (NMT)
                                     NLLB-200 · CTranslate2 · GPU

The bridge calls translation-service once per finalized caption and emits the translated captions itself. This service is not in the transcript path: if it is slow or down, original captions are completely unaffected.

Features

  • NLLB-200 translation via CTranslate2 (int8, GPU) — one model, all language pairs.
  • Shares one GPU with Voxtral vLLM (e.g. a single RTX 3090) — no second card needed.
  • Multi-target: translate each caption into several languages in one call.
  • Resilient: per-language failure isolation, timeouts + retry, backpressure — a failure in one language never affects the caption or the others.
  • Private by default: bearer-token auth; transcript/translation text is never logged or persisted.
  • Observable: Prometheus /metrics + a ready-made Grafana dashboard.

Requirements

  • An NVIDIA GPU with ~24 GB (e.g. RTX 3090/4090), shared with Voxtral.
  • NVIDIA driver ≥ 560 (the image is CUDA 12.6) and the NVIDIA Container Toolkit.
  • Docker (or Podman) with Compose.
  • A running Voxtral vLLM and the BBB LiveKit bridge (bbb-livekit-stt).

Deploy

Everything runs from prebuilt images — no source checkout on the server.

# 1. Get the compose file and env template.
curl -O https://raw.githubusercontent.com/virtUOS/bbb-live-translations/main/docker-compose.prod.yml
curl -o .env https://raw.githubusercontent.com/virtUOS/bbb-live-translations/main/.env.prod.example
$EDITOR .env          # set TRANSLATION_API_TOKENS (openssl rand -hex 32) and BIND_IP

# 2. Download + convert the model once (writes ./models, ~5 min).
docker run --rm -v "$PWD/models:/models" \
  ghcr.io/virtuos/bbb-live-translations/model-tools:latest
#   add  --size 600M  for the faster, lower-quality model

# 3. Start Voxtral + translation-service (they share the GPU).
docker compose -f docker-compose.prod.yml up -d

Then point the BBB bridge at the service (on its own server):

VOXTRAL_TRANSLATION_SERVICE_URL=http://<gpu-host>:8001      # or https:// via your proxy
VOXTRAL_TRANSLATION_SERVICE_TOKEN=<same as TRANSLATION_API_TOKENS>
VOXTRAL_TRANSLATION_TARGET_LANGUAGES=de,en,fr,es,pt,uk

Check it's up: curl -H "Authorization: Bearer <token>" http://<gpu-host>:8001/healthz → "model_loaded": true.

Full operations guide (GPU memory tuning, TLS, model swaps, load testing, metrics/Grafana): docs/runbook.md.

Configuration

Set in .env (see .env.prod.example for all options):

Variable Default Purpose
TRANSLATION_API_TOKENS — (required) Bearer token(s), shared with the bridge.
TRANSLATION_TARGET_LANGUAGES de,en,fr,es,pt,uk Languages to translate into.
NLLB_SIZE 1.3B 1.3B (quality) or 600M (latency).
BIND_IP 0.0.0.0 Interface to expose ports on — use a private/VPN address.
VLLM_GPU_MEMORY_UTILIZATION 0.75 vLLM's VRAM share; the rest is for translation.
HF_TOKEN — Only if the Voxtral model is gated for your account.

Languages

Translates into: German, English, French, Spanish, Portuguese, Ukrainian (configurable). Accepts source captions in any of: de, en, es, fr, hi, it, ja, pt, ru, zh, uk. Codes are ISO-639-1.

API

translation-service is an HTTP service (the bridge is its only client):

Endpoint Purpose
POST /translate_batch Translate one caption into multiple languages.
POST /translate Translate a single string.
GET /languages Supported source/target languages.
GET /healthz Model loaded, device, queue depth.
GET /metrics Prometheus metrics.

The request/response format is documented in docs/translation-http-contract.md (useful if you integrate a different bridge or STT frontend).

Security

  • Auth: a bearer token is required on every endpoint except /healthz//metrics.
  • Network: the service is meant to sit on a private/internal network; publish it only to the bridge host, and terminate TLS with your own reverse proxy for anything crossing hosts.
  • Privacy: captions and translations are never written to logs, metrics, or disk — only metadata (ids, languages, latencies).

Licensing

  • Code: GNU AGPL-3.0-or-later. Copyright © 2026 virtUOS, Universität Osnabrück. Because this is a network service, AGPL §13 applies: if you modify it and offer the modified version to users over a network, you must make your modified source available to those users.
  • Models: licensed separately. The default NLLB-200 models are CC-BY-NC-4.0 (non-commercial only) — deploying with them makes your deployment non-commercial. Every downloaded model's license is recorded in models/LICENSES.md.

About

Prototype to do live translations of live transcriptions in BigBlueButton

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages