Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -495,6 +495,9 @@ jobs:
dimos/e2e_tests/test_robot_picker_browser.py
dimos/e2e_tests/test_relay_auth_browser.py --no-cov

- name: Relay image build # docker/relay/Dockerfile must not rot silently
run: docker build -f docker/relay/Dockerfile .

tests:
if: |
!cancelled() &&
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ build/
/web/cockpit/.dist-*
/web/sdk/.dist-*

# Hosted relay: certificate and auth file next to docker/relay/compose.yaml
/docker/relay/config/

# Nix / direnv (symlink one of the tracked .envrc.* files to .envrc)
.direnv/
.envrc
Expand Down
27 changes: 27 additions & 0 deletions docker/relay/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# The hosted relay: web/ built into one image, run as the image's unprivileged
# deno user (uid 1993) with the certificate and auth file mounted at /etc/relay.
# Guide: docs/usage/relay_hosting.md; compose.yaml next to this file runs it.
# The Deno pin lives in dimos/utils/deno.py (CI greps it there); keep them equal.
FROM denoland/deno:2.6.10

# A server has no use for the daily version check against dl.deno.land.
ENV DENO_NO_UPDATE_CHECK=1
WORKDIR /app/web
# The build context is the repo root; Dockerfile.dockerignore next to this
# file narrows it to web/ (the root .dockerignore would drop package.json).
# WORKDIR created /app/web as root and --chown covers only the copied entries,
# so the directory itself needs the chown for deno to create node_modules.
COPY --chown=deno:deno web/ /app/web
RUN chown deno:deno /app/web
USER deno
RUN deno install --frozen && deno task -r build

EXPOSE 443/tcp 443/udp
# Docker lets the unprivileged user bind 443 (ip_unprivileged_port_start=0
# inside containers since 20.10). CMD holds only the file flags, so a compose
# `command:` can override those without repeating the rest.
ENTRYPOINT ["deno", "run", "--frozen", "--allow-net", "--allow-read=/app/web,/etc/relay", \
"relay/main.ts", "--host", "0.0.0.0", "--port", "443", \
"--cockpit-dir", "cockpit/dist", "--sdk-dir", "sdk/dist"]
CMD ["--cert", "/etc/relay/fullchain.pem", "--key", "/etc/relay/privkey.pem", \
"--auth-file", "/etc/relay/auth.json"]
15 changes: 15 additions & 0 deletions docker/relay/Dockerfile.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Replaces the repo-root .dockerignore for docker/relay/Dockerfile (BuildKit
# prefers <Dockerfile>.dockerignore): that one drops package.json, which the
# web build needs. Only web/ is copied; skip build products and local secrets.
*
!web
web/**/node_modules
web/**/dist
web/**/.dist-*
web/**/.build.lock
web/**/.env
web/**/.env.*
web/**/auth.json
web/**/*.pem
web/**/*.key
web/**/*.crt
18 changes: 18 additions & 0 deletions docker/relay/compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# The hosted relay (docs/usage/relay_hosting.md). From a checkout:
# docker compose -f docker/relay/compose.yaml up -d --build
# config/ (gitignored) holds fullchain.pem, privkey.pem, and auth.json, owned
# by uid 1993 (the image's deno user). The relay reads them at startup only.
services:
relay:
image: dimos-relay
build:
context: ../..
dockerfile: docker/relay/Dockerfile
ports:
- "443:443/tcp"
- "443:443/udp"
volumes:
- ./config:/etc/relay:ro
restart: unless-stopped
# The relay closes its sessions on SIGTERM and exits at once.
stop_grace_period: 5s
162 changes: 162 additions & 0 deletions docs/usage/relay_hosting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# Relay hosting

The relay on your own server, under your own name: robots dial out to it with a key, operators open it in a browser with a token, and the cockpit works from anywhere. This guide takes a fresh Ubuntu VM and a domain to `https://dimos-relay.example.com`. Section 9 does the same on a LAN without a domain. The relay itself needs nothing new; everything here is packaging: a container image ([`docker/relay/Dockerfile`](/docker/relay/Dockerfile)), a compose file ([`docker/relay/compose.yaml`](/docker/relay/compose.yaml)), and the steps around them. The flags it uses are explained in [Web SDK](/docs/usage/web_sdk.md).

## 1. What you get, and what you do not

- One relay process on one machine. The registry is in memory: after a restart every robot re-registers by itself within seconds and every open page reconnects.
- No accounts. The auth file holds robot keys, each bound to one robot id, and viewer tokens. A viewer token does everything (view, teleop, publish) on every robot behind the relay, so hand tokens out like passwords.
- The relay terminates TLS itself, on one port for HTTPS (TCP) and QUIC (UDP). No reverse proxy: an HTTP proxy cannot carry QUIC, and the relay advertises the WebTransport URL from the host the client dialed.
- The certificate and the auth file are read at startup. Changing either means a restart, which costs a few seconds of reconnection.

## 2. DNS and firewall

Point an A record at the VM (`dimos-relay.example.com -> <public IP>`). Open 443 TCP, 443 UDP, and 80 TCP for certbot (nothing listens on 80 between renewals). The cloud provider's own firewall (security group) needs the same rules; UDP 443 is the one people forget.

```bash
sudo ufw allow 22/tcp && sudo ufw allow 80/tcp && sudo ufw allow 443/tcp && sudo ufw allow 443/udp
sudo ufw enable
```

Without UDP 443 there is no WebTransport and, by design, no fallback: the page loads and never connects.

## 3. Certificate

Clone dimos onto the VM (the image is built from the checkout) and install Docker and certbot. Request the certificate with a deploy hook that installs the two PEM files into the relay's config directory, owned by uid 1993 (the image's `deno` user), and restarts the relay:

```bash
sudo apt install -y git certbot docker.io docker-compose-v2
sudo git clone https://github.com/dimensionalOS/dimos /srv/dimos
sudo mkdir /srv/dimos/docker/relay/config
sudo certbot certonly --standalone -d dimos-relay.example.com --deploy-hook \
'install -o 1993 -g 1993 -m 600 "$RENEWED_LINEAGE"/fullchain.pem "$RENEWED_LINEAGE"/privkey.pem /srv/dimos/docker/relay/config/ && docker compose -f /srv/dimos/docker/relay/compose.yaml restart relay'
```

The hook runs now (so `config/` holds `fullchain.pem` and `privkey.pem`; the restart is a no-op until the relay exists) and certbot saves it in the renewal configuration: every automatic renewal (certbot's timer, roughly every 60 days) installs the new files and restarts the relay, which reads certificates only at startup. `sudo certbot renew --dry-run` checks renewal but skips hooks, so rehearse the restart once by running the hook command by hand after step 5.

## 4. Auth file

`config/auth.json` maps robot ids to keys and viewer names to tokens, each 16 to 256 characters and no secret used twice. Generate them with `openssl rand -hex 32`:

```json
{
"robots": { "go2-lab": "<key>" },
"viewers": { "paul": "<token>" }
}
```

```bash
sudo chown 1993:1993 /srv/dimos/docker/relay/config/auth.json
sudo chmod 600 /srv/dimos/docker/relay/config/auth.json
```

A robot id here has to match the robot's `--robot-id` (its hostname when not given). Adding or rotating a secret is an edit plus `docker compose restart relay`. The relay refuses a malformed file with a message that names the entry, never the secret, and it never logs secrets.

## 5. Run

```bash
cd /srv/dimos/docker/relay
sudo docker compose up -d --build
curl https://dimos-relay.example.com/api/info
sudo docker compose logs -f relay
```

The first build takes a few minutes: Deno installs the web dependencies and builds the SDK and the cockpit inside the image. `/api/info` answers `{"wtUrl":"https://dimos-relay.example.com","v":...}`. The log opens with the ready line and then shows connections (`[relay] robot connected`, `[relay] viewer 1 authenticated as paul`), rejections with their reason (`[relay] robot go2-lab rejected: invalid robot key`), and `[relay] robot go2-lab disconnected` when one leaves. `restart: unless-stopped` brings the relay back after a reboot. A new dimos version is `git pull` and the same `up -d --build`.

## 6. Robot

The robot needs the key bound to its id, from the environment or the `.env` file of the checkout it runs from (the `--relay-key` flag exists but shows in the process list):

```bash
RELAY_KEY=<key> dimos run unitree-go2 --relay-url https://dimos-relay.example.com --robot-id go2-lab
```

It dials out, so it works behind NAT with no open ports. A wrong key is logged once as `auth_failed` and the bridge gives up rather than retrying: fix `RELAY_KEY` and start again.

## 7. Operator

Open `https://dimos-relay.example.com/`. The page asks for a viewer token, keeps it in the browser's local storage, and "Log out" in the status bar forgets it. With several robots registered the page lists them; "switch robot" reopens the list. `/api/stats` (per-robot, per-channel, and per-viewer counters, with viewer names) takes the same token as a bearer:

```bash
curl -H "Authorization: Bearer <token>" https://dimos-relay.example.com/api/stats
```

## 8. Without Docker

The same relay under systemd: the pinned Deno, a checkout owned by a service user, the built dists, and the right to bind 443.

```bash
sudo apt install -y git unzip
curl -fsSLo /tmp/deno.zip https://github.com/denoland/deno/releases/download/v2.6.10/deno-x86_64-unknown-linux-gnu.zip
sudo unzip -o -d /usr/local/bin /tmp/deno.zip
sudo useradd --system --create-home --home /srv/relay relay
sudo -u relay git clone https://github.com/dimensionalOS/dimos /srv/relay/dimos
sudo -u relay sh -c 'cd /srv/relay/dimos/web && deno install --frozen && deno task -r build'
sudo mkdir /etc/relay
```

The pin is `DENO_VERSION` in [`dimos/utils/deno.py`](/dimos/utils/deno.py#L32); an arm64 VM downloads `deno-aarch64-unknown-linux-gnu.zip` instead. Put `auth.json` in `/etc/relay` owned by `relay` with mode 600, and use `install -o relay -g relay -m 600 "$RENEWED_LINEAGE"/fullchain.pem "$RENEWED_LINEAGE"/privkey.pem /etc/relay/ && systemctl restart dimos-relay` as the certbot deploy hook. `/etc/systemd/system/dimos-relay.service`:

```ini
[Unit]
Description=dimos relay
After=network-online.target

[Service]
User=relay
WorkingDirectory=/srv/relay/dimos/web
ExecStart=/usr/local/bin/deno run --frozen --allow-net --allow-read=/srv/relay/dimos/web,/etc/relay relay/main.ts --host 0.0.0.0 --port 443 --cockpit-dir cockpit/dist --sdk-dir sdk/dist --cert /etc/relay/fullchain.pem --key /etc/relay/privkey.pem --auth-file /etc/relay/auth.json
AmbientCapabilities=CAP_NET_BIND_SERVICE
Restart=on-failure

[Install]
WantedBy=multi-user.target
```

`sudo systemctl enable --now dimos-relay`, then `journalctl -u dimos-relay -f` for the log.

## 9. LAN without a domain

On a LAN with no public name, sign your own certificate with mkcert. It creates a private CA and a certificate for the relay's address. The robot trusts the CA through `--relay-ca`. The auth file stays mandatory: the relay binds a non-loopback address only with certificate, key, and auth file together. The image and the compose file are the same as on a VM. Only the certificate step changes. `192.168.1.20` below is the relay machine's LAN address.

```bash
sudo apt install -y mkcert libnss3-tools
mkcert -install
mkdir -p docker/relay/config
mkcert -cert-file docker/relay/config/fullchain.pem -key-file docker/relay/config/privkey.pem 192.168.1.20
```

`libnss3-tools` is what lets `mkcert -install` add the CA to Firefox. Write `auth.json` as in section 4, give all three files to uid 1993 the same way, and start the relay as in section 5.

The robot needs the CA bundle: its QUIC client trusts certifi's roots, not the system store.

```bash
RELAY_KEY=<key> uv run dimos --simulation run unitree-go2-agentic-cockpit --relay-url https://192.168.1.20 \
--relay-ca "$(mkcert -CAROOT)/rootCA.pem" --robot-id go2-lab
```

Browsers are the catch. Firefox trusts the certificate for the page but drops QUIC through a root that is not built in. In `about:config` set `network.http.http3.disable_when_third_party_roots_found` to `false` and restart Firefox. Then open `https://192.168.1.20/` and enter the viewer token. Chromium refuses private roots for QUIC outright (`QUIC_CERT_ROOT_NOT_KNOWN`; no flag lifts it): the page loads over HTTPS and never connects. Chromium on a LAN needs a publicly trusted certificate, which a domain you own provides without any inbound port: point a name at the LAN address and use certbot's DNS challenge (`certbot certonly --manual --preferred-challenges dns -d relay.lan.example.com`).

Without Docker, section 8's command works with the mkcert files in place of the certbot ones.

## 10. Troubleshooting

- **`auth_failed`.** The relay's message says `missing` (the client sent no secret: `RELAY_KEY` not in the robot's environment, no token stored in the browser) or `invalid` (wrong secret, or a robot key used with another robot id). It is terminal: the bridge stops and the page shows the token form. Fix the secret and start the client again. The relay log names the rejected robot id or viewer.
- **`accept failed: aborted by peer: the application or application protocol caused the connection to be closed during the handshake`**, repeating every 8 s, with no `viewer N authenticated` line. A browser is retrying and closing the connection itself after the TLS check: with a private CA, Firefox without the `about:config` pref from section 9, or Chromium. The page shows "Waiting for a robot to register..." meanwhile, which also covers a page still connecting; the status bar shows the real state. The robot and `/api/stats` are unaffected.
- **Certificate errors.** Check the VM clock (`timedatectl`). The robot fails with `CERTIFICATE_VERIFY_FAILED` when the relay serves `cert.pem` instead of `fullchain.pem` (browsers fetch the missing intermediate, OpenSSL does not) or when `--relay-ca` points at the wrong CA.
- **UDP 443 blocked.** HTTPS works, the page loads and stays "reconnecting", the robot's startup attempts time out (`relay startup connection attempt N failed`) and it gives up. Check both firewalls for UDP; `sudo ss -ulnp | grep 443` on the VM shows the published UDP port.
- **Robot behind NAT.** Fine: it dials out, and the QUIC session sends keepalives every 4 s.
- **Relay behind NAT** (a home server). Forward TCP 443 and UDP 443 to it. `/api/info` advertises the name the client dialed, so nothing else changes.
- **Restarts.** `docker compose restart relay` is always safe: robots re-register within seconds, pages reconnect on their own. A robot killed without a clean close holds its id for up to 30 s (the QUIC idle timeout); its restart waits that out.

## 11. Checklist

What a hosted relay is verified with; repeat it after changes to the relay, the image, or this guide.

- [ ] Fresh Ubuntu VM with a domain, sections 2 to 5 top to bottom; `curl https://.../api/info` from a laptop.
- [ ] Replay robot on another, NATed network (`dimos --replay run unitree-go2 --relay-url ...`).
- [ ] Laptop on a third network, in Chromium and in Firefox: wrong token rejected, right token connects; robot picker with two robots; live map and video; teleop drives the replay robot.
- [ ] `docker compose restart relay` with the page open: the page reconnects, the robot re-registers.
- [ ] The deploy hook by hand: the relay restarts with the installed files; `sudo certbot renew --dry-run` passes.
- [ ] `/api/stats`: 401 without the token, 200 with it.
- [ ] Section 9 on a LAN: the robot registers with `--relay-ca`, Firefox with the pref connects, Chromium loads the page and never connects.
2 changes: 2 additions & 0 deletions docs/usage/web_sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ RELAY_KEY=<key> uv run dimos run unitree-go2 --relay-url https://dimos-relay.exa

The cockpit asks for the viewer token, keeps it in `localStorage`, and "log out" in the status bar forgets it. Your own page passes it to `connect({ url, token })`. `/api/stats` wants it as `Authorization: Bearer <token>` and drops its CORS header. A wrong key or token fails with `auth_failed` and neither client retries: fix the secret and restart. Edits to the file need a relay restart.

To host one on a VM with Docker and a Let's Encrypt certificate, or on a LAN with mkcert, follow [Relay hosting](/docs/usage/relay_hosting.md).

## Your first page

Create `ui/index.html`:
Expand Down
4 changes: 4 additions & 0 deletions web/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,10 @@ until "log out"; the SDK takes `connect({url, token})`). A wrong secret fails wi
which is terminal: neither client retries. `relay/auth.ts` compares in constant time and never logs
a secret; edits to the file need a restart.

Hosting a relay on a VM (container image, compose file, certbot, firewall, the auth file) is
[docs/usage/relay_hosting.md](../docs/usage/relay_hosting.md); `docker/relay/` holds the Dockerfile
and the compose file.

## Cockpit

```bash
Expand Down
Loading