Read when:
- adding or debugging tailnet reachability for a lease;
- deciding whether a host is provider-owned or only network-reachable;
- changing SSH, VNC, or coordinator bootstrap behavior over a tailnet.
Tailscale is an optional reachability layer, not a provider. Providers still own the machines (Hetzner, AWS, Azure, Google Cloud, Proxmox, static SSH hosts, and the rest). Tailscale only changes which host Crabbox dials when it runs SSH-backed work against a lease.
- Managed Linux leases can join a tailnet at creation time with
--tailscale. Tailscale provisioning is offered by the providers in the compiled capability catalog below. - Static (
provider=ssh) hosts are not joined by Crabbox; pointstatic.hostat a MagicDNS name or100.xaddress and assert reachability with--network tailscale. --tailscaleis rejected for non-Linux targets, for Blacksmith Testbox (Blacksmith owns connectivity), for static hosts, and for sandbox providers such as Sprites that expose SSH through their own proxy.
The compiled provider catalog declares Tailscale support for 11 providers.
- Bidirectional tailnet peers (including pond transport): aws, azure, digitalocean, gcp, hetzner, lambda, linode, ovh, scaleway, tencentcloud.
- Egress-only userspace access: islo. These providers do not advertise a bidirectional pond tailnet endpoint.
Create a managed Linux lease that joins the configured tailnet:
crabbox warmup --tailscale
crabbox run --tailscale -- pnpm test
crabbox run --tailscale --desktop --browser -- pnpm test:e2eChoose the connection path when acting on an existing lease (ssh, vnc,
screenshot, webvnc, status, inspect, and reused run --id leases):
crabbox ssh --id swift-crab --network auto
crabbox ssh --id swift-crab --network tailscale
crabbox vnc --id swift-crab --network tailscale --open
crabbox run --id swift-crab --network public -- pnpm test--network (config key network, default auto) selects how the SSH endpoint is
resolved:
auto: prefer the tailnet host when the lease carries Tailscale metadata and SSH is reachable there; otherwise fall back to the provider/public host.tailscale: require a tailnet host and fail clearly when this client cannot reach it over SSH.public: force the provider/public host (useful for debugging).
When auto falls back to the public host, Crabbox prints network fallback tailscale_unreachable and reports the selected path in ready/status output
(ready ssh=… network=public …) instead of silently switching.
network: auto
tailscale:
enabled: true
tags:
- tag:crabbox
hostnameTemplate: crabbox-{slug}
authKeyEnv: CRABBOX_TAILSCALE_AUTH_KEY
exitNode: build-host.example.ts.net
exitNodeAllowLanAccess: truetailscale.enabled (or --tailscale) requests a tailnet join for newly created
managed Linux leases. tailscale.network is an alias that sets the top-level
network mode used for target resolution on SSH-backed commands. Hostname templates
support {id}, {slug}, and {provider}; the rendered value is sanitized to a DNS
label and cut to 63 characters without a trailing hyphen. When enabled is set,
tags must contain at least one valid tag: value and
hostnameTemplate must be non-empty. A tag starts with tag: followed by an ASCII
letter, then only ASCII letters, digits, or dashes (for example tag:ci-1).
Underscores and names starting with digits or dashes are rejected before provisioning.
Environment overrides:
CRABBOX_TAILSCALE=1
CRABBOX_NETWORK=auto|tailscale|public
CRABBOX_TAILSCALE_TAGS=tag:crabbox,tag:ci
CRABBOX_TAILSCALE_HOSTNAME_TEMPLATE=crabbox-{slug}
CRABBOX_TAILSCALE_AUTH_KEY_ENV=CRABBOX_TAILSCALE_AUTH_KEY
CRABBOX_TAILSCALE_AUTH_KEY=<direct-provider only>
CRABBOX_TAILSCALE_EXIT_NODE=build-host.example.ts.net
CRABBOX_TAILSCALE_EXIT_NODE_ALLOW_LAN_ACCESS=1
Flag equivalents: --tailscale, --network, --tailscale-tags,
--tailscale-hostname-template, --tailscale-auth-key-env, --tailscale-exit-node,
and --tailscale-exit-node-allow-lan-access.
In direct-provider mode the auth key is read from the variable named by
tailscale.authKeyEnv (default CRABBOX_TAILSCALE_AUTH_KEY). Managed VM
providers normally use a one-off key. Islo requires a reusable, ephemeral key
because its snapshot-safe memory-only node identity must re-enroll after daemon
loss. Brokered mode does not need a local key; the coordinator mints one per lease
(see below).
Direct warmup and run refresh the lease's Tailscale address and readiness state in provider metadata, so later CLI processes can resolve the same tailnet endpoint. Refreshes preserve lifecycle state and expiry. A failed metadata read remains a warning; on Hetzner, diagnostic labels are shortened to the provider's allowed syntax so they cannot block subsequent lifecycle writes. Hetzner drops an FQDN longer than its 63-character label limit and uses the recorded tailnet IPv4 instead.
GCP stores address fields with a reversible, chunked label encoding. Each chunk
fits GCP's 63-character limit, including for long MagicDNS names. Previously
sanitized labels cannot be decoded unambiguously; run a metadata refresh (for
example, a run over --network public) to replace them from the guest.
The coordinator rejects metadata writes on released or expired leases and after confirmed provider cleanup. CLI refreshes also bind updates to the observed lease creation timestamp, preventing stale metadata from updating a replacement lease. Older API clients may omit this timestamp; their writes still receive the serialized cleanup and lifecycle checks.
Exit-node egress is opt-in per lease. The lease routes its outbound internet through an approved tailnet exit node after it joins Tailscale:
crabbox warmup --tailscale --tailscale-exit-node build-host.example.ts.net --tailscale-exit-node-allow-lan-access
crabbox run --tailscale --tailscale-exit-node 100.100.100.100 -- curl -4 https://ifconfig.meexitNode requires tailscale.enabled: true or --tailscale; otherwise Crabbox
rejects the request before provisioning so the requested egress route cannot be
silently omitted. exitNodeAllowLanAccess maps to Tailscale's LAN-access flag and
requires exitNode, even when Tailscale is disabled.
These dependencies are checked after lease-creation flags are resolved. Config
and environment settings can keep exit-node defaults with Tailscale disabled
and enable them per lease with --tailscale; unrelated commands such as
config show, list, and status can still load those defaults.
The exit node must already advertise exit-node capability and be approved in the
Tailscale admin console, and the tailnet policy must grant the lease's tags (for
example tag:crabbox) access to autogroup:internet through exit nodes.
Fresh nodes join the tailnet before selecting the exit node, so MagicDNS names can resolve against their peers. Enrollment completes only after selection succeeds. Islo recovery preserves a stopped daemon's exit-node preferences and keeps an incomplete selection unavailable to later workloads until a retry succeeds.
When an exit node is selected, network: auto prefers the tailnet host for bootstrap
because the provider/public SSH path can become asymmetric once the lease routes
outbound traffic through the exit node. Before sync or the remote command, Crabbox
verifies on the box that an exit node is selected in tailscale debug prefs and that the
node can reach the public internet. If that check fails, the run stops and reports the
exit-node egress failure (tailscale exit node … joined but remote internet egress failed), which usually means the exit node is not approved, the policy does not grant
autogroup:internet to the lease tag, or the exit-node machine is not forwarding.
When a lease is created through the coordinator, it mints a fresh auth key per lease using Tailscale OAuth. Secrets live in coordinator configuration:
CRABBOX_TAILSCALE_CLIENT_ID
CRABBOX_TAILSCALE_CLIENT_SECRET
CRABBOX_TAILSCALE_TAILNET optional, defaults to "-"
CRABBOX_TAILSCALE_TAGS default/allowed comma-separated tags (default tag:crabbox)
CRABBOX_TAILSCALE_ENABLED set 0 to force-disable, 1 to force-enable
CRABBOX_TAILSCALE_INSTALL_MODE package or pinned (default package)
CRABBOX_TAILSCALE_VERSION pinned static build version
CRABBOX_TAILSCALE_SHA256_AMD64 pinned amd64 archive checksum
CRABBOX_TAILSCALE_SHA256_ARM64 pinned arm64 archive checksum
Lease API requests that omit tailscaleTags or send an empty list use every tag
in CRABBOX_TAILSCALE_TAGS. Explicit non-empty lists must be a subset of those
allowed tags. The CLI sends its configured tags explicitly; this API default does
not override them.
The OAuth client's tags and each requested lease tag must also satisfy Tailscale's ownership rules:
- Single tag: give the OAuth client
tag:crabboxand setCRABBOX_TAILSCALE_TAGS=tag:crabbox. The tag sets match exactly. - Multi-tag exact match: an OAuth client with
tag:ci,tag:stagingcan mint a key requesting both tags together without extra ownership rules. - Multi-tag subsets: requesting only
tag:cifrom that two-tag client requirestag:cito own itself intagOwners; do the same for every subset tag. - Deployment-owner pattern: preferably give the OAuth client one narrow owner tag,
such as
tag:crabbox-deployer, and make that tag an owner of each workload tag that Crabbox may request.
Example deployment-owner policy:
{
"tagOwners": {
"tag:crabbox-deployer": ["autogroup:admin"],
"tag:ci": ["tag:crabbox-deployer"],
"tag:staging": ["tag:crabbox-deployer"]
}
}Then assign only tag:crabbox-deployer to the OAuth client and set
CRABBOX_TAILSCALE_TAGS=tag:ci,tag:staging. This keeps the client least-privilege
without requiring it to carry every workload tag. Do not broaden the OAuth client to
unrelated tags just to resolve an ownership error.
Flow:
- The CLI sends the requested Tailscale settings (enabled, tags, hostname, optional
exit node) in
CreateLease. - The coordinator validates the requested tags against
CRABBOX_TAILSCALE_TAGSand rejects any malformed tag or tag outside that set before minting a key or creating a machine. Explicit tag lists are never silently filtered; an empty list keeps the configured-default behavior. - The coordinator exchanges the OAuth client for a token and mints a one-off auth key that is non-reusable, ephemeral, pre-approved, tagged, and expires in 10 minutes.
- The key is injected only into the runner's cloud-init user-data.
- The runner installs Tailscale, runs
tailscale upwith the requested hostname and tags, and writes non-secret metadata under/var/lib/crabbox(tailscale-ipv4,-hostname,-fqdn,-version,-device-id, and exit-node markers). - After SSH readiness, the CLI reads that metadata and posts it back to the coordinator so the lease record carries the tailnet address.
The auth key is never stored in lease records, provider labels, run logs, or local config. The short-lived key can still appear in user-data at the provider, so the Worker only mints one-off ephemeral keys — never long-lived reusable keys.
The default package mode configures Tailscale's signed stable APT repository,
verifies and scopes its pinned keyring with signed-by, installs the current
package, and records the client version. Set
CRABBOX_TAILSCALE_INSTALL_MODE=pinned to download a static
archive, verify the configured SHA-256 checksum, and install the tailscale and
tailscaled binaries. Neither mode pipes a downloaded installer script into a
root shell.
On release, crabbox stop attempts a best-effort remote tailscale logout before
provider cleanup when SSH is still reachable. Coordinator cleanup does not delete a
device by an ID posted from the lease client; that metadata is diagnostic, not
trusted authorization for a privileged tailnet operation. One-off nodes are
ephemeral and age out in Tailscale if remote logout is unavailable.
If Tailscale rejects a requested subset or unowned tag, the coordinator returns
invalid_tailscale_tags with exact-match/tagOwners guidance, the failed
operation, and the HTTP status. Raw Tailscale response bodies are withheld from
coordinator responses.
Other Tailscale preparation failures return tailscale_unavailable before VM
allocation. The CLI reports the error without repeating creation and cancels its
ordinary create attempt. For an OAuth HTTP 401, repair the coordinator's Tailscale
OAuth credentials before retrying the command. A temporary Tailscale API outage
can be retried after the service recovers.
Preflight the coordinator without leasing a machine:
CRABBOX_TAILSCALE_ENABLED=0 scripts/live-tailscale-smoke.sh --json
CRABBOX_LIVE=1 CRABBOX_COORDINATOR=https://broker.example.com CRABBOX_ADMIN_TOKEN=replace-me scripts/live-tailscale-smoke.sh --jsonFor direct acquisition with an exit node and network: auto, readiness tries
the tailnet hostname first. After 30 seconds it also allows the provider/public
address to finish readiness, while continuing the tailnet wait within the same
readiness deadline. Explicit network: public retains the
public path, and automatic acquisition without an exit node is unchanged.
Crabbox continues to use OpenSSH with per-lease keys; Tailscale SSH is not used. Tailscale only changes the SSH endpoint from the public/provider host to the tailnet host.
Managed VNC remains loopback-bound and tunneled over SSH:
local localhost:5901 -> SSH -> remote 127.0.0.1:5900
Crabbox does not bind managed VNC to 100.x addresses and does not use Tailscale
Serve, Funnel, or noVNC for managed leases.
Static hosts are operator-managed. Point static.host at a MagicDNS name or 100.x
address and use --network tailscale to assert reachability:
provider: ssh
target: macos
static:
host: build-host.example.ts.net
user: alice
port: "22"
workRoot: /Users/alice/crabboxFor static hosts, --network tailscale only asserts reachability and probes SSH;
Crabbox does not install or join Tailscale on the host.