Repository navigation
boxlite run cannot attach on a deployed stack: box-level attach missing from the control plane #1609
Description
TL;DR
Deployed stacks refuse the box-level attach that boxlite run needs, since neither the API proxy nor the Go runner serves it, and #1611 now fixes both halves.
Details
Affects boxlite run alone. create, start, exec, cp and the SDKs work.
WS /api/v1/boxes/<box>/executions/<exec>/attach -> 101
WS /api/v1/boxes/<box>/attach -> 503
With Upgrade: websocket and --http1.1, only the box-level shape fails.
Two gaps:
apps/api/src/boxlite-rest/boxlite-ws-proxy.service.ts:ATTACH_PATHrequires/executions/<id>, somain.tsdestroys the socket unanswered.apps/runner/pkg/api/server.go: no/:boxId/attach, andExecManagerhas no main session. Fixing only the first yields 404.
Cause: #988 moved run to main-session attach and added the route to Rust serve, not the Go runner.
CI misses it: run_rest_e2e.rs targets local serve, run_cli_matrix.sh had no run case, and the stage e2e foreground run case is a strict xfail.
Linear: POL-953 (runner POL-660, API POL-373).
Runner fix (#1611)
- The runner serves
GET /v1/boxes/{boxId}/attach. The first attach opens the main session through the new FFI exportboxlite_box_attach. - The session id rides back on the 101 in
X-Boxlite-Execution-Id; later calls address it like any exec. - The reaper skips a main session and the lookup skips a finished one, mirroring the Rust server.
API fix (#1611)
ATTACH_PATHalso matches/api/v1[/<tenant>]/boxes/<id>/attach, andmatchAttachPathreports whether the upgrade targets the main session.- The prefix rewrite is already generic, so the runner receives
/v1/boxes/<id>/attach. - http-proxy-middleware v4 proxies through httpxy, which writes the runner 101 headers through (
ws-incoming.ts:147), sox-boxlite-execution-idreaches the client. A refusal keeps its status (:105). - A main-session attach skips AutoResume: its init is the user command, so resuming a stopped box would run the job again. The runner refuses a stopped box instead, as
boxlite servedoes (src/cli/src/commands/serve/mod.rs:1760-1764). An exec attach still wakes the box. openapi/box.openapi.yaml:824already documentsattachBox, header included.
SDK attach (#1611)
Every SDK names it attach, after Rust LiteBox::attach and the OpenAPI attachBox. Python already shipped attach(execution_id=None, stdin=True), so the rest take the same two choices: which session (the main one by default, or an exec by id, which only a REST runtime serves) and read-only.
- Rust:
box.attach(AttachOptions::main()),::execution(id),.read_only(), unchanged. - Python:
await box.attach(execution_id=None, stdin=True), unchanged;SyncBox.attachis new. - Node:
await box.attach(executionId?, stdin?), new. - Go:
box.Attach(ctx, &AttachOptions{ExecutionID, ReadOnly, Stdout, Stderr, OnStdout, OnStderr}). - C:
boxlite_box_attach(box, execution_id, read_only, &execution, &error); a NULL id is the main session.
Go and C name the flag read-only so its zero value stays writable; Python and Node keep stdin like docker-py.
Each binding is tested against the loopback box API: the main session and a read-only rejoin by id (?stdin=0).
Freeing a C or Go attach handle no longer kills the session; an exec handle still kills its unfinished process. Free waits for aborted output pumps first, so no callback follows the exit. Releasing a writable attach in any SDK still closes the session stdin, unlike the Kubernetes (stdinOnce) and docker (StdinOnce) defaults. Fixing that in the C FFI alone would split C and Go from Python and Node, so the core detach is #2045.
Test design
codecov/patch counted 40.5% of the runner half (gate 90%): the attach and exec success paths in exec_manager.go, the C FFI and the Go SDK need a live session.
A REST runtime yields a real box handle with no VM behind it. Each suite points one at a loopback box API serving the box lookup, POST /exec and both attach upgrades. A WebSocket plays stdout, stderr, then exit 7, or holds after stdout.
- Go SDK: completes the upgrade by hand, since the module has no WebSocket dependency.
- C SDK: a Rust lib test, with
tungstenite0.24 (already inCargo.lock) as a dev-dependency. - Runner: the gorilla upgrader it already depends on.
- API: the routing spec matches both shapes, and the proxy spec shows a main-session attach is proxied without AutoResume.
- Header forwarding has no committed test, since Jest cannot load the ESM-only httpxy; a one-off script on the installed library kept
x-boxlite-execution-idon the 101 and a 409 a 409.
Ten recorded mutation runs of the runner half each fail a new test: Go SDK sink mapping and refusal typing, C FFI stream handover and error code, runner session flag, stream wiring, exit recording and error wrap, controller slot release and 409 mapping. Routes, exec paths and the C null check had none.
Prior art for the pattern:
- client-go
websocket_test.go:59-61tests its exec stream client against anhttptestupgrade server, not a kubelet. - gorilla
client_server_test.go:72-74pairshttptest.NewServerwith anUpgraderwhose 101 carries a header (:111), as the runner suite does. - tokio-tungstenite
server-custom-accept.rs:109-147answers the 101 by hand withderive_accept_key, as the C suite does. - In this repo,
src/boxlite/src/rest/litebox.rs:2201drives the same route over a loopback listener, andsdks/go/rest_test.go:119-121points a REST runtime athttptest.
Rejected:
- An interface in front of
*boxlite.Box: a fake cannot return a live*Execution, so the wiring never runs. codecov.ymlexclusions: they hide the gap instead of closing it.
Residual gap: openMainSession (9 lines) reads the runner singleton, which tests cannot reset.
Not in this PR
- Cloud starts init before any attach.
POST /v1/boxeswaits forSTARTED(apps/api/src/boxlite-rest/boxlite-box.controller.ts:138-140), and the runner starts the box inCreate(apps/runner/pkg/boxlite/client.go:513-514). - The guest warns that a command exiting at once then loses its output and exit code (
src/guest/src/container/lifecycle.rs:246-249). Sorun --rmof a fast command can still fail on Cloud. - So the stage e2e case (
apps/e2e/cases/test_cli_entry.py:161-164) keeps its strict remote xfail: it is expected to fail through that ordering, not XPASS. - That failure is a 409 once the box has stopped,
Unavailablewhile it is stopping, or no output. The xfail reason now says so, and the mark comes off with the ordering fix. - The runner ignores the documented read-only
?stdin=0attach (openapi/box.openapi.yaml:840).
PR size
- feat(api,runner,sdk): serve box-level attach and expose it in every SDK #1611 adds 1432 code lines against the 400 cap (base
d3fb1fce); 550 are the C loopback test and 120 generated swagger. - Exception, as typed by the author: "No split is needed. The feature as a whole is already split into minimal modules; do not add extra splits that increase review cost."
Activity
fix(runner): serve box-level attach for the main command session
test: drive box-level attach through REST-backed loopback boxes
fix(api): proxy the main-session attach upgrade without waking the box
feat(sdk): expose attach with the same shape in every SDK
fix(runner): serve box-level attach for the main command session
test: drive box-level attach through REST-backed loopback boxes
fix(api): proxy the main-session attach upgrade without waking the box
feat(sdk): expose attach with the same shape in every SDK
fix(sdk): stop a freed C or Go attach from killing the session
docs(runner): document the main session id header on box attach