Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions docs/design/agent-workflows/projects/cold-turn-startup/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Cold-turn startup cost removal

This workspace plans two changes that each remove fixed wall-clock delay from the start of
a "cold" agent turn. Profiling on the live local runner on 2026-07-11 measured both. Together
they add up to roughly two seconds that the user waits before the model even begins to answer.

## Glossary

Read this once. Later files use these terms without re-defining them.

- **Runner**: the Node sidecar under `services/runner/`. It receives a `/run` request and
drives one agent turn. Written in TypeScript because the agent harnesses are Node libraries.
- **Harness**: the coding-agent program the runner drives (Pi, Claude Code, Codex). This plan
is about the Pi harness.
- **ACP**: Agent Client Protocol. The JSON-RPC protocol the runner speaks to a harness.
- **Adapter**: a small program that translates ACP on one side into a specific harness's own
interface on the other. For Pi the adapter is an npm package named `pi-acp`. It spawns the
real `pi` binary and bridges it to ACP.
- **sandbox-agent**: an npm package plus a compiled command-line binary (`@sandbox-agent/cli`).
The runner launches this binary; the binary chooses and launches the adapter. The runner
already ships one hand-written change to this package through pnpm's patch mechanism.
- **Probe**: a short child-process call the adapter runs at session start to gather version
information (`pi --version`, `npm view ...`). Each probe blocks the turn until it returns.
- **Cold turn**: the first turn of a conversation, or any turn where no warm harness session is
reused. A cold turn pays full session-startup cost. This plan only concerns cold turns.
- **Session**: one harness conversation. `createSession` opens it; this is where the Pi cold
start (and the probes) happen.
- **Mount**: attaching a remote object store as if it were a local directory, using a
user-space filesystem. The runner mounts the caller's durable working directory this way so
files the agent writes persist across turns.
- **geesefs / FUSE**: geesefs is the program that performs the mount. FUSE ("filesystem in
user space") is the Linux mechanism it uses. A FUSE mount placed over an existing directory
hides whatever that directory already contained.
- **cwd**: the working directory a harness session runs in. The runner sets it to the mounted
durable directory.
- **Daytona**: the remote sandbox provider. When a run is remote, the harness runs inside a
Daytona sandbox instead of on the runner host.
- **Patch (pnpm)**: a stored diff pnpm re-applies to a dependency's files on every install. The
runner keeps these under `services/runner/patches/`. A patch only affects the copy of the
package inside the runner's own `node_modules`.

## The two changes

- **Fix A, remove the Pi startup probes.** The Pi adapter runs three version probes at every
cold session start, serialized before the first model request. They cost about 1.6 seconds
and add nothing the user asked for. The change removes them. This is the higher-value and
better-understood change.
- **Fix B, stop paying for the durable mount on chat-only turns.** The runner mounts the
durable working directory before it opens the session, even for a turn that never touches a
file. The mount costs about 0.55 to 0.6 seconds. The change avoids that cost when no file is
used. Research below shows this is genuinely harder than it looks; the plan recommends
shipping Fix A first and treating Fix B as a follow-up.

## Reading order

1. `context.md` answers: what does the user experience today, and why does it happen.
2. `research.md` answers: exactly what the code does now, with file and line references, and
which facts changed the plan. Read this before proposing any edit.
3. `plan.md` answers: what to change, in what order, and how to test each step.
4. `open-questions.md` answers: what must be confirmed at runtime before or during the work,
because static reading of the code could not settle it.
5. `upstream-issue-draft.md` is the ready-to-file text for the upstream `pi-acp` issue asking
for a switch that disables the update-check probes (plan Stage 2b).
6. `status.md` tracks progress and decisions. It is the running source of truth.
51 changes: 51 additions & 0 deletions docs/design/agent-workflows/projects/cold-turn-startup/context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Context

## What the user experiences today

When a user sends the first message of a conversation, the agent takes about two seconds
longer to begin answering than the model itself needs. That delay is fixed setup work the
runner and the Pi adapter do before the first model request goes out. It repeats on every
cold turn. It is present both for local runs and for runs inside a Daytona sandbox, and it is
worse inside a sandbox because one of the probes reaches the public npm registry over the
network from a European sandbox.

Profiling on the live local runner on 2026-07-11 attributed the delay to two independent
causes:

- About 1.6 seconds comes from three version-check probes the Pi adapter runs at session
start (Fix A).
- About 0.55 to 0.6 seconds comes from the runner mounting the durable working directory
before it opens the session, even on a turn that never reads or writes a file (Fix B).

The two causes are unrelated in the code. They can ship separately.

## Why this work exists

Neither cost buys the user anything on a normal turn.

The Pi adapter probes exist to print a startup banner and an "upgrade available" notice. The
runner already throws that banner away: `services/runner/src/tracing/otel.ts` strips the
banner lines out of the reply after the fact. So the runner pays 1.6 seconds to produce text
it then deletes.

The durable mount exists so files the agent writes survive to the next turn. A chat-only turn
writes no files, so on those turns the mount is pure setup cost with no payoff. The difficulty
is that the runner cannot know in advance whether a turn will stay chat-only.

## Goals

- Remove the Pi adapter startup probes from the cold-turn path, for both local and Daytona
runs. Target saving: about 1.6 seconds per cold turn.
- Once the probes are gone, retire or shrink the banner-stripping code in `otel.ts`, since it
exists only to clean up after those probes.
- Investigate removing the eager durable mount from chat-only cold turns. Target saving: about
0.55 to 0.6 seconds per cold local chat turn. Ship this only if it can be made safe.

## Non-goals

- No change to what the agent can do, only to how fast a cold turn starts.
- No change to the durable-store contract or the wire protocol.
- No attempt to warm-pool Pi sessions or otherwise avoid cold starts entirely. That is a
separate, larger effort.
- No change to the Daytona snapshot build in this plan beyond what Fix A strictly needs to
reach the in-sandbox adapter copy.
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Open questions

These could not be settled by reading the code. Each names how to answer it and what depends on
the answer.

## 1. Which `pi-acp` copy actually runs at session start?

Static reading found two candidate copies: the runner's dependency `pi-acp@0.0.29` under
`node_modules`, reachable through the CLI's "PATH binary hint", and the CLI's own
registry-installed copy `pi-acp@0.0.31` under `~/.local/share/sandbox-agent/bin/agent_processes`.
The CLI's resolution order could pick either. The pnpm patch mechanism only reaches the first.

- How to answer: add a temporary marker to each `dist/index.js` and see which prints on a cold
local run, or enable the CLI's adapter-resolution logging, or trace the adapter child
process to see which `index.js` path it opens.
- Depends on it: Stage 2 (patch target) and Stage 4 (Daytona approach). This is the first task.

## 2. Does turning on `quietStartup` have any unwanted side effect?

`quietStartup` suppresses the whole startup banner, not only the version line. The runner
already strips that banner, so the user should see no change. Confirm nothing else in the
runner or the Pi extension reads the banner text or the startup-info metadata before it is
stripped.

- How to answer: grep the runner and the Pi extension for reads of the startup-info field
(`_meta.piAcp.startupInfo`) and confirm none depend on its content; then a cold local run
with `quietStartup` on, checking the reply is unchanged apart from the missing banner.
- Depends on it: Stage 1 shipping cleanly.

## 3. Is the harness cold-start window long enough to hide the 0.55 second mount?

The "overlap" variant of Fix B can only overlap the mount with the part of the cold start that
runs before workspace materialization, because `prepareWorkspace` depends on the mount being
complete. Whether that pre-workspace window is large enough to hide 0.55 seconds is a runtime
timing question.

- How to answer: instrument `acquireEnvironment` to log timestamps at mount begin, mount
complete, workspace-prep begin, and session-open begin on a cold local chat turn.
- Depends on it: whether Fix B's overlap variant is worth building, or whether only the full
lazy-mount design would pay off. This is the first task of the Fix B follow-up.

## 4. What is the correct Daytona delivery for a fixed adapter? (largely answered)

Upstream research answered the mechanism question: the CLI honors a pre-seeded
`agent_processes/pi/` directory ("already installed" short-circuit), so baking a fixed adapter
copy into the Daytona snapshot at that path is the preferred delivery, with a runner-hosted
registry JSON via `SANDBOX_AGENT_ACP_REGISTRY_URL` as the alternative. See `research.md`,
"How the installed adapter version is actually chosen".

What remains open is only verification: confirm the in-sandbox CLI's data directory path (the
in-sandbox equivalent of `~/.local/share/sandbox-agent/bin/agent_processes/pi/`) and that the
snapshot build can write there.

- How to answer: run a Daytona cold turn and locate the adapter install directory inside the
sandbox; then add the pre-seed to the snapshot build and confirm the CLI logs
"already installed" instead of a registry fetch.
- Depends on it: Stage 4.
Loading
Loading