feat(runner): deliver gateway tools to Claude on Daytona via an in-sandbox stdio MCP shim

docs/design/agent-workflows/documentation/adapters/claude-code.md

@@ -31,22 +31,38 @@ not when the harness name is something in particular. In practice the capability
 static per-harness fallback (`engines/sandbox_agent/capabilities.ts`): the daemon rarely fills
 a real `info.capabilities`, so the runner uses `mcpTools: true` for any non-Pi harness.
 
-The mechanism is a small MCP server named `agenta-tools` (`tools/tool-mcp-http.ts`, built by
-`tools/mcp-bridge.ts` `buildToolMcpServers`) that the runner serves on a loopback HTTP endpoint
-and attaches to the session as an ACP `type: "http"` MCP server. This is an Agenta tool DELIVERY
+The mechanism is a small MCP server named `agenta-tools`. This is an Agenta tool DELIVERY
 vehicle, not a user-declared MCP server: it carries the same gateway and code specs the Pi
-extension would register, just exposed over MCP because Claude cannot take a native tool. It runs
-in the already-running runner process (no runner-host child) and is reachable only from loopback;
-it holds only public metadata (names, descriptions, schemas) and a relay directory; the
+extension would register, just exposed over MCP because Claude cannot take a native tool. It
+holds only public metadata (names, descriptions, schemas) and a relay directory; the
 `call_ref`, the code, the scoped secrets, and the callback auth never reach it. When the model
 calls a tool, the server relays the request back to the runner over the file relay
 (`tools/relay.ts`), and the runner runs the private spec from memory and POSTs to `/tools/call`.
 The safety property is identical to Pi's: the provider key and the connection auth stay
-server-side, and the agent only ever asks Agenta to run a named tool.
-
-(This internal channel was disabled as collateral with the user-stdio-MCP disable in PR #4831 and
-restored over loopback HTTP by the gateway-tool-mcp project. It is independent of the user MCP
-capability below — the two toggle separately.)
+server-side, and the agent only ever asks Agenta to run a named tool. The name `agenta-tools`
+is reserved (the settings rules below are rendered against it), so a user-declared MCP server
+claiming it is refused up front.
+
+The channel has two transports, picked by where Claude runs:
+
+- **Local sandbox: loopback HTTP.** `buildToolMcpServers` (`tools/mcp-bridge.ts`, server in
+  `tools/tool-mcp-http.ts`) serves the channel on a loopback HTTP endpoint and attaches it as
+  an ACP `type: "http"` MCP server. It runs in the already-running runner process (no
+  runner-host child) and is reachable only from loopback. It also carries `client` tools,
+  paused in `tools/call`.
+- **Daytona: in-sandbox stdio shim.** Claude runs inside the sandbox, where the loopback URL
+  is unreachable, so the runner uploads the bundled shim (`tools/tool-mcp-stdio.ts`) plus a
+  public-specs JSON file into an ephemeral in-VM dir and advertises a typeless ACP stdio entry
+  the adapter spawns as `node tool-mcp-stdio.js`. The shim's `tools/call` writes relay request
+  files the runner-side relay loop executes server-side. Executable (gateway/callback) tools
+  only: a client tool on this path is refused up front (the shim cannot pause a call), and a
+  remote provider that is not Daytona refuses any custom tool. Contract details:
+  [runner-to-MCP interface page](../../interfaces/cross-service/runner-to-mcp-server.md).
+
+(This internal channel was disabled as collateral with the user-stdio-MCP disable in PR #4831,
+restored over loopback HTTP by the gateway-tool-mcp project, and extended to Daytona by the
+in-sandbox-tool-mcp project. It is independent of the user MCP capability below — the two
+toggle separately.)
 
 User-declared `mcp_servers` are a separate thing and effectively off today. They would reach
 Claude through `toAcpMcpServers` as additional ACP stdio servers, but only when

docs/design/agent-workflows/documentation/ground-truth.md

@@ -51,16 +51,20 @@ this page and the referenced code as the source of truth.
 - Callback tools route through `/tools/call`. On Daytona, Pi tool calls use the runner file
   relay.
 - Gateway/code tool delivery to non-Pi harnesses (Claude) exists through the internal MCP
-  channel, served over a loopback HTTP MCP endpoint the runner stands up (no runner-host child
-  process). User-declared MCP resolution is feature-gated (`AGENTA_AGENT_MCPS_ENABLED`, off by
-  default).
+  channel, on two transports: a loopback HTTP MCP endpoint the runner stands up on the local
+  sandbox (no runner-host child process), and an in-sandbox stdio MCP shim the runner uploads
+  on Daytona (`tools/tool-mcp-stdio.ts`; the harness spawns it, its `tools/call` writes relay
+  request files, and the runner-side relay loop executes server-side). The channel's name
+  `agenta-tools` is reserved; a user-declared MCP server claiming it is refused. User-declared
+  MCP resolution is feature-gated (`AGENTA_AGENT_MCPS_ENABLED`, off by default).
 - `client` tools (browser-fulfilled, e.g. `request_connection`) are delivered to Claude too on
   the local path: advertised over the same internal MCP channel and PAUSED in the `tools/call`
   handler (no JSON-RPC result + abort the request), then resumed from the browser result next
   turn — the same cross-turn pause Pi gets via the file relay, through one shared seam
   (`services/runner/src/engines/sandbox_agent/client-tools.ts`). On a remote sandbox the
-  loopback channel is unreachable, so a non-Pi run carrying ANY custom tool — client kind
-  included — is refused up front (`REMOTE_TOOLS_UNSUPPORTED_MESSAGE`), never dropped silently.
+  in-sandbox stdio shim has no pause path, so a non-Pi run carrying a client tool is refused
+  up front (`REMOTE_CLIENT_TOOLS_UNSUPPORTED_MESSAGE`); a remote provider that is not Daytona
+  refuses ANY custom tool (`REMOTE_TOOLS_UNSUPPORTED_MESSAGE`). Never dropped silently.
 
 ## Not Implemented
 
@@ -83,7 +87,7 @@ this page and the referenced code as the source of truth.
 - User-declared MCP transports split: remote (`http`) servers are delivered by the runner
   (`toAcpMcpServers`, #4834); stdio servers are disabled (`USER_MCP_UNSUPPORTED_MESSAGE`) because
   they launch a process on the runner host. The runner's own internal gateway-tool channel is a
-  separate thing and is delivered over loopback HTTP.
+  separate thing, delivered over loopback HTTP locally and the in-sandbox stdio shim on Daytona.
 - Trigger lifecycle, Compose.io trigger integration, and event-to-agent mapping are not
   implemented in the agent workflow code.
 - A persisted agent template object that separates `AGENTS.md`, skills, tools,

docs/design/agent-workflows/documentation/running-the-agent.md

@@ -176,6 +176,10 @@ sections).
   this non-zero auto-stop so a sandbox the runner leaks (a process KILL skips the per-run
   teardown) self-reaps instead of burning credit. Values below `1` fall back to the default
   (a `0` would re-disable auto-stop and reintroduce the leak).
+- `SANDBOX_AGENT_RELAY_MCP_BUNDLE`. Overrides the path of the in-sandbox stdio MCP shim bundle
+  the runner uploads to Daytona for non-Pi tool delivery. Default
+  `dist/tools/tool-mcp-stdio.js` (built by `pnpm run build:extension`). Trusted deployment
+  configuration — it selects code — never run or request configuration.
 - `AGENTA_RUNNER_SESSION_KEEPALIVE`. Gates session keep-alive: after a turn ends, the runner
   parks the live harness session and continues it on the next matching message in the same
   conversation, instead of cold-replaying the transcript. Default off. Local sandbox only;

docs/design/agent-workflows/documentation/tools.md

@@ -191,16 +191,26 @@ natively. Today that splits cleanly into two paths.
   list for Pi, so neither the synthetic `agenta-tools` server nor any user MCP server is
   attached.
 - **Claude and other ACP harnesses take MCP.** They cannot accept a native tool, so the runner
-  exposes the same resolved specs as a small synthetic MCP server named `agenta-tools`
-  (`services/agent/src/tools/mcp-bridge.ts` launches `services/agent/src/tools/mcp-server.ts`).
-  This bridge is given only public metadata (names, descriptions, schemas) and a relay
-  directory. It never receives the `call_ref`, the code, the scoped secrets, or the callback
-  auth. When the model calls a tool, the bridge relays the request back to the runner, and the
-  runner runs the private spec from memory. This `agenta-tools` server is a tool DELIVERY
-  vehicle, not a user MCP server: it carries gateway and code tools AND `client` tools (which it
-  pauses in `tools/call` rather than executing — see "Client tools" below), and it exists only on
-  the local Claude path (it is skipped on a remote sandbox, where its loopback URL is
-  unreachable).
+  exposes the same resolved specs as a small synthetic MCP server named `agenta-tools`. The
+  channel is given only public metadata (names, descriptions, schemas) and a relay directory.
+  It never receives the `call_ref`, the code, the scoped secrets, or the callback auth. When
+  the model calls a tool, the channel relays the request back to the runner, and the runner
+  runs the private spec from memory. This `agenta-tools` server is a tool DELIVERY vehicle,
+  not a user MCP server, and its name is reserved: a user-declared MCP server named
+  `agenta-tools` is refused up front (the Claude adapter renders permission rules against that
+  name). The channel has two transports, picked by where the harness runs. On the local
+  sandbox it is a loopback HTTP MCP server the runner serves
+  (`services/runner/src/tools/mcp-bridge.ts` `buildToolMcpServers` + `tool-mcp-http.ts`); it
+  carries gateway and code tools AND `client` tools (which it pauses in `tools/call` rather
+  than executing — see "Client tools" below). On Daytona the harness runs inside the sandbox,
+  where the loopback URL is unreachable, so the runner uploads an in-sandbox stdio MCP shim
+  (`services/runner/src/tools/tool-mcp-stdio.ts`, bundled by `build:extension`) plus a
+  public-specs JSON file into an ephemeral in-VM dir and advertises them as a stdio entry the
+  harness spawns; the shim's `tools/call` writes relay request files that the runner-side
+  relay loop executes server-side. The shim carries executable (gateway/callback) tools only;
+  client tools on a non-Pi remote run are refused up front, and a non-Daytona remote provider
+  fails closed (see "Client tools" below). The full transport and env contract lives in the
+  [runner-to-MCP interface page](../interfaces/cross-service/runner-to-mcp-server.md).
 
 Both paths funnel execution through one function, `runResolvedTool` in
 `services/agent/src/tools/dispatch.ts`. It is the single place that branches on `kind`, so how
@@ -249,11 +259,12 @@ file into a relay directory, the runner (which can reach Agenta) picks it up, pe
 `tools/relay-client.ts` (re-exported by `dispatch.ts`); the wire protocol (file suffixes,
 request and response shapes, byte-pinned request serialization) is `tools/relay-protocol.ts`;
 the runner-side loop is `startToolRelay` in `tools/relay.ts`. The two writer modules import
-node builtins only, so the Pi extension bundle and the future in-sandbox MCP shim (#5234)
-consume them from inside the sandbox. Same callback, same envelope, different delivery. The
-non-Pi internal MCP channel (a loopback HTTP MCP server the runner serves) uses this same relay
-even on local runs, because the harness calling it is kept blind to the private spec — only
-public metadata crosses the channel, and execution relays back to the runner.
+node builtins only, so the Pi extension bundle and the in-sandbox stdio MCP shim
+(`tools/tool-mcp-stdio.ts`) consume them from inside the sandbox. Same callback, same envelope, different delivery. The
+non-Pi internal MCP channel uses this same relay on both of its transports — the loopback HTTP
+server on local runs and the in-sandbox stdio shim on Daytona — because the harness calling it
+is kept blind to the private spec: only public metadata crosses the channel, and execution
+relays back to the runner.
 
 The relay files carry three guarantees. Publication is atomic in both directions: each side
 writes the full bytes to a temp name (`<final>.tmp.<nonce>`) and renames it to the final name
@@ -351,12 +362,16 @@ The pause itself is shared by both delivery paths through one seam
 - **Claude** calls the tool over the internal `agenta-tools` MCP server, and the runner pauses it
   inside the `tools/call` handler: it emits NO JSON-RPC result and aborts that in-flight request,
   so Claude cannot settle the call before the turn ends `paused`. The browser result resumes it
-  next turn (the MCP handler returns the stored output if the model re-calls). This is
-  local-only: on a remote sandbox the loopback MCP channel is unreachable, so a non-Pi run
-  carrying ANY custom tool — client kind included — is rejected up front
-  (`REMOTE_TOOLS_UNSUPPORTED_MESSAGE`), never delivered silently. (The ACP permission gate in
-  `acp-interactions.ts` keeps its own `kind: "client"` pause branch as a live fallback for a
-  harness that raises a permission gate carrying a resolved client spec.)
+  next turn (the MCP handler returns the stored output if the model re-calls). This pause is
+  local-only: the Daytona stdio shim has no pause path (the relay loop parks a client call and
+  writes no response file, so the shim would hang until the relay timeout and teach the model
+  the tool is broken), so a non-Pi remote run carrying a client tool is refused up front
+  (`REMOTE_CLIENT_TOOLS_UNSUPPORTED_MESSAGE`), never delivered silently. Executable tools DO
+  deliver on Claude+Daytona through the shim; a remote provider that is not Daytona still
+  refuses ANY custom tool (`REMOTE_TOOLS_UNSUPPORTED_MESSAGE`) until in-sandbox delivery is
+  proven there. (The ACP permission gate in `acp-interactions.ts` keeps its own
+  `kind: "client"` pause branch as a live fallback for a harness that raises a permission gate
+  carrying a resolved client spec.)
 
 A client tool's `render` hint can be `{ kind: "connect" }` (e.g. `request_connection`), the typed
 member of `RenderHint` that asks the frontend to draw the connect widget.
@@ -429,6 +444,18 @@ Two gates consult this same decision:
   runs, including its native builtins, and it gives Pi the same human-in-the-loop behavior
   Claude gets.
 
+Separately from the dialog gates, the relay re-checks every execute record with a runner-side
+execution guard (`buildRelayExecutionGuard` in
+`services/runner/src/engines/sandbox_agent/relay-guard.ts`), on EVERY harness: the relay dir is
+sandbox-writable, so a forged request file proves nothing about any dialog having run.
+`allow` passes and `deny` refuses identically everywhere. `ask` splits by harness: on Pi the
+guard consumes a dialog-recorded execution grant, so a forged or replayed record for an `ask`
+tool fails closed; on an MCP harness (Claude) the guard passes `ask`, because the harness's own
+dialog gates the call before it reaches the shim and the runner holds no grant for it. The
+stated residual: on the MCP path a forged relay file can still trigger an ask-tool without a
+dialog; reflecting the harness approval into the grant ledger is a documented follow-up. The
+hard deny boundary holds on every harness.
+
 Client tools are a carve-out from that same ladder. They are decided by the responder's
 `onClientTool` (consulted at the ACP gate on Claude, and by the relay on Pi), not by the
 policy-and-rules path above: a client tool with no `permission` of its own defaults to
@@ -616,19 +643,22 @@ never drift from the files that exist. The canonical playbook format lives in th
 | In-sandbox relay writer + wire protocol | `services/runner/src/tools/relay-client.ts`, `relay-protocol.ts` |
 | Relay wake sources (local `fs.watch`, Daytona watch exec) | `services/runner/src/tools/relay-watch.ts` |
 | Pi native delivery | `services/agent/src/extensions/agenta.ts` |
-| `agenta-tools` server for non-Pi harnesses | `services/agent/src/tools/mcp-bridge.ts`, `services/agent/src/tools/mcp-server.ts` |
+| `agenta-tools` channel for non-Pi harnesses (local loopback HTTP) | `services/runner/src/tools/mcp-bridge.ts`, `services/runner/src/tools/tool-mcp-http.ts` |
+| `agenta-tools` channel on Daytona (in-sandbox stdio shim: entrypoint, env contract, upload) | `services/runner/src/tools/tool-mcp-stdio.ts`, `services/runner/src/tools/tool-mcp-env.ts`, `services/runner/src/engines/sandbox_agent/tool-mcp-assets.ts` |
 | Capability probe | `services/agent/src/engines/sandbox_agent/capabilities.ts` |
 | Permission decision (shared by both gates) | `services/runner/src/permission-plan.ts` |
 | ACP responder (`ApprovalResponder`) | `services/runner/src/responder.ts` |
 | Tool relay enforcement | `services/runner/src/tools/relay.ts` |
+| Relay execution guard (every harness) | `services/runner/src/engines/sandbox_agent/relay-guard.ts` |
 
 ## Status and known gaps
 
 - **User MCP is effectively dead on the default path.** Resolution is off unless
   `AGENTA_AGENT_MCPS_ENABLED` is truthy, and even on, the runner drops user MCP for Pi. Pi and
   Agenta are the default harnesses, so `mcp_servers` is a silent no-op for most runs. It would
   reach Claude only. Do not confuse this with the `agenta-tools` server, which is an internal
-  tool-delivery vehicle for Claude, not a user MCP server.
+  tool-delivery vehicle for Claude, not a user MCP server; the name is reserved, and a
+  user-declared server claiming it is refused.
 - A tool's `permission` is honored on both harnesses now, including Pi's own builtins. Claude
   checks its rendered settings file first, then the ACP responder. Pi has no separate
   harness-side settings gate, so the relay decides everything Pi runs: gateway and code tools
@@ -638,7 +668,14 @@ never drift from the files that exist. The canonical playbook format lives in th
 - The `render` hint is plumbed end to end on the runner side, but full frontend projection of
   every render kind is still in progress.
 - Gateway calls on Daytona depend on the file relay, because the sandbox cannot reach Agenta
-  directly. The relay is also used by the non-Pi internal MCP channel on local runs.
+  directly. The relay also carries the non-Pi internal MCP channel on both transports: behind
+  the loopback HTTP server on local runs, and behind the in-sandbox stdio shim on Daytona.
+- **Non-Pi tools on a remote sandbox are Daytona-only, executable-only.** Executable
+  (gateway/callback) tools deliver on Claude+Daytona via the in-sandbox stdio shim. A client
+  tool on a non-Pi remote run refuses loud (`REMOTE_CLIENT_TOOLS_UNSUPPORTED_MESSAGE`; the
+  shim has no pause path), and a remote provider that is not Daytona refuses ANY custom tool
+  (`REMOTE_TOOLS_UNSUPPORTED_MESSAGE`) until in-sandbox delivery is proven there. Tracked in
+  [projects/in-sandbox-tool-mcp](../projects/in-sandbox-tool-mcp/).
 - **Code tools are standard-library-only.** The image ships `python3` and `node`, but the
   child env has no package install and no module path to the runner's dependencies, so a tool
   cannot import third-party packages.