Conversation start fails in sandboxes created on an older agent-server image (sandbox spec lookup returns None) #902

Summary

After an agent-server image bump, a conversation can no longer be started in a sandbox that was created on the previous image. The start task ends in ERROR with an empty detail. The sandbox itself is healthy and resumes fine. Only conversation start fails.

Details

LiveStatusAppConversationService._start_app_conversation looks up the sandbox's spec by image ID and asserts it exists:

sandbox_spec = await self.sandbox_spec_service.get_sandbox_spec(sandbox.sandbox_spec_id)
assert sandbox_spec is not None

With DynamicRemoteSandboxSpecService, specs come from runtime-api /api/warm-runtime-configs, which lists only the current image. Once the image is bumped, get_sandbox_spec(<old image>) returns None, the bare assert raises AssertionError with no message, and _exception_detail turns that into detail="".

The start path reads one field from the spec, working_dir (/workspace/project for both the old and the current spec). Resume doesn't use the spec at all, which is why the old sandbox wakes up normally.

How it was found

An OHE Beta automation that starts a conversation in a long-lived sandbox began failing with conversation start failed: right after Beta went 0.81.1 -> 0.82.0. That release included OpenHands-Cloud#1345 (agent-server 1.50.1-python -> 1.55.1-python) and #866 (SDK pins to 1.55.1). The sandbox was still on agent-server:1.50.1-python.

Evidence

• GET /api/v1/sandbox-specs?id=<...agent-server:1.50.1-python> returns [null]. The same call for :1.55.1-python returns a full spec.
• Control test: the same POST /api/v1/app-conversations (same user, trivial message, run: false) against two sandboxes:

• I could not read app-server logs, so the AssertionError itself is inferred from the code path and the empty detail, not seen in a traceback.
• Schema-level compatibility looks fine for the 1.50.1 -> 1.55.1 gap: a request built by 1.55.1 validates on the 1.50.1 server model, and real conversations from a live 1.50.1 agent-server parse with the 1.55.1 ConversationInfo. This is not an end-to-end test.

Related code that makes the same assumption

get_sandbox_spec returning None for an existing sandbox is treated differently in each caller:

Impact

• Any caller that starts a conversation in an existing sandbox on an older image: explicit sandbox_id (automations, SDK users, sub-conversations).
• Users on a sandbox grouping strategy other than the default NO_GROUPING: _find_running_sandbox_for_user does not filter by spec, so an old running sandbox can be selected and then fail.
• Default users (each conversation gets a fresh sandbox on the current image) are not affected.
• It recurs on every agent-server image bump. The set of affected sandboxes grows over time, and the next bump (SDK 1.56.0, chore(deps): bump SDK to 1.56.0 #876) will strand sandboxes on 1.55.1 the same way.

Proposed fix: three PRs

1. Fix conversation start (the outage). In _start_app_conversation, fall back to get_default_sandbox_spec() when the sandbox's spec is not advertised, as _get_agent_server_context already does. Log a warning naming the image and sandbox, and replace the bare assert with an explicit error where there is no default either. Add tests: a start against a sandbox whose image is not advertised, and the spec service returning None for an unadvertised image without mocks. Small and easy to backport.
2. One helper, and the other callers. Add a single helper for "the spec of an existing sandbox" (default fallback plus a warning) next to resolve_sandbox_spec, and use it in _get_agent_server_context, read_conversation_file, _restore_git_user_settings_after_resume, _resolve_archive_path and the PR 1 call site. While there, decide whether to persist working_dir on the stored sandbox record at creation, which would remove the "default spec has the same working_dir" assumption.
3. Grouping prefers current-image sandboxes (optional follow-up). Have _find_running_sandbox_for_user / _select_sandbox_by_strategy prefer sandboxes whose image is still advertised, so new conversations are not routed into old sandboxes when a current one exists. Intentionally kept out of PR 1.

Not PRs, but worth tracking

• After PR 1 ships, rerun the control test on Beta (start a conversation in an old-image sandbox) with the PR's image.
• Before the agent-server 1.56.0 bump, repeat the cross-version compatibility check. 1.56.0 changes agent settings and sub-agent handling, so the gap is larger than 1.50.1 -> 1.55.1.
• Open question: how long should old-image sandboxes be supported, and should the bump process say so in the release notes?

This issue was written by an AI agent (OpenHands) on behalf of @jpshackelford.