Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@ Architecture records describe OpenPI ownership boundaries, lifecycle seams, and
Keep unresolved alternatives and implementation proposals in [`../design/`](../design/). Promote a constraint through an accepted [`Decision`](../decisions/) before treating it as architecture policy.

Each new or materially revised architecture record should identify its evidence status, source revision, affected Pi primitive, current invariants, related Issues and Decisions, and any record it supersedes.

- [`WEB_MODEL_DISCOVERY.md`](WEB_MODEL_DISCOVERY.md) - bounded Web model snapshots and full-catalog discovery through Pi's model runtime
39 changes: 39 additions & 0 deletions docs/architecture/WEB_MODEL_DISCOVERY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Bounded Web model discovery

- Status: `validated`
- Created: 2026-09-09
- Verified: 2026-09-10
- Source boundary: implementation commit `c50af9d87bf2521710bef96bc788858c3cc3c143`, based on `5bf2fe29e52801d79826c2eb573be403f53285e5`
- Affected Pi primitive: Pi's available-model snapshot and active Web Session model selection
- Related Issue: [#459](https://github.com/openpi-dev/openpi/issues/459)
- Related PR: [#521](https://github.com/openpi-dev/openpi/pull/521)
- Related Decision: [0001 - documentation and evidence governance](../decisions/0001-documentation-and-evidence-governance.md)
- Supersedes: none

## Ownership

Pi remains the only source of truth for available models and the active Session model. `PiWebRuntime` reads `modelRuntime.getAvailableSnapshot()` and performs model selection through the active Pi Session. OpenPI does not persist a second model catalog, introduce a provider stack, or infer availability from browser state.

`PiWebAdapter` keeps the Web snapshot bounded to `WEB_MAX_MODELS` entries while retaining the current model. `snapshot.truncation.modelsOmitted` reports how many available models are absent after count or snapshot-byte limiting. The browser treats that value as projection evidence, not as a model total it computed locally.

## Discovery contract

When a snapshot omits models, the browser can query the authenticated `GET /api/models` endpoint. The request may include a query of at most 200 characters, a result limit no greater than 50, and the expected active Session id. A stale Session id fails with `409 SESSION_CHANGED`.

`WebRuntimeController.searchModels()` is a required runtime contract; Web Host has no optional unbounded fallback. The runtime searches provider, model id, name, and label in Pi's complete available-model snapshot through one shared projector. Results remain bounded by both the requested count and a 64 KiB response budget. The response reports `totalAvailable`, `totalMatches`, and `truncation.matchesOmitted`, while `truncation.bytes` equals the complete serialized response including totals and truncation metadata.

Provider and model id remain exact canonical selection identities and are never shortened into display strings. Only model name and label are bounded display projections. A model whose exact identity cannot fit in the response budget is omitted with truncation evidence rather than returned under a mutated identity. Selection sends the exact provider, model id, and active Session id through the existing model mutation path.

The model picker exposes search only when `modelsOmitted` is positive. It waits 250 ms after the latest input, cancels superseded requests, and rejects responses from older queries, snapshots, or Session/workspace epochs. Every accepted snapshot and `runtime_changed` event invalidates retained search results. If the picker remains open with a query, the new snapshot generation triggers one new debounced search. Waiting, empty, truncated-result, and error states are separate browser projections. Closing the picker or clearing the query cancels pending search work and restores the bounded snapshot list. An ordinary complete model list keeps the simpler selector without a search field.

## Review correction

The initial implementation at `bcfa2d73a8ae0b307018fe92b9325f1966839947` did not satisfy all invariants stated by this record. Its optional Host fallback did not enforce the byte budget, canonical provider/model identities were truncated to display bounds, byte evidence excluded response metadata, and search results survived same-Session catalog refreshes. PR #521 review identified these gaps. The source boundary above names the corrective implementation; the earlier commit is retained here as historical provenance, not as validated evidence.

## Evidence and limits

The adapter fixture projects 1,000 synthetic models into a 250-model snapshot, retains the current model, reports 750 omitted models, and stays inside the snapshot-byte bound. Runtime and Host tests cover the required search contract, full-catalog matching, exact long identities, result-count bounds, complete-response byte accounting, invalid limits, oversized queries, and stale Session rejection. Store and component tests cover debounce timing, cancellation, out-of-order responses, workspace and snapshot-generation changes, automatic re-query, empty results, errors, and exact model identity.

The production-WebHost Playwright scenario provides 250 visible synthetic models and one search-only model. It verifies the truncation message, bounded Session-scoped request, absence of the hidden model before search, and exact selection of that model afterward. The browser scenario intercepts deterministic model API responses; it is not a claim that a real provider account with 251 configured models was exercised. The adapter, runtime, Host, store, component, and browser tests form the complete evidence chain.

At the corrected source boundary, `bun run check` passed; `bun run test` reported 1,528 Node tests passed, zero failed, and one Windows-only test skipped, plus 138 Vitest tests passed; the final complete `bun run test:web:e2e` rerun reported 12 passed. An earlier complete E2E attempt hit an unrelated timeout inside the trajectory view's axe analysis; that test then passed alone in 1.8 seconds and in the final complete rerun.
265 changes: 262 additions & 3 deletions tests/web/app-render.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,10 @@ import { i18n } from "../../web/ui/src/i18n.ts";
import { WebClient } from "../../web/ui/src/protocol/client.ts";
import { createWebStore, webStore } from "../../web/ui/src/store/web-store.ts";

afterEach(cleanup);
afterEach(() => {
cleanup();
vi.useRealTimers();
});

it("keeps a workspace draft separate from the old Session UI and retains text after failed sending", async () => {
const initial = webStore.getState();
Expand Down Expand Up @@ -728,12 +731,12 @@ it("shows complete model identities before workspace selection", () => {
expect(modelButton.disabled).toBe(false);
fireEvent.click(modelButton);
expect(
screen.getByRole("menuitem", {
screen.getByRole("option", {
name: "Shared model (provider-alpha/a)",
}),
).toBeTruthy();
expect(
screen.getByRole("menuitem", {
screen.getByRole("option", {
name: "Shared model (provider-beta/b)",
}),
).toBeTruthy();
Expand Down Expand Up @@ -1061,3 +1064,259 @@ describe("thinking level picker", () => {
expect(screen.getByText(i18n.t("thinkingLevelMismatch"))).toBeTruthy();
});
});

it("debounces bounded model search when the snapshot omitted models", async () => {
vi.useFakeTimers();
const snapshot = activeSnapshot();
snapshot.runtime.status = "idle";
snapshot.models = [
{
provider: "provider-visible",
id: "visible",
name: "Visible model",
label: "Visible model",
current: true,
},
];
snapshot.truncation = {
...truncation,
modelsOmitted: 2,
truncated: true,
};
const baseStore = createWebStore();
const searchModels = vi.fn(async (_query: string) => {});
const selectModel = vi.fn(async (_value: string) => {});
const actions = {
...baseStore.getState().actions,
searchModels,
selectModel,
};
const initialSearch = baseStore.getState().modelSearch;
const props = {
snapshot,
selectedWorkspace: "/tmp",
sessionSwitching: false,
promptAdmissionPending: false,
liveRunning: false,
landing: false,
activeTurn: null,
turnCancellationPending: false,
turnTerminalStatus: null,
pendingFollowUpsReceipt: null,
thinkingPendingLevel: null,
actions,
modelSearch: initialSearch,
};
const { rerender } = renderWithI18n(createElement(Composer, props));

fireEvent.click(
screen.getByRole("button", {
name: "Visible model (provider-visible/visible)",
}),
);
expect(
screen.getByText(
"Showing 1 models. 2 more are available; search to find them.",
),
).toBeTruthy();
const searchInput = screen.getByPlaceholderText(
"Search provider, model name, or ID...",
);
fireEvent.change(searchInput, { target: { value: "h" } });
fireEvent.change(searchInput, { target: { value: "hidden" } });
expect(searchModels).not.toHaveBeenCalled();
await act(() => vi.advanceTimersByTimeAsync(249));
expect(searchModels).not.toHaveBeenCalled();
await act(() => vi.advanceTimersByTimeAsync(1));
expect(searchModels).toHaveBeenCalledOnce();
expect(searchModels).toHaveBeenCalledWith("hidden");

const refreshedSnapshot = {
...snapshot,
generatedAt: "2026-09-03T00:00:01Z",
};
rerender(
createElement(
I18nextProvider,
{ i18n },
createElement(Composer, {
...props,
snapshot: refreshedSnapshot,
}),
),
);
await act(() => vi.advanceTimersByTimeAsync(250));
expect(searchModels).toHaveBeenCalledTimes(2);
expect(searchModels).toHaveBeenLastCalledWith("hidden");

rerender(
createElement(
I18nextProvider,
{ i18n },
createElement(Composer, {
...props,
snapshot: refreshedSnapshot,
modelSearch: {
...initialSearch,
query: "hidden",
status: "ready",
models: [
{
provider: "provider-hidden",
id: "hidden/model",
name: "Hidden model",
label: "Hidden model",
current: false,
},
],
totalMatches: 1,
},
}),
),
);
fireEvent.click(
screen.getByRole("option", {
name: "Hidden model (provider-hidden/hidden/model)",
}),
);
expect(selectModel).toHaveBeenCalledWith("provider-hidden/hidden/model");
vi.useRealTimers();
});

it("cancels a pending model search when the picker closes", async () => {
vi.useFakeTimers();
const snapshot = activeSnapshot();
snapshot.runtime.status = "idle";
snapshot.models = [
{
provider: "provider-visible",
id: "visible",
name: "Visible model",
label: "Visible model",
current: true,
},
];
snapshot.truncation = {
...truncation,
modelsOmitted: 1,
truncated: true,
};
const baseStore = createWebStore();
const searchModels = vi.fn(async (_query: string) => {});
const actions = {
...baseStore.getState().actions,
searchModels,
};
renderWithI18n(
createElement(Composer, {
snapshot,
selectedWorkspace: "/tmp",
sessionSwitching: false,
promptAdmissionPending: false,
liveRunning: false,
landing: false,
activeTurn: null,
turnCancellationPending: false,
turnTerminalStatus: null,
pendingFollowUpsReceipt: null,
thinkingPendingLevel: null,
actions,
modelSearch: baseStore.getState().modelSearch,
}),
);

fireEvent.click(
screen.getByRole("button", {
name: "Visible model (provider-visible/visible)",
}),
);
const searchInput = screen.getByPlaceholderText(
"Search provider, model name, or ID...",
);
fireEvent.change(searchInput, { target: { value: "hidden" } });
fireEvent.keyDown(searchInput, { key: "Escape" });
await act(() => vi.advanceTimersByTimeAsync(250));

expect(searchModels).not.toHaveBeenCalled();
});

it("shows empty and error feedback for bounded model search", () => {
vi.useFakeTimers();
const snapshot = activeSnapshot();
snapshot.runtime.status = "idle";
snapshot.models = [
{
provider: "provider-visible",
id: "visible",
name: "Visible model",
label: "Visible model",
current: true,
},
];
snapshot.truncation = {
...truncation,
modelsOmitted: 1,
truncated: true,
};
const baseStore = createWebStore();
const initialSearch = baseStore.getState().modelSearch;
const props = {
snapshot,
selectedWorkspace: "/tmp",
sessionSwitching: false,
promptAdmissionPending: false,
liveRunning: false,
landing: false,
activeTurn: null,
turnCancellationPending: false,
turnTerminalStatus: null,
pendingFollowUpsReceipt: null,
thinkingPendingLevel: null,
actions: baseStore.getState().actions,
modelSearch: initialSearch,
};
const { rerender } = renderWithI18n(createElement(Composer, props));

fireEvent.click(
screen.getByRole("button", {
name: "Visible model (provider-visible/visible)",
}),
);
fireEvent.change(
screen.getByPlaceholderText("Search provider, model name, or ID..."),
{ target: { value: "missing" } },
);
rerender(
createElement(
I18nextProvider,
{ i18n },
createElement(Composer, {
...props,
modelSearch: {
...initialSearch,
query: "missing",
status: "ready",
totalMatches: 0,
},
}),
),
);
expect(screen.getByText("No matching models")).toBeTruthy();

rerender(
createElement(
I18nextProvider,
{ i18n },
createElement(Composer, {
...props,
modelSearch: {
...initialSearch,
query: "missing",
status: "error",
error: "Model lookup failed",
},
}),
),
);
expect(screen.getByRole("alert").textContent).toBe("Model lookup failed");
});
Loading
Loading