ci: Change host tests to use cached index - #5655
Merged
Merged
Conversation
Contributor
Preview deploymentsHost Test Results 1 files ±0 1 suites ±0 2h 21m 38s ⏱️ + 1m 30s Results for commit c312725. ± Comparison against earlier commit fcb7eb4. Realm Server Test Results 1 files ±0 1 suites ±0 14m 41s ⏱️ +23s Results for commit c312725. ± Comparison against earlier commit fcb7eb4. |
backspace
force-pushed
the
ci-host-index-cache-cs-12399
branch
from
July 31, 2026 17:25
5929660 to
8fa4b92
Compare
backspace
marked this pull request as ready for review
July 31, 2026 18:31
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8fa4b92b31
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Each of the 20 host-test shards, and live-test, spent ~2.5-3 min waiting for the base, skills and openrouter realms to index from scratch before testem launched. CI already snapshots that state; only local dev consumed it. A shard now imports the snapshot and lets its boot index reconcile it against the checkout. `discoverInvalidations` skips every file whose mtime matches its indexed row and revisits the rest, so the shard re-renders what this checkout changed — plus whatever the invalidation fan-out reaches from it — and keeps cached rows for everything else. `applyBatchUpdates` upserts only the invalidated URLs, so rows the pass never visits survive, and `_readiness-check` awaits the in-flight index, so tests cannot start against a half-reconciled realm. That reconciliation is what makes the cache safe for a PR that edits realm content: mtimes are content-derived on both sides, so a changed card is re-rendered rather than served from main's snapshot. - `ci:import-index` brings up the database and imports, as its own step ahead of the services. Not inside `test-services:host`: that task hands straight to `start-server-and-test`, so work in front of it delays Synapse and `register-realm-users` fails after its 24 attempts. - It asks for the host-scoped artifact, falling back to the full snapshot, and reports download and replay time separately with the compressed size. - `check-index-cache` gates on changes to how indexing or rendering behaves, where cached rows for untouched files can be wrong with nothing in their content to signal it. Push-to-main never uses the cache, so the memory baseline is not measured against a differently-seeded index. - `test-services:host` normalizes realm mtimes on every CI shard, cache or not, so `last-modified` and the "last saved" text the host renders from it read the same either way — Percy baselines come from main runs, which never import. A cache miss anywhere along that path leaves the shard indexing from scratch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
prerendered_html rows are produced by rendering cards through the host bundle, and reconciliation only revisits files whose mtimes changed — so a card this checkout did not touch keeps HTML built by main's host code. Search entries serve that HTML through their htmlQuery, which means a change to the code emitting it could leave tests asserting against main's markup and let a rendering regression pass. Send the host render path down the from-scratch route with the other index-behavior paths: the prerender and renderer components, isolated-render, the prerender lib helpers, the render route and its sub-routes, the render services, the render template, and the render utils. Client-side consumers of that HTML are left out — they read prerendered and freshly rendered HTML the same way, so a change there needs no reindex. Scoped to the render path rather than all of packages/host, which is this workflow's main path trigger and would return nearly every run to from-scratch indexing. Operator mode, the AI assistant and code submode never appear in a card's prerendered HTML — the same reasoning behind the deployed post-deploy reindex keying on a boxel-ui checksum rather than on host-bundle identity. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
backspace
force-pushed
the
ci-host-index-cache-cs-12399
branch
from
August 4, 2026 14:20
363bc34 to
fcb7eb4
Compare
lukemelia
approved these changes
Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It uses the reduced cache produced by #5662 to save some setup time for host tests. When files that touch the render pipeline are changed, the cache isn’t used in CI.
Analysis from Claude:
In a host shard log you can see the cache working, as the base and skills realms are ready immediately, the test realm is the one that needs indexing:
This suggests another possible improvement: including the test realm in the host test index cache. I’ll check that out with CS-12432.