feat(search): require versioned index envs, derive template + read patterns

[Joey0538]

Summary

• New pkg/searchindex package with StripVersion(name) → (base, version, ok) and StripVersionBase(name) → base helpers (single source of truth for the -v<N>$ parsing rule).
• search-sync-worker: MSG_INDEX_PREFIX, SPOTLIGHT_INDEX, USER_ROOM_INDEX are all now ,required (no silent empty defaults). Messages and spotlight values must end in -v<N> — validated at startup, fails fast with a clear error. User-room is intentionally unversioned (ES terms_lookup rejects wildcards, and the index is rebuildable from Mongo).
• Messages + spotlight collections: ES template name and index_patterns are now derived from the stripped base (e.g. messages-site-a_template + messages-site-a-*) so future v2/v3 indices auto-inherit the template. The actual write target (monthly index for messages, concrete versioned index for spotlight) stays unchanged.
• User-room: template name now derived from the env value instead of a hardcoded user_room_template, so each deployment gets a uniquely-named template. index_patterns is the single concrete name (matching the terms_lookup constraint).
• search-service: validates SEARCH_SPOTLIGHT_INDEX ends in -v<N> and derives a <base>-* read pattern at startup so message-search and room-search span all versions during reindex windows. Handler field renamed SpotlightIndex → SpotlightReadPattern to reflect the new semantics. SEARCH_USER_ROOM_INDEX flows through unchanged (concrete name required for terms_lookup).
• Dropped dead const UserRoomIndex = "user-room" and resolveUserRoomIndex empty-string fallback in search-service now that the env is required.
• docker-compose files for both services updated with values that match the new contract (spotlight-site-local-v1, user-room-mv-site-local).
• Plan doc at docs/superpowers/plans/2026-05-13-search-index-env-vars.md captures the design and reindex playbook.

Reindex flow this unlocks

1. Worker + search-service running on -v1. search-service queries <base>-* (matches v1).
2. Create <base>-v2 indices manually or via ES template auto-create.
3. _reindex v1 → v2 out of band.
4. Bump both services' envs to -v2, redeploy. Writes go to v2; reads via <base>-* see both.
5. Drop v1 when ready.

User-room doesn't get this flow by design — rebuild from Mongo subscriptions if its schema ever needs a breaking change.

Test plan

• Unit tests: make test — all packages green.
• Lint: make lint — 0 issues.
• Integration build: go vet -tags integration ./search-service/... ./search-sync-worker/... — clean.
• Integration tests: make test-integration SERVICE=search-sync-worker and make test-integration SERVICE=search-service (require Docker — to run in CI).
• Local smoke: docker compose up for both services brings them up with the new env values; ES shows three templates (messages-site-local_template, spotlight-site-local_template, user-room-mv-site-local_template).

Out of scope

• Production env coordination. Ops/IaC sets per-site values. Any prod site currently running the old spotlight-{site}-v1-chat value needs a coordinated rename (this PR only touches local-dev compose).
• Federation of spotlight/user-room. Only messages search uses cross-cluster *: patterns today.
• Reindex tooling automation — a future PR could add a make reindex target if needed.

https://claude.ai/code/session_01E2Cc6bj5Qo8GsYwZV8YEC2

---

Generated by Claude Code

Summary by CodeRabbit

• Documentation

  • Added plan describing standardized Elasticsearch index env-var contract and verification steps.

• Refactor

  • Index env vars are now required and validated; removed runtime fallback/default logic.
  • Versioned indices must use a -v<N> suffix; user-room index remains unversioned for terms_lookup compatibility.
  • Templates and index_patterns now derive from a stripped base name; spotlight queries use wildcard read patterns.

• Tests

  • Updated unit/integration tests and local compose configs to match new naming and validation rules.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds a new pkg/searchindex helper for parsing -v<N> suffixes, makes search index environment variables required, validates versioned indices for messages/spotlight, strips versions for template name and index_patterns generation (messages/spotlight), keeps user-room indices exact for terms_lookup, and wires a derived spotlight read pattern into the handler.

Changes

Search Index Environment-Variable Contract & Version Handling

Layer / File(s)	Summary
Implementation Plan & Searchindex Package docs/superpowers/plans/2026-05-13-search-index-env-vars.md, pkg/searchindex/version.go, pkg/searchindex/version_test.go	Adds a 10-task doc plan and a searchindex package with StripVersion / StripVersionBase and table-driven tests for parsing -v<N> suffixes.
search-sync-worker Configuration & Version Validation search-sync-worker/main.go, search-sync-worker/deploy/docker-compose.yml	Marks SPOTLIGHT_INDEX and USER_ROOM_INDEX required, removes SITE_ID fallback, and validates message/spotlight prefixes end with -v<N> using searchindex.StripVersion; updates worker docker-compose env values.
search-sync-worker Template Name & Index Pattern Generation search-sync-worker/messages.go, search-sync-worker/messages_test.go, search-sync-worker/spotlight.go, search-sync-worker/spotlight_test.go, search-sync-worker/user_room.go, search-sync-worker/user_room_test.go, search-sync-worker/inbox_integration_test.go	Message and spotlight template names and index_patterns are derived from version-stripped bases (wildcards like messages-site1-*, spotlight-site-a-*); user-room templates use the exact unversioned env value; tests updated accordingly.
search-sync-worker Docker Compose Configuration search-sync-worker/deploy/docker-compose.yml	Adds SPOTLIGHT_INDEX and USER_ROOM_INDEX env vars to the worker local docker-compose.
search-service Spotlight Pattern Derivation & Wiring search-service/main.go, search-service/handler.go, search-service/handler_test.go	Validates SEARCH_SPOTLIGHT_INDEX must have -v<N>, derives <base>-* SpotlightReadPattern at startup, renames/wires handler config and updates handler/tests to query using the derived pattern.
search-service User-Room Index Fallback Removal & Query Simplification search-service/store_es.go, search-service/store_es_test.go, search-service/query_messages.go, search-service/query_messages_test.go	Removes UserRoomIndex and resolver, requires concrete user-room index upstream, simplifies buildMessageQuery and terms-lookup comments, and updates tests to pass a non-empty user-room index.
search-service Integration Tests & Deployment search-service/integration_test.go, search-service/deploy/docker-compose.yml	Introduces a test-specific testUserRoomIndex for fixtures, replaces single spotlight index with wildcard pattern in tests, and updates docker-compose env values to match the new contract.
Docs & Verification Tasks docs/superpowers/plans/2026-05-13-search-index-env-vars.md	Lists verification steps, unit/integration checks, and deferred production coordination/reindex tasks.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• hmchangw/chat#166: Overlaps on spotlight index selection and handler wiring; related to the handler config change to use a read pattern.
• hmchangw/chat#109: Related prior work on spotlight and user-room collection template naming and index_patterns.
• hmchangw/chat#64: Earlier collection/template generation changes that this PR builds upon.

Suggested reviewers

• mliu33
• GITMateuszCharczuk

Poem

🐰 I trim the suffix, find the base,
Templates match with gentle grace.
Wildcards stretch where versions hide,
Exact names keep lookups tied.
Hopping indexes, springing pride.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 27.03% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely summarizes the main change: requiring versioned index environment variables and deriving template/read patterns from them.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/search-index-env-vars-5cts2

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}