Skip to content

fix(search): serve search.rooms from the ES spotlight index + correct frontend search contract#201

Merged
Joey0538 merged 6 commits into
mainfrom
claude/fix-search-rooms-contract
May 19, 2026
Merged

fix(search): serve search.rooms from the ES spotlight index + correct frontend search contract#201
Joey0538 merged 6 commits into
mainfrom
claude/fix-search-rooms-contract

Conversation

@Joey0538
Copy link
Copy Markdown
Collaborator

@Joey0538 Joey0538 commented May 19, 2026

Summary

Two related fixes to room/message search:

Backend — search.rooms served directly from ES (no Mongo hop)

search.rooms previously matched in the ES spotlight index, then re-hydrated every room from the Mongo subscriptions collection. The spotlight index is per-(account, room) and already carries roomId/roomName/roomType/siteId (search-sync-worker/spotlight.go), so the Mongo hop was redundant — a cross-store dependency, an extra round-trip, and a consistency window for data ES already had. Now the response is built directly from the spotlight hit.

  • pkg/model/search.go — add SiteID to SearchRoom (now an ES/wire projection, not a Mongo doc)
  • search-service/response.goparseRoomIDsparseRooms + toSearchRoom (reuses the existing generic rawResponse[T] envelope, mirroring the parseMessagesResponse/toSearchMessage pair)
  • search-service/handler.gosearchRooms returns ES-parsed rooms; Mongo path removed
  • search-service/store.go / store_mongo.goHydrateRooms and the subscriptions collection removed (added solely for this; searchApps does its own $lookup)
  • docs/client-api.md — updated (required: client-facing handler) — adds siteId, drops the "hydrated from MongoDB" wording

Frontend — corrected search wire contract

The chat-frontend was sending/parsing the wrong shapes against the backend:

  • Request payload field is query — the client was sending searchText, so every search submitted an empty query
  • Rooms filter is roomType (was scope; dropped the server-rejected app value)
  • Responses are {messages,total} / {rooms} (were results)
  • Room hit field is name (was roomName)
  • Dropped phantom total/userId; added editedAt?/updatedAt?; kept siteId (now legitimately returned)
  • Updated roomFromSearchHit, the 4 consumer components, and all affected tests

Also

Validation

  • make lint 0 issues; make test (all services, race) green; gosec PASS
  • go vet -tags=integration ./search-service/ clean (rooms integration tests converted to seed only the spotlight index — Mongo container removed from that fixture)
  • Frontend: npm run typecheck clean; npm test 540/540 pass
  • /simplify run on the branch: reuse clean, efficiency a net win (removes a per-request cross-store hop); the one must-fix (an unreachable nil guard) and a stale test name were addressed
  • govulncheck/semgrep could not run in the dev sandbox (no vuln.go.dev access; broken Python) — CI validates these

Test plan

  • CI sast job green (gosec + govulncheck + semgrep)
  • make test-integration SERVICE=search-service green (validates the ES-only search.rooms path end-to-end)
  • Manual: room/message search returns results in the UI (rooms typeahead, in-room search, global search pane, member-picker channel search)

Generated by Claude Code

Summary by CodeRabbit

  • New Features

    • Message search results now surface edit/update timestamps for clearer message history.
  • Improvements

    • Room and message search results and filters updated for more consistent labels and behavior (e.g., room names display consistently).
    • Search now returns results in spotlight relevance order with more complete room data (includes site info).
  • Documentation

    • Updated Search Rooms docs to reflect Elasticsearch-backed results and the revised response schema.

Review Change Stack

claude added 3 commits May 19, 2026 07:28
search.rooms previously matched in ES then re-hydrated each room from
the Mongo `subscriptions` collection. The spotlight index already
carries roomId/roomName/roomType/siteId per (account, room), so the
Mongo hop was redundant — it added a cross-store dependency, a round
trip, and a consistency window for fields ES already has. Serve the
response directly from the spotlight hit; drop HydrateRooms and the
subscriptions collection (added solely for this — apps does its own
$lookup). Add SiteID to model.SearchRoom.

Frontend: correct the search.rooms / search.messages wire contract.
The payload field is `query` (was incorrectly sending `searchText`),
the rooms filter is `roomType` (was `scope`, and dropped the
server-rejected `app` value), responses are `{messages,total}` /
`{rooms}` (were `results`), and the room hit field is `name` (was
`roomName`). Drop phantom fields; keep siteId now that the backend
returns it. Update consumers, tests, and docs/client-api.md.
The `// #nosec G402 -- ...` line directly above already states the
justification; the trailing `//nolint:gosec // ...` restated it in
slightly different words. Both directives remain (standalone gosec
and golangci-lint are independent mechanisms) — only the duplicated
prose is removed. Addresses the pending simplify nit from #197.
simplify review: parseRooms always returns a non-nil slice (nil only
with a non-nil error, handled above), so the `if rooms == nil` guard
in searchRooms was dead defensive code — removed. Renamed
TestSearchRoomsResponseJSON_EmptySubscriptions → _EmptyRooms now that
the path no longer touches subscriptions.
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 19, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4017dcb5-85b9-4cda-a6d2-f2c002a060c4

📥 Commits

Reviewing files that changed from the base of the PR and between 6e19170 and e0bb239.

📒 Files selected for processing (5)
  • .github/workflows/ci.yml
  • chat-frontend/src/components/MainApp/SearchResultsPane/SearchResultsPane.test.jsx
  • docker-local/setup.sh
  • docs/client-api.md
  • search-service/integration_test.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • docs/client-api.md
  • chat-frontend/src/components/MainApp/SearchResultsPane/SearchResultsPane.test.jsx
  • search-service/integration_test.go

📝 Walkthrough

Walkthrough

This PR migrates room search from MongoDB subscription hydration to direct Elasticsearch spotlight index projection, with corresponding updates to frontend search API contracts and consumers. Frontend search APIs now use explicit request payloads and renamed response fields (resultsmessages/rooms). Backend eliminates the Mongo hydration path entirely, relying on spotlight ES fields. All frontend components and tests updated accordingly.

Changes

Search API refactor and spotlight ES integration

Layer / File(s) Summary
Frontend search API contracts
chat-frontend/src/api/searchMessages/index.ts, chat-frontend/src/api/searchRooms/index.ts
SearchMessageHit drops userId and adds optional timestamp fields; SearchMessagesResponse.results renamed to messages. SearchRoomsArgs replaces scope with roomType; SearchRoomHit.roomName renamed to name, becomes simplified; SearchRoomsResponse changes from { total, results } to { rooms }. Both APIs now use destructured parameters and explicit request payloads with query key.
Backend data models
pkg/model/search.go, pkg/model/model_test.go
SearchRoom struct adds SiteID, removes bson tags (keeps json), documents as spotlight ES projection. Tests verify JSON marshaling including SiteID and empty rooms response.
Backend ES response parsing
search-service/response.go, search-service/response_test.go
New parseRooms replaces parseRoomIDs, extracting full room metadata (name, type, siteId) from spotlight hits into model.SearchRoom. Tests validate parsing, empty handling, malformed JSON, and order preservation.
Backend handler: ES-only search
search-service/handler.go, search-service/handler_test.go
Handler searchRooms now calls parseRooms directly on ES response, eliminating Mongo hydration. Tests expect ES-sourced fields without Mongo enrichment; added test for empty ES result.
Backend store cleanup
search-service/store.go, search-service/store_mongo.go, search-service/mock_store_test.go
MongoStore interface removes HydrateRooms method. mongoStore drops subscriptions collection and initialization. Mocks updated to remove HydrateRooms and align with new interface.
Frontend SearchBar
chat-frontend/src/components/MainApp/AppHeader/SearchBar/SearchBar.jsx, .test.jsx
Uses roomType filter, reads resp.rooms, renders hit.name. Tests mock new response shape and verify new request payload.
Frontend in-room search
chat-frontend/src/components/MainApp/ChatPage/InRoomSearch/InRoomSearch.jsx, .test.jsx
Reads message results from resp.messages. Tests mock and verify updated request/response shapes.
Frontend SearchResultsPane
chat-frontend/src/components/MainApp/SearchResultsPane/SearchResultsPane.jsx, .test.jsx
Uses roomType filter, reads room results from resp.rooms and message results from resp.messages, renders hit.name. Tests verify both response shapes and UI interactions.
Frontend MemberPicker
chat-frontend/src/components/MainApp/ChatPage/ManageMembersDialog/MemberPicker/MemberPicker.jsx, .test.jsx
Channel search uses roomType filter, reads resp.rooms, renders r.name. Tests mock updated shapes and verify request structure.
Frontend room formatting utility
chat-frontend/src/lib/roomFormat.js, .test.js
roomFromSearchHit maps hit.name instead of hit.roomName. Test fixture updated to match.
Backend integration tests
search-service/integration_test.go
Setup refactored to use ES-only data without Mongo; handler wired without Mongo store. Tests assert complete SearchRoom objects from spotlight index without Mongo hydration.
Documentation and cleanup
docs/client-api.md, pkg/oidc/oidc.go, pkg/searchengine/factory.go, .github/workflows/ci.yml, docker-local/setup.sh
Client API docs describe room search from spotlight ES index with siteId. CI git-fetch depth changed to full history; docker-local credentials file mode relaxed; minor TLS comment clarifications.

Sequence Diagram

sequenceDiagram
  participant Client
  participant Frontend
  participant SearchService
  participant Elasticsearch
  Client->>Frontend: user search (query, roomType?)
  Frontend->>SearchService: `.search.rooms` / `.search.messages` RPC with {query, roomType, size}
  SearchService->>Elasticsearch: spotlight index search request
  Elasticsearch-->>SearchService: ES hits (ordered)
  SearchService-->>Frontend: SearchRoomsResponse { rooms: [...] } or SearchMessagesResponse { messages: [...] }
  Frontend-->>Client: render results (hit.name / message hits)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • hmchangw/chat#166: Configures spotlight index wiring used by the searchRooms handler refactored in this PR.
  • hmchangw/chat#116: Prior changes to searchRooms handling and spotlight parsing overlap with this migration.

Suggested reviewers

  • mliu33
  • yenta

🐰 Hop hop, the search is now a breeze,
Spotlight shines from Elasticsearch with ease,
No more Mongo hydration chains to unwound,
Room projections bloom directly from the ground,
APIs aligned in perfect harmony.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.81% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly summarizes the main changes: serving search.rooms from the Elasticsearch spotlight index and correcting the frontend search contract.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/fix-search-rooms-contract

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/client-api.md (1)

1578-1633: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Search Rooms error docs still contain stale references.

In this updated Search Rooms section, the error anchor and backend wording are inconsistent with the current contract:

  • #5-error-envelope-reference should point to section 6.
  • internal reason still says ES or MongoDB backend failure, but this endpoint is now ES-only.
📄 Suggested doc patch
-See [Error envelope](`#5-error-envelope-reference`).
+See [Error envelope](`#6-error-envelope-reference`).

-| `internal`    | ES or MongoDB backend failure (transient or permanent). The raw error is never leaked to the client. |
+| `internal`    | ES backend failure (transient or permanent). The raw error is never leaked to the client. |
As per coding guidelines: "`docs/client-api.md`: If changes touch a client-facing handler ... update `docs/client-api.md` in the same PR to reflect the new request/response schema, error cases, and triggered events."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/client-api.md` around lines 1578 - 1633, Update the Search Rooms docs to
fix stale references and backend wording: replace the
`#5-error-envelope-reference` anchor with the correct
`#6-error-envelope-reference` and change the `internal` error reason text from
"ES or MongoDB backend failure (transient or permanent)." to reflect ES-only
(e.g., "Elasticsearch backend failure (transient or permanent). The raw error is
never leaked to the client."). Ensure the error table entries for `bad_request`
and `internal` remain accurate (keep mention that raw errors are not leaked) and
that `roomType` validation notes still reject `"app"` and unrecognized values.
🧹 Nitpick comments (2)
chat-frontend/src/components/MainApp/SearchResultsPane/SearchResultsPane.test.jsx (1)

40-43: ⚡ Quick win

Strengthen request contract assertion for room search.

Line 42 only checks query; it won’t catch regressions in roomType/size for this path.

Proposed test update
     expect(request).toHaveBeenCalledWith(
       'chat.user.alice.request.search.rooms',
-      expect.objectContaining({ query: 'gen' })
+      { query: 'gen', roomType: 'all', size: 50 }
     )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@chat-frontend/src/components/MainApp/SearchResultsPane/SearchResultsPane.test.jsx`
around lines 40 - 43, The test currently asserts that the mocked request was
called with 'chat.user.alice.request.search.rooms' and only checks { query:
'gen' }, which misses regressions in roomType/size; update the assertion in
SearchResultsPane.test.jsx to expect the request payload to include the full
contract (e.g., include roomType and size alongside query) by replacing
expect.objectContaining({ query: 'gen' }) with an object matcher that asserts {
query: 'gen', roomType: <expectedValue>, size: <expectedValue> } (or use
expect.objectContaining with all three keys) for the
'chat.user.alice.request.search.rooms' call to ensure room search parameters are
validated.
search-service/integration_test.go (1)

1051-1057: ⚡ Quick win

Add a cross-account leakage assertion to this ES-only path.

Now that search.rooms no longer re-hydrates through Mongo subscriptions, the auth boundary lives entirely in the spotlight query. This test only seeds Alice's docs, so it won't catch a missing userAccount filter.

🧪 Suggested test hardening
 func TestIntegration_SearchRooms_HappyPath(t *testing.T) {
 	f := setupRoomsFixture(t)

 	const account = "alice"
 	now := time.Now().UTC()

 	// Seed spotlight docs for two rooms alice is in.
 	seedDoc(t, f.esURL, "spotlight-subs-test", "spot-r1", map[string]any{
 		"roomId":      "r1",
 		"roomName":    "engineering-announcements",
 		"roomType":    "channel",
 		"userAccount": account,
 		"siteId":      "site-local",
 		"joinedAt":    now.Add(-48 * time.Hour).Format(time.RFC3339),
 	})
 	seedDoc(t, f.esURL, "spotlight-subs-test", "spot-r2", map[string]any{
 		"roomId":      "r2",
 		"roomName":    "engineering-random",
 		"roomType":    "channel",
 		"userAccount": account,
 		"siteId":      "site-local",
 		"joinedAt":    now.Add(-24 * time.Hour).Format(time.RFC3339),
 	})
+	seedDoc(t, f.esURL, "spotlight-subs-test", "spot-r3", map[string]any{
+		"roomId":      "r3",
+		"roomName":    "engineering-secret",
+		"roomType":    "channel",
+		"userAccount": "mallory",
+		"siteId":      "site-local",
+		"joinedAt":    now.Add(-12 * time.Hour).Format(time.RFC3339),
+	})

 	reqBytes, err := json.Marshal(model.SearchRoomsRequest{Query: "engineering"})
 	require.NoError(t, err)

 	msg, err := f.clientNATS.Request(subject.SearchRooms(account), reqBytes, 10*time.Second)
 	require.NoError(t, err)

 	var resp model.SearchRoomsResponse
 	require.NoError(t, json.Unmarshal(msg.Data, &resp))

 	require.Len(t, resp.Rooms, 2, "both rooms matching 'engineering' must be returned")
 	byID := map[string]model.SearchRoom{}
 	for _, r := range resp.Rooms {
 		byID[r.RoomID] = r
 	}
 	assert.Equal(t, model.SearchRoom{RoomID: "r1", Name: "engineering-announcements", RoomType: "channel", SiteID: "site-local"}, byID["r1"])
 	assert.Equal(t, model.SearchRoom{RoomID: "r2", Name: "engineering-random", RoomType: "channel", SiteID: "site-local"}, byID["r2"])
+	_, leaked := byID["r3"]
+	assert.False(t, leaked, "rooms from other accounts must not leak")
 }

As per coding guidelines, "Tests must cover happy path, error paths, edge cases (empty collections, boundary conditions), and invalid input."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@search-service/integration_test.go` around lines 1051 - 1057, The ES-only
test currently seeds only Alice's docs and asserts two rooms are returned, but
doesn't guard against cross-account leakage; update the test that inspects
resp.Rooms (model.SearchRoom) to also assert that every returned room is owned
by Alice's account (e.g., check a userAccount/account field on each resp.Rooms
entry or ensure no room with another account ID appears), or alternatively seed
a room belonging to a different account and assert it is NOT returned;
specifically modify the assertions around resp.Rooms/byID (and the spotlight
query setup) so the test fails if results lack the required userAccount filter.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@docs/client-api.md`:
- Around line 1578-1633: Update the Search Rooms docs to fix stale references
and backend wording: replace the `#5-error-envelope-reference` anchor with the
correct `#6-error-envelope-reference` and change the `internal` error reason
text from "ES or MongoDB backend failure (transient or permanent)." to reflect
ES-only (e.g., "Elasticsearch backend failure (transient or permanent). The raw
error is never leaked to the client."). Ensure the error table entries for
`bad_request` and `internal` remain accurate (keep mention that raw errors are
not leaked) and that `roomType` validation notes still reject `"app"` and
unrecognized values.

---

Nitpick comments:
In
`@chat-frontend/src/components/MainApp/SearchResultsPane/SearchResultsPane.test.jsx`:
- Around line 40-43: The test currently asserts that the mocked request was
called with 'chat.user.alice.request.search.rooms' and only checks { query:
'gen' }, which misses regressions in roomType/size; update the assertion in
SearchResultsPane.test.jsx to expect the request payload to include the full
contract (e.g., include roomType and size alongside query) by replacing
expect.objectContaining({ query: 'gen' }) with an object matcher that asserts {
query: 'gen', roomType: <expectedValue>, size: <expectedValue> } (or use
expect.objectContaining with all three keys) for the
'chat.user.alice.request.search.rooms' call to ensure room search parameters are
validated.

In `@search-service/integration_test.go`:
- Around line 1051-1057: The ES-only test currently seeds only Alice's docs and
asserts two rooms are returned, but doesn't guard against cross-account leakage;
update the test that inspects resp.Rooms (model.SearchRoom) to also assert that
every returned room is owned by Alice's account (e.g., check a
userAccount/account field on each resp.Rooms entry or ensure no room with
another account ID appears), or alternatively seed a room belonging to a
different account and assert it is NOT returned; specifically modify the
assertions around resp.Rooms/byID (and the spotlight query setup) so the test
fails if results lack the required userAccount filter.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7fca3973-1374-41da-8764-e974c6e9492c

📥 Commits

Reviewing files that changed from the base of the PR and between a55a505 and 6e19170.

📒 Files selected for processing (25)
  • chat-frontend/src/api/searchMessages/index.ts
  • chat-frontend/src/api/searchRooms/index.ts
  • chat-frontend/src/components/MainApp/AppHeader/SearchBar/SearchBar.jsx
  • chat-frontend/src/components/MainApp/AppHeader/SearchBar/SearchBar.test.jsx
  • chat-frontend/src/components/MainApp/ChatPage/InRoomSearch/InRoomSearch.jsx
  • chat-frontend/src/components/MainApp/ChatPage/InRoomSearch/InRoomSearch.test.jsx
  • chat-frontend/src/components/MainApp/ChatPage/ManageMembersDialog/MemberPicker/MemberPicker.jsx
  • chat-frontend/src/components/MainApp/ChatPage/ManageMembersDialog/MemberPicker/MemberPicker.test.jsx
  • chat-frontend/src/components/MainApp/SearchResultsPane/SearchResultsPane.jsx
  • chat-frontend/src/components/MainApp/SearchResultsPane/SearchResultsPane.test.jsx
  • chat-frontend/src/lib/roomFormat.js
  • chat-frontend/src/lib/roomFormat.test.js
  • docs/client-api.md
  • pkg/model/model_test.go
  • pkg/model/search.go
  • pkg/oidc/oidc.go
  • pkg/searchengine/factory.go
  • search-service/handler.go
  • search-service/handler_test.go
  • search-service/integration_test.go
  • search-service/mock_store_test.go
  • search-service/response.go
  • search-service/response_test.go
  • search-service/store.go
  • search-service/store_mongo.go
💤 Files with no reviewable changes (2)
  • search-service/store.go
  • search-service/mock_store_test.go

claude added 2 commits May 19, 2026 07:44
- docs/client-api.md (search.rooms): fix stale #5#6 error-envelope
  anchor (every other ref uses #6); the `internal` reason no longer
  says "ES or MongoDB" — this endpoint is ES-only now.
- SearchResultsPane.test.jsx: assert the full search.rooms wire
  payload {query,roomType,size}, not just query.
- integration_test.go: seed a room owned by another account and
  assert it does not leak. With Mongo hydration removed, the
  spotlight userAccount term filter is the sole access boundary —
  this guards that regression directly.
The "Detect affected integration targets" step did
`git fetch --depth=1 origin <base>` then a three-dot
`origin/<base>...HEAD` diff. Three-dot needs a merge base, but a
depth-1 base fetch has no history, so the step intermittently
failed with "no merge base" (exit 128), blocking every downstream
job. Drop --depth=1; checkout already uses fetch-depth: 0, so a
full base fetch resolves the merge base reliably.
Copy link
Copy Markdown
Collaborator

@GITMateuszCharczuk GITMateuszCharczuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, confirmed its running just fine on codespaces

Service containers run as non-root (uid 10001, since the runtime
hardening in #197) and bind-mount docker-local/backend.creds
read-only at /etc/nats/backend.creds. setup.sh wrote it 0600, so the
in-container user got "permission denied" on `make up`. Generate it
0644 instead. Safe only because this is a throwaway local-dev
credential created by this script; the .env file stays 0600 (read by
the compose CLI on the host, never mounted).
@Joey0538 Joey0538 merged commit 7b5747b into main May 19, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants