Skip to content

fix(admin): extend server-side content search to the content picker and MCP#752

Open
edrpls wants to merge 2 commits into
emdash-cms:mainfrom
edrpls:fix/content-list-search
Open

fix(admin): extend server-side content search to the content picker and MCP#752
edrpls wants to merge 2 commits into
emdash-cms:mainfrom
edrpls:fix/content-list-search

Conversation

@edrpls
Copy link
Copy Markdown
Contributor

@edrpls edrpls commented Apr 24, 2026

What does this PR do?

Extends the server-side content list search (?q=) — which shipped via #1226 — to two surfaces that still post-filtered in memory:

  • ContentPickerModal (used when linking content from the editor) now pushes its search box to the server (search option → ?q=) instead of filtering only the rows already loaded, so it finds entries anywhere in a large collection. It uses keepPreviousData so the list doesn't flash to empty between keystrokes, and keeps load-more available while searching (results can span multiple pages).
  • MCP content_list tool gains a q parameter, so agents can search a collection server-side rather than post-filtering a single page of results.

This branch was rebased onto main after #1226 merged an independent implementation of the core feature. The overlapping core-search work (handler, repository, schemas, ContentList, router, API client, index migration) is now provided by #1226; this PR keeps only the picker + MCP extensions on top of it.

Related: #1219, #1226

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (or targeted tests for my change)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (if applicable). Do not include messages.po changes except in translation PRs — a workflow extracts catalogs on merge to main.
  • I have added a changeset (if this PR changes a published package)
  • New features link to an approved Discussion: https://github.com/emdash-cms/emdash/discussions/...

AI-generated code disclosure

  • This PR includes AI-generated code — model/tool: Claude Opus 4.8 (Claude Code)

Screenshots / test output

MCP integration tests pass, including a new case asserting content_list filters by q:

Test Files  1 passed (1)
     Tests  27 passed (27)

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 24, 2026

🦋 Changeset detected

Latest commit: c4cb0e9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 14 packages
Name Type
emdash Patch
@emdash-cms/admin Patch
@emdash-cms/cloudflare Patch
@emdash-cms/sandbox-workerd Patch
@emdash-cms/fixture-perf-site Patch
@emdash-cms/perf-demo-site Patch
@emdash-cms/cache-demo-site Patch
@emdash-cms/auth Patch
@emdash-cms/blocks Patch
@emdash-cms/gutenberg-to-portable-text Patch
@emdash-cms/x402 Patch
create-emdash Patch
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

Scope check

This PR changes 1,912 lines across 29 files. Large PRs are harder to review and more likely to be closed without review.

If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs.

See CONTRIBUTING.md for contribution guidelines.

@github-actions
Copy link
Copy Markdown
Contributor

Lunaria Status Overview

🌕 This pull request will trigger status changes.

Learn more

By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.

You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

File Note
packages/admin/src/locales/ar/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/de/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/en/messages.po Source changed, localizations will be marked as outdated.
packages/admin/src/locales/es-419/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/eu/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/fa/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/fr/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/ja/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/ko/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/pseudo/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/pt-BR/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/zh-CN/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/zh-TW/messages.po Localization changed, will be marked as complete.
Warnings reference
Icon Description
🔄️ The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented Apr 24, 2026

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@752

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@752

@emdash-cms/auth-atproto

npm i https://pkg.pr.new/@emdash-cms/auth-atproto@752

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@752

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@752

@emdash-cms/contentful-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/contentful-to-portable-text@752

emdash

npm i https://pkg.pr.new/emdash@752

create-emdash

npm i https://pkg.pr.new/create-emdash@752

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@752

@emdash-cms/plugin-cli

npm i https://pkg.pr.new/@emdash-cms/plugin-cli@752

@emdash-cms/plugin-types

npm i https://pkg.pr.new/@emdash-cms/plugin-types@752

@emdash-cms/registry-client

npm i https://pkg.pr.new/@emdash-cms/registry-client@752

@emdash-cms/registry-lexicons

npm i https://pkg.pr.new/@emdash-cms/registry-lexicons@752

@emdash-cms/sandbox-workerd

npm i https://pkg.pr.new/@emdash-cms/sandbox-workerd@752

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@752

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@752

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@752

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@752

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@752

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@752

@emdash-cms/plugin-field-kit

npm i https://pkg.pr.new/@emdash-cms/plugin-field-kit@752

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@752

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@752

commit: c4cb0e9

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

This PR has been inactive for 14 days. It will be closed automatically in 7 days if there is no further activity.

If you're still working on this, please push an update or leave a comment.

@edrpls
Copy link
Copy Markdown
Contributor Author

edrpls commented May 16, 2026

Rebased on upstream main (f28eb7b). Resolved orthogonal conflicts with #750 (sort headers, since merged) — <ContentList> now takes both sort/onSortChange and searchQuery/onSearchChange props; router threads sort + debounced search through the same TanStack Query key; fetchContentList accepts both orderBy/order and q; describe("orderBy"), describe("search"), and describe("sortable headers") test blocks coexist. Locale .po files reset to upstream for Lunaria re-extraction.

Local test runs after rebase, on top of f28eb7b:

  • packages/core/tests/database/repositories/content.test.ts: 51/51 pass (includes 6 new search tests)
  • packages/core/tests/unit/api/content-handlers.test.ts: 33/33 pass
  • packages/admin/tests/components/ContentList.test.tsx: 38/38 pass

Bumping out of stale-warning territory.

@edrpls
Copy link
Copy Markdown
Contributor Author

edrpls commented May 18, 2026

Rebased on upstream main (aaf021c) to clear conflicts after #751 merged.

Conflicts resolved (all orthogonal to #751's now-landed total/pagination machinery — same files, adjacent additions):

  • packages/admin/src/components/ContentList.tsx — kept total + clampedPage/paginatedItems from main alongside new searchQuery/onSearchChange props. The auto-fetch effect now bails on client-side search only (!serverSideSearch && searchQuery), so server-driven search can still page forward into the filtered set.
  • packages/admin/src/router.tsx — both total={total} and searchQuery/onSearchChange props pass through; the existing useInfiniteQuery key already threads sort + q: debouncedSearch so cache invalidation is correct.
  • packages/core/src/database/repositories/content.ts — auto-merged. findMany now runs count(type, where) in parallel via Promise.all, and count itself resolves searchSpec so the denominator respects an active query.
  • packages/core/tests/database/repositories/content.test.ts + tests/unit/api/content-handlers.test.ts — kept both describe("total" / "list total") and describe("search" / "list search") blocks side-by-side instead of choosing one.

Local verification against aaf021c:

Suite Result
packages/core repo + handler tests (content.test.ts + content-handlers.test.ts) 88 / 88
packages/core dialect tests (dialect-compat + dialect-runtime-imports) 17 / 17
packages/admin ContentList.test.tsx 39 / 39
pnpm --filter emdash --filter @emdash-cms/admin typecheck clean

@github-actions github-actions Bot added the review/needs-review No maintainer or bot review yet label May 30, 2026
@ascorbic ascorbic added the bot:review Trigger an emdashbot code review on this PR label May 31, 2026
Copy link
Copy Markdown
Contributor

@emdashbot emdashbot Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach judgment: This is the right fix for a real bug. Filtering search client-side against an incomplete accumulator was fundamentally broken for any collection larger than a single API page. Moving the filter to the server via a q parameter is the correct architecture, and the repository implementation is careful about SQL safety (parameterized pragma_table_info, sql.ref() for columns, LIKE escape, input truncation).

What I checked: API schema, route wiring, ContentRepository.findMany/count, listTableColumns dialect helper, admin ContentList and ContentPickerModal, MCP tool exposure, and the test coverage.

Headline conclusion: The server-side query building is clean and safe, but the admin UI integration missed three edge cases where server-side search changes the meaning of items (from "loaded pages" to "filtered result set"). These aren't catastrophic, but they break pagination and create a UX trap when a search returns zero results.

Fixes needed:

  1. ContentList pagination denominator (effectiveTotal) should trust the server total in server-side search mode.
  2. The search input must remain visible even when a server-side search returns zero results.
  3. The "empty collection" state should not appear for a zero-result server-side search.
  4. ContentPickerModal should not hide the load-more button during search.

@github-actions github-actions Bot added review/awaiting-author Reviewed; waiting on the author to respond and removed review/needs-review No maintainer or bot review yet labels May 31, 2026
edrpls added a commit to edrpls/emdash that referenced this pull request Jun 2, 2026
Addresses review feedback on PR emdash-cms#752 where server-side search changed
the meaning of `items` from "loaded pages" to "filtered result set",
breaking pagination and creating a zero-result UX trap.

- ContentList denominator now trusts the server `total` in server-side
  search mode (it already reflects the filtered count) instead of
  collapsing to the loaded page size while a query is active.
- The search input stays mounted when a server-side search returns zero
  results, so the user can still edit or clear the query.
- A zero-result server-side search shows the "no results" message rather
  than the "empty collection / create your first one" CTA.
- ContentPickerModal keeps the load-more button visible during search,
  since server-side search results are paginated too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added review/needs-rereview Author pushed changes since the last review and removed review/awaiting-author Reviewed; waiting on the author to respond labels Jun 2, 2026
@edrpls
Copy link
Copy Markdown
Contributor Author

edrpls commented Jun 2, 2026

Thanks for the careful review — all four UI edge cases are fixed in 37749f3:

  1. Pagination denominatoreffectiveTotal now trusts the server total whenever server-side search is active (serverSideSearch || !searchQuery). The server's total already reflects the filtered set, since findMany passes search through to count, so the denominator no longer collapses to the loaded page size mid-search.
  2. Search input visibility — the input now stays mounted when there's content or an active server-side query (items.length > 0 || (serverSideSearch && searchQuery)), so a zero-result search no longer hides the box and traps the user.
  3. Empty-collection vs. no-results — the "create your first one" CTA is now gated on items.length === 0 && !searchQuery; a zero-result server-side search falls through to the "No results for …" message instead.
  4. ContentPickerModal load-more — dropped the !searchQuery guard on the load-more button so paginated search results can be loaded further.

Added three ContentList tests covering the server-side denominator, input persistence, and the no-results-vs-empty-collection distinction. Full suite green (42/42).

@edrpls edrpls force-pushed the fix/content-list-search branch from 37749f3 to 59d9e82 Compare June 2, 2026 00:49
edrpls added a commit to edrpls/emdash that referenced this pull request Jun 2, 2026
Addresses review feedback on PR emdash-cms#752 where server-side search changed
the meaning of `items` from "loaded pages" to "filtered result set",
breaking pagination and creating a zero-result UX trap.

- ContentList denominator now trusts the server `total` in server-side
  search mode (it already reflects the filtered count) instead of
  collapsing to the loaded page size while a query is active.
- The search input stays mounted when a server-side search returns zero
  results, so the user can still edit or clear the query.
- A zero-result server-side search shows the "no results" message rather
  than the "empty collection / create your first one" CTA.
- ContentPickerModal keeps the load-more button visible during search,
  since server-side search results are paginated too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor Author

@edrpls edrpls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated high-effort review (/code-review high) of the server-side search change. 9 findings below as inline comments, ranked by severity: 2 correctness gaps that matter for an i18n CMS, a count/denominator mismatch, two picker state races, a perf regression on D1, a Postgres dialect-parity issue, a debounce UI flash, and one reuse cleanup. None block the core fix; the non-ASCII and custom-field ones are the most worth addressing before merge.

Comment thread packages/core/src/database/repositories/content.ts Outdated
Comment thread packages/core/src/database/repositories/content.ts Outdated
Comment thread packages/admin/src/components/ContentList.tsx
Comment thread packages/admin/src/components/ContentPickerModal.tsx Outdated
Comment thread packages/core/src/database/repositories/content.ts Outdated
Comment thread packages/core/src/database/dialect-helpers.ts Outdated
Comment thread packages/admin/src/components/ContentPickerModal.tsx Outdated
Comment thread packages/admin/src/components/ContentList.tsx Outdated
Comment thread packages/core/src/database/repositories/content.ts Outdated
edrpls added a commit to edrpls/emdash that referenced this pull request Jun 2, 2026
…spec once

Two fixes from the high-effort review of PR emdash-cms#752:

- ContentList: in server-side search mode the row-count line above
  pagination reported the loaded-items count, not the server's filtered
  `total`, so it contradicted the pagination denominator until every page
  was fetched. renderItemCount now uses `total` for the match count in
  server-side mode (client-side mode still uses the locally-filtered count,
  since `total` there reflects the unfiltered set).

- ContentRepository: a searched list resolved the search predicate twice —
  once in findMany and once in count — each running a table-column
  introspection query, doubling round-trips on D1. count now accepts an
  optional pre-resolved searchSpec; findMany passes its own through, so the
  introspection runs once per request.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@edrpls
Copy link
Copy Markdown
Contributor Author

edrpls commented Jun 2, 2026

@emdashbot — ran a high-effort multi-agent review on this branch and pushed fixes for the two low-risk findings. Summary below, with a couple of design calls that need a maintainer decision before I go further.

✅ Fixed in d087bc86

  • Search count line contradicted the pagination denominator. In server-side mode renderItemCount reported the loaded item count ("20 items matching…") while the pager used the server total ("1 / 8"). Now uses total for the match count in server-side mode; client-side mode still uses the local filtered count. Test added.
  • Double table-column introspection per search. findMany resolved the search predicate and then count resolved it again, so each searched request ran listTableColumns twice (4 D1 round-trips instead of 2). count now takes an optional pre-resolved searchSpec and findMany passes its own through — introspection runs once.

❓ Need a maintainer decision (design altitude)

  1. Non-ASCII case-insensitive search is broken on SQLite. The pattern is lowercased in JS (full Unicode) but the column uses SQLite's LOWER(), which folds ASCII only (no ICU/COLLATE configured). A row titled ÉCOLE won't match école. For a CMS that ships i18n/RTL as a headline feature this seems significant — but the fix (FTS-backed search, or ICU/unaccent on PG + a documented SQLite caveat) is a bigger call than I want to make unilaterally. Worth blocking the merge, or acceptable as a known limitation + follow-up?
  2. Search only targets hardcoded ["title","name","slug"], ignoring the existing per-field _emdash_fields.searchable flag (read elsewhere via FTSManager.getSearchableFields). A collection whose primary text field is headline/body is silently unsearchable. Should the list q filter derive searchable columns from the field schema instead of a constant? (This also overlaps conceptually with the FTS layer — want them unified or kept separate?)

🟡 Lower-priority findings (left for your call — happy to fix any in this PR)

  • Postgres dialect parity: listTableColumns hardcodes table_schema = 'public' while sibling columnExists/indexExists use current_schema() (migration 038 documents this footgun). On a non-public-schema PG deployment, search silently returns the entire unfiltered collection. One-line fix; flagging since tableExists/listTablesLike share the hardcode (mixed precedent).
  • ContentPickerModal: load-more results wiped by a background refetch (no staleTime; refocus/invalidation re-settles to page 1 and the accumulator-reset effect discards appended rows). Cleaner as useInfiniteQuery.
  • ContentPickerModal: load-more cursor/query mismatch during the 300ms debounce window (old cursor + new q).
  • ContentList: search-box unmount + empty-collection CTA flash for ~300ms after clearing a zero-result query (router passes the immediate searchQuery while items lag on the debounce). Cosmetic.
  • Cleanup: escapeLike is duplicated verbatim in media.ts and options.ts (+ regex equivalents in redirect.ts/comment.ts) — worth one shared helper.

Full details are in the inline comments on review #4405665999. Let me know how you'd like to handle #1 and #2 and I'll proceed.

The core server-side content search (?q=) landed via emdash-cms#1226. Extend it to
two surfaces that still post-filtered in memory:

- ContentPickerModal now pushes its search box to the server (search
  option -> ?q=), so it finds entries anywhere in a large collection
  instead of only the rows already loaded. Uses keepPreviousData to
  avoid flashing to empty between keystrokes and keeps load-more
  available while searching.
- MCP content_list gains a q parameter, so agents search server-side
  rather than post-filtering a page of results.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@edrpls edrpls force-pushed the fix/content-list-search branch from d087bc8 to 349a6f2 Compare June 2, 2026 19:19
@edrpls edrpls changed the title fix(admin): make content list search work across the whole collection fix(admin): extend server-side content search to the content picker and MCP Jun 2, 2026
@github-actions github-actions Bot added size/M and removed size/L labels Jun 2, 2026
Replace the manual accumulator (allItems/nextCursor mirrored from a
useQuery) with useInfiniteQuery, matching the ContentList pattern. The
accumulator could lose loaded pages on any background refetch (window
focus, cache invalidation) and, with keepPreviousData, could fire
load-more with a stale cursor against a freshly-changed search. Deriving
the list straight from query pages removes both windows; search stays in
the query key so changing it starts a fresh page chain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot mentioned this pull request Jun 2, 2026
18 tasks
@edrpls
Copy link
Copy Markdown
Contributor Author

edrpls commented Jun 2, 2026

Update — scope narrowed after rebase. Since my review note above, #1226 merged an independent implementation of the core server-side content search (?q=), including the locale-aware list index migration. I've rebased this branch onto main and dropped the overlapping core work; this PR is now just the two surfaces #1226 didn't cover:

  • ContentPickerModal — pushes its search box to the server instead of in-memory filtering.
  • MCP content_list — gains a q parameter so agents search server-side.

That moots most of the items above. Re-mapping each:

Design calls — now #1226's territory

  1. Non-ASCII case-insensitive search on SQLite — no longer in this PR; the core search ships via fix: server-side content list search + locale-aware list indexes #1226. If the ASCII-only LOWER() folding is still worth addressing, it's a follow-up against fix: server-side content list search + locale-aware list indexes #1226's implementation rather than a blocker here. Happy to open a separate issue.
  2. Hardcoded ["title","name","slug"] vs the field schemafix: server-side content list search + locale-aware list indexes #1226 partly addresses this: its resolveSearchColumns derives columns from _emdash_fields (adds title/name only when the collection defines them) rather than a flat constant. Whether to further key off the searchable flag / unify with the FTS layer is best tracked against that code.

Lower-priority findings

  • Postgres table_schema = 'public' hardcode — moot; the listTableColumns helper that had it was part of the discarded implementation and is gone.
  • ContentPickerModal load-more wiped by background refetchfixed in c4cb0e99 — the picker now derives its list from useInfiniteQuery pages instead of a manual accumulator, so a refocus/invalidation can't drop loaded pages.
  • ContentPickerModal load-more cursor/query mismatchfixed in the same commit — search lives in the query key, so a search change starts a fresh page chain rather than firing load-more with a stale cursor.
  • ContentList zero-result CTA flash — moot; that component's search now ships via fix: server-side content list search + locale-aware list indexes #1226.
  • escapeLike duplication — moot here (the copy was in the discarded code); still a reasonable standalone cleanup if anyone wants to pick it up.

All inline threads have been resolved accordingly. No maintainer decision is blocking this PR anymore — it's a focused follow-up on top of #1226. 🤖 update drafted with Claude Code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/admin area/core bot:review Trigger an emdashbot code review on this PR cla: signed overlap review/needs-rereview Author pushed changes since the last review size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants