Repository navigation
feat: DB-side fuzzy job search (drop the in-memory index) - #8
Merged
Merged
Conversation
added 14 commits
June 17, 2026 07:30
felipebalbi
enabled auto-merge
June 17, 2026 17:51
Resolves the spec section 9 cross-branch conflict: the dashboard work that landed on main reads live.index for its Recent activity list, but this branch removes the in-memory JobIndex. Repoint api/dashboard.rs::get recent_jobs to WebDb::jobs_list_page(None, 0, 8) (SQL, newest-first) and refresh the now-stale in-memory-index comments in api_dashboard.rs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves job fuzzy search — plus the "↑ N new" pill and the live poller's
change-detection — off the full-history in-memory
JobIndexinpaavo-weband into SQLite, so the web process no longer holds the whole job history in
RAM just to search it. SQLite drives iteration and returns only the requested
page.
fuzzy_scoreSQLite scalar function(
SkimMatcherV2, registered on every connection) ranks matches; a subsequenceLIKE '%a%l%m%c%x%'pre-filter decides membership. Soalmcxstill matchesand ranks
alice … mcxa266, identically to the old in-memory matcher — but theexpensive UDF runs only on matched rows, and the pagination
COUNT(*)calls itzero times.
JobIndexdeleted./api/jobsreads SQLite directly (fuzzy page +search_count, or anas_of-pinned time-ordered page for a blank query); the"N new" pill is a
COUNT(*) WHERE submitted_at > as_of; the poller'schange-detection switches to a bounded
GROUP BY stateactivity_digest(no full-table materialization).
fuzzy-matchermovespaavo-web → paavo-db.DB-side scan.
No schema migration, no
paavo-proto/ wire change, nopaavodchange, no/api/eventscontract change. Design + plan:docs/superpowers/specs/2026-06-17-db-side-fuzzy-search-design.md,docs/superpowers/plans/2026-06-17-db-side-fuzzy-search.md.Notable design points / trade-offs
search_countandsearch_index_pageuse abyte-identical lowercased haystack and the same subsequence pattern, so a page
can never reference a row the count excludes. A divergence-guard test asserts
the
LIKEmembership set equals thefuzzy_score IS NOT NULLset.committed (debounced) keystroke — fine at lab scale; the
activity_digestcanmiss two offsetting same-tick state transitions and self-heals on the next poll
(it's a refetch hint, never a correctness input).
feat/dashboard-sql-aggregateslands after this, its"Recent activity" list should read
JobRow::list_index_page(None, 0, 8)ratherthan reintroduce
JobIndex.Test plan
cargo fmt --all -- --check— cleancargo clippy --workspace --all-targets -- -D warnings— cleancargo test --workspace— green (362 passed, 0 failed; 6 hardware/tooling-gated ignores)as_ofpin,count_newerboundary,activity_digestchange-detection, theLIKE≡
fuzzy_scoredivergence guard, search pagination;/api/jobsend-to-end through SQL (
api_jobs); digest-driven SSE bump (api_events).trunk build --release(crates/paavo-web-ui) — builds (workspace-excluded wasm32)PAAVO_FAKE_RUNNER=1daemon +paavo-web; on/jobs, typingalmcxranks analice … mcxa266row first, clearing thebox returns to the time-ordered list, the "↑ N new" pill updates on new
submissions, and the query fires only ~250 ms after typing stops.