Skip to content

perf(persistence): Move pinned messages pagination to SQL#2689

Open
VelikovPetar wants to merge 25 commits into
masterfrom
feature/FLU-487_move_pinned_message_pagination_to_sql
Open

perf(persistence): Move pinned messages pagination to SQL#2689
VelikovPetar wants to merge 25 commits into
masterfrom
feature/FLU-487_move_pinned_message_pagination_to_sql

Conversation

@VelikovPetar
Copy link
Copy Markdown
Contributor

@VelikovPetar VelikovPetar commented May 26, 2026

Submit a pull request

Linear: FLU-487
Review after: #2679

CLA

  • I have signed the Stream CLA (required).
  • The code changes follow best practices
  • Code changes are tested (add some information if not applicable)

Description of the pull request

  • Refactor the pinned messages pagination to be performed on SQL level, instead of fetching all thread messages from DB, then paginating in-memory.
  • Fix limit always being applied from the beginning of the fetched messages
  • Add support for greaterThanOrEqual/lessThanOrEqual pagination

Summary by CodeRabbit

  • New Features

    • Improved performance for pinned message pagination through database-level filtering optimization.
  • Bug Fixes

    • Fixed pinned message pagination to correctly handle inclusive boundary conditions.
    • Fixed cursor-based pagination to accurately position results after the specified pivot point.

# Conflicts:
#	packages/stream_chat_persistence/CHANGELOG.md
…ture/FLU-485_optimize_read_message_from_db_part2

# Conflicts:
#	packages/stream_chat_persistence/CHANGELOG.md
#	packages/stream_chat_persistence/lib/src/dao/message_dao.dart
…b' into feature/FLU-485_optimize_read_message_from_db_part2
@VelikovPetar VelikovPetar changed the title perf(llc): Move thread messages pagination to SQL perf(llc): Move pinned messages pagination to SQL May 26, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 26, 2026

Review Change Stack

📝 Walkthrough

Walkthrough

This PR refactors pinned message pagination in PinnedMessageDao.getMessagesByCid from in-memory trimming to SQL-level cursor pagination using (createdAt, id) tuple predicates. Updates include a new _lookupCursor helper, deterministic test timestamps, comprehensive pagination edge-case coverage, and changelog entries.

Changes

Pinned Message Cursor Pagination

Layer / File(s) Summary
SQL-level cursor pagination with tuple predicates
packages/stream_chat_persistence/lib/src/dao/pinned_message_dao.dart
getMessagesByCid now resolves (createdAt, id) cursor tuples from cached messages, applies SQL-level predicates for all boundary cases (lessThan, lessThanOrEqual, greaterThan, greaterThanOrEqual), applies LIMIT at the database level, and conditionally reverses results for backward pagination. New _lookupCursor helper looks up the (createdAt, id) tuple for a given message id while enforcing thread visibility constraints.
Deterministic timestamp-based test setup
packages/stream_chat_persistence/test/src/dao/pinned_message_dao_test.dart
_prepareTestData now uses shared baseTime with per-index second offsets instead of DateTime.now() for strictly monotonic test timestamps. Quoted and thread reply message timestamps updated to use the same deterministic scheme.
Pagination test coverage and edge cases
packages/stream_chat_persistence/test/src/dao/pinned_message_dao_test.dart
Extended existing pagination test assertions to verify cursor-trimmed message boundaries. Added comprehensive new getMessagesByCid pagination test group covering cursor behavior, missing-id no-ops, default limit, inclusive pivot handling, and tied createdAt splitting. Added hydration test validating paginated results retain fully hydrated latest and own reactions.
Changelog documentation
packages/stream_chat_persistence/CHANGELOG.md
Added upcoming-changes entry documenting database-level pagination filtering. Extended fixed-section with entries for inclusive boundary handling and correct forward-cursor behavior with limit.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

  • renefloor

Poem

🐰 Hop into SQL's zen garden,
where cursors bloom in ordered pairs,
(createdAt, id) dance in twilight—
backward, forward, tied with care.
Tests now tick in steady seconds,
no more flaky timestamps here! 🌟

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: moving pinned messages pagination from in-memory to SQL level, which is the primary performance improvement across all modified files.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/FLU-487_move_pinned_message_pagination_to_sql

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@VelikovPetar VelikovPetar changed the title perf(llc): Move pinned messages pagination to SQL perf(persistence): Move pinned messages pagination to SQL Jun 2, 2026
…e_pinned_message_pagination_to_sql

# Conflicts:
#	packages/stream_chat_persistence/CHANGELOG.md
#	packages/stream_chat_persistence/lib/src/dao/message_dao.dart
#	packages/stream_chat_persistence/lib/src/dao/pinned_message_dao.dart
#	packages/stream_chat_persistence/test/src/dao/message_dao_test.dart
#	packages/stream_chat_persistence/test/src/dao/pinned_message_dao_test.dart
@VelikovPetar VelikovPetar marked this pull request as ready for review June 5, 2026 14:22
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/stream_chat_persistence/lib/src/dao/pinned_message_dao.dart`:
- Around line 246-255: The cursor lookup currently calls
_lookupCursor(messagePagination?.lessThan/lessThanOrEqual/greaterThan/greaterThanOrEqual)
which resolves by id only and can match cursors from other channels; update
_lookupCursor to accept a cid/channelCid parameter (e.g., _lookupCursor(cid,
cursorId)) and change the calls inside getMessagesByCid to pass the active
channel's cid so lookups are scoped to that channel; also update the other
occurrence (the block around the second set of calls) to pass cid when invoking
_lookupCursor and adjust the helper implementation to filter by channelCid when
resolving cursor ids.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4373489d-563d-4324-afce-ee6ada4ddc00

📥 Commits

Reviewing files that changed from the base of the PR and between a2dc959 and f6b1b11.

📒 Files selected for processing (3)
  • packages/stream_chat_persistence/CHANGELOG.md
  • packages/stream_chat_persistence/lib/src/dao/pinned_message_dao.dart
  • packages/stream_chat_persistence/test/src/dao/pinned_message_dao_test.dart

Comment on lines +246 to +255
final (
lessThanCursor,
lessThanOrEqualCursor,
greaterThanCursor,
greaterThanOrEqualCursor,
) = await (
_lookupCursor(messagePagination?.lessThan),
_lookupCursor(messagePagination?.lessThanOrEqual),
_lookupCursor(messagePagination?.greaterThan),
_lookupCursor(messagePagination?.greaterThanOrEqual),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Scope cursor lookup to the active channel.

_lookupCursor resolves by id only. If a cursor id exists in another channel, it can still produce a cutoff and incorrectly filter this channel instead of behaving like a no-op. Please scope lookup by channelCid and pass cid from getMessagesByCid.

Suggested fix
-    ) = await (
-      _lookupCursor(messagePagination?.lessThan),
-      _lookupCursor(messagePagination?.lessThanOrEqual),
-      _lookupCursor(messagePagination?.greaterThan),
-      _lookupCursor(messagePagination?.greaterThanOrEqual),
+    ) = await (
+      _lookupCursor(cid, messagePagination?.lessThan),
+      _lookupCursor(cid, messagePagination?.lessThanOrEqual),
+      _lookupCursor(cid, messagePagination?.greaterThan),
+      _lookupCursor(cid, messagePagination?.greaterThanOrEqual),
     ).wait;
...
-  Future<({DateTime createdAt, String id})?> _lookupCursor(String? id) async {
+  Future<({DateTime createdAt, String id})?> _lookupCursor(
+    String cid,
+    String? id,
+  ) async {
     if (id == null) return null;
     final createdAt = await (selectOnly(pinnedMessages)
           ..addColumns([pinnedMessages.createdAt])
           ..where(pinnedMessages.id.equals(id))
+          ..where(pinnedMessages.channelCid.equals(cid))
           ..where(
             pinnedMessages.parentId.isNull() |
                 pinnedMessages.showInChannel.equals(true),
           ))
         .map((row) => row.read(pinnedMessages.createdAt))
         .getSingleOrNull();

Also applies to: 358-366

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stream_chat_persistence/lib/src/dao/pinned_message_dao.dart` around
lines 246 - 255, The cursor lookup currently calls
_lookupCursor(messagePagination?.lessThan/lessThanOrEqual/greaterThan/greaterThanOrEqual)
which resolves by id only and can match cursors from other channels; update
_lookupCursor to accept a cid/channelCid parameter (e.g., _lookupCursor(cid,
cursorId)) and change the calls inside getMessagesByCid to pass the active
channel's cid so lookups are scoped to that channel; also update the other
occurrence (the block around the second set of calls) to pass cid when invoking
_lookupCursor and adjust the helper implementation to filter by channelCid when
resolving cursor ids.

@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.99%. Comparing base (a2dc959) to head (f6b1b11).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2689      +/-   ##
==========================================
+ Coverage   65.96%   65.99%   +0.03%     
==========================================
  Files         425      425              
  Lines       26926    26950      +24     
==========================================
+ Hits        17761    17785      +24     
  Misses       9165     9165              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant