fix(query-planner): out-of-schema-range $gt/$lt silently drops boundary documents#8601
fix(query-planner): out-of-schema-range $gt/$lt silently drops boundary documents#8601sravan27 wants to merge 3 commits into
Conversation
A `$gt`/`$lt` bound that lies outside the indexed field's schema minimum/maximum was clamped to the boundary when building the index string but kept exclusive, so the boundary document was silently dropped from index-based results: e.g. `$gt: -10` on a field with `minimum: 0` excluded the doc with value 0, and `$lt: 110` with `maximum: 100` excluded 100. Storages that resolve the query purely from the index (memory, localstorage, foundationdb, denokv) returned wrong results with no error. Treat a lower bound below the field minimum as INDEX_MIN (inclusive) and an upper bound above the field maximum as INDEX_MAX (inclusive). Exact-boundary exclusives ($gt at minimum, $lt at maximum) and all in-range bounds are unchanged. Validated end-to-end against the memory storage; adds a query-correctness regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@sravan27 according to the CI this test does not fail, even without your fix. Are you sure you are testing the correct behavior? Which storage is broken for you? |
|
You're right. The original storage-correctness test still passed without the source change, so it was not proving the bug cleanly. I pushed commit 5c8bd12 to add a direct query-planner regression instead. It asserts the actual planned bounds:
Local verification on this branch:
Result: So the issue this PR now tests is specifically the query planner producing an exclusive clamped/open-ended bound, not the broader storage result helper. If you prefer, I can also remove the storage-correctness case and keep only the direct planner test. |
|
only keep the storage-correctness test |
Summary
A
$gt/$ltrange bound that lies entirely outside the indexed field's schemaminimum/maximumsilently drops the boundary document. The out-of-range bound is clamped to the boundary when the index string is built (getNumberIndexString), but the bound stays exclusive — so the boundary document is excluded. Storages that resolve the query purely from the index (selectorSatisfiedByIndex: true— memory, localstorage, foundationdb, denokv) return wrong results with no error and no matcher fallback.Reproduce
Field
scorewith{ minimum: 0, maximum: 100 }, docsscore ∈ {0, 1, 50, 99, 100}:$gte: -10/$lte: 110are unaffected, because the inclusive bound includes the clamped boundary — which is why this slipped through (the existing boundary test only checks bounds at the limit, not beyond it).Fix
In the query planner, a lower bound below the field
minimumbecomesINDEX_MIN(inclusive) and an upper bound above the fieldmaximumbecomesINDEX_MAX(inclusive). Exact-boundary exclusives ($gtat the minimum,$ltat the maximum) and all in-range bounds are unchanged.Tests
Adds a
rx-storage-query-correctnesscase covering both bug cases plus regression guards ($gtat minimum still excludes it,$ltat maximum still excludes it, in-range bounds unchanged). Validated end-to-end against the memory storage before/after the fix.