indexed nearest-by join - #1

Draft

sezruby wants to merge 10 commits into

mainfrom

Commits on May 12, 2026

feat(knn): Phase 0 foundation — LanceProbe primitive + metric types

sezruby
and
claude
committed
ad53c4b
feat(knn): staged RDD pipeline + IndexedNearestJoin.apply + bounded TopKHeap

sezruby
and
claude
committed
e70a8a6
feat(knn): Phase 1.5 — fragment-grouped probing for multi-task parallelism

sezruby
and
claude
committed
209b8f7
feat(knn): 3-exec Catalyst-visible staged plan with AQE-visible merge shuffle

sezruby
and
claude
committed
edac519
feat(knn): df.kNearestJoin DataFrame extension method

sezruby
and
claude
committed
30fc916
feat(knn): Phase 3 hardening — refineFactor, prefilter pushdown, IVF-PQ recall

sezruby
and
claude
committed
e8b4ec1
feat(knn): Spark 4.2 SQL integration — IndexedNearestByJoinRule

sezruby
and
claude
committed
d24262c
test(knn-bench): benchmark suite — synthetic, Wikipedia perf, SIFT/Cohere recall, SQL

sezruby
and
claude
committed
7e03dce
docs(knn): design, impl plan, reviewer guide, ANN proposal, benchmark results

sezruby
and
claude
committed
dfcd46e

Commits on May 13, 2026

test(knn-bench): closer-to-RewriteNearestByJoin baseline + cluster scaling sweep

sezruby
and
claude
committed
e22230a