fix(java): hold Dataset read lock when passing Dataset into native calls

[zhangyue19921010]

Problem

Dataset guards its native handle with a ReentrantReadWriteLock: its own methods take the read lock, and close() takes the write lock. However, entry points outside the class pass the Dataset object into JNI without acquiring this lock:

• Fragment: newScan / deleteRows / countRows / mergeColumns / updateColumns
• LanceScanner.create / AsyncScanner.create
• SqlQuery.intoBatchRecords
• CommitBuilder.execute (dataset branch)
• Compaction.planCompaction / commitCompaction, CompactionTask.execute
• VectorTrainer.trainIvfCentroids / trainPqCodebook
• DatasetDeltaBuilder.build
• memwal: ShardWriter.create, LsmScanner.fromSnapshots, LsmPointLookupPlanner / LsmVectorSearchPlanner constructors

So Dataset.close() is not mutually exclusive with these calls. A concurrent close() hits a flaw in jni-rs take_rust_field: if another thread holds the handle mutex, try_lock fails and the boxed BlockingDataset is freed on the early-return path — a native use-after-free. Under contention (e.g. 16 threads Fragment.countRows() vs close()) this aborts the JVM (SIGABRT, malloc corruption); it also leaves both the Rust field and the Java handle in a corrupt state, crashing later unrelated calls. Any JVM sharing one Dataset across threads with a close/eviction path (e.g. Spark executors) is affected.

Fix

Follow the existing locking mechanism instead of introducing a new one:

• Add Dataset.acquireReadLock(): acquires the shared read lock and rejects closed datasets, mirroring the readLock + handle != 0 check Dataset's own methods already use.
• Wrap every external native call that borrows the Dataset in try (LockManager.ReadLock readLock = dataset.acquireReadLock()) { ... }.

close() (write lock) now waits for in-flight native calls; callers racing with close() either complete safely or fail cleanly with IllegalArgumentException("Dataset is closed"). The read lock is reentrant, so nested paths like Dataset.newScan → LanceScanner.create are safe.

[lance-gatekeeper]

✅ Gate recommendation: approve.

The remaining public JNI bypass is fixed: commit compaction preserves its public signature behind the dataset read-lock boundary and keeps the raw native entry point private. The change now covers the reported close/use race at the Java/native boundary.