fix(index): isolate null-only zone maps in version 1

docs/src/format/index/scalar/zonemap.md

@@ -8,8 +8,14 @@ zones that cannot contain matching values.
 Zone maps are "inexact" filters - they can definitively exclude zones but may include
 false positives that require rechecking.
 
-In addition, since finding NULLs is a common query pattern, the index also maintains a
-bitmap of null rows which allows it to return exact results for IS NULL queries.
+In addition, since finding NULLs is a common query pattern, current writers maintain a
+bitmap of null rows that allows exact `IS NULL` and `IS NOT NULL` queries. Legacy
+version-0 indices may omit this bitmap and provide only inexact null pruning.
+
+Each index operates in one of two modes:
+
+- **Ordered:** stores minimum and maximum values.
+- **Null-only:** stores only top-level null information for non-orderable values.
 
 ## Index Details
 
@@ -25,37 +31,71 @@ The zone map index stores zone statistics in a single file:
 
 ### Zone Statistics File Schema
 
-| Column        | Type       | Nullable | Description                             |
-|---------------|------------|----------|-----------------------------------------|
-| `min`         | {DataType} | true     | Minimum value in the zone               |
-| `max`         | {DataType} | true     | Maximum value in the zone               |
-| `null_count`  | UInt32     | false    | Number of null values in the zone       |
-| `nan_count`   | UInt32     | false    | Number of NaN values (for float types)  |
-| `fragment_id` | UInt64     | false    | Fragment containing this zone           |
-| `zone_start`  | UInt64     | false    | Starting row offset within the fragment |
-| `zone_length` | UInt32     | false    | Number of rows in this zone             |
+| Column        | Type            | Nullable | Description                                      |
+|---------------|-----------------|----------|--------------------------------------------------|
+| `min`         | {DataType}/Null | true     | Minimum value, or Null in null-only mode         |
+| `max`         | {DataType}/Null | true     | Maximum value, or Null in null-only mode         |
+| `null_count`  | UInt32          | false    | Number of top-level null values in the zone      |
+| `nan_count`   | UInt32          | false    | Number of NaN values (for float types)           |
+| `fragment_id` | UInt64          | false    | Fragment containing this zone                    |
+| `zone_start`  | UInt64          | false    | Starting row offset within the fragment          |
+| `zone_length` | UInt64          | false    | Inclusive row-offset span of this zone           |
+
+```python
+extrema_type = value_type if supports_min_max else pa.null()
+
+pa.schema([
+    pa.field("min", extrema_type, nullable=True),
+    pa.field("max", extrema_type, nullable=True),
+    pa.field("null_count", pa.uint32(), nullable=False),
+    pa.field("nan_count", pa.uint32(), nullable=False),
+    pa.field("fragment_id", pa.uint64(), nullable=False),
+    pa.field("zone_start", pa.uint64(), nullable=False),
+    pa.field("zone_length", pa.uint64(), nullable=False),
+])
+```
 
 ### Schema Metadata
 
-| Key                 | Type   | Description                               |
-|---------------------|--------|-------------------------------------------|
-| `rows_per_zone`     | String | Number of rows per zone (default: "8192") |
-| `null_bitmap`       | UInt32 | Index of null bitmap global buffer        |
+| Key                 | Type   | Description                                             |
+|---------------------|--------|---------------------------------------------------------|
+| `rows_per_zone`     | String | Number of rows per zone (default: "8192")              |
+| `null_bitmap`       | String | Index of null bitmap global buffer                      |
+| `data_type`         | String | Index of logical data type buffer for null-only indices |
 
 ### Global Buffers
 
 | Metadata Key        | Description                                                |
 |---------------------|------------------------------------------------------------|
 | `null_bitmap`       | A serialized RowAddrTreeMap specifying which rows are null |
+| `data_type`         | An Arrow IPC schema containing the indexed logical type    |
+
+```python
+pa.schema([
+    pa.field("value", value_type, nullable=True),
+])
+```
+
+`zonemap.lance` is the index entry point. Each record is one zone located by
+`fragment_id`, `zone_start`, and `zone_length`. The zone length is
+`last_row_offset - first_row_offset + 1`; it includes gaps between live row offsets.
 
 ## Accelerated Queries
 
-The zone map index provides inexact results for the following query types (nullability queries
-return exact results):
+Ordered zone maps provide inexact results for `Equals`, `Range`, and `IsIn`
+queries by retaining zones whose minimum and maximum may contain a match.
+Null-only indices do not accelerate these predicates.
+
+When a complete `null_bitmap` is present, `IsNull` and `IsNotNull` return exact
+row selections in both modes. If a legacy version-0 index omits the bitmap,
+`IsNull` returns an `AtMost` zone selection that requires rechecking, and
+`IsNotNull` remains a refinement or scan because negating an inexact null query
+would be unsafe.
+
+`ZoneMapIndexDetails.supports_min_max` records the mode. An absent value means ordered
+mode for compatibility with existing version-0 zone maps.
+
+## Index Versions
 
-| Query Type | Description               | Operation                                   | Result Type |
-|------------|---------------------------|---------------------------------------------|-------------|
-| **Equals** | `column = value`          | Includes zones where min ≤ value ≤ max      | AtMost      |
-| **Range**  | `column BETWEEN a AND b`  | Includes zones where ranges overlap         | AtMost      |
-| **IsIn**   | `column IN (v1, v2, ...)` | Includes zones that could contain any value | AtMost      |
-| **IsNull** | `column IS NULL`          | Includes zones where null_count > 0         | Exact       |
+- Version 0 stores ordered zone maps.
+- Version 1 stores null-only zone maps so version-0 readers fall back to scanning.

protos/index_old.proto

@@ -45,6 +45,10 @@ message ZoneMapIndexDetails {
   // true means IS NULL is exact and IS NOT NULL can be answered without a
   // full scan.
   optional bool has_null_bitmap = 3;
+  // Whether this index stores ordered minimum and maximum values. Absent means
+  // true for compatibility with version-0 zone maps, which only supported
+  // orderable scalar fields. False identifies a null-only zone map.
+  optional bool supports_min_max = 4;
 }
 message InvertedIndexDetails {
   enum DocumentGranularity {