Skip to content

feat: target exact V2 versions in compaction - #8584

Draft
Xuanwo wants to merge 1 commit into
xuanwo/mixed-file-versions-4-operationsfrom
xuanwo/mixed-file-versions-5-compaction
Draft

feat: target exact V2 versions in compaction#8584
Xuanwo wants to merge 1 commit into
xuanwo/mixed-file-versions-4-operationsfrom
xuanwo/mixed-file-versions-5-compaction

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add an optional exact data storage version to compaction options
  • carry the resolved target through distributed rewrite results
  • allow binary copy only when every selected input matches the output target
  • make TryBinaryCopy reencode mismatches and ForceBinaryCopy report target, actual version, and path

Stack

  1. feat: add mixed data file version capability #8580 - capability reservation and commit gates
  2. feat: validate mixed V2 data file versions #8581 - capability activation, storage validation, and exact read dispatch
  3. feat: support per-operation V2 write targets #8582 - per-operation write targets
  4. feat: propagate exact V2 versions across dataset operations #8583 - dataset operation propagation
  5. feat: target exact V2 versions in compaction #8584 - compaction targets and binary copy policy
  6. feat: expose exact V2 versions in bindings #8585 - bindings, documentation, and rollout contract

Testing

  • homogeneous and mixed input compaction
  • explicit and fallback targets
  • TryBinaryCopy and ForceBinaryCopy mismatch behavior
  • cargo check and clippy workspace gates

@github-actions github-actions Bot added A-python Python bindings A-java Java bindings + JNI A-deps Dependency updates A-encoding Encoding, IO, file reader/writer enhancement New feature or request labels Aug 17, 2026
@Xuanwo
Xuanwo force-pushed the xuanwo/mixed-file-versions-5-compaction branch from 3d8b737 to a57cc2f Compare August 18, 2026 18:37
@Xuanwo
Xuanwo force-pushed the xuanwo/mixed-file-versions-5-compaction branch from a57cc2f to 70b3fbb Compare August 18, 2026 18:43
@Xuanwo
Xuanwo force-pushed the xuanwo/mixed-file-versions-5-compaction branch from 70b3fbb to 3845284 Compare August 20, 2026 10:27
@Xuanwo
Xuanwo force-pushed the xuanwo/mixed-file-versions-5-compaction branch 2 times, most recently from 4f820b4 to e9be694 Compare August 20, 2026 14:15
@Xuanwo
Xuanwo force-pushed the xuanwo/mixed-file-versions-5-compaction branch from e9be694 to 0c0c0a2 Compare August 20, 2026 14:26
@Xuanwo
Xuanwo force-pushed the xuanwo/mixed-file-versions-5-compaction branch 2 times, most recently from 737adec to e055405 Compare August 20, 2026 15:15
@Xuanwo
Xuanwo force-pushed the xuanwo/mixed-file-versions-5-compaction branch from e055405 to 6e82e37 Compare August 20, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-deps Dependency updates A-encoding Encoding, IO, file reader/writer A-java Java bindings + JNI A-python Python bindings enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant