Add Delta Lake 4.2 support with basic features

[jihoonson]

Fixes #15665.

Description

This PR adds RAPIDS Accelerator support for Delta Lake 4.2.0 with Scala 2.13 on:

• Apache Spark 4.0.1
• Apache Spark 4.1.1

The change adds the delta-42x module and the Delta 4.2-specific provider, runtime shim, catalog, write, MERGE, OPTIMIZE, REORG, and auto-compaction implementations.

A Spark-version-specific plugin build can include multiple compatible Delta integration modules. At runtime, the plugin checks both the installed Delta Lake version and the Spark version, then loads the matching Delta runtime shim. For example, the Spark 4.0.1 build can select either the Delta 4.0 or 4.2 implementation, while the Spark 4.1.1 build can select either the Delta 4.1 or 4.2 implementation.

The change also:

• Adapts GPU write, table creation and replacement, MERGE statistics, and transaction handling to Delta 4.2 API changes.
• Shares compatible implementations across Delta 3.3–4.2 and Delta 4.0–4.2 while preserving version-specific behavior.
• Updates the Maven modules, Scala 2.13 build files, CI integration-test matrix, and supported-version documentation.

The following Delta 4.2 write features are not yet supported on GPU and fall back to CPU:

• replaceOn and replaceUsing
• targetAlias
• Null-intolerant dynamic partition overwrite
• Catalog-managed table reads and writes
• Server-side-planned table replacement

Testing

Added or updated coverage for:

• Runtime provider and shim selection for supported Spark and Delta combinations.
• Rejection of unsupported Spark and Delta combinations.
• Delta 4.2 runtime-specific GPU write construction.
• CPU fallback for the new Delta 4.2 write options.
• MERGE ... WHEN NOT MATCHED BY SOURCE behavior on Delta 4.1 and 4.2.
• Regression coverage for existing supported Delta versions.

Local Delta integration-suite results:

• Spark 3.5.5 / Delta 3.3.1: 605 passed, 196 skipped
• Spark 4.0.1 / Delta 4.0.1: 610 passed, 191 skipped
• Spark 4.0.1 / Delta 4.2.0: 615 passed, 186 skipped
• Spark 4.1.1 / Delta 4.1.0: 611 passed, 190 skipped
• Spark 4.1.1 / Delta 4.2.0: 615 passed, 186 skipped

Performance results

I ran CTAS, UPDATE, DELETE, MERGE, COMPACT, and LIQUID CLUSTERING queries against the NDS store sales Delta Lake table at sf=1k. Details of the benchmark queries can be found in here. These queries were tested with my workstation, which has a RTX A5500 GPU. I compared runtime between Delta 4.1 and Delta 4.2 using the same jar. 2 queries in the below showed a meaningful improvement. No regression was observed.

Name = liquid_cluster
Means = 407751.4, 354743.4
Time diff = 53008.0
Speedup = 1.1494263177271233
T-Test (test statistic, p value, df) = 2.390793833968829, 0.04380153368406999, 8.0
T-Test Confidence Interval = 1879.93305330562, 104136.06694669438
ALERT: significant change has been detected (p-value < 0.05)
ALERT: improvement in performance has been observed
--------------------------------------------------------------------
Name = merge
Means = 36207.0, 31232.0
Time diff = 4975.0
Speedup = 1.1592917520491803
T-Test (test statistic, p value, df) = 4.752692362147525, 0.001440257135978383, 8.0
T-Test Confidence Interval = 2561.1323017640293, 7388.86769823597
ALERT: significant change has been detected (p-value < 0.05)
ALERT: improvement in performance has been observed

Checklists

Documentation

• Updated for new or modified user-facing features or behaviors
• No user-facing change

Testing

• Added or modified tests to cover new code paths
• Covered by existing tests
  (Please provide the names of the existing tests in the PR description.)
• Not required

Performance

• Tests ran and results are added in the PR description
• Issue filed with a link in the PR description
• Not required

[jihoonson]

I'm going to run some performance tests and update the PR. Draft until then.

[jihoonson]

build

[NvTimLiu]

The change for CICD looks good to me, +1

[NvTimLiu]

Pom files changes LGTM, +1

[jihoonson]

build

[jihoonson]

build

[jihoonson]

@greptile-apps please review

[greptile-apps]

Greptile Summary

Adds Delta Lake 4.2 support for Scala 2.13 builds on Spark 4.0.1 and 4.1.1.

• Adds the Delta 4.2 provider, runtime shim, catalog, write, MERGE, OPTIMIZE, REORG, and auto-compaction implementations.
• Refactors compatible Delta functionality into shared source ranges while retaining version-specific adapters.
• Updates Maven profiles, runtime selection, CI configuration, documentation, and CPU-fallback coverage for unsupported Delta 4.2 features.

Confidence Score: 5/5

The PR appears safe to merge based on the reviewed runtime selection, build wiring, compatibility guards, and test coverage.

No concrete changed-code failure remained after checking the Delta runtime matrix, module packaging alignment, shared implementation refactors, and unsupported-feature fallback paths.

Important Files Changed

Filename	Overview
delta-lake/common/src/main/delta-io/scala/org/apache/spark/sql/delta/rapids/DeltaRuntimeShim.scala	Selects version-specific runtime shims using both Spark and Delta versions, with matching supported and rejected matrix tests.
delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/Delta42xProvider.scala	Registers Delta 4.2 GPU behavior and guards catalog-managed and server-side replacement paths for CPU fallback.
delta-lake/delta-42x/src/main/scala/org/apache/spark/sql/delta/rapids/delta42x/GpuMergeIntoCommand42x.scala	Adapts the established GPU MERGE flow to Delta 4.2 while retaining Delta transaction and metric handling.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/GpuDeltaCatalogBase.scala	Generalizes GPU table creation and replacement across Delta 3.3–4.2 through version-specific catalog hooks.
pom.xml	Includes the compatible Delta modules in the Spark 4.0.1 and 4.1.1 build profiles.
integration_tests/src/main/python/delta_lake_write_test.py	Adds coverage for Delta 4.2 write construction and expected CPU fallback behavior.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Spark workload] --> B[Read Spark and Delta versions]
  B --> C{Compatible combination?}
  C -- No --> D[Normal CPU Delta path]
  C -- Yes --> E[Select Delta 4.0, 4.1, or 4.2 runtime shim]
  E --> F{Operation supported on GPU?}
  F -- No --> D
  F -- Yes --> G[GPU Delta command]
  G --> H[Delta transaction and commit]

Loading

_{Reviews (1): Last reviewed commit: "fix cdf write for delta 42" | Re-trigger Greptile}

[Copilot]

🔵 Needs a closer look

The PR introduces large, cross-cutting version-shim and runtime-selection changes across many Delta/Spark combinations, which warrants final human verification beyond the specific issues noted.

Pull request overview

Adds RAPIDS Accelerator integration for OSS Delta Lake 4.2.0 across Spark 4.0.1 and 4.1.1 builds, including a new delta-42x adapter module and runtime shim selection logic so a single Spark-version build can load the appropriate Delta adapter at runtime based on the installed Delta version.

Changes:

• Introduces the Delta 4.2 adapter module (delta-42x) with runtime shim, provider, catalog, and GPU write plumbing.
• Refactors shared Delta 3.3–4.2 / 4.0–4.2 code paths (write operations metadata, command shims, CDF planning strategy wiring, post-commit auto-compaction hooks).
• Updates unit/integration test coverage, Maven module wiring (including Scala 2.13 builds), CI scripts, and supported-version documentation.

File summaries

File	Description
tests/src/test/spark401/scala/org/apache/spark/sql/delta/rapids/DeltaRuntimeShimSuite.scala	Adds Spark 4.0.1/4.1.1 shim/provider selection tests including Delta 4.2 assertions.
tests/pom.xml	Adds Spark 4.0.1 (buildver 401) test profile and parameterizes Delta test dependency properties.
scala2.13/tests/pom.xml	Mirrors test profile/property updates for Scala 2.13 build.
scala2.13/pom.xml	Adds delta-42x module + Delta 4.2 coordinates for Spark 4.0.1 and Spark 4.1.1 Scala 2.13 builds.
scala2.13/delta-lake/delta-42x/pom.xml	New Scala 2.13 Delta 4.2 adapter Maven module definition and shared source wiring.
scala2.13/delta-lake/delta-41x/pom.xml	Extends shared-source roots to include new Delta 33x–42x and 40x–42x common code.
scala2.13/delta-lake/delta-40x/pom.xml	Extends shared-source roots to include new Delta 33x–42x and 40x–42x common code.
scala2.13/delta-lake/delta-33x/pom.xml	Extends shared-source roots to include new Delta 33x–42x common code.
pom.xml	Adds delta-42x module + Delta 4.2 coordinates for non-Scala-2.13 root build profiles.
jenkins/spark-tests.sh	Expands Delta integration-test matrix to exercise Delta 4.2 on Spark 4.0.1 and 4.1.1.
integration_tests/src/main/python/delta_lake_write_test.py	Adds Delta 4.2 write-option CPU fallback coverage (replaceOn/replaceUsing/targetAlias/null-intolerant DPO).
integration_tests/src/main/python/delta_lake_utils.py	Adds Delta 4.2 runtime detection helper and centralizes “NOT MATCHED BY SOURCE GPU-supported” predicate.
integration_tests/src/main/python/delta_lake_merge_test.py	Updates skip conditions to cover OSS Delta 4.1+ behavior consistently.
delta-lake/README.md	Documents Delta 4.2.0 support and Spark compatibility mapping.
delta-lake/delta-42x/src/main/scala/org/apache/spark/sql/delta/rapids/delta42x/GpuWriteIntoDelta42x.scala	Introduces Delta 4.2-specific GPU WriteIntoDelta implementation with distinct FQCN for shaded jars.
delta-lake/delta-42x/src/main/scala/org/apache/spark/sql/delta/rapids/delta42x/GpuCreateDeltaTableCommand.scala	Delta 4.2 GPU create/replace table command with catalog-managed table validations and behavior.
delta-lake/delta-42x/src/main/scala/org/apache/spark/sql/delta/rapids/delta42x/Delta42xRuntimeShim.scala	Adds Delta 4.2 runtime shim: provider/catalog wiring, transaction hooks, and operation-metadata builders.
delta-lake/delta-42x/src/main/scala/org/apache/spark/sql/delta/hooks/GpuAutoCompact.scala	Adds Delta 4.2-specific auto-compaction hook implementation.
delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/OptimizeTableCommandMeta.scala	Delta 4.2 OPTIMIZE command GPU tagging and conversion logic (incl. catalog-owned fallback).
delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/MergeIntoCommandMeta.scala	Delta 4.2 MERGE command meta + GPU conversion wiring (incl. NOT MATCHED BY SOURCE support).
delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/GpuDeltaCatalog.scala	Delta 4.2 GPU catalog integration, including UC-related identifier handling and table lookup.
delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/DeltaReorgTableCommandMeta.scala	Delta 4.2 REORG TABLE GPU tagging and conversion wiring (incl. catalog-owned fallback).
delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/Delta42xProvider.scala	Delta 4.2 provider: rules/strategy wiring and GPU conversion logic for Delta write-related commands.
delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/Delta42xConfigChecker.scala	Delta 4.2 config validation for unsupported write options and catalog-owned table restrictions.
delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/Delta42xCDFRelationStrategy.scala	Delta 4.2 CDF strategy using version-pinned CDF relation shim.
delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/Delta42xCDFRelationShim.scala	Reflection-based shim for Delta 4.2 CDF batch schema snapshot access.
delta-lake/delta-41x/src/main/scala/org/apache/spark/sql/delta/rapids/GpuWriteIntoDelta.scala	Routes write operation metadata construction through DeltaRuntimeShim to unify 4.x semantics.
delta-lake/delta-41x/src/main/scala/org/apache/spark/sql/delta/rapids/delta41x/GpuCreateDeltaTableCommand.scala	Switches to shared 40x–42x GPU create-table base class.
delta-lake/delta-41x/src/main/scala/org/apache/spark/sql/delta/rapids/delta41x/Delta41xRuntimeShim.scala	Adds explicit GPU write factory + operation-metadata builders + post-commit hook wiring for 4.1.
delta-lake/delta-41x/src/main/scala/org/apache/spark/sql/delta/hooks/GpuAutoCompact.scala	Renames/adjusts auto-compact hook for Delta 4.1-specific semantics.
delta-lake/delta-41x/src/main/scala/com/nvidia/spark/rapids/delta/delta41x/GpuDeltaCatalog.scala	Updates catalog wiring to use new shared base types and CTAS catalog-create behavior.
delta-lake/delta-41x/src/main/scala/com/nvidia/spark/rapids/delta/delta41x/Delta41xProvider.scala	Updates provider to use shared CDF strategy mechanism.
delta-lake/delta-40x/src/main/scala/org/apache/spark/sql/delta/rapids/GpuWriteIntoDelta.scala	Routes write operation metadata construction through DeltaRuntimeShim to unify 4.x semantics.
delta-lake/delta-40x/src/main/scala/org/apache/spark/sql/delta/rapids/delta40x/GpuCreateDeltaTableCommand.scala	Switches to shared 40x–42x GPU create-table base class.
delta-lake/delta-40x/src/main/scala/org/apache/spark/sql/delta/rapids/delta40x/Delta40xRuntimeShim.scala	Adds explicit GPU write factory + operation-metadata builders + post-commit hook wiring for 4.0.
delta-lake/delta-40x/src/main/scala/com/nvidia/spark/rapids/delta/delta40x/GpuDeltaCatalog.scala	Updates catalog wiring to use new shared base types.
delta-lake/delta-40x/src/main/scala/com/nvidia/spark/rapids/delta/delta40x/Delta40xProvider.scala	Updates provider to use shared CDF strategy mechanism.
delta-lake/delta-40x/pom.xml	Extends shared-source roots to include new Delta 33x–42x and 40x–42x common code.
delta-lake/delta-33x/src/main/scala/org/apache/spark/sql/delta/rapids/delta33x/Delta33xRuntimeShim.scala	Adds explicit GPU write factory + operation-metadata builders for Delta 3.3.x shim.
delta-lake/delta-33x/src/main/scala/com/nvidia/spark/rapids/delta/delta33x/GpuDeltaCatalog.scala	Updates catalog wiring to use new shared write interface type.
delta-lake/delta-33x/src/main/scala/com/nvidia/spark/rapids/delta/delta33x/Delta33xProvider.scala	Updates provider to use shared CDF strategy mechanism.
delta-lake/delta-33x/pom.xml	Extends shared-source roots to include new Delta 33x–42x common code.
delta-lake/delta-24x/src/main/scala/org/apache/spark/sql/delta/rapids/delta24x/Delta24xRuntimeShim.scala	Adds explicit GPU write factory wiring to older shims for consistency.
delta-lake/delta-23x/src/main/scala/org/apache/spark/sql/delta/rapids/delta23x/Delta23xRuntimeShim.scala	Adds explicit GPU write factory wiring to older shims for consistency.
delta-lake/delta-22x/src/main/scala/org/apache/spark/sql/delta/rapids/delta22x/Delta22xRuntimeShim.scala	Adds explicit GPU write factory wiring to older shims for consistency.
delta-lake/delta-21x/src/main/scala/org/apache/spark/sql/delta/rapids/delta21x/Delta21xRuntimeShim.scala	Adds explicit GPU write factory wiring to older shims for consistency.
delta-lake/delta-20x/src/main/scala/org/apache/spark/sql/delta/rapids/delta20x/Delta20xRuntimeShim.scala	Adds explicit GPU write factory wiring to older shims for consistency.
delta-lake/common/src/main/delta-io/scala/org/apache/spark/sql/delta/rapids/DeltaRuntimeShim.scala	Reworks runtime shim selection to be based on installed Delta + Spark versions; adds write factories + operation-metadata APIs.
delta-lake/common/src/main/delta-io/scala/com/nvidia/spark/rapids/delta/GpuDeltaDataSource.scala	Routes GPU write command construction through the runtime shim factory.
delta-lake/common/src/main/delta-io/scala/com/nvidia/spark/rapids/delta/DeltaIOProvider.scala	Routes GPU write command construction through the runtime shim factory.
delta-lake/common/src/main/delta-40x/scala/org/apache/spark/sql/delta/hooks/GpuAutoCompact.scala	Names Delta 4.0 auto-compact hook explicitly and aligns tableId behavior.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/shims/ShimOptimisticTransaction.scala	Adds shim class to normalize optimistic transaction construction for Delta 4.0–4.2.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/ShimDeltaInvariantCheckerExec.scala	Adds shim for CPU invariant checker exec construction across versions.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/GpuWriteIntoDeltaBase.scala	Introduces shared Delta 4.0–4.2 GPU write base using self-types to avoid binary-compat issues.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/GpuUpdateCommand.scala	Adds shared GPU UpdateCommand wrapper for classic-session shims.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/GpuOptimisticTransaction.scala	Makes auto-compact hook version-injected and removes hard-coded hook dependency.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/GpuMergeStats.scala	Extracts MERGE stats structures into shared source for reuse across versions.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/GpuDeltaFileFormatWriter.scala	Adds Spark 4.x wrapper delegating task attempt context creation to shared base logic.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/GpuDeltaCatalog4x.scala	Generalizes shared Delta 4.x catalog base types to accept 4.2 wiring.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/GpuDeleteCommand.scala	Adds shared GPU DeleteCommand wrapper for classic-session shims.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/GpuCreateDeltaTableCommand40x42xBase.scala	Renames/extends shared create-table base to cover Delta 4.0–4.2.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/DeltaMdcShims.scala	Documents and bridges Spark 4.0 vs 4.1 MDC construction differences.
delta-lake/common/src/main/delta-40x-42x/scala/org/apache/spark/sql/delta/rapids/ClassicSparkCommandShims.scala	Adds classic-session-specific shims for Spark 4.x Delta command execution.
delta-lake/common/src/main/delta-40x-42x/scala/com/nvidia/spark/rapids/delta/shims/StatsExprShim.scala	Adds Spark 4.0 expression unwrapping shim for RuntimeReplaceable task-time evaluation.
delta-lake/common/src/main/delta-40x-42x/scala/com/nvidia/spark/rapids/delta/shims/MetadataShims.scala	Adds shared accessors for Delta statistics metadata keys.
delta-lake/common/src/main/delta-40x-42x/scala/com/nvidia/spark/rapids/delta/common/UpdateCommandMeta.scala	Adds shared UpdateCommand meta for 4.x runtimes.
delta-lake/common/src/main/delta-40x-42x/scala/com/nvidia/spark/rapids/delta/common/GpuDeltaParquetFileFormat2.scala	Adds shared DV-aware parquet file format wrapper.
delta-lake/common/src/main/delta-40x-42x/scala/com/nvidia/spark/rapids/delta/common/GpuDeltaParquetFileFormat.scala	Adds shared thin wrapper delegating Delta 4.x file format behavior.
delta-lake/common/src/main/delta-40x-42x/scala/com/nvidia/spark/rapids/delta/common/DeltaDynamicPartitionOverwriteCommandMeta.scala	Adds shared DPO command meta for 4.x runtimes.
delta-lake/common/src/main/delta-40x-42x/scala/com/nvidia/spark/rapids/delta/common/DeleteCommandMeta.scala	Adds shared DeleteCommand meta for 4.x runtimes.
delta-lake/common/src/main/delta-40x-41x/scala/org/apache/spark/sql/delta/rapids/GpuMergeIntoCommand.scala	Extracts MERGE stats code to shared module and aligns command mixins.
delta-lake/common/src/main/delta-40x-41x/scala/com/nvidia/spark/rapids/delta/common/OptimizeTableCommandMeta.scala	Uses shared Delta command interface for optimized-table log discovery.
delta-lake/common/src/main/delta-33x/scala/org/apache/spark/sql/delta/rapids/GpuWriteIntoDelta.scala	Aligns older shim’s GPU write interface type to new shared trait.
delta-lake/common/src/main/delta-33x/scala/org/apache/spark/sql/delta/hooks/GpuAutoCompact.scala	Ensures tableId behavior is consistent with newer shims.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/GpuDeltaDynamicPartitionOverwriteCommand.scala	Routes GPU write command construction through the runtime shim factory.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/GpuWriteIntoDeltaLike.scala	Introduces shared GPU write interface aligned with Delta’s WriteIntoDeltaLike.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/GpuUpdateCommandBase.scala	Switches DeltaCommand dependency to a shared GPU Delta command interface.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/GpuOptimizeTableCommand.scala	Switches DeltaCommand dependency to a shared GPU Delta command interface.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/GpuOptimisticTransactionBase.scala	Adds shared optimized-write planning path for GPU Delta transactions.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/GpuDeltaReorgTableCommand.scala	Adds shared GPU REORG TABLE implementation (PURGE) via GpuOptimizeTableCommand.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/GpuDeltaCommandUtils.scala	Adds shared createTableRelation helper used by the GPU Delta command interface.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/GpuDeltaCommandLike.java	Adds a shared interface to normalize DeltaCommand API surface across versions (incl. createTableRelation).
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/GpuDeleteCommandBase.scala	Switches DeltaCommand dependency to a shared GPU Delta command interface.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/GpuCreateDeltaTableCommandBase.scala	Enhances shared create-table logic with catalog-managed validations and shim-based operation metadata.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/DeltaTrampoline.scala	Minor formatting/compat adjustment in shared trampoline helper.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/DeltaRuntimeShimBase.scala	Adds shared runtime shim base used by 3.3.x and 4.x shims.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/DeltaCommandShims.scala	Adds a shared abstraction for Spark 3.x vs Spark 4.x command session/DF APIs.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/common/GpuDeltaFileFormatWriterBase.scala	Adds shared task attempt context creation logic for Delta GPU file writing.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/rapids/commands/GpuOptimizeExecutor.scala	Switches DeltaCommand dependency to a shared GPU Delta command interface.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/hooks/GpuAutoCompact.scala	Refactors auto-compact base to use version-specific tableId access and hook invocation.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/deletionvectors/RapidsStoredBitmap.scala	Adds GPU-side helpers for loading serialized deletion vectors into host memory.
delta-lake/common/src/main/delta-33x-42x/scala/org/apache/spark/sql/delta/deletionvectors/RapidsDeletionVectorStore.scala	Adds a simplified DV store implementation for host-side DV loading and checksum validation.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/GpuDeltaCatalogBase.scala	Refactors GPU Delta catalog creation to use shim-based write factory and improved table lookup hooks.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/DeltaWriteUtils.scala	Adds shared optimized-write enablement logic copied from Delta to preserve behavior.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/common/UpdateCommandMetaBase.scala	Adds shared GPU tagging base for UpdateCommand across supported versions.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/common/RapidsRowIndexFilters.scala	Minor cleanup in shared row index filters source.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/common/OptimizeTableCommandMetaBase.scala	Adds shared GPU tagging base for OptimizeTableCommand across versions.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/common/MergeIntoCommandMetaBase.scala	Adds shared GPU tagging base for MergeIntoCommand across versions.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/common/DeltaReorgTableCommandMetaBase.scala	Adds shared base type for REORG TABLE meta implementations.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/common/DeltaProviderBase.scala	Generalizes CDF strategy wiring so each adapter can supply the correct per-version strategy.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/common/DeltaDynamicPartitionOverwriteCommandMetaBase.scala	Ensures delta write option map is passed into tagging for DPO for correctness.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/common/DeltaCDFRelationStrategyBase.scala	Adds shared planning logic for CDF relation replanning; concrete strategies supply version-specific DF builder.
delta-lake/common/src/main/delta-33x-42x/scala/com/nvidia/spark/rapids/delta/common/DeleteCommandMetaBase.scala	Adds shared GPU tagging base for DeleteCommand across versions.
delta-lake/common/src/main/delta-33x-41x/scala/com/nvidia/spark/rapids/delta/common/DeltaReorgTableCommandMeta.scala	Refactors REORG meta to use shared base and shared GPU Delta command interface.
delta-lake/common/src/main/delta-33x-41x/scala/com/nvidia/spark/rapids/delta/common/DeltaCDFRelationStrategy.scala	Refactors existing CDF strategy to use the new shared base strategy implementation.
delta-lake/common/src/main/delta-20x-24x/scala/org/apache/spark/sql/delta/rapids/GpuWriteIntoDeltaLike.scala	Introduces/aligns shared GPU write trait for older Delta shims.
delta-lake/common/src/main/delta-20x-24x/scala/org/apache/spark/sql/delta/rapids/GpuWriteIntoDelta.scala	Mixes in the shared GPU write interface for older Delta shims.

Review details

Suppressed comments (1)

delta-lake/delta-42x/src/main/scala/com/nvidia/spark/rapids/delta/delta42x/Delta42xProvider.scala:201

• Same issue as above: the thrown IllegalStateException message is truncated and doesn't explain the supported table implementations for GPU conversion.

• Files reviewed: 82/112 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[greptile-apps]

Too many files changed for review (112 files, 100 file limit).

Bypass the limit by tagging @greptile-apps to review.

[jihoonson]

build

[gerashegalov]

isDynamicPartitionOverwriteMode can throw

https://github.com/delta-io/delta/blob/9f600c42dd9bc383c025ee6e43f8199571a2783b/spark/src/main/scala/org/apache/spark/sql/delta/commands/WriteIntoDelta.scala#L119

Suggested change

	toBooleanOption(cpuWrite.options.isDynamicPartitionOverwriteMode),
	toBooleanOption(Try(cpuWrite.options.isDynamicPartitionOverwriteMode).getOrElse(false)),

[jihoonson]

Good catch. Fixed.