Skip to content

Update dependencies for renamed cudf-spark-private artifacts - #15881

Merged
NvTimLiu merged 3 commits into
NVIDIA:mainfrom
NvTimLiu:private-artifacatId-rename
Sep 6, 2026
Merged

Update dependencies for renamed cudf-spark-private artifacts#15881
NvTimLiu merged 3 commits into
NVIDIA:mainfrom
NvTimLiu:private-artifacatId-rename

Conversation

@NvTimLiu

@NvTimLiu NvTimLiu commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Issue: part of #15882.

This change updates the Scala 2.12 and Scala 2.13 private-plugin Maven coordinates from rapids-4-spark-private_* to cudf-spark-private_*, renames the corresponding Maven version property,
updates CI dependency-cache lookup paths, and loads private build metadata from cudf-spark-private-version-info.properties.

The dependency contents and Spark execution paths are unchanged.

Performance testing is not required because this changes dependency coordinates and metadata resource naming only; no Spark execution or GPU processing path changes.

Checklists

Documentation

  • Updated for new or modified user-facing features or behaviors
  • No user-facing change

Testing

  • Added or modified tests to cover new code paths
  • Covered by existing tests
    (Please provide the names of the existing tests in the PR description.)
  • Not required

Performance

  • Tests ran and results are added in the PR description
  • Issue filed with a link in the PR description
  • Not required

@NvTimLiu
NvTimLiu requested a review from a team as a code owner September 3, 2026 04:07
@greptile-apps

greptile-apps Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR consistently migrates the private plugin dependency and build-information resource to the new cudf-spark-private naming.

  • Updates Maven coordinates and version properties for Scala 2.12 and 2.13.
  • Updates snapshot checksum and workflow lookups used for dependency-cache invalidation.
  • Updates the private build-information resource loaded during plugin initialization.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename Overview
pom.xml Renames the shared private dependency version property used by the Scala 2.12 build.
scala2.13/pom.xml Keeps the generated Scala 2.13 dependency property aligned with the main build.
sql-plugin/pom.xml Migrates the Scala 2.12 SQL plugin dependency to the renamed private artifact.
scala2.13/sql-plugin/pom.xml Applies the same private dependency migration to the Scala 2.13 SQL plugin.
aggregator/pom.xml Updates the Scala 2.12 aggregator input to the renamed private artifact.
scala2.13/aggregator/pom.xml Keeps the Scala 2.13 aggregator input aligned with the main aggregator.
.github/workflows/mvn-verify-check.yml Evaluates the renamed private dependency property when constructing verification matrices.
.github/workflows/mvn-verify-check/get-deps-sha1.sh Tracks the renamed private snapshot coordinate when generating dependency cache checksums.
sql-plugin/src/main/scala/com/nvidia/spark/rapids/Plugin.scala Loads private build metadata using the resource name associated with the renamed artifact.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  P[Root Maven property] --> S[SQL plugin dependency]
  P --> A[Aggregator dependency]
  S --> J[Packaged plugin JAR]
  A --> J
  W[Verification workflow] --> H[Snapshot checksum helper]
  H --> C[Dependency cache key]
  J --> R[Private build-info resource]
  R --> B[Plugin initialization]
Loading

Reviews (4): Last reviewed commit: "Load renamed private build info resource" | Re-trigger Greptile

@NvTimLiu
NvTimLiu force-pushed the private-artifacatId-rename branch from 1f79850 to d471bd0 Compare September 3, 2026 04:47
@NvTimLiu NvTimLiu self-assigned this Sep 3, 2026
@NvTimLiu

NvTimLiu commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

build

@NvTimLiu NvTimLiu added the build Related to CI / CD or cleanly building label Sep 4, 2026
@NvTimLiu
NvTimLiu requested review from GaryShen2008 and sameerz and a lite review from Copilot September 4, 2026 02:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are limited to Maven coordinates and the corresponding CI checksum lookup, and repository-wide searches show no remaining references to the old private artifactId.

Pull request overview

Updates this repository’s build-time dependency coordinates to consume the renamed private plugin artifacts (rapids-4-spark-private_*cudf-spark-private_*) across both Scala 2.12 and 2.13 builds, and keeps CI cache invalidation working by updating the snapshot checksum lookup script accordingly.

Changes:

  • Switch SQL plugin and aggregator Maven dependencies to cudf-spark-private_${scala.binary.version} for Scala 2.12 and 2.13.
  • Update the mvn-verify dependency SHA1 helper to query the renamed private snapshot artifact path.
File summaries
File Description
sql-plugin/pom.xml Updates Scala 2.12 SQL plugin dependency artifactId for the private plugin.
scala2.13/sql-plugin/pom.xml Updates Scala 2.13 SQL plugin dependency artifactId for the private plugin.
aggregator/pom.xml Updates Scala 2.12 aggregator dependency artifactId for the private plugin.
scala2.13/aggregator/pom.xml Updates Scala 2.13 aggregator dependency artifactId for the private plugin.
.github/workflows/mvn-verify-check/get-deps-sha1.sh Updates CI cache-key dependency SHA1 lookup to follow the renamed private snapshot artifactId.
Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@NvTimLiu

NvTimLiu commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

Build PASS:
dee78dcc-affa-464a-92dd-db3f79594356
27b7eaae-6840-4af3-9dc8-13cf33803413

Comment thread aggregator/pom.xml Outdated
@NvTimLiu

NvTimLiu commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

build

GaryShen2008
GaryShen2008 previously approved these changes Sep 4, 2026
Update the Scala 2.12 and 2.13 SQL plugin and aggregator dependencies to use the renamed cudf-spark-private artifact prefix.

Point the Maven verification checksum helper at the renamed snapshot artifact so dependency cache invalidation continues to track private plugin updates.

Signed-off-by: timl <timl@nvidia.com>
Align the Maven version property with the cudf-spark-private artifact coordinate introduced by the private artifact rename.

Rename the root property and the SQL plugin and aggregator dependency references for Scala 2.12, then regenerate the corresponding Scala 2.13 POMs. Update both Maven verification workflow paths so snapshot cache keys and shim selection continue to read the private dependency version.

Blossom and other external callers that override or evaluate spark-rapids-private.version must migrate to cudf-spark-private.version as part of the coordinated rollout.

Signed-off-by: timl <timl@nvidia.com>
Signed-off-by: timl <timl@nvidia.com>
@NvTimLiu

NvTimLiu commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

build

@NvTimLiu

NvTimLiu commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

pre-merge Build PASS:
4a0aadd9-dd5b-404a-b78d-f58719a48180

@NvTimLiu
NvTimLiu merged commit 1aaef70 into NVIDIA:main Sep 6, 2026
59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Related to CI / CD or cleanly building

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants