Add advisory audit for NVIDIA API deprecations [reduced-it]

[gerashegalov]

Related to #15825.

Description

Deprecation annotations in NVIDIA-owned dependencies can land before cudf-spark can consume an
artifact containing the replacement API. With fatal Scala warnings, that ordering breaks unrelated
cudf-spark pull requests even though no compatible migration is available yet.

This change keeps deprecations from NVIDIA-owned cuDF Java, cudf-spark-jni, and
cudf-spark-private APIs visible as compiler information while retaining fatal handling for
third-party deprecations. Because cudf-spark-private does not have one artifact-specific namespace,
the policy covers its NVIDIA-owned package roots and an exact PartialAggUtils bridge symbol; an
adjacent Spark symbol remains fatal as a regression control.

The Maven pull-request workflow also gains an optional fan-in audit that downloads completed
matrix-job logs and reports compiler deprecations in the job summary and annotations. Findings or
incomplete log collection fail this optional check so contributors inspect it, while the report is
still uploaded and the build jobs remain authoritative. Log collection validates GitHub signed
redirects and never forwards the repository token to the signed download URL. Contributor
documentation now describes the staged cross-repository deprecation lifecycle. Both the compiler-
policy fixture checker and log audit run through Maven's existing Jython 2.7.3 dependency, so this
change does not add a host Python prerequisite; the existing resource-nesting check remains on
Python 3 for a dedicated follow-up port.

Validation performed:

• mvn -N -Dbuildver=330 verify
• mvn -N -f scala2.13/pom.xml -Dbuildver=350 verify
• Jython 2.7.3 deprecation-policy fixtures using the actual Scala 2.12.21 and Scala 2.13.18 POM arguments
• 18 audit unit tests under Jython 2.7.3 and 38 compatibility tests under Python 3
• Jython audit fixtures: clean logs pass; NVIDIA deprecation findings fail the optional audit
• Scala 2.12 and 2.13 Scalastyle, deprecation-policy, and with-resource nesting executions
• git diff --check

Checklists

Documentation

• Updated for new or modified user-facing features or behaviors
• No user-facing change

Testing

• Added or modified tests to cover new code paths
• Covered by existing tests
  (Please provide the names of the existing tests in the PR description.)
• Not required

Performance

• Tests ran and results are added in the PR description
• Issue filed with a link in the PR description
• Not required

[greptile-apps]

Greptile Summary

This PR makes deprecations from selected NVIDIA-owned APIs advisory while preserving fatal treatment for third-party deprecations, and adds an optional GitHub Actions audit that collects diagnostics from completed matrix jobs.

• Adds synchronized Scala 2.12 and 2.13 compiler warning policies and fixture validation.
• Adds a Jython-compatible log collector, report generation, annotations, and unit tests.
• Documents the staged cross-repository API deprecation lifecycle.
• Removes the obsolete hybrid dependency property from both Scala POM views.

Confidence Score: 4/5

The implementation appears technically sound, but the explicit performance-checklist requirement must be satisfied before merging.

The audit and compiler-policy changes have focused validation, and the recent removal of the hybrid property leaves no dangling references. Merge readiness is blocked only by the missing verifiable rationale for marking performance testing as unnecessary.

Files Needing Attention: .github/workflows/mvn-verify-check.yml

Important Files Changed

Filename	Overview
.github/workflows/mvn-verify-check.yml	Adds the optional matrix-log deprecation audit and Actions-read permission.
pom.xml	Adds Scala 2.12 warning policy and Jython audit/check executions while removing the obsolete hybrid property.
scala2.13/pom.xml	Keeps the generated Scala 2.13 build aligned with the compiler policy and validation configuration.
scripts/check_deprecation_policy.py	Compiles fixtures to verify NVIDIA deprecations remain visible but nonfatal while control cases remain fatal.
scripts/deprecation_audit.py	Securely downloads matrix logs, extracts deprecation diagnostics, and emits summaries, annotations, and JSON.
scripts/tests/test_deprecation_audit.py	Covers parsing, origin classification, redirect handling, retries, reporting, and failure behavior.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Matrix[Maven matrix jobs] --> Logs[Completed job logs]
  Logs --> Audit[Jython deprecation audit]
  Audit --> Parse[Parse and deduplicate diagnostics]
  Parse --> Summary[GitHub job summary and annotations]
  Parse --> Artifact[Raw JSON report]
  Parse --> Result{Findings or incomplete collection?}
  Result -- Yes --> Fail[Fail optional audit]
  Result -- No --> Pass[Pass audit]

Loading

_{Reviews (7): Last reviewed commit: "Merge branch 'main' into codex/nvidia-de..." | Re-trigger Greptile}

[gerashegalov]

build

[gerashegalov]

build

[gerashegalov]

build