Skip to content

[AURON #2524] Support unix_date and date_from_unix_date natively - #2526

Merged
slfan1989 merged 1 commit into
apache:masterfrom
Sigma-Ma:Auron-2524-support-unix-date-and-date-from-unix-date-natively
Sep 23, 2026
Merged

slfan1989 merged 1 commit into
apache:masterfrom
Sigma-Ma:Auron-2524-support-unix-date-and-date-from-unix-date-natively

Conversation

@Sigma-Ma

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #2524

Rationale for this change

Spark's unix_date and date_from_unix_date expressions currently fall back to Spark evaluation.

What changes are included in this PR?

  • Convert both expressions to the existing native Date32/Int32 casts through Spark shims.
  • Keep Spark 3.0 builds compatible, since these functions were introduced in Spark 3.1.
  • Add Spark regression tests for nulls, round trips and integer boundaries with ANSI mode enabled and disabled, and Rust tests for array and scalar casts.

Are there any user-facing changes?

Both functions can now execute natively. Their SQL syntax and results are unchanged.

How was this patch tested?

  • cargo test -p datafusion-ext-exprs cast::test::test_epoch_date_cast -- --exact.
  • ./build/mvn -Ppre -Pspark-3.5 -Pscala-2.12 -pl spark-extension-shims-spark test -Dsuites=org.apache.auron.AuronFunctionSuite -DskipTests=false -DskipBuildNative.
  • ./dev/reformat --check.

The new Spark tests failed on the original code because the projection fell back to Spark, and passed after the change.

Was this patch authored or co-authored using generative AI tooling?

  • Yes
  • No

Generated-by: OpenAI Codex (GPT-6)

@slfan1989 slfan1989 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@slfan1989
slfan1989 merged commit f2b12b4 into apache:master Sep 23, 2026
134 checks passed
@Sigma-Ma
Sigma-Ma deleted the Auron-2524-support-unix-date-and-date-from-unix-date-natively branch September 23, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support unix_date and date_from_unix_date natively

2 participants