From 14a5f038a6d28f516cdd7699274c4386b648d8e6 Mon Sep 17 00:00:00 2001 From: Sylvain Utard Date: Wed, 30 Sep 2026 09:43:36 +0200 Subject: [PATCH 1/2] Truncate TIMESTAMPTZ arithmetically in ICU date_trunc for fixed-length parts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit NOTE: this is a fork-only change. The ICU date_trunc code on upstream duckdb/main has changed since this fork point, so this commit will not be upstreamed as-is. Before proposing anything upstream, check whether main still has the per-row ICU cost measured below; it may already be addressed there, and the patch would need rework against the new code either way. Problem date_trunc(part, TIMESTAMPTZ) went through an icu::Calendar for every row: setTime, set fields, recompute. With a constant 'day' part on 10M rows in America/New_York, that took ~0.2s against ~0.01s for naive TIMESTAMP. Change When the part is a constant of fixed wall-time length (microseconds, milliseconds, second/epoch, minute, hour, day and its aliases), bind swaps in ArithmeticTruncFunction, which works on µs directly: - Below a minute, the result is floor(utc, unit). tzdata offsets are whole seconds, so the time zone and calendar cannot affect the result. This applies to every calendar. - Minute: keep the instant's offset (ICU's PreserveOffsets), floor the wall time, subtract the same offset. - Hour and day: floor the wall time, then re-resolve the offset at the truncated wall time, like ICU does. Gaps take the offset before a transition and overlaps the offset after it, which matches ICU's default UCAL_WALLTIME_LAST. Minute and above are only enabled for the gregorian calendar. Everything else (variable parts, coarser parts, other calendars, zones that don't expose their transitions) keeps the existing ICU path, whose function body is unchanged apart from a small TruncWithICU helper. The offsets come from a new ZoneOffsets table (icu-zone-offsets.{hpp,cpp}) built at bind time from icu::BasicTimeZone transitions. It is a sorted list of constant-offset segments with sentinels at both ends. One lookup serves both instants and wall times, and a one-segment cursor skips the binary search for clustered input. Zones with recurring rules are covered up to 2250; later instants, and instants within three days of the int64 limits, fall back to ICU per row. Build rejects zones where wall times would not increase monotonically, or where an offset is a day or more. The second check is what makes a single bounds check enough to rule out overflow. The table is immutable and shared between bind data copies through a shared_ptr. Performance (Apple Silicon, release build) - 10M rows, America/New_York, all threads: ICU path 0.202s, arithmetic path 0.0075s. Naive TIMESTAMP is 0.0086s. - 100M sorted rows, one thread, per row: day 2.2 ns, hour 2.2 ns, minute 2.0 ns, second 1.5 ns. A plain max(ts) scan is 0.42 ns. - 20M shuffled rows, one thread: 27 ns per row, against 255 ns for ICU. Random order misses the cursor, so each row does two binary searches over about 700 segments. A bucket index could fix this, but it was left out to keep ZoneOffsets simple. Tests test_icu_datetrunc.test gets explicit cases (UTC, a fixed offset, Asia/Kolkata history, New York spring-forward and fall-back, a session TimeZone change between queries, a non-gregorian calendar). It also gets a differential check: a part stored in a column forces the ICU path, and every constant-part result must match it. That runs across 11 zones and the islamic-civil calendar, over ~13.5k instants covering transition windows, 1883, 1945, the 1970 epoch, the 2250 end of the table and the infinities. Deliberately introduced bugs (no hour re-resolution, an off-by-one floor, an off-by-one segment end) each make it fail. Four micro-benchmarks under benchmark/micro/timestamp/ compare naive TIMESTAMP with TIMESTAMPTZ in UTC, a fixed offset and America/New_York. Co-authored-by: Cursor --- .../timestamp/date_trunc_timestamp.benchmark | 18 ++ .../date_trunc_timestamptz.benchmark | 22 ++ .../date_trunc_timestamptz_fixed.benchmark | 22 ++ .../date_trunc_timestamptz_new_york.benchmark | 22 ++ extension/icu/CMakeLists.txt | 3 +- extension/icu/icu-datetrunc.cpp | 152 +++++++++++++- extension/icu/icu-zone-offsets.cpp | 76 +++++++ extension/icu/include/icu-zone-offsets.hpp | 74 +++++++ .../timestamp/test_icu_datetrunc.test | 193 ++++++++++++++++++ 9 files changed, 574 insertions(+), 8 deletions(-) create mode 100644 benchmark/micro/timestamp/date_trunc_timestamp.benchmark create mode 100644 benchmark/micro/timestamp/date_trunc_timestamptz.benchmark create mode 100644 benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark create mode 100644 benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark create mode 100644 extension/icu/icu-zone-offsets.cpp create mode 100644 extension/icu/include/icu-zone-offsets.hpp diff --git a/benchmark/micro/timestamp/date_trunc_timestamp.benchmark b/benchmark/micro/timestamp/date_trunc_timestamp.benchmark new file mode 100644 index 000000000000..83dad1e77bd2 --- /dev/null +++ b/benchmark/micro/timestamp/date_trunc_timestamp.benchmark @@ -0,0 +1,18 @@ +# name: benchmark/micro/timestamp/date_trunc_timestamp.benchmark +# description: date_trunc('day') on 10 million naive TIMESTAMP values +# group: [timestamp] +# +# Baseline for date_trunc_timestamptz*.benchmark. ns/row = timing_seconds * 100. + +name TIMESTAMP date_trunc day +group timestamp +subgroup date_trunc + +load +CREATE TABLE ts AS SELECT TIMESTAMP '2024-01-01 00:00:00' + to_seconds(i) AS ts FROM range(10000000) t(i); + +run +SELECT count(DISTINCT date_trunc('day', ts)) FROM ts; + +result I +116 diff --git a/benchmark/micro/timestamp/date_trunc_timestamptz.benchmark b/benchmark/micro/timestamp/date_trunc_timestamptz.benchmark new file mode 100644 index 000000000000..5da3a7dc44f1 --- /dev/null +++ b/benchmark/micro/timestamp/date_trunc_timestamptz.benchmark @@ -0,0 +1,22 @@ +# name: benchmark/micro/timestamp/date_trunc_timestamptz.benchmark +# description: date_trunc('day') on 10 million TIMESTAMPTZ values +# group: [timestamp] +# +# ns/row = timing_seconds * 1e9 / 10000000 = timing_seconds * 100. +# Compare with date_trunc_timestamp.benchmark on the same instants. + +name TIMESTAMPTZ date_trunc day (UTC) +group timestamp +subgroup date_trunc + +require icu + +load +SET TimeZone='UTC'; +CREATE TABLE ts AS SELECT (TIMESTAMP '2024-01-01 00:00:00' + to_seconds(i))::TIMESTAMPTZ AS ts FROM range(10000000) t(i); + +run +SELECT count(DISTINCT date_trunc('day', ts)) FROM ts; + +result I +116 diff --git a/benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark b/benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark new file mode 100644 index 000000000000..ce9ea875ddf2 --- /dev/null +++ b/benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark @@ -0,0 +1,22 @@ +# name: benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark +# description: date_trunc('day') on 10 million TIMESTAMPTZ values in a fixed-offset zone +# group: [timestamp] +# +# ns/row = timing_seconds * 100. Etc/GMT-5 is a constant +05:00 offset with no transitions. + +name TIMESTAMPTZ date_trunc day (fixed offset) +group timestamp +subgroup date_trunc + +require icu + +load +SET TimeZone='UTC'; +CREATE TABLE ts AS SELECT (TIMESTAMP '2024-01-01 00:00:00' + to_seconds(i))::TIMESTAMPTZ AS ts FROM range(10000000) t(i); +SET TimeZone='Etc/GMT-5'; + +run +SELECT count(DISTINCT date_trunc('day', ts)) FROM ts; + +result I +116 diff --git a/benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark b/benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark new file mode 100644 index 000000000000..207431c0218d --- /dev/null +++ b/benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark @@ -0,0 +1,22 @@ +# name: benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark +# description: date_trunc('day') on 10 million TIMESTAMPTZ values across America/New_York transitions +# group: [timestamp] +# +# ns/row = timing_seconds * 100. The instants span the 2024 spring-forward transition. + +name TIMESTAMPTZ date_trunc day (America/New_York) +group timestamp +subgroup date_trunc + +require icu + +load +SET TimeZone='UTC'; +CREATE TABLE ts AS SELECT (TIMESTAMP '2024-01-01 00:00:00' + to_seconds(i))::TIMESTAMPTZ AS ts FROM range(10000000) t(i); +SET TimeZone='America/New_York'; + +run +SELECT count(DISTINCT date_trunc('day', ts)) FROM ts; + +result I +117 diff --git a/extension/icu/CMakeLists.txt b/extension/icu/CMakeLists.txt index a5bb58a9424f..67e30a03ba2a 100644 --- a/extension/icu/CMakeLists.txt +++ b/extension/icu/CMakeLists.txt @@ -25,7 +25,8 @@ set(ICU_EXTENSION_FILES icu-table-range.cpp icu-strptime.cpp icu-timebucket.cpp - icu-timezone.cpp) + icu-timezone.cpp + icu-zone-offsets.cpp) build_static_extension(icu ${ICU_EXTENSION_FILES}) link_threads(icu_extension "") diff --git a/extension/icu/icu-datetrunc.cpp b/extension/icu/icu-datetrunc.cpp index 554e01c0442e..6a39c6be3fc2 100644 --- a/extension/icu/icu-datetrunc.cpp +++ b/extension/icu/icu-datetrunc.cpp @@ -1,11 +1,17 @@ #include "include/icu-datetrunc.hpp" #include "include/icu-datefunc.hpp" +#include "include/icu-zone-offsets.hpp" +#include "duckdb/common/limits.hpp" +#include "duckdb/common/types/interval.hpp" #include "duckdb/common/types/timestamp.hpp" #include "duckdb/common/vector_operations/binary_executor.hpp" +#include "duckdb/execution/expression_executor.hpp" #include "duckdb/planner/expression/bound_function_expression.hpp" #include "duckdb/main/extension/extension_loader.hpp" +#include + namespace duckdb { struct ICUDateTrunc : public ICUDateFunc { @@ -123,6 +129,141 @@ struct ICUDateTrunc : public ICUDateFunc { calendar->set(UCAL_ERA, era); } + static timestamp_t TruncWithICU(icu::Calendar *calendar, part_trunc_t truncator, timestamp_t input) { + auto micros = SetTime(calendar, input); + truncator(calendar, micros); + return GetTimeUnsafe(calendar, micros); + } + + //! The length in µs of the parts that always span the same amount of wall time, or 0. + static int64_t FixedPartLength(DatePartSpecifier part) { + switch (part) { + case DatePartSpecifier::DAY: + case DatePartSpecifier::DOW: + case DatePartSpecifier::ISODOW: + case DatePartSpecifier::DOY: + case DatePartSpecifier::JULIAN_DAY: + return Interval::MICROS_PER_DAY; + case DatePartSpecifier::HOUR: + return Interval::MICROS_PER_HOUR; + case DatePartSpecifier::MINUTE: + return Interval::MICROS_PER_MINUTE; + case DatePartSpecifier::SECOND: + case DatePartSpecifier::EPOCH: + return Interval::MICROS_PER_SEC; + case DatePartSpecifier::MILLISECONDS: + return Interval::MICROS_PER_MSEC; + case DatePartSpecifier::MICROSECONDS: + return 1; + default: + return 0; + } + } + + //! A constant part of fixed length, truncated with arithmetic on µs instead of ICU calls. + struct ArithmeticTruncData : public BindData { + ArithmeticTruncData(ClientContext &context, DatePartSpecifier part, int64_t unit_p) + : BindData(context), truncator(TruncationFactory(part)), unit(unit_p) { + } + + //! Truncates the instants the arithmetic cannot handle + part_trunc_t truncator; + int64_t unit; + //! Unset below a minute: offsets are whole seconds, so the instant truncates like its wall time. + shared_ptr offsets; + + unique_ptr Copy() const override { + return make_uniq(*this); + } + }; + + //! The per-execution state of an arithmetic truncation. + struct ArithmeticTruncator { + explicit ArithmeticTruncator(const ArithmeticTruncData &data) : unit(data.unit), offsets(data.offsets.get()) { + } + + const int64_t unit; + const ZoneOffsets *const offsets; + ZoneOffsets::Cursor instant_cursor; + ZoneOffsets::Cursor wall_cursor; + + //! Offsets and units are under a day, so truncation moves an instant by less than three days. + static constexpr int64_t MIN_SAFE = NumericLimits::Minimum() + 3 * Interval::MICROS_PER_DAY; + static constexpr int64_t MAX_SAFE = NumericLimits::Maximum() - 3 * Interval::MICROS_PER_DAY; + + static int64_t Floor(int64_t value, int64_t unit) { + const auto remainder = value % unit; + return value - (remainder < 0 ? remainder + unit : remainder); + } + + bool TryTrunc(int64_t utc, int64_t &result) { + if (utc < MIN_SAFE || utc > MAX_SAFE) { + return false; + } + if (!offsets) { + result = Floor(utc, unit); + return true; + } + int64_t offset; + if (!offsets->OffsetAtInstant(utc, instant_cursor, offset)) { + return false; + } + const auto floored = Floor(utc + offset, unit); + // Like ICU, keep the instant's offset below an hour (PreserveOffsets) and re-resolve it from an hour up. + if (unit >= Interval::MICROS_PER_HOUR && !offsets->OffsetAtWallTime(floored, wall_cursor, offset)) { + return false; + } + result = floored - offset; + return true; + } + }; + + template + static void ArithmeticTruncFunction(DataChunk &args, ExpressionState &state, Vector &result) { + auto &info = state.expr.Cast().bind_info->Cast(); + ArithmeticTruncator truncator(info); + CalendarPtr calendar; + UnaryExecutor::Execute(args.data[1], result, args.size(), [&](T input) { + if (!Timestamp::IsFinite(input)) { + return input; + } + int64_t truncated; + if (truncator.TryTrunc(input.value, truncated)) { + return timestamp_t(truncated); + } + if (!calendar) { + calendar.reset(info.calendar->clone()); + } + return TruncWithICU(calendar.get(), info.truncator, input); + }); + } + + template + static unique_ptr BindDateTrunc(ClientContext &context, ScalarFunction &bound_function, + vector> &arguments) { + if (!arguments[0]->IsFoldable()) { + return Bind(context, bound_function, arguments); + } + const auto part_value = ExpressionExecutor::EvaluateScalar(context, *arguments[0]); + DatePartSpecifier part; + if (part_value.IsNull() || !TryGetDatePartSpecifier(part_value.ToString(), part) || !FixedPartLength(part)) { + return Bind(context, bound_function, arguments); + } + + auto data = make_uniq(context, part, FixedPartLength(part)); + if (data->unit >= Interval::MICROS_PER_MINUTE) { + // Only the offsets are modelled, so other calendars stay on ICU. + if (std::strcmp(data->calendar->getType(), "gregorian") == 0) { + data->offsets = ZoneOffsets::Build(data->calendar->getTimeZone()); + } + if (!data->offsets) { + return Bind(context, bound_function, arguments); + } + } + bound_function.SetFunctionCallback(ArithmeticTruncFunction); + return std::move(data); + } + template static void ICUDateTruncFunction(DataChunk &args, ExpressionState &state, Vector &result) { D_ASSERT(args.ColumnCount() == 2); @@ -143,9 +284,7 @@ struct ICUDateTrunc : public ICUDateFunc { auto truncator = TruncationFactory(GetDatePartSpecifier(specifier)); UnaryExecutor::Execute(date_arg, result, args.size(), [&](T input) { if (Timestamp::IsFinite(input)) { - auto micros = SetTime(calendar.get(), input); - truncator(calendar.get(), micros); - return GetTimeUnsafe(calendar.get(), micros); + return TruncWithICU(calendar.get(), truncator, input); } else { return input; } @@ -156,9 +295,7 @@ struct ICUDateTrunc : public ICUDateFunc { part_arg, date_arg, result, args.size(), [&](string_t specifier, T input) { if (Timestamp::IsFinite(input)) { auto truncator = TruncationFactory(GetDatePartSpecifier(specifier.GetString())); - auto micros = SetTime(calendar.get(), input); - truncator(calendar.get(), micros); - return GetTimeUnsafe(calendar.get(), micros); + return TruncWithICU(calendar.get(), truncator, input); } else { return input; } @@ -168,7 +305,8 @@ struct ICUDateTrunc : public ICUDateFunc { template static ScalarFunction GetDateTruncFunction(const LogicalTypeId &type) { - return ScalarFunction({LogicalType::VARCHAR, type}, LogicalType::TIMESTAMP_TZ, ICUDateTruncFunction, Bind); + return ScalarFunction({LogicalType::VARCHAR, type}, LogicalType::TIMESTAMP_TZ, ICUDateTruncFunction, + BindDateTrunc); } static void AddBinaryTimestampFunction(const string &name, ExtensionLoader &loader) { diff --git a/extension/icu/icu-zone-offsets.cpp b/extension/icu/icu-zone-offsets.cpp new file mode 100644 index 000000000000..5dd4946ea640 --- /dev/null +++ b/extension/icu/icu-zone-offsets.cpp @@ -0,0 +1,76 @@ +#include "include/icu-zone-offsets.hpp" + +#include "duckdb/common/types/date.hpp" +#include "duckdb/common/types/interval.hpp" +#include "unicode/basictz.h" +#include "unicode/tzrule.h" +#include "unicode/tztrans.h" + +namespace duckdb { + +static int64_t RuleOffset(const icu::TimeZoneRule &rule) { + return (int64_t(rule.getRawOffset()) + rule.getDSTSavings()) * Interval::MICROS_PER_MSEC; +} + +static bool IsUnderADay(int64_t offset) { + return offset > -Interval::MICROS_PER_DAY && offset < Interval::MICROS_PER_DAY; +} + +shared_ptr ZoneOffsets::Build(const icu::TimeZone &zone) { + const auto basic_zone = dynamic_cast(&zone); + if (!basic_zone) { + return nullptr; + } + + // Recurring rules produce transitions forever, so stop at a year well past scan-scale data. + const int64_t coverage_end = int64_t(Date::FromDate(2250, 1, 1).days) * Interval::MICROS_PER_DAY; + // Early enough to precede every tzdata transition, late enough for exact µs conversion. + UDate search_from = -1.0e15; + + icu::TimeZoneTransition transition; + bool has_next = basic_zone->getNextTransition(search_from, false, transition); + int64_t offset; + if (has_next) { + offset = RuleOffset(*transition.getFrom()); + } else { + int32_t raw = 0; + int32_t dst = 0; + UErrorCode status = U_ZERO_ERROR; + basic_zone->getOffset(0, false, raw, dst, status); + if (U_FAILURE(status)) { + return nullptr; + } + offset = (int64_t(raw) + dst) * Interval::MICROS_PER_MSEC; + } + if (!IsUnderADay(offset)) { + return nullptr; + } + + auto result = make_shared_ptr(); + auto &segments = result->segments; + segments.push_back(Segment {NumericLimits::Minimum(), NumericLimits::Minimum(), offset}); + for (; has_next; has_next = basic_zone->getNextTransition(search_from, false, transition)) { + search_from = transition.getTime(); + const auto utc = int64_t(search_from) * Interval::MICROS_PER_MSEC; + const auto offset_after = RuleOffset(*transition.getTo()); + const bool past_coverage = utc >= coverage_end; + if (!past_coverage && offset_after == segments.back().offset) { + continue; + } + if (!IsUnderADay(offset_after)) { + return nullptr; + } + // A fall-back longer than the time since the previous transition would make wall times non-monotonic. + if (utc + offset_after <= segments.back().start_local) { + return nullptr; + } + segments.push_back(Segment {utc, utc + offset_after, offset_after}); + if (past_coverage) { + return std::move(result); + } + } + segments.push_back(Segment {NumericLimits::Maximum(), NumericLimits::Maximum(), 0}); + return std::move(result); +} + +} // namespace duckdb diff --git a/extension/icu/include/icu-zone-offsets.hpp b/extension/icu/include/icu-zone-offsets.hpp new file mode 100644 index 000000000000..a5816026754c --- /dev/null +++ b/extension/icu/include/icu-zone-offsets.hpp @@ -0,0 +1,74 @@ +//===----------------------------------------------------------------------===// +// DuckDB +// +// icu-zone-offsets.hpp +// +// +//===----------------------------------------------------------------------===// + +#pragma once + +#include "duckdb.hpp" +#include "unicode/timezone.h" + +#include + +namespace duckdb { + +//! The UTC offsets of an ICU time zone as a table of constant-offset segments, so instants and wall times +//! (both in µs) can be converted with arithmetic instead of per-value ICU calls. +//! +//! A transition starts at the instant `utc` and at the wall time `utc + offset_after`. Wall times in a gap +//! therefore take the offset before the transition and wall times in an overlap the offset after it, +//! which is how ICU resolves them under its default UCAL_WALLTIME_LAST. +class ZoneOffsets { +public: + //! The last segment a lookup hit, so that clustered inputs skip the binary search. + struct Cursor { + int64_t lo = 0; + int64_t hi = 0; + int64_t offset = 0; + }; + + //! Builds the table, or returns nullptr if the zone does not expose its transitions or has an offset of a day + //! or more. Zones with recurring rules are only covered up to a fixed year. + static shared_ptr Build(const icu::TimeZone &zone); + + //! The offset of the instant `utc`, or false past the covered range. + bool OffsetAtInstant(int64_t utc, Cursor &cursor, int64_t &offset) const { + return Lookup<&Segment::start_utc>(utc, cursor, offset); + } + + //! The offset that resolves the wall time `local`, or false past the covered range. + bool OffsetAtWallTime(int64_t local, Cursor &cursor, int64_t &offset) const { + return Lookup<&Segment::start_local>(local, cursor, offset); + } + +private: + struct Segment { + int64_t start_utc; + int64_t start_local; + int64_t offset; + }; + + //! Both starts strictly increase. The first segment starts at INT64_MIN and the last one only marks + //! the end of the covered range. + vector segments; + + template + bool Lookup(int64_t key, Cursor &cursor, int64_t &offset) const { + if (key < cursor.lo || key >= cursor.hi) { + const auto next = std::upper_bound(segments.begin(), segments.end(), key, + [](int64_t k, const Segment &segment) { return k < segment.*START; }); + if (next == segments.end()) { + return false; + } + const auto &segment = *(next - 1); + cursor = Cursor {segment.*START, (*next).*START, segment.offset}; + } + offset = cursor.offset; + return true; + } +}; + +} // namespace duckdb diff --git a/test/sql/function/timestamp/test_icu_datetrunc.test b/test/sql/function/timestamp/test_icu_datetrunc.test index 324107a0bb8a..9e312bafb5c4 100644 --- a/test/sql/function/timestamp/test_icu_datetrunc.test +++ b/test/sql/function/timestamp/test_icu_datetrunc.test @@ -246,3 +246,196 @@ query I select date_trunc('day', '2024-05-06 16:09:28+05:00'::timestamptz) >= '2024-05-06 17:19:18+05:20'::date as r ---- True + +# Constant-part truncation on UTC, a fixed offset, and a zone whose offset is constant only after its last transition. +statement ok +SET Calendar='gregorian'; + +statement ok +SET TimeZone='UTC'; + +query IIII +SELECT date_trunc('day', TIMESTAMPTZ '2024-06-15 15:45:30.123456+00'), + date_trunc('hour', TIMESTAMPTZ '2024-06-15 15:45:30.123456+00'), + date_trunc('minute', TIMESTAMPTZ '2024-06-15 15:45:30.123456+00'), + date_trunc('second', TIMESTAMPTZ '2024-06-15 15:45:30.123456+00'); +---- +2024-06-15 00:00:00+00 2024-06-15 15:00:00+00 2024-06-15 15:45:00+00 2024-06-15 15:45:30+00 + +query II +SELECT date_trunc('day', TIMESTAMPTZ '1969-12-31 23:30:00+00'), date_trunc('hour', TIMESTAMPTZ '1969-12-31 23:30:45+00'); +---- +1969-12-31 00:00:00+00 1969-12-31 23:00:00+00 + +query II +SELECT date_trunc('day', '-infinity'::TIMESTAMPTZ), date_trunc('hour', 'infinity'::TIMESTAMPTZ); +---- +-infinity infinity + +statement ok +SET TimeZone='Etc/GMT-5'; + +query IIII +SELECT date_trunc('day', TIMESTAMPTZ '2024-06-15 20:00:00+00'), + date_trunc('hour', TIMESTAMPTZ '2024-06-15 20:00:00+00'), + date_trunc('minute', TIMESTAMPTZ '2024-06-15 20:10:45+00'), + date_trunc('second', TIMESTAMPTZ '2024-06-15 20:10:45.9+00'); +---- +2024-06-16 00:00:00+05 2024-06-16 01:00:00+05 2024-06-16 01:10:00+05 2024-06-16 01:10:45+05 + +# Asia/Kolkata is +05:30 today, with earlier offsets. Local midnight is not UTC midnight. +statement ok +SET TimeZone='Asia/Kolkata'; + +query III +SELECT date_trunc('day', TIMESTAMPTZ '1942-09-01 00:00:00+00'), + date_trunc('hour', TIMESTAMPTZ '1905-06-01 12:00:00+00'), + date_trunc('day', TIMESTAMPTZ '2024-01-01 00:00:00+00'); +---- +1942-09-01 01:00:00+06:30 1905-06-01 17:00:00+05:21 2024-01-01 00:00:00+05:30 + +query IIII +SELECT date_trunc('day', TIMESTAMPTZ '2024-06-15 15:45:30.123456+00'), + date_trunc('hour', TIMESTAMPTZ '2024-06-15 15:45:30.123456+00'), + date_trunc('minute', TIMESTAMPTZ '2024-06-15 15:45:30.123456+00'), + date_trunc('second', TIMESTAMPTZ '2024-06-15 15:45:30.123456+00'); +---- +2024-06-15 00:00:00+05:30 2024-06-15 21:00:00+05:30 2024-06-15 21:15:00+05:30 2024-06-15 21:15:30+05:30 + +# America/New_York spring forward: 2024-03-10 02:00 EST becomes 03:00 EDT. +# 07:00Z is 03:00 EDT; local midnight is 05:00Z, not 04:00Z. +statement ok +SET TimeZone='America/New_York'; + +query IIII +SELECT ts, date_trunc('day', ts), date_trunc('hour', ts), date_trunc('minute', ts) +FROM (VALUES + (TIMESTAMPTZ '2024-03-10 05:30:00+00'), + (TIMESTAMPTZ '2024-03-10 06:30:00+00'), + (TIMESTAMPTZ '2024-03-10 06:59:59+00'), + (TIMESTAMPTZ '2024-03-10 07:00:00+00'), + (TIMESTAMPTZ '2024-03-10 07:30:00+00') +) v(ts); +---- +2024-03-10 00:30:00-05 2024-03-10 00:00:00-05 2024-03-10 00:00:00-05 2024-03-10 00:30:00-05 +2024-03-10 01:30:00-05 2024-03-10 00:00:00-05 2024-03-10 01:00:00-05 2024-03-10 01:30:00-05 +2024-03-10 01:59:59-05 2024-03-10 00:00:00-05 2024-03-10 01:00:00-05 2024-03-10 01:59:00-05 +2024-03-10 03:00:00-04 2024-03-10 00:00:00-05 2024-03-10 03:00:00-04 2024-03-10 03:00:00-04 +2024-03-10 03:30:00-04 2024-03-10 00:00:00-05 2024-03-10 03:00:00-04 2024-03-10 03:30:00-04 + +# Fall back: 2024-11-03 02:00 EDT becomes 01:00 EST. The overlap resolves the way ICU does. +query IIII +SELECT ts, date_trunc('day', ts), date_trunc('hour', ts), date_trunc('second', ts) +FROM (VALUES + (TIMESTAMPTZ '2024-11-03 04:30:00+00'), + (TIMESTAMPTZ '2024-11-03 05:00:00+00'), + (TIMESTAMPTZ '2024-11-03 05:30:00+00'), + (TIMESTAMPTZ '2024-11-03 05:59:59+00'), + (TIMESTAMPTZ '2024-11-03 06:00:00+00'), + (TIMESTAMPTZ '2024-11-03 06:30:00+00') +) v(ts); +---- +2024-11-03 00:30:00-04 2024-11-03 00:00:00-04 2024-11-03 00:00:00-04 2024-11-03 00:30:00-04 +2024-11-03 01:00:00-04 2024-11-03 00:00:00-04 2024-11-03 01:00:00-05 2024-11-03 01:00:00-04 +2024-11-03 01:30:00-04 2024-11-03 00:00:00-04 2024-11-03 01:00:00-05 2024-11-03 01:30:00-04 +2024-11-03 01:59:59-04 2024-11-03 00:00:00-04 2024-11-03 01:00:00-05 2024-11-03 01:59:59-04 +2024-11-03 01:00:00-05 2024-11-03 00:00:00-04 2024-11-03 01:00:00-05 2024-11-03 01:00:00-05 +2024-11-03 01:30:00-05 2024-11-03 00:00:00-04 2024-11-03 01:00:00-05 2024-11-03 01:30:00-05 + +# Bind data follows the session TimeZone and does not reuse an offset from the previous query. +statement ok +SET TimeZone='UTC'; + +query I +SELECT date_trunc('day', TIMESTAMPTZ '2024-03-10 07:00:00+00'); +---- +2024-03-10 00:00:00+00 + +statement ok +SET TimeZone='America/New_York'; + +query I +SELECT date_trunc('day', TIMESTAMPTZ '2024-03-10 07:00:00+00'); +---- +2024-03-10 00:00:00-05 + +statement ok +SET TimeZone='UTC'; + +query I +SELECT date_trunc('day', TIMESTAMPTZ '2024-03-10 07:00:00+00'); +---- +2024-03-10 00:00:00+00 + +# Non-gregorian calendars stay on ICU. Month boundaries differ from the Gregorian calendar. +statement ok +SET Calendar='islamic-civil'; + +query II +SELECT date_trunc('day', TIMESTAMPTZ '2024-03-10 12:00:00+00'), date_trunc('month', TIMESTAMPTZ '2024-03-10 12:00:00+00'); +---- +1445-08-29 00:00:00+00 1445-08-01 00:00:00+00 + +statement ok +SET Calendar='gregorian'; + +# Constant parts are truncated arithmetically; a part column forces the ICU path. The two must agree. +statement ok +CREATE TABLE trunc_parts AS +SELECT unnest(['microseconds', 'milliseconds', 'second', 'epoch', 'minute', 'hour', 'day', 'julian']) AS part; + +# Dense windows around transitions (New York 2024, 1883 and 1945, Lord Howe 2024, the 1970 epoch, the 2250 end +# of the transition table), plus a sparse sweep across four centuries. Strides carry µs so every part matters. +statement ok +CREATE TABLE trunc_instants AS +SELECT TIMESTAMPTZ '2024-03-10 04:00:00+00' + to_microseconds(i * 37000123) AS ts FROM range(1000) t(i) +UNION ALL SELECT TIMESTAMPTZ '2024-11-03 03:00:00+00' + to_microseconds(i * 41000077) FROM range(1000) t(i) +UNION ALL SELECT TIMESTAMPTZ '1883-11-18 12:00:00+00' + to_microseconds(i * 29000001) FROM range(1000) t(i) +UNION ALL SELECT TIMESTAMPTZ '1945-09-30 04:00:00+00' + to_microseconds(i * 29000013) FROM range(1000) t(i) +UNION ALL SELECT TIMESTAMPTZ '2024-10-05 14:00:00+00' + to_microseconds(i * 17000019) FROM range(1000) t(i) +UNION ALL SELECT TIMESTAMPTZ '2024-04-06 14:00:00+00' + to_microseconds(i * 19000023) FROM range(1000) t(i) +UNION ALL SELECT TIMESTAMPTZ '1969-12-31 12:00:00+00' + to_microseconds(i * 86399999) FROM range(1000) t(i) +UNION ALL SELECT TIMESTAMPTZ '2249-10-01 00:00:00+00' + to_microseconds(i * 9000000007) FROM range(1000) t(i) +UNION ALL SELECT TIMESTAMPTZ '1850-01-01 00:00:00+00' + to_microseconds(i * 2592031000003) FROM range(5500) t(i) +UNION ALL SELECT unnest(['-infinity'::TIMESTAMPTZ, 'infinity'::TIMESTAMPTZ, NULL]); + +statement ok +CREATE MACRO trunc_mismatches() AS TABLE +SELECT * FROM ( + SELECT part, ts, date_trunc(part, ts) AS icu, CASE part + WHEN 'microseconds' THEN date_trunc('microseconds', ts) + WHEN 'milliseconds' THEN date_trunc('milliseconds', ts) + WHEN 'second' THEN date_trunc('second', ts) + WHEN 'epoch' THEN date_trunc('epoch', ts) + WHEN 'minute' THEN date_trunc('minute', ts) + WHEN 'hour' THEN date_trunc('hour', ts) + WHEN 'day' THEN date_trunc('day', ts) + WHEN 'julian' THEN date_trunc('julian', ts) + END AS arithmetic + FROM trunc_parts, trunc_instants +) WHERE icu IS DISTINCT FROM arithmetic; + +foreach zone UTC Etc/GMT-5 America/New_York Australia/Lord_Howe Asia/Kolkata Asia/Kathmandu America/St_Johns Europe/Dublin Pacific/Apia Africa/Casablanca Antarctica/Troll + +statement ok +SET TimeZone='${zone}'; + +query IIII +SELECT * FROM trunc_mismatches(); +---- + +endloop + +# Calendars other than gregorian keep ICU from a minute up, but sub-minute parts never depend on the calendar. +statement ok +SET Calendar='islamic-civil'; + +statement ok +SET TimeZone='America/New_York'; + +query IIII +SELECT * FROM trunc_mismatches(); +---- + +statement ok +SET Calendar='gregorian'; From 9457f02a582c405636b5f2d7c149b6b8f33d252e Mon Sep 17 00:00:00 2001 From: Sylvain Utard Date: Wed, 30 Sep 2026 10:46:38 +0200 Subject: [PATCH 2/2] lint --- benchmark/micro/timestamp/date_trunc_timestamp.benchmark | 1 + benchmark/micro/timestamp/date_trunc_timestamptz.benchmark | 1 + benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark | 1 + .../micro/timestamp/date_trunc_timestamptz_new_york.benchmark | 1 + 4 files changed, 4 insertions(+) diff --git a/benchmark/micro/timestamp/date_trunc_timestamp.benchmark b/benchmark/micro/timestamp/date_trunc_timestamp.benchmark index 83dad1e77bd2..90d0615aac23 100644 --- a/benchmark/micro/timestamp/date_trunc_timestamp.benchmark +++ b/benchmark/micro/timestamp/date_trunc_timestamp.benchmark @@ -1,6 +1,7 @@ # name: benchmark/micro/timestamp/date_trunc_timestamp.benchmark # description: date_trunc('day') on 10 million naive TIMESTAMP values # group: [timestamp] + # # Baseline for date_trunc_timestamptz*.benchmark. ns/row = timing_seconds * 100. diff --git a/benchmark/micro/timestamp/date_trunc_timestamptz.benchmark b/benchmark/micro/timestamp/date_trunc_timestamptz.benchmark index 5da3a7dc44f1..a7e895274f70 100644 --- a/benchmark/micro/timestamp/date_trunc_timestamptz.benchmark +++ b/benchmark/micro/timestamp/date_trunc_timestamptz.benchmark @@ -1,6 +1,7 @@ # name: benchmark/micro/timestamp/date_trunc_timestamptz.benchmark # description: date_trunc('day') on 10 million TIMESTAMPTZ values # group: [timestamp] + # # ns/row = timing_seconds * 1e9 / 10000000 = timing_seconds * 100. # Compare with date_trunc_timestamp.benchmark on the same instants. diff --git a/benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark b/benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark index ce9ea875ddf2..a1aad291de56 100644 --- a/benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark +++ b/benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark @@ -1,6 +1,7 @@ # name: benchmark/micro/timestamp/date_trunc_timestamptz_fixed.benchmark # description: date_trunc('day') on 10 million TIMESTAMPTZ values in a fixed-offset zone # group: [timestamp] + # # ns/row = timing_seconds * 100. Etc/GMT-5 is a constant +05:00 offset with no transitions. diff --git a/benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark b/benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark index 207431c0218d..423c18a0cf82 100644 --- a/benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark +++ b/benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark @@ -1,6 +1,7 @@ # name: benchmark/micro/timestamp/date_trunc_timestamptz_new_york.benchmark # description: date_trunc('day') on 10 million TIMESTAMPTZ values across America/New_York transitions # group: [timestamp] + # # ns/row = timing_seconds * 100. The instants span the 2024 spring-forward transition.