Skip to content

[fix](timezone) resolve ISO 8601 Z designator to UTC without system Zulu alias - #70724

Open
abhinav-phi wants to merge 1 commit into
apache:masterfrom
abhinav-phi:fix-69326-utc-z-designator
Open

abhinav-phi wants to merge 1 commit into
apache:masterfrom
abhinav-phi:fix-69326-utc-z-designator

Conversation

@abhinav-phi

Copy link
Copy Markdown
Contributor

Background

BE datetime parsing resolves time zone names through TimezoneUtils, which scans the system tzdata (/usr/share/zoneinfo, overridable via TZDIR) into a name cache and keeps a separate cache of fixed-offset zones. ISO 8601 / RFC 3339 timestamps end in Z as the UTC designator, and today that designator only resolves when the system tzdata scan finds a legacy top-level Zulu file.

What problem does this PR solve?

Issue Number: close #69326
Related PR: None
PR Type: Bugfix

Problem Summary: On hosts whose tzdata ships no top-level Zulu file (for example Ubuntu 24.04 ships only Etc/Zulu), find_cctz_time_zone("Z") and parse_tz_offset_string("Z") both fail. A valid 2015-01-01T03:47:41Z input then converts to NULL, and a VARIANT path predefined as DATETIME filters the NULL out, so the field silently disappears from the stored VARIANT value while the load still reports success. The fix treats z/zulu as UTC aliases in normalize_timezone_name and seeds the name cache with UTC for both names unconditionally in load_timezones_to_cache, so the designator no longer depends on any system tzfile. Most of the diff is the regression test.

Details

  • Frequency: every load or query that feeds a Z-suffixed timestamp into BE datetime parsing on hosts without the legacy Zulu tzfile; the VARIANT typed-path drop is the visible data-loss symptom.
  • Root cause: load_timezones_to_cache() installed the z cache alias only conditionally (if (find("zulu")) emplace("z")), and normalize_timezone_name() did not accept z at all, so with no system Zulu file both the cache path and the offset-parse path failed.
  • Why existing tests missed it: the test environment's tzdata contains a Zulu entry, so the conditional alias was always installed there; nothing exercised the "no Zulu file" configuration.
  • Fix: normalize_timezone_name now maps z to UTC alongside utc/etc/utc/zulu, and load_timezones_to_cache unconditionally emplaces z and zulu as cctz::utc_time_zone() instead of copying from a system-provided zulu entry. find_cctz_time_zone("Z") now works on an empty named cache too, because the normalized name falls through to the parse_tz_offset_string UTC shortcut.
  • Other paths and branches: try_get_fixed_offset_seconds benefits automatically, since a Z-resolved zone is cctz::utc_time_zone() whose name is UTC and reports offset 0. No other call sites changed.

Testing

New unit test TimezoneUtilsTest.UtcZDesignatorResolvesToUtc in be/test/util/timezone_utils_test.cpp: it asserts parse_tz_offset_string("Z"/"z") succeeds, then runs find_cctz_time_zone("Z"/"z"/"zulu") both with an emptied cache and with the preloaded cache, asserting the resolved zone is UTC in each case. On master (before the fix) parse_tz_offset_string("Z") returns false and the uncached find_cctz_time_zone("Z") fails, so the test fails before the fix and passes after it. No local BE build is available in my environment (Windows host), so the suite itself runs in CI.

Notes for Reviewers

The conditional zulu-copy block in load_timezones_to_cache was replaced rather than extended; emplace keeps any system-provided zulu entry intact since UTC has the same value. CST special case untouched.

Release note

Timestamps ending in Z now load and query correctly on hosts without the legacy Zulu tzfile, and VARIANT DATETIME typed paths no longer silently drop such fields.

@abhinav-phi

Copy link
Copy Markdown
Contributor Author

hi @ixzc, this is the pr for issue #69326 which you approved.. z designator now resolves to utc directly in normalize_timezone_name, the cache seeds z and zulu unconditionally, and the unit test covers both cached and uncached paths.. let me know if anything needs changing.. thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] UTC Z timestamp parsing depends on the Zulu tzdata alias and can drop VARIANT fields

1 participant