Skip to content

Require matching dtypes for extension comparisons - #11016

Open
connortsui20 wants to merge 6 commits into
developfrom
ct/ext-types/comparison-consistency
Open

connortsui20 wants to merge 6 commits into
developfrom
ct/ext-types/comparison-consistency

Conversation

@connortsui20

@connortsui20 connortsui20 commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Summary

Extension comparisons could compare raw storage on the array path while scalar execution rejected the same operands. Applies one dtype requirement across comparison paths.

Changes

Checks logical dtypes at binding and before comparison fast paths or encoding kernels run. Regression tests cover batch, row, and constant execution, encoded timestamps, and DuckDB filters. DataFusion literal conversion is handled separately in #11067.

Formatting passes with CI's pinned nightly-2026-09-10 for vortex-datetime-parts and vortex-duckdb. Tests, Clippy, and benchmarks were not rerun locally after cleanup.

API Changes

Comparison operands must have matching logical dtypes, ignoring nullability. Comparisons between an extension and its storage dtype, or between different extension dtypes, now return an error. This includes timestamps with different units or time zones.

@connortsui20 connortsui20 added the changelog/break A breaking API change label Oct 29, 2026 — with ChatGPT Codex Connector
@connortsui20
connortsui20 added this pull request to stack #11017 October 29, 2026 22:05
@codspeed

codspeed Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 6 benchmarks

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚡ 3 improved benchmarks
❌ 6 regressed benchmarks
✅ 2309 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ CPU Simulation bench_compare_varbin[(50000, 2)] 132.1 µs 162.2 µs -18.57%
❌ CPU Simulation bench_compare_varbin[(50000, 64)] 133.9 µs 164.1 µs -18.39%
❌ CPU Simulation bench_compare_sliced_dict_varbinview[(1000, 10000)] 76.9 µs 89.7 µs -14.25%
❌ CPU Simulation bench_compare_sliced_dict_varbinview[(7500, 10000)] 198.6 µs 228.3 µs -12.99%
❌ CPU Simulation bench_compare_sliced_dict_varbinview[(10000, 10000)] 204.8 µs 234.6 µs -12.7%
❌ Walltime starts_with_avx2[8, VarBinView] 68.5 µs 76.3 µs -10.23%
⚡ CPU Simulation filter_powerlaw_by_random[10000] 684 ns 314.1 ns ×2.2
⚡ CPU Simulation take_filter_primitive_small_uncached_random_mask_random_indices[4096, 10] 24 µs 20.6 µs +16.46%
⚡ Walltime starts_with_avx2[64, VarBin] 61 µs 53.5 µs +14.15%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/ext-types/comparison-consistency (db0b297) with develop (96fc43d)

Open in CodSpeed

@connortsui20
connortsui20 marked this pull request as draft October 29, 2026 22:22
@connortsui20
connortsui20 force-pushed the ct/ext-types/comparison-consistency branch from 2c3048c to deb59a4 Compare October 30, 2026 13:43
@connortsui20
connortsui20 force-pushed the ct/ext-types/comparison-consistency branch 2 times, most recently from b1e9533 to 5bed318 Compare October 30, 2026 19:35
Base automatically changed from ct/ext-types/path-consistency to develop October 30, 2026 19:45
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 force-pushed the ct/ext-types/comparison-consistency branch from 5bed318 to 2461528 Compare October 30, 2026 19:45
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant