[joseph-isaacs]

Summary

Follows #11047.

The benchmark binaries no longer decide which scan version a benchmark uses. That choice now lives in the Python bench orchestrator (vx-bench), which lists the scan version of every benchmark.

Changes

• duckdb-bench and datafusion-bench: --scan-v2 is a plain bool that defaults to false. The binaries no longer default it from the benchmark name (TPC-H, TPC-DS, ClickBench) or read VORTEX_SCAN_V2=0. duckdb-bench always sets the scan version from the flag, so VORTEX_SCAN_V2 no longer affects it either.

• bench-orchestrator: BENCHMARK_SCAN_V2 in config.py maps every benchmark to its scan version:

  Benchmark	Vortex scan
  tpch, tpcds, clickbench	V2
  appian, clickbench-sorted, fineweb, gh-archive, job, polarsignals, public-bi, statpopgen, spatialbench, vortex	V1

  For DataFusion and DuckDB, BenchmarkExecutor.build_command passes --scan-v2 true or --scan-v2 false from this map. Lance commands don't get the flag. vx-bench run prints the scan version (Vortex scan: V2) in its run header.

• Tests: test_every_benchmark_has_a_scan_version fails if a new benchmark is added without a scan version. test_build_command_selects_scan_version checks the flag for every benchmark on both engines. The Lance test now also checks that the flag is omitted.

CI goes through vx-bench run, so the scan versions CI uses don't change. A binary run directly now uses V1 unless you pass --scan-v2 true.

[codspeed]

Merging this PR will degrade performance by 17.95%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 1 improved benchmark
❌ 4 regressed benchmarks
✅ 2313 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Walltime	filter_neon[64]	1.9 µs	2.8 µs	-34.34%
❌	Walltime	threshold_neon[64]	2.6 µs	3.4 µs	-24.84%
❌	Walltime	filter_neon[80]	2.5 µs	3.2 µs	-20.88%
❌	Walltime	threshold_neon[80]	3.4 µs	4.1 µs	-15.45%
⚡	Walltime	lt_i64_nullable_avx512	8.3 µs	7.4 µs	+12.66%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ji/vibrant-clarke-5b971d (af654ec) with develop (37d9e8e)}