Skip to content

Preserve declared T-SQL cast types during frontend compilation - #1073

Merged
carli2 merged 4 commits into
masterfrom
feat/tsql-expression-types
Oct 11, 2026
Merged

carli2 merged 4 commits into
masterfrom
feat/tsql-expression-types

Conversation

@carli2

@carli2 carli2 commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

CAST and CONVERT currently lose their declared SQL target type: integer casts are described as BIGINT, and extra arithmetic/boolean operations are used to communicate conversion types. Preserve the frontend's declared (expr, type, collation) contract through the existing shared expression compiler, which emits only the conversion formula.

  • Remove the outer integer-division, floating and double-negation type markers. Nested casts, NULL, CASE and ISNULL retain their declared contracts.
  • Describe TINYINT/SMALLINT/INT/BIGINT with the protocol's existing 1/2/4/8-byte widths while retaining native int64 values internally.
  • Compile window arguments before the shared planner creates aggregate identities; carry known window result contracts separately. Window, grouping, joins, subqueries and DML continue through the existing planner and storage operations.
  • Consume frontend annotations in compound logical inputs and ordinary INSERT VALUES, preserving quoted data. No runtime type annotation is emitted.

Based on master 465364107, which includes #1069. The PR changes two existing frontend modules, two existing functional test suites and one benchmark scheduling weight; other dialect parsers, scm/ and storage/ receive no changes.

Validation: fresh interpreter and JIT builds; focused suites only, with 107/107 native frontend cases and 22/22 shared type-fusion cases passing in both modes. Exact-numeric specifications retain all 50 critical gates: 54/128 cases pass overall and 74 existing noncritical specifications remain unsupported. No critical flags or timing limits were weakened. Scheme formatter checks and SQL regression guard pass. Full CI is green on 31ffcfe: all 35 executed checks succeeded, with the unrelated PHP job skipped. Local validation used focused suites only.

The annotation lowering now retains unchanged expression graphs and rebuilds only paths containing frontend contracts. It does not clone annotation-free sources/stages. New regressions cover unchanged input, NULL replacement and quoted syntax.

Manual A/B for the reported arithmetic suite used seven fixed fresh fixture pairs, the unchanged workload/seed, warmup: 0, four samples and total aggregation. Before this adjustment: 34.379 -> 37.853 ms (+10.1%). After the adjustment against current master: 37.128 -> 38.853 ms (+4.6%); all six suite cases remain below the 20% gate. The original CI observation was 53.005 -> 72.575 ms (+36.9%) and was not reproduced locally. The other failed jobs reported an occupied port and a JIT GC error (found pointer to free object). A focused seven-pair JIT run reproduced that GC error on the candidate in pair seven; The corresponding seven master runs and fourteen additional unchanged-master runs passed. Four fixed additional master-only runs with GOGC=20 reproduced the same fatal GC error in run four, in the same ordered permission-membership query, on unchanged master 4653641 using the identical JIT binary. This establishes a separate existing JIT/GC failure; its specific root cause is not yet diagnosed. No core changes or threshold relaxations are included.

The subsequent JIT shard 1 measurement timed out at the unchanged 15-minute deadline while verifying two initially suspect suites. The ordered-prefix verification passed; range-aggregate verification was interrupted. Its retained process timing artifacts project all seven pairs to 268.473 seconds. Per the existing tests/README.md scheduling policy, update only that suite's performance_shard_weight from 2 to ceil(268.473 / 15) = 18. This separates it from the ordered-prefix suite, preserves assignment of every discovered suite exactly once and changes no query, setup, expectations, sample count, 20% gate or CI timeout. Two existing focused scheduling contract tests and the regression guard pass.

Independent seven-pair JIT A/B verification of range-first-aggregate used the unchanged fixture/seed, zero warmup and eleven samples with total aggregation (one cold query plus ten reuses). The initially suspect dependent-lookup workload measured 73.106 -> 71.668 ms (-2.0%); the largest regression across all six cases was +1.4%, with every assertion and performance gate passing. No failed trials were discarded. The previous head's full interpreter/JIT suites, both Go test modes, cross-JIT checks, upgrade, packaging, storage alternatives and RPM all passed; full CI also passed for the scheduling update on 31ffcfe.

The scheduling-update run then reported a different interpreter CASE-aggregate regression: 25.327 -> 45.239 ms (+78.6%) after seven fixtures. Retained timings are strongly bimodal on both sides (roughly 24/49 ms). Focused seven-pair verification with the unchanged snapshot/workload passed all eight performance cases; that cold CASE case measured 17.064 -> 19.480 ms (+14.2%), zero warmup and one sample as specified. IR, reordered plan, physical plan and executable Scheme code are byte-for-byte identical between master and this branch for the exact query. One unchanged CI rerun passed, with that CASE workload at 20.201 -> 21.288 ms (+5.4%). No thresholds, assertions or sample configuration were altered; the original failed trial records are retained.

Remaining follow-ups, specified by the inherited compatibility suites:

  • Extend compile-time type/collation propagation through derived relations, UNION branches, aggregates and window stage outputs; resolve catalog-dependent window contracts after binding. Unknown result contracts still use the existing text fallback. Literal types must come from SQL spelling, and parameter types from declared contracts, independently of runtime carriers.
  • Add text, binary, temporal and numeric conversion targets with existing native representations. Implement assignment coercions and implicit conversion rules in the frontend; retain explicit unsupported-shape errors.
  • Extend DDL, defaults, constraints and identity syntax through existing schema options/operators. Implement additional batch/session commands and migration script constructs documented in the compatibility fixtures.
  • Complete typed parameter execution, result description, catalog procedures and remaining protocol callbacks using the released protocol dependency.
  • Generic multi-precision integers and decimal coefficients with compile-time scale correction remain a separate, deferred design. Finite dialect precision/rounding boundary specifications stay noncritical; this PR adds no numeric runtime tags or storage checks.

This is an incremental frontend feature, not a claim that all compatibility specifications or complete installation workloads already pass.

Focused master reproducer for the independent JIT/GC failure: use a JIT-enabled build and run GOGC=20 python3 run_sql_tests.py tests/performance/tenant-document-membership.yaml --fail-fast repeatedly with fresh data stores. All four stress outcomes were retained (three passes, one fatal failure); no failing fixture was discarded. The master log is /tmp/pr1073-master-jit-membership-gc20-four/tenant-document-membership-04-A.memcp.log.

@carli2
carli2 force-pushed the feat/tsql-expression-types branch from ba6252b to 891db22 Compare October 11, 2026 00:07
@carli2
carli2 marked this pull request as ready for review October 11, 2026 01:07
@carli2
carli2 merged commit b5086cf into master Oct 11, 2026
42 of 44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant