cranelift: fold ctz/clz comparisons against zero into direct LSB / sign-bit tests

[ggreif]

Four mid-end ISLE rules in `opts/icmp.isle` for the boolean-context cases — when `ctz`/`clz` flows into a comparison against zero (the consumer cares only "is it zero?", not the numeric value):

```
ctz(X) == 0 iff (X & 1) != 0 ; LSB of X set
ctz(X) != 0 iff (X & 1) == 0 ; LSB of X clear
clz(X) == 0 iff X <signed 0 ; MSB of X set (X is negative)
clz(X) != 0 iff X >=signed 0 ; MSB of X clear (X is non-negative)
```

The bit-counting instruction is DCE'd; backend emits a single-cycle `test reg, imm` (LSB case) or `test reg, reg; js` (sign case) instead of `TZCNT/BSF/LZCNT/BSR` + `TEST` + `JCC` — saves ~3 cycles per occurrence on Intel x86_64 (TZCNT/LZCNT are 3-cycle latency with a false GPR dep), proportionally more on JIT-less backends.

Why this matters in practice

The poster-child workload is the Motoko runtime's discriminator test on every `Nat`/`Int` operation:

• Compact (scalar) integers: low bit clear — fast path is plain Wasm i32/i64 arithmetic.
• Heap-allocated big integers (via libtommath): low bit set (skew tag).

Every arithmetic op begins with this LSB test. The Motoko codegen (`src/codegen/instrList.ml:97-100`) already emits the LSB-test-of-AND-1 pattern as `(ctz X)` — unconditionally, no flag gate — so every moc-compiled wasm running on wasmtime today does TZCNT + TEST + JCC on the hot path of every numeric op. The Rust RTS / GC paths that work on the same tagged pointer scheme see the same pattern.

With these rules in place, cranelift collapses the comparison back to a single `test r, 1` — restoring the original cost of the discriminator and unlocking measurable speed-ups for every Motoko canister on a wasmtime-based IC subnet (and any other wasm that produces this shape).

The clz / sign-bit half exists for the same reason on the rare paths that test sign before dispatching; structurally parallel rewrite, ships in the same patch.

The converse fold on the wasm-byte-savings side is in WebAssembly/binaryen#8562 (LSB→ctz under `-Os`); landing it there together with this in cranelift gives byte savings without cycle cost.

Filetest covers i32/i64 ctz and clz in both eq and ne forms plus a negative case (`ctz(X) == 4` must not trigger — that's a numeric-value test on the count, a different rewrite family).

[ggreif]

Sketched an extension to also catch the wasm-emitted shape brif (ireduce.i32 (ctz.i64 X)) (and clz), which is what frontends like Motoko's moc produce — wasm's if takes an i32 condition, so the i64 LSB test always flows through ireduce and brif directly, with no icmp interposed for the egraph rules in this PR to match.

Scope creep: the natural place is each backend's is_nonzero helper (x64 inst.isle:3806-3826, aarch64 inst.isle:4659-4670, plus riscv64 and s390x), where rules like

(rule (is_nonzero (ctz val @ (value_type (ty_32_or_64 ty))))
  (CondResult.CC (x64_test ty val (RegMemImm.Imm 1)) (CC.Z)))

(rule (is_nonzero (clz val @ (value_type (ty_32_or_64 ty))))
  (CondResult.CC (x64_test ty val val) (CC.NS)))

would lower brif (ctz X) to test X, 1; jz and brif (clz X) to test X, X; jns directly, plus an (ireduce _ (ctz/clz _)) variant for the wasm path.

That's 4× backend files + filetests, different reviewers per arch, and a different review audience from this egraph PR. Punting on the amendment and filing a separate follow-up instead.

[ggreif]

Concrete real-world workload for the clz boolean-context fold: Motoko's classical-persistence backend already emits the bare i32.clz; if shape for Int <-> 0 sign tests (e.g. Prim.abs). The compiler-side peephole (shrU 31; if -> clz; if(swap) and and 0x80000000; if -> clz; if(swap) in motoko/src/codegen/instrList.ml) generates this directly because their target is i32 wasm without a wrap.

So the JIT-side fold here is the natural meeting point for classical-Motoko output: clz directly into brif, no icmp, no ireduce. The rules in this PR don't yet catch that shape (it's brif (clz X), not icmp eq (clz X) 0), but the equivalent backend lowering (per the previous comment) would close that gap end-to-end.

[github-actions]

Subscribe to Label Action

cc @cfallin, @fitzgen

Details

This issue or pull request has been labeled: "cranelift", "isle"

Thus the following users have been cc'd because of the following labels:

• cfallin: isle
• fitzgen: isle

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[alexcrichton]

Thanks for this! Are the backend rules necessary with the mid-end egraph rules? I'd expect that the egraph rewrites would be sufficient and the backends largely wouldn't need to change, unless they need to emit a new instruction pattern which isn't currently recognized.

One thing you may also want to do is to add something in tests/disas/*.wat. That's a wasm-level test which ensures that the right assembly will be generated which is a bit of an end-to-end test. Not required, but if you're curious that'd be a good way to validate that this is all kicking in correctly.

[ggreif]

Added tests/disas/ctz-clz-bool-condition.wat (commit 0519796) per your suggestion — covers if/select/eqz consumers over (ctz/clz X) eq/ne 0 for i32 and i64, plus a numeric-comparison negative test ((ctz X) == 4).

Empirical answer to your first question — the egraph rules in this PR are complete for the icmp-mediated case but don't catch the wasm-natural bare form:

icmp-mediated (this PR's target — collapses correctly):

;; if_ctz_eq0_i32:        testl $1, %edx; jne
;; select_ctz_eq0_i32:    testl $1, %edx; cmovne
;; eqz_ctz_eq0_i32:       testl $1, %edx; sete
;; if_clz_eq0_i32:        testl %edx, %edx; jl

bare if (ctz X) (no icmp — what wasm frontends like Motoko emit directly):

;; if_ctz_bare_i32:       bsfl %edx, %r9d; cmovel; testl %r9d, %r9d; jne
;; if_clz_bare_i32:       bsrl + cmovel + 0x1f-sub + test  ;; 5 insns

negative test (correctly not collapsed — confirms the rules don't over-fire on numeric comparisons):

;; if_ctz_eq4_i32:        bsfl %edx, %r9d; cmpl $4, %r9d; je

So the egraph rules cover their intended shape end-to-end. The bare form would need backend is_nonzero specializations as a follow-up — happy to land this PR with just the egraph rules and tackle the bare form separately, if that scoping makes sense to you.

[alexcrichton]

Ah yes good point, and yeah sounds good to me. Thanks for your work here! Happy to review backend-specific changes as well