Add MLX backend for Apple Silicon neural network inference - #16

Closed

ChinChangYang wants to merge 130 commits into

feature/mlx-backend

Commits on Jan 15, 2026

Add MLX backend for Apple Silicon

ChinChangYang
and
claude
committed
def265b
Optimize MLX backend with logaddexp and addmm

ChinChangYang
and
claude
committed
3540214
Skip mask ops in MLX backend when requireExactNNLen=true

ChinChangYang
and
claude
committed
2c9f18b

Commits on Jan 16, 2026

Add graph compilation to MLX backend for improved throughput

ChinChangYang
and
claude
committed
50cfcbc
Add FP16 support to MLX backend

ChinChangYang
and
claude
committed
611476b

Commits on Apr 22, 2026

Harden MLX backend CMake configuration

ChinChangYang
and
claude
committed
b8f227b

Commits on May 19, 2026

Commits on May 20, 2026

Add SP2 design spec — MLX Winograd autotuner

ChinChangYang
and
claude
committed
3b4a472
Strengthen SP2 acceptance test #1

ChinChangYang
and
claude
committed
c131116
Add SP2 implementation plan

ChinChangYang
and
claude
committed
e67e438
SP2 Task 1: split WinogradConfig into per-stage InputTransform/OutputUntransform

ChinChangYang
and
claude
committed
6bbe07c
SP2 Task 2: MLXWinogradTuneParams struct + plain-text persistence

ChinChangYang
and
claude
committed
e0e1be8
SP2 Task 2 review fixes: improve parser diagnostics + add boundary test

ChinChangYang
and
claude
committed
5cc1145
SP2 Task 3: measurement primitive for tuner candidates

ChinChangYang
and
claude
committed
16aab33
SP2 Task 3 review fixes: warmup discipline + channel-alias bug + linkage cleanup

ChinChangYang
and
claude
committed
2a2fd3c
SP2 Task 4: grid search + loadOrAutoTune

ChinChangYang
and
claude
committed
3ca7907
SP2 spec amendment: search-works assertion (a) calibrated to hardware reality

ChinChangYang
and
claude
committed
d226391
SP2 Task 4: grid search + loadOrAutoTune (search-works test calibrated)

ChinChangYang
and
claude
committed
ef51a95
SP2 Task 4 review fixes: real seed override + comment accuracy + dedup + version derive

ChinChangYang
and
claude
committed
8c49c14
SP2 Task 5: wire tuner into ComputeHandle/Model/ConvLayer

ChinChangYang
and
claude
committed
4712493
SP2 Task 5 review fix: makeCacheKey includes actual tg0/tg1 quadruple

ChinChangYang
and
claude
committed
8d7d65b
SP2 acceptance: MLX-fp32 Winograd with tuner-cached configs >= Metal

ChinChangYang
and
claude
committed
6097808
Add SP3 design spec — MLX Winograd fp16 with selective fp32 accumulation

ChinChangYang
and
claude
committed
bb729a5
SP3 spec amendment: BN fp32 intermediate REQUIRED + paired-t gate

ChinChangYang
and
claude
committed
f011943
Add SP3 implementation plan

ChinChangYang
and
claude
committed
87a0853
SP3 Task 1: templatize Winograd kernels on T (fp16/fp32 dispatch)

ChinChangYang
and
claude
committed
09f06b4
SP3 Task 2: BatchNormLayer fp32 intermediate (required for accuracy)

ChinChangYang
and
claude
committed
678ee7d
SP3 Task 3: ConvLayer drops !useFP16 gate, threads dtype to Winograd

ChinChangYang
and
claude
committed
e7ca5a9
SP3 Task 4: tuner cache filename gains _fp16/_fp32 dtype suffix

ChinChangYang
and
claude
committed
6d52ed8
SP3 Task 5: ComputeHandle wires useFP16 into tuner; comments updated

ChinChangYang
and
claude
committed
71ebdac
SP3 Task 6: bench harness — env-var fp16 hooks + paired-t output

ChinChangYang
and
claude
committed
5c29b58
SP3 Task 7: acceptance orchestrator (two paired-t arms + testgpuerror)

ChinChangYang
and
claude
committed
0f4fd12
SP3 Auto -> fp16 flip: mlxUseFP16 = auto now means fp16

ChinChangYang
and
claude
committed
1ca9ac9
SP3 acceptance: MLX-fp16 Winograd >= Metal-fp16, > MLX-fp32 (SP2)

ChinChangYang
and
claude
committed
727366c
SP3 post-acceptance hardening: Model::apply() fp16 guard + fp16 search-works test

ChinChangYang
and
claude
committed
36a8818
Add SP4 design spec — aggressive F(2,3) autotuner

ChinChangYang
and
claude
committed
ff41df7
Add SP4 implementation plan

ChinChangYang
and
claude
committed
bbf2974
SP4 Task 1: bump tuner schema to v2 — add wpt/vw/gridOrder/matmulOrient fields

ChinChangYang
and
claude
committed
8f29342
SP4 Task 2: thread WPT/VW/GRID_ORDER/MATMUL_ORIENT template args through winogradConv2d

ChinChangYang
and
claude
committed
81ad9a1
SP4 Task 3: WPT loop in both transform kernels with tail guards

ChinChangYang
and
claude
committed
891a227
SP4 Task 4: VW (vector-width) packing in both transform kernels — Cfast only

ChinChangYang
and
claude
committed
ee6121d
SP4 Task 5: gridOrder Tfast branch in both transform kernels

ChinChangYang
and
claude
committed
4e9794f
SP4 Task 6: matmulOrient Tpd path in kernels + host filter

ChinChangYang
and
claude
committed
95c0157
SP4 Task 7: expanded candidate sets + per-model validity filtering

ChinChangYang
and
claude
committed
dc3ceb0
SP4 Task 8: Joint pass A — (wpt, vw) sweep with top-3 carry-through

ChinChangYang
and
claude
committed
a8ef3b7
SP4 Task 9: Joint pass B — (tg0, tg1) sweep over top-(wpt, vw) configs

ChinChangYang
and
claude
committed
a10030e
SP4 Task 10: refinement — coordinate-descent around the joint-pass winner

ChinChangYang
and
claude
committed
fc46d68
SP4 Task 11: hierarchical driver — orient → gridOrder → joint A → joint B → refine → end-to-end

ChinChangYang
and
claude
committed
f01b6d7
SP4 Task 12: wire matmulOrient + new tuner fields through ConvLayer / Model

ChinChangYang
and
claude
committed
8a8959d
SP4 Task 13: bad-seed convergence test — refinement moves ≥30% on at least one stage

ChinChangYang
and
claude
committed
894da25
SP4 Task 14: cache version-mismatch retune test

ChinChangYang
and
claude
committed
0f12a57
SP4 Task 15: acceptance orchestrator — SP3 vs SP4 paired-t + testgpuerror + wall-time

ChinChangYang
and
claude
committed
6763df0
SP4 acceptance: aggressive F(2,3) autotuner gates — all 5 PASS

ChinChangYang
and
claude
committed
e1714f1
SP4 cleanup: dead struct removal + stale comments

ChinChangYang
and
claude
committed
ee311c2
Add SP5 — Winograd tuner simplification spec

ChinChangYang
and
claude
committed
cc6ba06
Add SP5 — Winograd tuner simplification implementation plan

ChinChangYang
and
claude
committed
ec0659a
SP5 Task 1: Add flat per-stage sweep functions (not yet wired)

ChinChangYang
and
claude
committed
2f5aeae

Commits on May 21, 2026

SP5 Task 2: Switch loadOrAutoTune to flat sweep; delete Joint-A/B/refine

ChinChangYang
and
claude
committed
7c50049
SP5 Task 3: Drop output VW (kernel + data model + tests)

ChinChangYang
and
claude
committed
a00be53
SP5 Task 4: Drop output gridOrder (kernel + data model + tests)

ChinChangYang
and
claude
committed
3bc39c2
SP5 Task 5: Drop matmulOrient (enum + kernel + weight layout + backend threading)

ChinChangYang
and
claude
committed
b16a81f
SP5 Task 6: Drop global gridOrder field from MLXWinogradTuneParams

ChinChangYang
and
claude
committed
3703817
SP5 Task 7: Bump cache schema to v3

ChinChangYang
and
claude
committed
927ac64
SP5 Task 8: v3 roundtrip + isValid invariant tests

ChinChangYang
and
claude
committed
a35fc42
SP5 Task 9: Output-kernel monomorphic smoke test

ChinChangYang
and
claude
committed
55d4146
SP5 Task 10: Gated flat-sweep convergence test

ChinChangYang
and
claude
committed
f6d4450
SP5 Task 11: Replace bench_sp4_acceptance.sh with SP5 version

ChinChangYang
and
claude
committed
4cf652d
SP5: Final doc/comment polish

ChinChangYang
and
claude
committed
51c9db6
chore: remove subsumed 4-field tuner roundtrip test

ChinChangYang
and
claude
committed
2f6daf5
Add MLX tests relocation design spec

ChinChangYang
and
claude
committed
52fef8c
Add MLX tests relocation implementation plan

ChinChangYang
and
claude
committed
12b42bd
Move runMLXWinotunerTests body to mlxwinotuner.cpp (not yet wired)

ChinChangYang
and
claude
committed
f6e8134
Move runMLXWinogradTests body to mlxbackend.cpp (not yet wired)

ChinChangYang
and
claude
committed
3102323
Relocate MLX aux tests: wire one-shot guard, revert testnn.cpp/tests.h to master

ChinChangYang
and
claude
committed
b152ab6
Task 3 polish: add <cstdio>, fix stale forward-decl comments

ChinChangYang
and
claude
committed
38c0be6
MLX backend: assert on ACTIVATION_MISH_SCALE8; document why Mish is FP16-safe

ChinChangYang
and
claude
committed
1199024
Spec: MLX tuner baked-default baseline anchor

ChinChangYang
and
claude
committed
cc9fdef
Plan: MLX tuner baked-default baseline anchor

ChinChangYang
and
claude
committed
3e19cd7
MLX tuner: baseline_ms / delta_pct in flatSweepInput log line

ChinChangYang
and
claude
committed
c7d4e4b
MLX tuner: baseline_ms / delta_pct in flatSweepOutput log line

ChinChangYang
and
claude
committed
efed6de
MLX tuner: gated baseline-consistency check

ChinChangYang
and
claude
committed
72364c4
MLX tuner: document %+.1f ↔ regex contract and Test 2 scope

ChinChangYang
and
claude
committed
429bd68
Add MLX tuner shape-diagnostic design spec

ChinChangYang
and
claude
committed
723d4cf
Add MLX tuner shape-diagnostic implementation plan

ChinChangYang
and
claude
committed
1baa4c7
Add per-slot median scoring primitives to MLX tuner

ChinChangYang
and
claude
committed
49e6dcc
Append per-slot median timings to MLX flat-sweep log lines

ChinChangYang
and
claude
committed
f80a9df
Add conv-3x3 shape distribution formatter for MLX tuner

ChinChangYang
and
claude
committed
10f01f9
Log MLX conv-3x3 shape distribution at model load

ChinChangYang
and
claude
committed
ad7edde
Add per-slot numeric-consistency gated test for MLX tuner

ChinChangYang
and
claude
committed
67fe303
Add MLX tuner adaptive-scoring design spec

ChinChangYang
and
claude
committed
88d7897
Add MLX tuner adaptive-scoring implementation plan

ChinChangYang
and
claude
committed
2ab5bc2
Add planShapeRotation pure-function for adaptive scoring

ChinChangYang
and
claude
committed
3cb2df5
Extract buildConv3x3Histograms shared helper

ChinChangYang
and
claude
committed
1ef276f
Add histogram fields to ModelInfoForTuning (additive)

ChinChangYang
and
claude
committed
cbf329e
Rewrite MLX tuner scoring to adaptive per-shape rotation

ChinChangYang
and
claude
committed
545d0f6
Remove deprecated mid/maxConvChannels3x3 from ModelInfoForTuning

ChinChangYang
and
claude
committed
f2f93ab
Clean up residual per-slot references after PER_SHAPE rename

ChinChangYang
and
claude
committed
c656f84
Spec: MLX backend comment cleanup

ChinChangYang
and
claude
committed
208dd45
Spec: expand Commit 2 scope to all SP/Task sites

ChinChangYang
and
claude
committed
022acaf
Plan: MLX backend comment cleanup

ChinChangYang
and
claude
committed
9c7083c
Fix stale comments and close rounding-repair test gap

ChinChangYang
and
claude
committed
5a9433b
De-tag historical SP/Task references in MLX comments

ChinChangYang
and
claude
committed
a16479e
Rewrite MLX winograd block comments in present tense

ChinChangYang
and
claude
committed
9e0a8c7
Remove MLX design specs, plans, and SP-named bench scripts

ChinChangYang
and
claude
committed
2a229c2
Eliminate -Wshadow warnings in MLX Winograd test code

ChinChangYang
and
claude
committed
feb5304
Document batchSize=8 tuner pinning rationale

ChinChangYang
and
claude
committed
89f6b7f

Commits on May 22, 2026

Strip internal codenames and dangling spec references from MLX files

ChinChangYang
and
claude
committed
4f0ed4b
Move runMLXWinogradTests into dedicated mlxtests.cpp

ChinChangYang
and
claude
committed
39e94b0
Move runMLXWinotunerTests into mlxtests.cpp

ChinChangYang
and
claude
committed
845f0d3
Strip sp5/task10 codenames from mlxtests temp-file paths

ChinChangYang
and
claude
committed
fb302fc
List MLX and METAL in the dummy-backend warning
ChinChangYang
committed
a854ce6
Hoist MLX minimum version into MLX_MIN_VERSION variable
ChinChangYang
committed
0717fcc
Tighten MLX platform/arch validation error messages
ChinChangYang
committed
446910d
Remove unverified MLX python-fallback discovery path

ChinChangYang
and
claude
committed
89eedbc
Clean up MLX backend CMake version and deployment-target config

ChinChangYang
and
claude
committed
890b857
Document mlxUseFP16 in analysis/match/contribute configs

ChinChangYang
and
claude
committed
d7b0cd0
Drop host-specific FP16 throughput numbers from MLX config comments

ChinChangYang
and
claude
committed
874bce6
Fix MLX batched pass-policy stride for human-SL nets

ChinChangYang
and
claude
committed
db6c01b

Commits on May 23, 2026

Remove bench_mlx_honest.sh paired-benchmark helper

ChinChangYang
and
claude
committed
c9bf1a8