Skip to content

perf(metal): opt-in DV_METAL_FP16 reduced-precision inference#4

Merged
BenjaminDEMAILLE merged 2 commits into
IPNP-BIPN:apple-silicon-native-v2-prfrom
nh13:nh_dv-3-fp16
Jun 24, 2026
Merged

perf(metal): opt-in DV_METAL_FP16 reduced-precision inference#4
BenjaminDEMAILLE merged 2 commits into
IPNP-BIPN:apple-silicon-native-v2-prfrom
nh13:nh_dv-3-fp16

Conversation

@nh13

@nh13 nh13 commented Jun 22, 2026

Copy link
Copy Markdown

Draft — depends on #2, and should not merge until a GIAB F1/concordance run confirms accuracy. Default behavior is unchanged (gated on DV_METAL_FP16=1).

Lets MPSGraph use FP16 fast-math (optimizationLevel1 + reducedPrecisionFastMath=Default) instead of the forced Level0/FP32 path.

Preliminary validation (chr20, HG002, 4731 examples)

  • Metadata and record order are identical to FP32.
  • fp16-vs-fp32 genotype-call agreement is 60/4731 (1.27%) — exactly the FP32 backend's own run-to-run nondeterminism baseline, i.e. reduced precision adds no divergence beyond what the default metal backend already exhibits.
  • Speedup is modest at this scale (the call_variants stage is I/O-bound reading the example shards, which masks the GPU backbone gain).

A true GIAB F1/concordance run is the merge gate before this default is ever flipped.

Not included: ANE

A real ANE path is not added — the 7-channel Inception-v3 input is rejected by the ANE, so MPSGraph stays on the GPU. The existing ane_speculate scaffold is untouched.

@nh13

nh13 commented Jun 22, 2026

Copy link
Copy Markdown
Author

Concordance gate cleared. chr20 / HG002 (50×) scored against GIAB v4.2.1 truth in the confident regions via RTG vcfeval: fp16 and fp32 are identical.

Precision Recall F1 TP FP FN
fp32 0.9993 0.9955 0.9974 81,755 55 368
fp16 0.9993 0.9955 0.9974 81,755 55 368

At the call level the two are not merely close but identical: 97,527 PASS calls each, 0 differing (locus + ref + alt + GT). The ~1.3% genotype jitter measured earlier lives entirely at borderline RefCall/NoCall sites that never surface as PASS variants; high-confidence calls are robust to fp16 precision.

Caveats: single chr20 sample on GRCh37, one fp32 draw (the default metal backend is run-to-run nondeterministic, so fp32-vs-fp32 also jitters at borderline sites), confident regions only. DV_METAL_FP16 remains opt-in / default-off, so merging this does not change default behavior.

Marking ready for review.

@BenjaminDEMAILLE

Copy link
Copy Markdown

Heads-up @nh13: I had to squash apple-silicon-native-v2-pr from 273 commits down to 1, because Google's CLA bot can't auto-scan PRs with more than 250 commits and this branch is the head of the upstream PR google#1085. The squash is content-identical (same tree hash 810ff855), only the history was flattened.

Side effect: this rewrote the base of your 4 open PRs (#3, #4, #6, #7), so they now show an inflated diff against r1.10. A rebase onto the new base fixes it cleanly (no real conflicts, the content is unchanged). The old base tip was 0cd41f859de445f5d8a60524b86b54bb418a043b, so:

git fetch origin
git rebase --onto origin/apple-silicon-native-v2-pr 0cd41f859de445f5d8a60524b86b54bb418a043b <your-branch>
git push --force-with-lease

Sorry for the churn, and thanks again for all the fixes. Your authorship is preserved via a Co-Authored-By trailer on the squashed commit.

nh13 added 2 commits June 24, 2026 11:16
Gate MPSGraph precision on DV_METAL_FP16: when set, use optimizationLevel1
+ reducedPrecisionFastMath=Default (FP16 Winograd / operand conversion)
instead of the forced Level0/FP32 path. Default (unset) is unchanged and
remains contractual full FP32.

Validated on chr20 (HG002, 4731 examples): metadata and record order are
identical to FP32, and fp16-vs-fp32 genotype-call agreement is 60/4731
(1.27%) — exactly the FP32 backend's own run-to-run nondeterminism
baseline, i.e. reduced precision adds no divergence beyond what the
default metal backend already exhibits. A true ANE path is not added: the
7-channel Inception-v3 input is rejected by the ANE, so MPSGraph stays on
the GPU. Any default flip should still be gated behind GIAB concordance.
Reword the nonstandard 'FP19/TF32' to 'TF32 (19-bit)' — TF32 is the 19-bit
operand format; it is not FP16 (already named as the Winograd path).
@BenjaminDEMAILLE BenjaminDEMAILLE merged commit 46b5281 into IPNP-BIPN:apple-silicon-native-v2-pr Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants