Repository navigation
Conversation
psiddh
requested review from
SS-JIA,
digantdesai and
larryliu0820
as code owners
November 12, 2026 01:30
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24971
Note: Links to docs will display an error until the docs builds have been completed. ⏳ 3 Pending, 2 Unrelated FailuresAs of commit 611136d with merge base 765e973 ( BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
pytorch-bot
Bot
added
the
module: vulkan
meta-cla
Bot
added
the
CLA Signed
Contributor
|
@psiddh has exported this pull request. If you are a Meta employee, you can view the originating Diff in D131277775. |
This PR needs a
|
psiddh
added a commit
to psiddh/executorch
that referenced
this pull request
Nov 12, 2026
…#24971) Summary: Pull Request resolved: pytorch#24971 The quantized linear pattern detector matched every linear with a dequantized constant weight, while replace_quantized_linear_patterns only has custom ops for 8-bit statically quantized activations and for 4-bit group-quantized weights (weight-only or with dynamically quantized activations). The partitioner delegates every node of a match, so an 8-bit weight-only linear (torchao IntxWeightOnlyConfig int8) was delegated with its torchao.dequantize_affine node left in place. The Vulkan runtime has no dequantize_affine kernel (the quantize/dequantize op implementations were removed in pytorch#15750), so the program failed to initialize on device. The registry entries for torchao.quantize_affine and torchao.dequantize_affine still claimed both ops as supported, which also delegated them outside of patterns. The detector now reports a match only when a custom op exists for it and the runtime can run it. The weight-only 4-bit kernel packs the weights in blocks of 4 output channels (Q4gswLinear.cpp checks N % 4 == 0 at prepack), so weight-only 4-bit linears with other output sizes are no longer matched; previously they exported and then failed at init. The match now also records the activation's quantize node in the dynamically quantized pattern, which the partitioner previously picked up only through the stale quantize_affine registry entry, and the two torchao registry entries are removed. Unmatched linears now stay outside of the delegate. With torchao quantization this surfaces at export as "Missing out variants: {'torchao::dequantize_affine'}" instead of a failure on device. WebGPU uses the same patterns, so the same applies there. Not changed: the quantized_decomposed quantize_per_channel / quantize_per_token / dequantize_per_channel / dequantize_per_token and choose_qparams (tensor, per_token_asymmetric) registry entries also have no runtime kernel since pytorch#15750. I left them because the VulkanQuantizer dynamic flow may still depend on them being claimed; they are worth a look by the backend owners. This diff was written with the help of Claude Code (Claude Opus 5.5). Differential Revision: D131277775
psiddh
force-pushed
the
export-D131277775
branch
from
November 12, 2026 02:06
6bcc3a4 to
4227dc1
Compare
meta-codesync
Bot
changed the title
SS-JIA
approved these changes
Nov 12, 2026
SS-JIA
left a comment
SS-JIA
left a comment
Contributor
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
…ch#24810) Summary: Pull Request resolved: pytorch#24810 Adds `examples/models/phonon`: Gradium's Apache-2.0 Phonon 31M English text-to-speech model exported to one `.pte`, with a backend-agnostic C++ runner (`phonon_runner`) and a Python runner (`run_phonon.py`). It follows `nemotron3_diarization`: one export script, CMake presets with `make phonon-{cpu,cuda,mlx,vulkan}`, CI jobs in `pull.yml`, `mlx.yml` and `cuda.yml`, and a short README in the same shape. CUDA writes `aoti_cuda_blob.ptd` beside the `.pte`. Validated against the official runtime on XNNPACK, Vulkan, WebGPU, CUDA and MLX. Exports default to FP32 on every backend; `--dtype bf16` is available on CUDA and MLX. WebGPU runs in the browser: the README gives the `--backend webgpu --webgpu-compat-ops` export and links the Text to Speech demo of the ExecuTorch WebGPU Playground. Also fixes an XNNPACK delegate use-after-free that Phonon hits: `XNNCompiler::compileModel` frees the named-data buffer of every packed filter/bias right after `xnn_create_runtime_v4`, even when another node still reads that constant. Phonon's `StreamingConvTranspose1d` feeds the transposed-conv bias to the conv and to the held-back tail subtraction, so a program loaded from a file (C++ runner, pybindings) segfaulted in the first `mimi_decode` call. A packed value that another node also reads now stays alive; an unknown node type keeps every constant alive. Same class of bug as the PReLU fix (pytorch#21480). Differential Revision: D128116963
…#24929) Summary: Pull Request resolved: pytorch#24929 Follow-up to D128116963. The C++ runner gets `--stream`, which writes 16-bit mono PCM to stdout as each 80 ms frame is decoded instead of writing the WAV at the end, so playback can start after the first frame (`phonon_runner ... --stream | aplay -q -f S16_LE -r 24000 -c 1`). The runner already decoded frame by frame through a callback; this only routes the samples to stdout and moves the summary line to stderr. `--qlinear` now accepts only the modes that each backend has a kernel for (XNNPACK 8da4w/8da8w, CUDA 4w/8w, Vulkan and WebGPU 4w/8da4w, MLX 4w/8w) and rejects the rest at export instead of failing at runtime. Quantization covers the flow LM's transformer and flow net; the input projection, the one-output EOS head and the Mimi codec stay in the export dtype. The EOS head is skipped because the Vulkan and WebGPU 4-bit linear does not yet handle an output size that is not a multiple of 4. CUDA 4w uses the packed int4 kernel, which needs `--dtype bf16` and packs on the GPU, so two `torch.arange` calls in the attention take the input's device. MLX 4w uses asymmetric int4: symmetric int4 keeps the words but delays the end-of-speech decision, adding up to two seconds of silence at the end of a sentence, while asymmetric int4 keeps the timing of FP32. The README documents both features with the sizes per backend. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: mergennachin Differential Revision: D131277773
… a multiple of 4 (pytorch#24930) Summary: Pull Request resolved: pytorch#24930 make_linear_q4gsw_op and make_linear_dq8ca_q4gsw_op pad the weight scales and the bias of a 4-bit linear to a multiple of 4 output channels, but they do not pad the packed weights. The CompositeExplicitAutograd implementation of et_vk.linear_q4gsw (also used by et_vk.linear_dq8ca_q4gsw) dequantized the unpadded weights with the padded scales, so any lowering that re-traced the graph failed for such a linear, for example the 512 -> 1 end-of-speech head of a TTS model: RuntimeError: shape '[1, 16, 1]' is invalid for input of size 64 The implementation now drops the padded scale and bias rows. This affects the export-time reference implementation only; the dynamically quantized 8-bit activation, 4-bit weight kernel already handles any number of output channels on device. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: SS-JIA Differential Revision: D131277774
…#24971) Summary: Pull Request resolved: pytorch#24971 The quantized linear pattern detector matched every linear with a dequantized constant weight, while replace_quantized_linear_patterns only has custom ops for 8-bit statically quantized activations and for 4-bit group-quantized weights (weight-only or with dynamically quantized activations). The partitioner delegates every node of a match, so an 8-bit weight-only linear (torchao IntxWeightOnlyConfig int8) was delegated with its torchao.dequantize_affine node left in place. The Vulkan runtime has no dequantize_affine kernel (the quantize/dequantize op implementations were removed in pytorch#15750), so the program failed to initialize on device. The registry entries for torchao.quantize_affine and torchao.dequantize_affine still claimed both ops as supported, which also delegated them outside of patterns. The detector now reports a match only when a custom op exists for it and the runtime can run it. The weight-only 4-bit kernel packs the weights in blocks of 4 output channels (Q4gswLinear.cpp checks N % 4 == 0 at prepack), so weight-only 4-bit linears with other output sizes are no longer matched; previously they exported and then failed at init. The match now also records the activation's quantize node in the dynamically quantized pattern, which the partitioner previously picked up only through the stale quantize_affine registry entry, and the two torchao registry entries are removed. Unmatched linears now stay outside of the delegate. With torchao quantization this surfaces at export as "Missing out variants: {'torchao::dequantize_affine'}" instead of a failure on device. WebGPU uses the same patterns, so the same applies there. Not changed: the quantized_decomposed quantize_per_channel / quantize_per_token / dequantize_per_channel / dequantize_per_token and choose_qparams (tensor, per_token_asymmetric) registry entries also have no runtime kernel since pytorch#15750. I left them because the VulkanQuantizer dynamic flow may still depend on them being claimed; they are worth a look by the backend owners. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: SS-JIA Differential Revision: D131277775
psiddh
force-pushed
the
export-D131277775
branch
from
November 13, 2026 08:49
4227dc1 to
611136d
Compare
psiddh
added a commit
to psiddh/executorch
that referenced
this pull request
Nov 13, 2026
…#24971) Summary: Pull Request resolved: pytorch#24971 The quantized linear pattern detector matched every linear with a dequantized constant weight, while replace_quantized_linear_patterns only has custom ops for 8-bit statically quantized activations and for 4-bit group-quantized weights (weight-only or with dynamically quantized activations). The partitioner delegates every node of a match, so an 8-bit weight-only linear (torchao IntxWeightOnlyConfig int8) was delegated with its torchao.dequantize_affine node left in place. The Vulkan runtime has no dequantize_affine kernel (the quantize/dequantize op implementations were removed in pytorch#15750), so the program failed to initialize on device. The registry entries for torchao.quantize_affine and torchao.dequantize_affine still claimed both ops as supported, which also delegated them outside of patterns. The detector now reports a match only when a custom op exists for it and the runtime can run it. The weight-only 4-bit kernel packs the weights in blocks of 4 output channels (Q4gswLinear.cpp checks N % 4 == 0 at prepack), so weight-only 4-bit linears with other output sizes are no longer matched; previously they exported and then failed at init. The match now also records the activation's quantize node in the dynamically quantized pattern, which the partitioner previously picked up only through the stale quantize_affine registry entry, and the two torchao registry entries are removed. Unmatched linears now stay outside of the delegate. With torchao quantization this surfaces at export as "Missing out variants: {'torchao::dequantize_affine'}" instead of a failure on device. WebGPU uses the same patterns, so the same applies there. Not changed: the quantized_decomposed quantize_per_channel / quantize_per_token / dequantize_per_channel / dequantize_per_token and choose_qparams (tensor, per_token_asymmetric) registry entries also have no runtime kernel since pytorch#15750. I left them because the VulkanQuantizer dynamic flow may still depend on them being claimed; they are worth a look by the backend owners. This diff was written with the help of Claude Code (Claude Opus 5.5). Reviewed By: SS-JIA Differential Revision: D131277775
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
The quantized linear pattern detector matched every linear with a dequantized constant weight, while replace_quantized_linear_patterns only has custom ops for 8-bit statically quantized activations and for 4-bit group-quantized weights (weight-only or with dynamically quantized activations). The partitioner delegates every node of a match, so an 8-bit weight-only linear (torchao IntxWeightOnlyConfig int8) was delegated with its torchao.dequantize_affine node left in place. The Vulkan runtime has no dequantize_affine kernel (the quantize/dequantize op implementations were removed in #15750), so the program failed to initialize on device. The registry entries for torchao.quantize_affine and torchao.dequantize_affine still claimed both ops as supported, which also delegated them outside of patterns.
The detector now reports a match only when a custom op exists for it and the runtime can run it. The weight-only 4-bit kernel packs the weights in blocks of 4 output channels (Q4gswLinear.cpp checks N % 4 == 0 at prepack), so weight-only 4-bit linears with other output sizes are no longer matched; previously they exported and then failed at init. The match now also records the activation's quantize node in the dynamically quantized pattern, which the partitioner previously picked up only through the stale quantize_affine registry entry, and the two torchao registry entries are removed.
Unmatched linears now stay outside of the delegate. With torchao quantization this surfaces at export as "Missing out variants: {'torchao::dequantize_affine'}" instead of a failure on device. WebGPU uses the same patterns, so the same applies there.
Not changed: the quantized_decomposed quantize_per_channel / quantize_per_token / dequantize_per_channel / dequantize_per_token and choose_qparams (tensor, per_token_asymmetric) registry entries also have no runtime kernel since #15750. I left them because the VulkanQuantizer dynamic flow may still depend on them being claimed; they are worth a look by the backend owners.
This diff was written with the help of Claude Code (Claude Opus 5.5).
Reviewed By: SS-JIA
Differential Revision: D131277775