Skip to content

Add shared elementwise LUT fusion - #24985

Open
usamahz wants to merge 1 commit into
pytorch:mainfrom
usamahz:codex/mletorch-2880-elementwise-lut
Open

usamahz wants to merge 1 commit into
pytorch:mainfrom
usamahz:codex/mletorch-2880-elementwise-lut

Conversation

@usamahz

@usamahz usamahz commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Pointwise quantized chains currently leave separate operations for backends to lower, limiting LUT fusion and duplicating graph search. Add a shared pass that finds single-input, single-output pointwise regions and evaluates each region as one lookup table. Arm uses the pass for int8 and int16 tables, while Cortex-M configures its backend-specific lowering through a subclass. The pass fuses unary operators and add, subtract, multiply, and divide with scalar constants.

Table evaluation keeps FP32 intermediates and quantizes only the final entries using backend rounding. Region boundaries preserve standalone requantization, avoid fusing pure Q/DQ, and retain valid suffixes after multi-input operations.

For review, start with backends/transforms/fuse_elementwise_to_lut.py, then the Arm and Cortex-M pass integrations and their tests.

Jira: MLETORCH-2880

Authored with AI assistance using OpenAI Codex.

Test plan

  • 116 local tests passed using these focused commands (29 Arm and 87 shared/Cortex-M). CMSIS-NN was used for quantization and SDPA coverage; five FVP-only cases were deselected.
pytest -q backends/arm/test/passes/test_arm_fuse_elementwise_to_lut.py backends/arm/test/passes/test_insert_table_ops_lowering.py backends/arm/test/passes/test_insert_table_ops_pass.py
pytest -q backends/transforms/test/test_fuse_elementwise_to_lut.py backends/cortex_m/test/test_cortex_m_fuse_elementwise_to_lut.py backends/cortex_m/test/test_activation_lut.py
pytest -q backends/cortex_m/test/misc/test_quantization.py
pytest -q backends/cortex_m/test/ops/test_sdpa.py -k 'not test_implementation_sdpa'
  • Four representative Arm models passed lowering checks.
  • Local pre-commit and pre-push checks passed.
  • Arm extended CI passed for patch set 6.

Full TOSA reference and FVP validation were unavailable locally.

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

Find single-input, single-output pointwise regions in a shared pass and
lower each region directly to one lookup table. Extend Arm table
insertion to use the region search, including int16 tables and their
output rescale, and configure Cortex-M lowering through a subclass.

Fuse add, subtract, multiply and divide with one-element constants.
Evaluate regions in FP32, preserving input and constant dequantization
and quantizing only the final table entries with backend rounding.
Keep graph search independent of backend operators and quantization
conventions. Preserve standalone requantization and unary input
boundaries, and retain valid suffixes after multi-input operations.

Cover integer code domains, branches, constants, mutation boundaries,
explicit Q/DQ, Arm int16 interpolation and both Cortex-M layouts.

Authored with AI assistance using OpenAI Codex.

Jira: MLETORCH-2880
Signed-off-by: Usamah Zaheer <usamah.zaheer@arm.com>
Change-Id: I79816eec03db314c38d90faa8a6ef9fa4719f83c
@usamahz usamahz added the release notes: arm Changes to the ARM backend delegate label Nov 12, 2026
@pytorch-bot

pytorch-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24985

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 9792a0e with merge base f8373ec (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 12, 2026
@github-actions github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels Nov 12, 2026
@usamahz usamahz added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label Nov 12, 2026
@usamahz usamahz self-assigned this Nov 12, 2026
@zingo zingo added the help wanted Extra attention is needed label Nov 13, 2026
@zingo

zingo commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

Hi @rascani and @digantdesai this adds a file outside Arm :)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. help wanted Extra attention is needed module: arm Issues related to arm backend partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: arm Changes to the ARM backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants