Skip to content

perf(rocm): enable Liger fused RMSNorm/SwiGLU for AMD DFlash configs - #779

Open
zhentaocc wants to merge 1 commit into
sgl-project:mainfrom
zhentaocc:perf/amd-dflash-liger-kernels
Open

perf(rocm): enable Liger fused RMSNorm/SwiGLU for AMD DFlash configs#779
zhentaocc wants to merge 1 commit into
sgl-project:mainfrom
zhentaocc:perf/amd-dflash-liger-kernels

Conversation

@zhentaocc

@zhentaocc zhentaocc commented Aug 20, 2026

Copy link
Copy Markdown

Summary

Turns on Liger fused RMSNorm/SwiGLU (model.use_liger_kernel: true) for the two AMD ROCm Qwen3.5-4B DFlash example configs. Validated as a clean win on MI355X (gfx950) with real specforge train, not a profiler estimate.

  • examples/configs/offline/colocated/qwen3.5-4b-dflash-offline-amd.yaml
  • examples/configs/online/disaggregated/external/qwen3.5-4b-dflash-online-amd.yaml

Measured on MI355X (Qwen3.5-4B DFlash, offline, batch_size=2, S=2048, num_anchors=512)

Method: container based on sglang-v0.5.14 (torch 2.9.1+rocm7.2.0), single card, max_steps=200, discard first 20 warmup steps, steady-state mean of perf/global_samples_per_second (n=19); peak VRAM polled across all 8 cards (others idle). Same seed, loss checked step-by-step.

tokens/s (samples/s) s/step peak VRAM loss @ step200
use_liger_kernel=false 12.68 0.623 35.37 GiB 7.0105
use_liger_kernel=true 13.97 0.565 30.18 GiB 7.0101

+10.2% throughput, −5.2 GiB / −15% peak memory, loss parity.

Note: the CUDA-profiler-estimated FLCE (fused Linear+CE) tier was dropped — it is numerically correct but 2.5× slower on gfx950 (Liger's chunked recompute overhead outweighs the logits memory-bandwidth savings). Fused AdamW was also measured (+0.9%, within noise) and dropped. Only RMSNorm/SwiGLU is enabled here.

Depends on #773

This PR does not include the flex-backend fix; it must be merged on top of #773 (or after #773 lands). On torch<2.11, use_liger_kernel=true drives flex-attention through the inductor autotune lowering path, where the unpatched kernel_options={\"BACKEND\":\"TRITON\"} leaks a bare TRITON identifier → NameError: 'TRITON' is not defined. #773 gates that on torch>=2.11. Merging this config change before #773 would break the AMD configs on ROCm torch<2.11.

Test plan

Turn on model.use_liger_kernel for the two AMD Qwen3.5-4B DFlash example
configs (offline + online). This routes the draft model's RMSNorm and
SwiGLU MLP through Liger's fused Triton kernels.

Validated on a single idle MI355X (gfx950), torch 2.9.1+rocm7.2.0, real
`specforge train` (offline, batch_size=2, S=2048, 200 steps, steady-state
mean over 19 post-warmup steps):

  use_liger_kernel=false -> 12.68 samples/s, 0.623 s/step, 35.37 GiB peak
  use_liger_kernel=true  -> 13.97 samples/s, 0.565 s/step, 30.18 GiB peak

  => +10.2% throughput, -5.2 GiB (-15%) peak memory, loss parity to 4
     decimals (step200 7.0105 vs 7.0101).

Depends on the flex-backend fix (gate BACKEND=TRITON on torch>=2.11): on
torch<2.11 the Liger path drives flex-attention through the inductor
autotune lowering, which crashes with NameError: 'TRITON' is not defined
without that fix.
@zhentaocc
zhentaocc force-pushed the perf/amd-dflash-liger-kernels branch from 893794a to 17cee84 Compare August 21, 2026 02:22
@zhentaocc
zhentaocc marked this pull request as ready for review August 21, 2026 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant