Rocm gfx1100 integration - #283
Open
Calandracas606 wants to merge 5 commits into
Open
Conversation
Add compat_rocm.cuh with HIP polyfills for CUDA intrinsics (__shfl, warp sync, dp4a, half2 hmax/hmin). Stub MMA tensor-core ops with abort(). Guard cooperative-group and CUDA-graph paths. bindings.cpp uses #if !defined(USE_ROCM) / #else to split CUDA-only vs ROCm-compatible bindings into a clean whitelist. Kernel fixes: - rope.cu: fix shared-memory race in RoPE norm (only lane 0 writes sum) - graph.cuh: alias cudaKernelNodeParams to hipKernelNodeParams on ROCm - quant/reconstruct.cu: guard ptx.cuh include, hipify launch syntax - quant/codebook.cuh: explicit #ifdef instead of LOP3_6A macro - compat_rocm.cuh: dp4a uses unsigned bytes (was signed int8_t) - cuda_drv: HIP driver API shims Co-authored-by: openhands <openhands@all-hands.dev>
build_config.py excludes CUDA-only source files from the ROCm build. setup.py adds -DUSE_ROCM, -O3, and -Wno-register when building for ROCm. arch_list.py provides ROCm GPU architecture detection. Co-authored-by: openhands <openhands@all-hands.dev>
Instead of scattering IS_ROCM / has_ext checks across module files, all platform dispatch is centralized in ext.py: - Missing BC_* constructors return None (via _BCNone sentinel) - Missing C++ functions (silu_mul, rms_norm, gated_rms_norm, etc.) are replaced with pure-PyTorch implementations from ext_fallbacks.py - Fused sampler is disabled via env var on ROCm Module changes are minimal: - gated_delta_net.py: bc_split reflects bc availability (1 line) - exl3.py: guard bc.run_alloc with None check (1 line) - config.py: use_mgemm returns False when exl3_mgemm unavailable (3 lines) Zero changes to mamba2.py, mlp.py, rmsnorm.py, attn.py, etc. Co-authored-by: openhands <openhands@all-hands.dev>
Change test device defaults from cuda:1/cuda:2 to cuda:0 (the only GPU on the ROCm test machine). Remove old platform-specific test files that tested the previous IS_ROCM dispatch pattern (now handled by ext.py). Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds basic support for rocm.
So far only Qwen35 models work. This is because I wanted to implement a thin vertical slice.
This suite of tests passes:
Performance is not optimized at all. Cuda kernels which are too complex to port in the PR are replaced by pure pytorch fallbacks. This PR is just adding the minimal changes to get it up and running. Performance optimizations will follow.
Qwen3.5-9B-4.00bpw
Prefill (4K context) ~ 1,138 tok/s
Decode ~ 14.3 tok/s
Qwen3.6-27B-4.00bpw
Prefill (4K context) ~ 500 tok/s
Decode ~ 4.33 tok/s
The code in this PR was written with LLM assistance. GLM-5.2 with OpenHands harness.