[Not ready] MSCCLPP EP implementation - #852

Draft

Binyang Li (Binyang2014) wants to merge 134 commits into

mainfrom

feature/ep

Commits on Aug 21, 2026

atomic add

Changho Hwang (chhwang)
authored and
Binyang Li (Binyang2014)
committed
b414793
Add Expert-Parallel (MoE dispatch/combine) extension under src/ext/ep

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
ca8e3d1
src/ext/ep: port low-latency dispatch/combine kernels

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
9b754a5
ext/ep: fix multi-rank intranode dispatch+combine

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
483a788
ep: fix port-channel rank ordering for internode HT dispatch

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
08dd8ef
ep: fix internode combine in multirank test

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
182df20
ep: document internode HT validation on 2x H100x8

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
6dcf647
ext/ep: unfilter LL sync + add LL multirank test (intra-node WIP)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
860d438
ext/ep tests: add optional HT benchmark pass

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
e32a7a0
ext/ep tests: time dispatch and combine separately in MSCCLPP_EP_BENCH

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
c7266d6
ext/ep: fix LL IB atomicAdd alignment by widening signaling buffers to int64

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
28d1ba4
ext/ep: restore self CUDA-IPC connection (was needed by HT/LL paths)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
f4bb33c
ext/ep: keep self slots in LL sema+port-channel loops (fixes cross-node)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
4abd257
tests/ep: size HT buffers for bench hidden so bench phase fits

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
2e173fd
ext/ep: LL intra-node fast path via CUDA IPC + MemoryChannel

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
fd0588f
tests/ep: LL bench combine uses recv_tokens×hidden for payload bytes

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
940e45e
tests/ep: LL bench prints per_rank_bw and accepts size env vars

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
aa53a60
tests/ep: HT benches also print per_rank_bw

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
5edb531
ep/ll: bump grid size to better utilize SMs

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
5adf817
ep/ll: gate SM-count grid bump behind IPC path

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
890948a
tests/ep: hoist combine output tensor out of the timed loop

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
7f21c3f
ep/ll: use 1 expert per SM with 32 warps per block

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
0a83ef3
ep/ll: keep (3,10) on PortChannel path

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
1fb09b7
tests/ep: align intranode HT bench with NCCL-EP accounting

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
9eb4d55
tests/ep: align internode HT bench with NCCL-EP accounting

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
2b47f7f
ep: shard PortChannels across multiple ProxyServices

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
1632d63
ep: env-tunable + arch-aware num_proxy_services (default 8 on Hopper, 1 on Blackwell)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
c566fd8
ep tests: clean shutdown to silence TCPStore/HeartbeatMonitor noise

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
8a4e8b2
ep tests: report dispatch/combine min, avg, max time and use avg for BW

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
353c9a9
ext/ep: refresh README to reflect current LL, proxy sharding, and bench harness

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
221c5d5
tests/ep: add NCCL-EP six-metric BW breakdown (send/recv x total/nvl/rdma)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
b3cbc48
tests/ep: disable NCCL HeartbeatMonitor to silence mpirun shutdown noise

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
bf878b3
tests/ep: intranode send-side counts unique (token, dst_node) to match NCCL-EP

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
ee5a23a
ext/ep: optional preallocated outputs for low_latency_dispatch

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
46719d1
Potential fix for pull request finding

authored and
Binyang Li (Binyang2014)
committed
c1464f5
Potential fix for pull request finding

authored and
Binyang Li (Binyang2014)
committed
589d21c
ext/ep: add MIT license headers to EP sources and tests

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
920e52d
ext/ep: remove outdated single-rank smoke test

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
745d170
ext/ep: remove unused mscclpp_ep CMake target

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
53627a9
ext/ep: fix SWITCH_* macros and add missing standard headers

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
0b09140
ext/ep: add pragma once to event.hpp and update validation docs

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
ccfd7f7
core: TODO notes on CUDA-IPC atomicAdd context/flush caveats

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
97648ff
ext/ep: apply clang-format and black to fix CI lint failures

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
0c498f7
EP GB200 (4 GPUs/node) support

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
84bb4da
ext/ep: route LL internode atomics over NVL72 fabric (Proposal A)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
e4f0461
ext/ep: NVLS HT B2 phases 1-3 (notify_dispatch barrier + counter fast path)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
f28cbfc
ext/ep: WIP Phase 4 NVLS HT B2 fabric-IPC data path

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
b51d122
ext/ep: WIP Phase 4 NVLS HT B2 third multimem barrier in notify_dispatch

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
6499d50
ext/ep: WIP Phase 4 NVLS HT B2 cooperative-copy + relaxed multimem.red

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
3490251
ext/ep: WIP Phase 4 normalize fence/sync ordering in combine writer

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
4f5198f
ext/ep: WIP Phase 4 fabric-VA cross-node tail/head counter (bypass NVLS multimem.red)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
16c483f
ext/ep: WIP Phase 4 fix NVLS self-overcount + cached_notify NVLS barrier

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
cf1c0cd
ext/ep: HT perf - lower lazy head-feedback threshold to chunk/4

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
ffad22d
ext/ep: fix kRDMASender epilogue tail-write race (unblocks chunk_send>16)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
8249663
ext/ep: lift cached_notify nc cap, strided-warp head fixup (HT 442/399 GB/s)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
00261b6
ext/ep: 8x unroll cross-node int4 copy (HT 645/612 GB/s)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
74d6d09
ext/ep: LL escalate IPC layout when num_experts exceeds coop grid cap

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
bfe0056
test/ext/ep: make HT test Config env-driven

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
61bb62a
test/ext/ep: HT tests — env-driven cfg + allgather bookkeeping

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
d6137de
test/ext/ep: HT — scale combine tolerance with bf16 ulp

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
93bbbb1
test/ext/ep: intranode HT bench — cached-mode iter loop

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
6063473
test/ext/ep: intranode HT — parameterize num_sms and NVL chunk sizes

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
2677cf9
ext/ep: gate GB200-specific changes for non-GB200 builds

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
00927b0
ext/ep: README — GB200 build + runtime notes

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
a7954ae
ext/ep: README — split HT/LL mpirun, add torch install steps

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
95c47d5
ext/ep: README — clarify GB200 env-var defaults

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
89a371f
ext/ep: README — refresh Layout, reorder GB200 build, drop torchvision

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
3c5521e
ext/ep: README - enable MSCCLPP_EP_BENCH in test commands; trim unused mpirun env vars

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
39c10b7
ext/ep: README - add explicit bench shape (TOKENS/HIDDEN/EXPERTS/TOPK) to examples

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
d54d20d
ext/ep: README - drop --mca TCP-only flags from default mpirun examples

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
affc950
ext/ep: README - document GB200 runtime env (NCCL_IB_DISABLE=1, NCCL_MNNVL_ENABLE=0, MSCCLPP_HCA_DEVICES, rtprio, nvidia-imex checks)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
652fa1f
ext/ep: README - update Status with GB200 16-node validation + NVLS fabric IPC

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
4602f12
ext/ep: README - document GB200 NVLS variants in LL transport table + GB200 validation rows

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
573bb47
ext/ep: README - add Migration Phase 5 (Azure GB200 NVL72 port)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
31e4603
tests/ep: Unify the name of EP benchmark variables

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
2d89e52
ext/ep: gate NVLS HT B2 on cross-host fabric IPC support (H100 fix)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
9929aec
ext/ep: fix CUDA 11.8 build + apply clang-format/black

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
894bece
Fix python lint formatting in internode multirank test

authored and
Binyang Li (Binyang2014)
committed
eecbc09
Revert "Clean up completed communicator receives (#804)" - thread-safety regression causing cudaErrorIllegalAddress on HT internode GB200

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
192b39e
ext/ep/buffer: P2 - shard proxy on Blackwell multi-node by default (num_proxies = local_world_size when num_rdma_ranks > 1)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
3fb6a5f
ep/internode: P4b - tune kNumDispatchRDMASenderWarps 7->6 (HT 2n dispatch 728->689us on GB200)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
26f74ea
ep: add MSCCLPP_EP_DISPATCH_NCCLEP build guard (EP_DISPATCH_NCCLEP) for NCCL-EP-ported warp-specialized HT dispatch (default OFF)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
a02dfa1
ep: scaffold guarded dispatch_ncclep kernel (NCCL-EP port baseline) + launch-site selection

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
09f5beb
ep(ncclep): increment 1 - same-GPU fused direct-write to recv_x

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
1da1f4b
ep(ncclep): increment 2 - deepen drain-copy MLP unroll 5->28

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
fa90026
ep(ncclep): increment-3 de-risk - DRAIN_NOOP probe + SKIP_VERIFY test gate

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
5936919
ep(ncclep): increment 3 - cross-GPU peer-map direct-write (eliminate receiver hidden drain)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
b6dcdb9
ep(ncclep): increment 4a - VMM unicast recv pool (TMA-eligible peer mapping)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
32448d6
ep(ncclep): increment 5 - sender direct-write dispatch (kEpDirect, dispatch-only WIP)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
b79ed13
ep(ncclep): inc5 ring-slot shrink + read-once direct write

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
f6bf781
ep(ncclep): inc5 keep full rdma clean for combine under kEpDirect

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
276fef1
ep(ncclep): inc5 combine-direct gather (kEpDirect)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
8a9e9b6
ep(intranode): sender direct-write dispatch (MSCCLPP_EP_INTRA_DIRECT)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
4548a65
ep(ncclep): inc5 combine-gather correctness at >32 ranks (16n)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
38ecbd0
docs(ep): document MSCCLPP_EP_DIRECT and MSCCLPP_EP_INTRA_DIRECT for GB200

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
a1de83c
MoE Commnucator design doc (#818)
Binyang Li (Binyang2014)
committed
b3f6fc0
ep: high-throughput (HT) MoE backend on the nanobind MoECommunicator (#825)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
42fc400
update read me
Binyang Li (Binyang2014)
committed
907f1e2
Binyli/ep revise (#828)
Binyang Li (Binyang2014)
committed
ab9282b
Update ep test. Enable cuda graph for ep testing (#829)
Binyang Li (Binyang2014)
committed
2316677
ep: unified low-latency EP benchmark + LL combine SM-scaling fix (#831)

authored and
Binyang Li (Binyang2014)
committed
b8ef6a1
ep(bench): mscclpp_ep_bench uses NCCL-EP-style random top-k routing (#832)

committed
8632b99
Optimized LL implementation (#834)

Binyang Li (Binyang2014)
and
Copilot
committed
b88d79d
Support hidden size 6656 in low-latency EP (#835)

Binyang Li (Binyang2014)
and
Copilot
committed
cbe3d81
Refactor HT EP for direct fabric domains (#837)
Binyang Li (Binyang2014)
committed
042082d
Enable efficient 16-rank HT EP

Binyang Li (Binyang2014)
and
Copilot
committed
fb2fb97
Add token major layout support (#841)
Binyang Li (Binyang2014)
committed
7bedae6
Adjust token major layout (#844)

Binyang Li (Binyang2014)
and
Copilot
committed
ed07c0d
Refactor HT mode (#846)

Binyang Li (Binyang2014)
and
Copilot
committed
e5f0a07
Add LL MXFP8 scale-32 dispatch (#847)
Binyang Li (Binyang2014)
committed
3cd2e99
Add native LL MXFP8 dispatch (#848)
Binyang Li (Binyang2014)
committed
9a15b6b
Harden LL token metadata handling (#849)
Binyang Li (Binyang2014)
committed
c904e0b
ep(bench): unified in-process Python EP benchmark (mscclpp vs NCCL-EP) (#836)

committed
8676e4d
fix(ep-bench): strip C++ template args in kineto kernel-name match (#856)

Qinghua Zhou (seagater)
authored and
Binyang Li (Binyang2014)
committed
6921cb5
Support rank major (#857)

Binyang Li (Binyang2014)
and
Copilot
committed
7002c84
ep: address implementation review feedback

Binyang Li (Binyang2014)
and
Copilot
committed
5dccbe6
style(ep): format HT combine launch macro

Binyang Li (Binyang2014)
and
Copilot
committed
62ca35e
wip

Caio Rocha (caiocbr)
authored and
Binyang Li (Binyang2014)
committed
b62a066
wip

Caio Rocha (caiocbr)
authored and
Binyang Li (Binyang2014)
committed
9574d04
move review to skill
Binyang Li (Binyang2014)
committed
29005c0
Potential fix for pull request finding 'CodeQL / Multiplication result converted to larger type'

Binyang Li (Binyang2014)
and
github-advanced-security[bot]
committed
21bb092
Add bulk asynchronous copy primitives

Changho Hwang (chhwang)
authored and
Binyang Li (Binyang2014)
committed
4f7d878
Make bulk reduction element types nameable from host code

Changho Hwang (chhwang)
authored and
Binyang Li (Binyang2014)
committed
f690c89
Fix docs generation, add Python capability query, drop duplicate check

Changho Hwang (chhwang)
authored and
Binyang Li (Binyang2014)
committed
33d632e
Silence unused-private-field warning on ROCm

Changho Hwang (chhwang)
authored and
Binyang Li (Binyang2014)
committed
9ae0e75
Use maybe_unused for architecture-gated kernel parameters

Changho Hwang (chhwang)
authored and
Binyang Li (Binyang2014)
committed
545e42c
Optimizing Memory Footprint (#869)

Caio Rocha (caiocbr)
and
Binyang Li (Binyang2014)
committed
5417603
use mscclpp api for tma data transfer (#872)
Binyang Li (Binyang2014)
committed
2996999
Fix low-latency EP epoch race with host-assigned dispatch epochs (#875)

Caio Rocha (caiocbr)
authored and
Binyang Li (Binyang2014)
committed
5eed219
Refactor EP runtime architecture (#874)

Binyang Li (Binyang2014)
and
Copilot
committed
1e7cc33
Add direct send for rank major layout (#880)

Binyang Li (Binyang2014)
and
Copilot
committed
ca4106b
Resolve feature/ep rebase conflicts with main

Binyang Li (Binyang2014)
and
Copilot
committed
8e688ee
ep(bench): unified in-process Python EP benchmark — review-comment follow-ups (#858)

authored
2eddba5

Commits on Aug 22, 2026

ep(bench): add mscclpp high-throughput (rank-major) backend to the unified Python EP benchmark (#860)

authored
3d02152

Uh oh!

Commits

Commits on Aug 21, 2026

Commits on Aug 22, 2026