feat(megatron): add nccl_comm_warmup to avoid iteration-1 NCCL cudaMalloc OOM (#6387)#9602

Jintao-Huang merged 1 commit into

modelscope:mainmodelscope/ms-swift:mainfrom

yuchenwang3:feat/nccl-comm-warmupyuchenwang3/ms-swift:feat/nccl-comm-warmupCopy head branch name to clipboard