Update H100 and MI300X distributed benchmark data

[frank-2077]

Update H100 official and MI300X/ROCm distributed benchmark results.

Test environments: MI300X with ROCm 7.14 and PyTorch 2.12; H100 80GB with CUDA 13.0 and PyTorch 2.13. Both use BF16 with M=32, H=4096, and I=12288.

[coderabbitai]

Important

Review skipped

Too many files!

This PR contains 139 files, which is 39 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: d28a8e70-0433-44d6-9385-20599a436bde

📥 Commits

Reviewing files that changed from the base of the PR and between ccb70e3 and c328645.

⛔ Files ignored due to path filters (14)

• benchmarks/results/pr325_rocm_mi300x/collective_overhead.png is excluded by !**/*.png
• benchmarks/results/pr325_rocm_mi300x/distributed_ffn_overhead.png is excluded by !**/*.png
• benchmarks/results/pr325_rocm_mi300x/single_gpu_overhead.png is excluded by !**/*.png
• benchmarks/results/pr328_rocm_mi300x/distributed_logp_latency.png is excluded by !**/*.png
• benchmarks/results/pr328_rocm_mi300x/single_gpu_grid.png is excluded by !**/*.png
• benchmarks/results/pr328_rocm_mi300x/single_gpu_grid_kernels.png is excluded by !**/*.png
• benchmarks/results/pr328_rocm_mi300x/single_gpu_latency.png is excluded by !**/*.png
• benchmarks/results/pr328_rocm_mi300x/single_gpu_memory.png is excluded by !**/*.png
• benchmarks/results/ws2_rocm_mi300x/distributed_cp_latency.png is excluded by !**/*.png
• benchmarks/results/ws2_rocm_mi300x/exactness_matrix.png is excluded by !**/*.png
• benchmarks/results/ws2_rocm_mi300x/single_gpu_grid.png is excluded by !**/*.png
• benchmarks/results/ws2_rocm_mi300x/single_gpu_latency.png is excluded by !**/*.png
• benchmarks/results/ws2_rocm_mi300x/single_gpu_memory.png is excluded by !**/*.png
• benchmarks/results/ws2_rocm_mi300x/tp_degree_invariance.png is excluded by !**/*.png

📒 Files selected for processing (139)

• .github/workflows/ci.yml
• .gitignore
• _dev_notes/rocm_logprob_implementation_summary.md
• benchmarks/benchmark_rocm_attention.py
• benchmarks/benchmark_rocm_collectives.py
• benchmarks/benchmark_rocm_det_gemm_leaf.py
• benchmarks/benchmark_rocm_ffn.py
• benchmarks/benchmark_rocm_logp.py
• benchmarks/benchmark_ws2_rocm_attention.py
• benchmarks/profile_rocm_ffn.py
• benchmarks/results/pr319_rocm_mi300x/distributed/report.md
• benchmarks/results/pr319_rocm_mi300x/distributed/strict_rccl_ag_rs_w2.json
• benchmarks/results/pr319_rocm_mi300x/distributed/strict_rccl_ag_rs_w4.json
• benchmarks/results/pr319_rocm_mi300x/distributed/strict_rccl_ag_rs_w8.json
• benchmarks/results/pr325_rocm_mi300x/cuda_cpu_comparison.json
• benchmarks/results/pr325_rocm_mi300x/report.md
• benchmarks/results/pr325_rocm_mi300x/results.json
• benchmarks/results/pr328_rocm_mi300x/report.md
• benchmarks/results/pr328_rocm_mi300x/results.json
• benchmarks/results/ws2_cpu/report.md
• benchmarks/results/ws2_cpu/results.json
• benchmarks/results/ws2_rocm_mi300x/PR_DESCRIPTION.md
• benchmarks/results/ws2_rocm_mi300x/report.md
• benchmarks/results/ws2_rocm_mi300x/results.json
• csrc/cuda/attention/deterministic_attention.cu
• csrc/cuda/distributed/deterministic_collective.cu
• csrc/deterministic_logp_kernel.cu
• csrc/hip/hip_deterministic_logp_kernel.hip
• csrc/ops.cpp
• csrc/rocm/distributed/deterministic_collective.hip
• docs/benchmarking/README.md
• docs/design/rocm-deterministic-collectives.md
• docs/operators/grpo-loss.md
• examples/__init__.py
• examples/vime_qwen3_8b_rocm_ablation/README.md
• examples/vime_qwen3_8b_rocm_ablation/rocm_python_entrypoint.sh
• examples/vime_qwen3_8b_rocm_ablation/run.py
• examples/vime_rocm_attention_ablation/README.md
• examples/vime_rocm_attention_ablation/__init__.py
• examples/vime_rocm_attention_ablation/launch_arm.sh
• examples/vime_rocm_attention_ablation/run.py
• examples/vime_rocm_attention_ablation/tis_metrics.py
• examples/vime_rocm_attention_ablation/validate_artifacts.py
• pyproject.toml
• rl_engine/_C.pyi
• rl_engine/distributed/__init__.py
• rl_engine/distributed/collectives.py
• rl_engine/distributed/rocm_collectives.py
• rl_engine/distributed/transport_collectives.py
• rl_engine/integrations/__init__.py
• rl_engine/integrations/framework_operators.py
• rl_engine/integrations/linear_logp.py
• rl_engine/integrations/megatron_runtime.py
• rl_engine/integrations/rocm_ablation.py
• rl_engine/integrations/runtime.py
• rl_engine/integrations/vime/__init__.py
• rl_engine/integrations/vime/attention.py
• rl_engine/integrations/vime/linear_logp_provider.py
• rl_engine/integrations/vime/logp.py
• rl_engine/integrations/vllm_runtime.py
• rl_engine/kernels/attention_contract.py
• rl_engine/kernels/attention_preprocess.py
• rl_engine/kernels/loss_contract.py
• rl_engine/kernels/ops/cuda/attention/__init__.py
• rl_engine/kernels/ops/cuda/attention/cp_comm.py
• rl_engine/kernels/ops/cuda/attention/deterministic_attn.py
• rl_engine/kernels/ops/cuda/attention/flashinfer_paged_attention.py
• rl_engine/kernels/ops/cuda/loss/batch_invariant_logp.py
• rl_engine/kernels/ops/cuda/loss/vocab_parallel_logp.py
• rl_engine/kernels/ops/cuda/rotary_embedding/rope.py
• rl_engine/kernels/ops/matmul/__init__.py
• rl_engine/kernels/ops/matmul/det_gemm.py
• rl_engine/kernels/ops/pytorch/attention/ablation.py
• rl_engine/kernels/ops/pytorch/attention/cp_attention.py
• rl_engine/kernels/ops/pytorch/ffn/ffn.py
• rl_engine/kernels/ops/pytorch/loss/batch_invariant_logp.py
• rl_engine/kernels/ops/pytorch/loss/distributed_grpo_loss.py
• rl_engine/kernels/ops/pytorch/loss/vocab_parallel_logp.py
• rl_engine/kernels/ops/rocm/attention/__init__.py
• rl_engine/kernels/ops/rocm/attention/flash_attn.py
• rl_engine/kernels/ops/rocm/attention/strict_runtime.py
• rl_engine/kernels/ops/rocm/loss/__init__.py
• rl_engine/kernels/ops/rocm/loss/vocab_parallel_logp.py
• rl_engine/kernels/ops/rocm/matmul/__init__.py
• rl_engine/kernels/ops/rocm/matmul/det_gemm.py
• rl_engine/kernels/ops/rocm/rotary_embedding/__init__.py
• rl_engine/kernels/ops/rocm/rotary_embedding/rope.py
• rl_engine/kernels/ops/triton/activation/swiglu.py
• rl_engine/kernels/ops/triton/attention/__init__.py
• rl_engine/kernels/ops/triton/attention/deterministic_attn.py
• rl_engine/kernels/ops/triton/ffn/__init__.py
• rl_engine/kernels/ops/triton/ffn/ffn.py
• rl_engine/kernels/ops/triton/loss/batch_invariant_logp.py
• rl_engine/kernels/ops/triton/loss/linear_logp.py
• rl_engine/kernels/ops/triton/loss/vocab_parallel_logp.py
• rl_engine/kernels/ops/triton/matmul/det_gemm.py
• rl_engine/kernels/registry.py
• rl_engine/testing/distributed_logprob_comparison.py
• rl_engine/testing/logprob_comparison.py
• rl_engine/testing/logprob_drift.py
• scripts/ws2_p2p_nccl_attention_reference_check.py
• scripts/ws2_pr7_flashinfer_attention_check.py
• setup.py
• tests/conftest.py
• tests/distributed/test_det_gemm_simulated_tp.py
• tests/distributed/test_deterministic_all_gather.py
• tests/distributed/test_deterministic_all_reduce.py
• tests/distributed/test_deterministic_reduce_scatter.py
• tests/distributed/test_qwen_ffn_topology.py
• tests/distributed/test_rocm_attention_transport.py
• tests/distributed/test_rocm_strict_attention_cp.py
• tests/distributed/test_transport_deterministic_collective.py
• tests/test_attention_ablation.py
• tests/test_attention_correctness.py
• tests/test_attention_dispatch.py
• tests/test_build_platform_collectives.py
• tests/test_det_gemm.py
• tests/test_deterministic_attention_cuda.py
• tests/test_distributed_grpo_loss.py
• tests/test_distributed_logprob_comparison.py
• tests/test_flashinfer_pr7_attention.py
• tests/test_framework_runtime_adapters.py
• tests/test_grpo_loss_contract.py
• tests/test_linear_logp.py
• tests/test_logprob_comparison.py
• tests/test_logprob_contract.py
• tests/test_qwen_ffn.py
• tests/test_rocm_aiter_api_contract.py
• tests/test_rocm_collective_benchmark.py
• tests/test_rocm_e2e_ablation.py
• tests/test_rocm_logprob_backend.py
• tests/test_rocm_packed_ffn.py
• tests/test_rocm_strict_paged_attention.py
• tests/test_triton_deterministic_attention.py
• tests/test_vime_attention_provider.py
• tests/test_vime_linear_logp_provider.py
• tests/test_vime_logprob_provider.py
• tests/test_vime_rocm_attention_topology.py
• tests/test_vocab_parallel_logp.py

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

_{Comment @coderabbitai help to get the list of available commands.}

[Flink-ddd]

LGTM

[maxiaosong1124]

LGTM!