Skip to content

perf(gb300): Refresh Qwen3.5 FP4 GB300 dynamo-trt STP/MTP recipes from srt-slurm / 刷新基于 srt-slurm 的 Qwen3.5 FP4 GB300 dynamo-trt STP/MTP 配方 #2730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cquil11 merged 4 commits into from
Aug 27, 2026
Merged

perf(gb300): Refresh Qwen3.5 FP4 GB300 dynamo-trt STP/MTP recipes from srt-slurm / 刷新基于 srt-slurm 的 Qwen3.5 FP4 GB300 dynamo-trt STP/MTP 配方 #2730

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e09f640
perf(gb300): Refresh Qwen3.5 FP4 GB300 dynamo-trt STP/MTP recipes fro…
richardhuo-nv Aug 25, 2026
e8dbc69
docs(gb300): Fill in perf-changelog PR link / 补充 perf-changelog 中的 PR 链接
richardhuo-nv Aug 25, 2026
4ed3feb
Merge branch 'main' into rihuo/qwen3.5-fp4-gb300-trt-recipe-refresh
richardhuo-nv Aug 25, 2026
6fff90a
chore: sync main and restore append-only changelog [skip-sweep]
cquil11 Aug 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 169 additions & 0 deletions ...es/trtllm/qwen3.5/gb300-fp4/disagg/8k1k/mtp/ctx11_gen1_dep8_batch128_eplb0_mtp3_1229.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
name: disagg-gb300-11p1d-dep1-dep8-c1229-mtp
model:
path: qwen3.5-fp4
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc24
precision: fp4

dynamo:
install: true
wheel: 1.4.0.dev20260807

identity:
container:
image: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc24
frameworks:
tensorrt_llm: 1.3.0rc24
slurm:
time_limit: 04:00:00
health_check:
max_attempts: 540
interval_seconds: 10
resources:
gpu_type: gb300
gpus_per_node: 4
prefill_nodes: 3
prefill_workers: 11
gpus_per_prefill: 1
decode_nodes: 2
decode_workers: 1
gpus_per_decode: 8
backend:
type: trtllm
sequential_node_start: 2
prefill_environment:
TLLM_LOG_LEVEL: INFO
TRTLLM_SERVER_DISABLE_GC: '1'
TRTLLM_WORKER_DISABLE_GC: '1'
TRTLLM_ENABLE_PDL: '1'
NCCL_GRAPH_MIXING_SUPPORT: '0'
MIMALLOC_PURGE_DELAY: '0'
TRT_LLM_DISABLE_LOAD_WEIGHTS_IN_PARALLEL: '1'
CTX_LOAD_STAGGER_S: '180'
UCX_TLS: cuda_ipc,cuda_copy,sm,self,tcp
decode_environment:
TLLM_LOG_LEVEL: INFO
TRTLLM_SERVER_DISABLE_GC: '1'
TRTLLM_WORKER_DISABLE_GC: '1'
TRTLLM_ENABLE_PDL: '1'
NCCL_GRAPH_MIXING_SUPPORT: '0'
MIMALLOC_PURGE_DELAY: '0'
UCX_TLS: cuda_ipc,cuda_copy,sm,self,tcp
trtllm_config:
prefill:
cache_transceiver_config:
backend: NIXL
kv_transfer_timeout_ms: 600000
max_tokens_in_buffer: 16384
transceiver_runtime: PYTHON
cuda_graph_config: null
disable_overlap_scheduler: true
enable_attention_dp: true
kv_cache_config:
dtype: fp8
enable_block_reuse: false
free_gpu_memory_fraction: 0.7
max_batch_size: 2
max_num_tokens: 16896
max_seq_len: 8448
moe_config:
backend: CUTEDSL
moe_expert_parallel_size: 1
nvfp4_gemm_config:
allowed_backends:
- cutlass
- cublaslt
- cutedsl
- cuda_core
pipeline_parallel_size: 1
print_iter_log: true
speculative_config:
decoding_type: MTP
max_draft_len: 3
tensor_parallel_size: 1
trust_remote_code: true
decode:
cache_transceiver_config:
backend: NIXL
kv_transfer_timeout_ms: 600000
max_tokens_in_buffer: 16384
transceiver_runtime: PYTHON
cuda_graph_config:
batch_sizes:
- 1
- 2
- 4
- 8
- 12
- 16
- 20
- 24
- 28
- 32
- 36
- 40
- 44
- 48
- 52
- 56
- 60
- 64
- 68
- 72
- 76
- 80
- 84
- 88
- 92
- 96
- 100
- 104
- 108
- 112
- 116
- 120
- 124
- 128
enable_padding: true
enable_attention_dp: true
enable_lm_head_tp_in_adp: true
kv_cache_config:
dtype: fp8
enable_block_reuse: false
free_gpu_memory_fraction: 0.8
max_batch_size: 128
max_num_tokens: 512
max_seq_len: 9472
moe_config:
backend: CUTEDSL
use_low_precision_moe_combine: true
moe_expert_parallel_size: 8
num_postprocess_workers: 4
nvfp4_gemm_config:
allowed_backends:
- cutlass
- cublaslt
- cutedsl
- cuda_core
pipeline_parallel_size: 1
print_iter_log: true
speculative_config:
decoding_type: MTP
max_draft_len: 3
stream_interval: 100
tensor_parallel_size: 8
trust_remote_code: true
frontend:
type: dynamo
enable_multiple_frontends: true
env:
ETCD_LEASE_TTL: '120'
DYN_TOKENIZER: "fastokens"

benchmark:
type: sa-bench
isl: 8192
osl: 1024
concurrencies:
- 1229
req_rate: inf
num_prompts_mult: 16
151 changes: 151 additions & 0 deletions ...pes/trtllm/qwen3.5/gb300-fp4/disagg/8k1k/mtp/ctx17_gen2_dep8_batch64_eplb0_mtp3_1126.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
name: disagg-gb300-17p2d-dep1-dep8-c1126-mtp
model:
path: qwen3.5-fp4
container: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc24
precision: fp4

dynamo:
install: true
wheel: 1.4.0.dev20260807

identity:
container:
image: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc24
frameworks:
tensorrt_llm: 1.3.0rc24
slurm:
time_limit: 04:00:00
health_check:
max_attempts: 540
interval_seconds: 10
resources:
gpu_type: gb300
gpus_per_node: 4
prefill_nodes: 5
prefill_workers: 17
gpus_per_prefill: 1
decode_nodes: 4
decode_workers: 2
gpus_per_decode: 8
backend:
type: trtllm
sequential_node_start: 2
prefill_environment:
TLLM_LOG_LEVEL: INFO
TRTLLM_SERVER_DISABLE_GC: '1'
TRTLLM_WORKER_DISABLE_GC: '1'
TRTLLM_ENABLE_PDL: '1'
NCCL_GRAPH_MIXING_SUPPORT: '0'
MIMALLOC_PURGE_DELAY: '0'
TRT_LLM_DISABLE_LOAD_WEIGHTS_IN_PARALLEL: '1'
CTX_LOAD_STAGGER_S: '180'
UCX_TLS: cuda_ipc,cuda_copy,sm,self,tcp
decode_environment:
TLLM_LOG_LEVEL: INFO
TRTLLM_SERVER_DISABLE_GC: '1'
TRTLLM_WORKER_DISABLE_GC: '1'
TRTLLM_ENABLE_PDL: '1'
NCCL_GRAPH_MIXING_SUPPORT: '0'
MIMALLOC_PURGE_DELAY: '0'
UCX_TLS: cuda_ipc,cuda_copy,sm,self,tcp
trtllm_config:
prefill:
cache_transceiver_config:
backend: NIXL
kv_transfer_timeout_ms: 600000
max_tokens_in_buffer: 16384
cuda_graph_config: null
disable_overlap_scheduler: true
enable_attention_dp: true
kv_cache_config:
dtype: fp8
enable_block_reuse: false
free_gpu_memory_fraction: 0.7
max_batch_size: 2
max_num_tokens: 16896
max_seq_len: 8448
moe_config:
backend: CUTEDSL
moe_expert_parallel_size: 1
nvfp4_gemm_config:
allowed_backends:
- cutlass
- cublaslt
- cutedsl
- cuda_core
pipeline_parallel_size: 1
print_iter_log: true
speculative_config:
decoding_type: MTP
max_draft_len: 3
tensor_parallel_size: 1
trust_remote_code: true
decode:
cache_transceiver_config:
backend: NIXL
kv_transfer_timeout_ms: 600000
max_tokens_in_buffer: 16384
cuda_graph_config:
batch_sizes:
- 1
- 2
- 4
- 8
- 12
- 16
- 20
- 24
- 28
- 32
- 36
- 40
- 44
- 48
- 52
- 56
- 60
- 64
enable_padding: true
enable_attention_dp: true
enable_lm_head_tp_in_adp: true
kv_cache_config:
dtype: fp8
enable_block_reuse: false
free_gpu_memory_fraction: 0.8
max_batch_size: 64
max_num_tokens: 256
max_seq_len: 9472
moe_config:
backend: CUTEDSL
use_low_precision_moe_combine: true
moe_expert_parallel_size: 8
num_postprocess_workers: 4
nvfp4_gemm_config:
allowed_backends:
- cutlass
- cublaslt
- cutedsl
- cuda_core
pipeline_parallel_size: 1
print_iter_log: true
speculative_config:
decoding_type: MTP
max_draft_len: 3
stream_interval: 100
tensor_parallel_size: 8
trust_remote_code: true
frontend:
type: dynamo
enable_multiple_frontends: true
env:
ETCD_LEASE_TTL: '120'
DYN_TOKENIZER: "fastokens"

benchmark:
type: sa-bench
isl: 8192
osl: 1024
concurrencies:
- 1126
req_rate: inf
num_prompts_mult: 16
Loading