Tune B200 DSV4 SGLang AgentX prefill scheduling / 调整 B200 DSV4 SGLang AgentX 预填充调度 - #2718
Tune B200 DSV4 SGLang AgentX prefill scheduling / 调整 B200 DSV4 SGLang AgentX 预填充调度#2718nvpohanh wants to merge 12 commits into
Conversation
调整 B200 DSV4 AgentX 预填充调度。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
更新性能变更日志中的 PR 链接。
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 206cb37. Configure here.
修正性能变更日志中的 PR 链接回填。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32720253003 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32720253003 |
…agentic-prefill-scheduling # Conflicts: # perf-changelog.yaml
更新 B200 DSV4 AgentX 高并发性能曲线配置。
简化 B200 DSV4 AgentX 启动参数分支。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32924641821 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32930559234 |
统一使用 0.90 的静态显存比例，为 B200 FP4 indexer 保留瞬时工作区。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32958611057 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32958611057 |
将 c96 纳入较小的分块预填充预算和紧凑 CUDA Graph 配置，以避免 FP4 索引器工作区内存不足。
同步最新主分支，保留 PR 2718 的 B200 SGLang AgentX 配置和性能变更记录。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33043221347 |
将 TP8 HiCache 比例降至 2.75，同时保留 DEP8 比例 8，避免 NScale 1.7 TiB 主机内存上的 V4 分页池初始化失败。\n\n改用支持小数的比例上限检查，并在性能变更日志中记录实测的主机内存限制。
同步最新主分支，并按追加式规则保留 PR 2718 的性能变更日志条目在文件末尾。\n\n合并主分支新增的 Qwen3.8-Flash-Next 配置，同时保留 NScale TP HiCache 主机内存修复。
再次同步快速更新的主分支，并保留 PR 2718 的性能变更日志条目在最新主分支条目之后。\n\n该同步包含 Qwen3.8 B300 变更日志链接修正，不改变 B200 NScale 调优配置。
1 participant
[by Codex]
Refresh the B200 DeepSeek V4 SGLang AgentX submission with validated high-concurrency HiCache profiles:
--prefill-decode-intervalto DEP8 profiles only; use interval 24 for concurrency 64, 96, and 128.--chunked-prefill-size 49152,--cuda-graph-max-bs 32, and--load-balance-method total_requests.--mem-fraction-static 0.90across every TP and DP profile. The official c128 sweep showed that 0.91 left insufficient transient workspace for the FP4 indexer.Full-hour validation reached 34.587 tok/s/user and 43,382.65 tok/s/GPU at concurrency 128, and 28.916 tok/s/user and 50,468.23 tok/s/GPU at concurrency 160. No SGLang or AIPerf source changes are required.
使用经过验证的高并发 HiCache 配置刷新 B200 DeepSeek V4 SGLang AgentX 提交：
--prefill-decode-interval；并发 64、96 和 128 使用 interval 24。--chunked-prefill-size 49152、--cuda-graph-max-bs 32和--load-balance-method total_requests。--mem-fraction-static 0.90。官方 c128 sweep 表明，0.91 无法为 FP4 indexer 保留足够的瞬时工作区。完整一小时验证中，并发 128 达到 34.587 tok/s/user 和 43,382.65 tok/s/GPU，并发 160 达到 28.916 tok/s/user 和 50,468.23 tok/s/GPU。不需要修改 SGLang 或 AIPerf 源代码。
Note
Low Risk
Changes are limited to benchmark shell flags, YAML scenario matrices, and changelog documentation with no production runtime or auth/data-path code.
Overview
Refreshes the B200 DeepSeek V4 SGLang AgentX benchmark recipe and nvidia-master search space for validated high-concurrency HiCache runs—benchmark/config only, no framework code changes.
HiCache: Replicated TP8 profiles default ratio 2.75 (fits ~1.7 TiB NScale hosts); DP8 keeps ratio 8. Ratio caps are validated with awk so fractional limits work.
DP-attention (DEP8): Drops
--enable-prefill-delayerin favor of--prefill-decode-interval(24 by default, 20 at conc 160). Router policy switches from consistent_hashing to cache_aware, with--balance-abs-threshold 32at conc 160.--mem-fraction-static 0.90applies to all profiles. At conc 96/128/160, chunked prefill shrinks to preserve FP4 indexer HBM; conc 128/160 add total_requests DP balancing, 1s load snapshots, and conc 96+ cap--cuda-graph-max-bsat 32. Adds multithreaded model load via--model-loader-extra-config.Config sweep: Non-DP HiCache conc list tops out at 16; DP HiCache runs at 64, 96, 128, 160 with the sglang router. perf-changelog documents the tuning rationale.
Reviewed by Cursor Bugbot for commit e128497. Bugbot is set up for automated code reviews on this repo. Configure here.