Skip to content

[Fix] Improve MTP compatibility and mitigate MoE load imbalance by skipping pad tokens - #1952

Open
tina-wen wants to merge 5 commits into
InternLM:mainfrom
tina-wen:mem-opt-bugfix
Open

[Fix] Improve MTP compatibility and mitigate MoE load imbalance by skipping pad tokens#1952
tina-wen wants to merge 5 commits into
InternLM:mainfrom
tina-wen:mem-opt-bugfix

Refine MTP recomputation and FSDP scheduling

f7efe1e
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs