Skip to content

Feat: pack the residue columns of the ep_pp_dp packing into two-segment pairs - #1764

Merged
majieyue merged 1 commit into
intelligent-machine-learning:masterfrom
majieyue:feat/rerank-two-way-mixed-columns
Nov 17, 2026
Merged

majieyue merged 1 commit into
intelligent-machine-learning:masterfrom
majieyue:feat/rerank-two-way-mixed-columns

Conversation

@majieyue

@majieyue majieyue commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

The ep_pp_dp topology-rerank packing fills each PP column with the
largest-remaining psw group first. That works well while groups can own
whole columns, but once every group holds fewer than pp slots the
greedy starts a column from the big fragment and then chips the 4-slot
piece off the NEXT group that still has slots — which leaves the last
residue columns with no 8-slot pairing partner and forces an avoidable
THREE-segment column.

Assuming case (9 psws x 128 nodes, pp=12,
ep_workers=4, N=1152): 18 pure columns consume 24 slots of every psw,
leaving 9 x 8 residue slots to fill exactly 6 residue columns. The
online rerank logged one mixed column as
{SG3: 4, SG5: 4, SG8: 4} (three psw groups in one PP column → 2 extra
stage boundaries), while the residue profile admits a strictly two-way
packing: 6 columns of [8+4] — 6 groups exhaust themselves as the 8-slot
half and 3 groups are spent as a 4-slot "double-small" half in each of
two adjacent columns.

Changes

_ep_pp_dp_group_sequence (master/elastic_training/topology_rerank.py),
residue column filling only — whole (pure) columns, the EP-slot straddle
accounting and the crumb fill are untouched.

  • Whole-column phase (some group holds at least all the slots the
    column still needs): unchanged — it owns the column alone.
  • Residue phase: accumulate the column from the SMALLEST remaining
    group, preferring an exact fit for what the column still needs (ties
    by the smallest group id). Every fragment group is consumed inside at
    most two columns, so residue columns stay strictly two-segment
    whenever such a packing exists; otherwise the loop degrades to the
    plain smallest-first accumulation that still terminates with a valid
    (best-effort) layout.

Each take provably consumes ≥1 slot: the smallest-first selection is
made over groups with remaining > 0 only (the old max(...) never
picked an empty group, but carrying that generic choice into the
residue phase would loop when it selects a drained key).

Layout effect (backtest on the ling-max shape)

Scenario old greedy new packing
9 psws x 128, pp=12, ep_workers=4 18 pure + 5 x [8+4] + 1 x 3-segment, 7 boundaries 18 pure + 6 x [4|8] two-segment, 6 boundaries
same shape with effective EP=128 (slot = 16 nodes) 5 x [8+4] + 1 x 3-segment, 7 boundaries 6 x [4|8], 6 boundaries
one segment shrinks to 64 nodes + a default group 5 plus 1 mixed triples, 7 boundaries 6 x two-segment, 6 boundaries
drifted psws (127/129, crumb path) n/a unchanged — the crumb-level 1/288 EP straddle is a node-granularity effect, out of scope

Per-pipeline-stage DP composition also becomes smoother: every psw
contributes a steady 2-3 slots to each stage row instead of the old
3/3/2/3/3/... uneven counts.

EP behavior is unchanged: every EP slot stays inside one psw group
(0/288 and 0/72 straddles in all ideal layouts, before and after).

Tests

  • test_topology_rerank.py: the 26 pre-existing sequence-level pins are
    byte-for-byte unchanged (pure columns, crumb fill, straddle
    reporting, creation-order invariants); the new regression test
    test_ep_pp_dp_residues_pack_into_two_segment_columns locks the
    ling-max shape: every EP slot intra-group, every column ≤2 segments,
    exactly 6 mixed columns and exactly 6 crossing boundaries.
  • test_node_group_strategy / test_job_manager / test_soft_group /
    test_pod_scaler all at baseline.

@codecov

codecov Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.27%. Comparing base (4cdbdb3) to head (bd79547).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1764      +/-   ##
==========================================
- Coverage   82.30%   82.27%   -0.03%     
==========================================
  Files         255      255              
  Lines       25435    25439       +4     
==========================================
- Hits        20935    20931       -4     
- Misses       4500     4508       +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

…nt pairs

The ep_pp_dp greedy took residue slots from the largest-remaining psw
group first, which drifted the last leftover columns into a psw mix of
3+ segments (e.g. the ling-max job showed [SG3|SG5|SG8] in one PP
column). Residue sizes cannot always own a whole column, so accumulate
each residue column from the SMALLEST remaining group (exact fit
preferred) instead: every fragment group is then spent inside at most
two columns and the residue columns stay strictly two-segment whenever
the sizes allow it.

For 9 psws x 8 residue slots (pp=12, ep_workers=4) this packs the
residues into 6 [4|8] columns (6 PP boundaries) instead of 5 mixed
columns + 1 three-segment column (7 boundaries); the pure columns, the
EP-slot straddle handling and the crumb fill are unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
@majieyue
majieyue force-pushed the feat/rerank-two-way-mixed-columns branch from ba07293 to bd79547 Compare November 16, 2026 18:40

@duck7216 duck7216 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@majieyue
majieyue merged commit a177312 into intelligent-machine-learning:master Nov 17, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants