Skip to content

fix(muon): avoid EP/FSDP AGRS OOM - #2069

Open
jayhenry wants to merge 2 commits into
InternLM:mainfrom
jayhenry:fix_muon_oom
Open

fix(muon): avoid EP/FSDP AGRS OOM#2069
jayhenry wants to merge 2 commits into
InternLM:mainfrom
jayhenry:fix_muon_oom

Conversation

@jayhenry

@jayhenry jayhenry commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Expose MuonConfig.remainder_strategy="pad_all2all" as a temporary workaround for remainder batches when all-to-all is available.
  • Fix the global shard dimension passed to AGRS and all-to-all when EP and FSDP shard the same tensor dimension.
  • Add a safe mechanism-level reproduction, an EP+FSDP regression test, and update xtuner_muon_oom.md.

Root cause

For GLM-5.2 with 512 ranks and EP8, the expert FSDP group has 64 ranks. The 76 same-shaped fused_w2 parameters form a remainder batch of 12. Before this fix, AGRS used the full DTensor dimension (256 * 6144) instead of the EP-local dimension (256 * 6144) / 8, producing a 72 GiB buffer and a second 72 GiB flatten copy.

Validation

  • PYTHONPATH=. python -m pytest tests/optim/test_muon.py -q — 12 passed.
  • CPU and 4-rank CUDA runs of .dev_scripts/repro_muon_agrs_ep_oom.py completed successfully under the local GPU lock.
  • The new regression test fails on the parent revision (Newton-Schulz sees 12 rows) and passes with this change (6 rows).

The two commits are intentionally separate: the first is the temporary configuration workaround, and the second is the permanent dimension fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant