[GLM-5.2] Improve optimizer and checkpoint coverage

[jayhenry]

Summary

• Allow the GLM-5.2 SFT example to select AdamW or Muon through OPTIMIZER while keeping AdamW as the default.
• Add FSDP2 + EP4 + MTP HuggingFace checkpoint round-trip coverage.
• Freeze DSAIndexer parameters because its forward path runs under no_grad, preventing DCP resume from expecting optimizer states that are absent from trained checkpoints.
• Collect temporary DCP objects, release cached device memory, and report per-rank allocated/reserved/free memory after resume.
• Add LoadCheckpointConfig.offload_optimizer_first_step as an opt-in control for cold resume memory: Trainer moves optimizer state to CPU after checkpoint load and moves it back to the training device after the first gradient reduction, using TrainEngine.put_optimizer_to_device directly.

Root cause

A fresh training process creates Adam states only after its cold first forward/backward. A resumed process instead loads those states before that cold step, adding about 25.8 GB per rank to the compilation/backward peak. The resulting memory pressure first surfaced in NCCL gradient all-reduce; DeepEP and watchdog timeouts were downstream failures. Emptying the allocator cache was valid but insufficient because the restored optimizer tensors are live state rather than cache.

Testing

• tests/engine/test_glm52_moe_train_engine.py::TestGlm52CheckpointEngine::test_dcp_round_trip_preserves_model_and_optimizer: passed on two H200 GPUs. The real GLM-5.2 EP2 engine loads DCP state, keeps optimizer tensors on CPU through forward/backward/all-reduce, moves them back to CUDA, and completes a valid AdamW step.
• tests/train/test_trainer.py::test_resume_and_load_checkpoint_cfg: passed; the enabled switch produces exactly one cpu -> cuda transfer around the first resumed step.
• tests/train/test_trainer.py::TestTrainerSaveHF::test_resume: passed; the default-disabled path preserves existing resume behavior.
• Ruff lint/format and repository pre-commit checks, including mypy: passed.

These changes make GLM-5.2 checkpoint resume match both the no-grad DSA execution path and the memory lifecycle of a fresh process while keeping first-step optimizer offload explicitly configurable.

[jayhenry]

@claude review

[jayhenry]

@claude review code, and approve if it's fine