Phases of Muon: When Muon Eclipses SignSGD
Abstract
Muon and related spectral optimizers have recently demonstrated strong empirical performance as scalable stochastic methods, often outperforming Adam. Yet their behavior remains poorly understood. We analyze stochastic Muon, which orthogonalizes an exponential moving average of minibatch gradients, on a high-dimensional matrix-valued least squares problem, and we compare it with stochastic SignSGD, a simple baseline comparable to Adam. We derive explicit deterministic dynamics that provide a tractable framework for studying learning behavior. We show that Muon has non-linear sensitivity to the effective batch Beff=B(1+β)/(1−β). With exact whitening and small Beff Muon fails to resolve smaller eigenmodes, slowing down convergence. Momentum and finite Newton--Shulz iterations can act as a ridge which lessen this effect. This behavior is in contrast with SignSGD which has no transition, leading to different optimal learning rates, but with matching convergence rate up to a constant factor in the isotropic setting. For anisotropic data, large batch setting Muon performs a square-root preconditioning with respect to the data covariance spectrum while SignSGD provides no preconditioning for generic covariances. We analyze time to target loss in a power law covariance model and derive a phase diagram in the data exponent α and target exponent ζ. Muon is favored for steep covariance spectra with rich targets (α>2, ζ<α+1), SignSGD is favored for flat spectra (α<1), and in other regions the winner depends on how the target loss scales with dimension. Our results shed light on why replacing Adam with Muon is not trivial even from a pure optimization perspective.
Related
- When Is Muon Optimal? Radial Fragility and Minimax Geometry
- MUON-RT Reusing the Teacher's Trajectory for Momentum-Free Muon for LLM Self-Distillation
- Muon-C: Operator-Aligned Muon for Convolutional Kernels
- Spacetimes of maximal proper acceleration and muon mean lifetime
- Development of a Physics-Informed Neural Framework, MEOWN, for Rapid Prediction of Muon Stopping Sites in Crystalline Materials, for understanding Quantum Magnet employing Muon Spectroscopy
BibTeX
@misc{paquette2026phases,
title = {{Phases of Muon: When Muon Eclipses SignSGD}},
author = {Elliot Paquette and Noah Marshall and Lucas Benigni and Guangyuan Wang and Atish Agarwala and Courtney Paquette},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.09552},
eprint = {2605.09552},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.09552},
}
RIS
TY - UNPB
TI - Phases of Muon: When Muon Eclipses SignSGD
AU - Elliot Paquette
AU - Noah Marshall
AU - Lucas Benigni
AU - Guangyuan Wang
AU - Atish Agarwala
AU - Courtney Paquette
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2605.09552
UR - https://arxiv.org/abs/2605.09552
AB - Muon and related spectral optimizers have recently demonstrated strong empirical performance as scalable stochastic methods, often outperforming Adam. Yet their behavior remains poorly understood. We analyze stochastic Muon, which orthogonalizes an exponential moving average of minibatch gradients, on a high-dimensional matrix-valued least squares problem, and we compare it with stochastic SignSGD, a simple baseline comparable to Adam. We derive explicit deterministic dynamics that provide a tractable framework for studying learning behavior. We show that Muon has non-linear sensitivity to the effective batch B_eff=B(1+β)/(1−β). With exact whitening and small B_eff Muon fails to resolve smaller eigenmodes, slowing down convergence. Momentum and finite Newton--Shulz iterations can act as a ridge which lessen this effect. This behavior is in contrast with SignSGD which has no transition, leading to different optimal learning rates, but with matching convergence rate up to a constant factor in the isotropic setting. For anisotropic data, large batch setting Muon performs a square-root preconditioning with respect to the data covariance spectrum while SignSGD provides no preconditioning for generic covariances. We analyze time to target loss in a power law covariance model and derive a phase diagram in the data exponent α and target exponent ζ. Muon is favored for steep covariance spectra with rich targets (α>2, ζ<α+1), SignSGD is favored for flat spectra (α<1), and in other regions the winner depends on how the target loss scales with dimension. Our results shed light on why replacing Adam with Muon is not trivial even from a pure optimization perspective.
ER -