Constant Step Sizes for Muon-Type Spectral Optimizers on Strongly Convex Quadratics
Abstract
Muon and related optimizers update a weight matrix with a spectrally transformed gradient. In this work, we study such spectral optimizers with a constant step size on strongly convex matrix quadratics, which isolate the effect of the update from the confounds of nonconvexity, noise, and step size schedules. We analyze an update using two quantities: the gain, which is the factor by which it amplifies the gradient along each singular direction, and the length, which is the norm of the update. These two quantities place the popular spectral updates in three regimes with different behavior: (i) with a bounded gain, the iterates behave like preconditioned gradient descent and converge for step sizes set by the largest curvature; (ii) with an unbounded gain but a vanishing length, they still converge, with the admissible step sizes set by the total curvature instead; and (iii) with a non-vanishing length, they do not converge for almost every initialization. Interestingly, the practical variants of Muon fall in all three regimes, depending only on how the update is normalized and scaled and not on the polynomial that approximates the polar factor. Experiments validate these findings.
Related
- R-Linear Convergence of Barzilai-Borwein Methods on Strictly Convex Quadratics via Closed-Cone Homogeneous Dynamics
- Global and Unconditional R-Linear Convergence of Rank-Compressed Weighted LMSD Sweeps for Strictly Convex Quadratics
- Doubly exponential convergence of the cyclic steepest descent method for strictly convex quadratics in arbitrary dimensions
- Strongly linearly convex exhaustion of a class of ℂ-convex domains
- Silver Rate Is (Almost) Optimal for Gradient Descent: The Strongly Convex Case
BibTeX
@misc{jawanpuria2026constant,
title = {{Constant Step Sizes for Muon-Type Spectral Optimizers on Strongly Convex Quadratics}},
author = {Pratik Jawanpuria and Bhabatosh Garai and Bamdev Mishra},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.27747},
eprint = {2610.27747},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.27747},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Constant Step Sizes for Muon-Type Spectral Optimizers on Strongly Convex Quadratics
AU - Pratik Jawanpuria
AU - Bhabatosh Garai
AU - Bamdev Mishra
PY - 2026
DA - 2026/10/20
PB - arXiv
DO - 10.48550/arxiv.2610.27747
UR - https://arxiv.org/abs/2610.27747
AB - Muon and related optimizers update a weight matrix with a spectrally transformed gradient. In this work, we study such spectral optimizers with a constant step size on strongly convex matrix quadratics, which isolate the effect of the update from the confounds of nonconvexity, noise, and step size schedules. We analyze an update using two quantities: the gain, which is the factor by which it amplifies the gradient along each singular direction, and the length, which is the norm of the update. These two quantities place the popular spectral updates in three regimes with different behavior: (i) with a bounded gain, the iterates behave like preconditioned gradient descent and converge for step sizes set by the largest curvature; (ii) with an unbounded gain but a vanishing length, they still converge, with the admissible step sizes set by the total curvature instead; and (iii) with a non-vanishing length, they do not converge for almost every initialization. Interestingly, the practical variants of Muon fall in all three regimes, depending only on how the update is normalized and scaled and not on the polynomial that approximates the polar factor. Experiments validate these findings.
ER -