Muon-C: Operator-Aligned Muon for Convolutional Kernels
Abstract
Muon replaces matrix momentum with an approximately orthogonal polar direction, but its geometry depends on the matrix representation. For convolution, standard unfolding describes a local patch map rather than the convolution operator. We introduce Muon-C, an operator-aligned optimizer that represents kernel momentum as frequency-wise channel-transfer matrices, polarizes these blocks independently, and uses a critical Fourier grid to return updates exactly to the original finite kernel support. We show that the new geometry arises from combining the block partition and Fourier coordinates. The exact-polar direction is a linear minimization oracle under the critically sampled convolution norm. Its worst-case guarantee relative to the continuous convolution-operator norm is never weaker than unfolding and is strictly stronger for 3×3 kernels. On CIFAR-10 flow matching with matched applied-update RMS, Muon-C reaches 9.87 FID at 40k iterations, compared with 22.26 for unfolded Muon and 51.31 for Adam. It reaches their final quality using 0.62× and 0.64× their model FLOPs, respectively. Under equal tuning budgets, Muon-C achieves 3.42 FID. Gains persist across data scales and transfer to classification across convolutional architectures.
Related
- Regularization and compression of integral operator kernels with tensor paraproducts
- Hierarchical Time Series Forecasting with Operator-Valued Kernels
- A Degeneracy-Aligned and Discrete Maximum-Principle- Compatible Peridynamic Differential Operator for the Heston Stochastic Volatility Model
- A Nonlinear Physics-based Reduced Order Model with Convolutional-based Operator Compression
- Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels
BibTeX
@misc{qing2026muonc,
title = {{Muon-C: Operator-Aligned Muon for Convolutional Kernels}},
author = {Jiaxin Qing and Lexin Li},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.09676},
eprint = {2609.09676},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.09676},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Muon-C: Operator-Aligned Muon for Convolutional Kernels
AU - Jiaxin Qing
AU - Lexin Li
PY - 2026
DA - 2026/09/09
PB - arXiv
DO - 10.48550/arxiv.2609.09676
UR - https://arxiv.org/abs/2609.09676
AB - Muon replaces matrix momentum with an approximately orthogonal polar direction, but its geometry depends on the matrix representation. For convolution, standard unfolding describes a local patch map rather than the convolution operator. We introduce Muon-C, an operator-aligned optimizer that represents kernel momentum as frequency-wise channel-transfer matrices, polarizes these blocks independently, and uses a critical Fourier grid to return updates exactly to the original finite kernel support. We show that the new geometry arises from combining the block partition and Fourier coordinates. The exact-polar direction is a linear minimization oracle under the critically sampled convolution norm. Its worst-case guarantee relative to the continuous convolution-operator norm is never weaker than unfolding and is strictly stronger for 3×3 kernels. On CIFAR-10 flow matching with matched applied-update RMS, Muon-C reaches 9.87 FID at 40k iterations, compared with 22.26 for unfolded Muon and 51.31 for Adam. It reaches their final quality using 0.62× and 0.64× their model FLOPs, respectively. Under equal tuning budgets, Muon-C achieves 3.42 FID. Gains persist across data scales and transfer to classification across convolutional architectures.
ER -