Q-MINO: A Minimal-Norm Method for Quantization-Aware Training
Abstract
The Straight-Through Estimator (STE) is a widely used heuristic for Quantization-Aware Training (QAT), but its surrogate gradients can exhibit substantial mismatch with the underlying quantized objective, leading to noisy updates and parameter oscillations, particularly in ultra-low-bit regimes. We propose the Quantization-Aware Minimal-Norm Optimizer (Q-MINO), a temporal bundle method that combines gradient consensus, state-drift regularization, and an alignment constraint to construct stabilized, minimum-norm update directions from recent optimization states. Q-MINO solves the resulting constrained subproblem using a warm-started Frank--Wolfe procedure with a feasible fallback initialization. Theoretically, via a stochastic Lyapunov Kurdyka--Łojasiewicz (KL) framework, we show that Q-MINO achieves asymptotic neighborhood convergence. Moreover, we detail numerical experiments with Q-MINO at various quantizations.
Related
- QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction
- Beyond Shadow Weights: Quantization-Aware Training as Quantized-Endpoint Descent
- Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement
- Minimal Lagrangian surfaces in the two-dimensional complex hyperbolic quadric via the loop group method
- Should I Use This Synthetic Dataset for Training? How to Test with Minimal Real Data
BibTeX
@misc{li2026qmino,
title = {{Q-MINO: A Minimal-Norm Method for Quantization-Aware Training}},
author = {Don Li},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.00738},
eprint = {2610.00738},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.00738},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Q-MINO: A Minimal-Norm Method for Quantization-Aware Training
AU - Don Li
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2610.00738
UR - https://arxiv.org/abs/2610.00738
AB - The Straight-Through Estimator (STE) is a widely used heuristic for Quantization-Aware Training (QAT), but its surrogate gradients can exhibit substantial mismatch with the underlying quantized objective, leading to noisy updates and parameter oscillations, particularly in ultra-low-bit regimes. We propose the Quantization-Aware Minimal-Norm Optimizer (Q-MINO), a temporal bundle method that combines gradient consensus, state-drift regularization, and an alignment constraint to construct stabilized, minimum-norm update directions from recent optimization states. Q-MINO solves the resulting constrained subproblem using a warm-started Frank--Wolfe procedure with a feasible fallback initialization. Theoretically, via a stochastic Lyapunov Kurdyka--Łojasiewicz (KL) framework, we show that Q-MINO achieves asymptotic neighborhood convergence. Moreover, we detail numerical experiments with Q-MINO at various quantizations.
ER -