ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights
Abstract
We introduce ShamAN-Q, a sub-1-bit post-training quantization method that extends NanoQuant by replacing each its diagonal reconstruction geometry with a tractable dense curvature metric, using a general paradigm popularized by the Shampoo optimizer. For each linear weight, ShamAN-Q fits a Kronecker product to the empirical Fisher information matrix of a small calibration set by Kullback--Leibler minimization, forming a Mahalanobis reconstruction loss from the result. The continuous ADMM updates from NanoQuant become solutions to Sylvester equations, while its discrete projection and deployment format remain unchanged. Because the curvature is local to a given set of weights, ShamAN-Q re-measures the input curvature statistic for each layer immediately before layer factorization, periodically refreshing all statistics on the partially quantized model. ShamAN-Q also redistributes the uniform rank from NanoQuant across layers at the same total number of bits. On Qwen3-Base, ShamAN-Q lowers WikiText-2 perplexity at ≈1 bpw from 27.56 to 22.96 (0.6B), 19.21 to 16.72 (1.7B), and 14.29 to 13.80 (4B) while matching or improving zero-shot accuracy on the Eleuther LM Evaluation Harness. On 0.6B, ShamAN-Q at ≈0.8 bpw matches the published perplexity of NanoQuant at ≈1.0 bpw.
Related
- Minimax Optimality of Shampoo for Nonstationary Stochastic Optimization
- A Statistical Viewpoint on Adam, Shampoo and Muon
- The reverse Hölder inequality for 𝒜p(·) weights with applications to matrix weights
- Selberg sieve weights and sign changes of Kloosterman sums. I. Uniform asymptotics for shifted weights
- On weighted estimates involving singular weights for the Dirac equation
BibTeX
@misc{mei2026shamanq,
title = {{ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights}},
author = {Jonathan Mei and Sang Hyub Kim and Oliver Knitter and Chi Chen and Martin Roetteler},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38521},
eprint = {2609.38521},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38521},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - ShamAN-Q: Shampoo Augmented NanoQuant for Sub-1-bit LLM Weights
AU - Jonathan Mei
AU - Sang Hyub Kim
AU - Oliver Knitter
AU - Chi Chen
AU - Martin Roetteler
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.38521
UR - https://arxiv.org/abs/2609.38521
AB - We introduce ShamAN-Q, a sub-1-bit post-training quantization method that extends NanoQuant by replacing each its diagonal reconstruction geometry with a tractable dense curvature metric, using a general paradigm popularized by the Shampoo optimizer. For each linear weight, ShamAN-Q fits a Kronecker product to the empirical Fisher information matrix of a small calibration set by Kullback--Leibler minimization, forming a Mahalanobis reconstruction loss from the result. The continuous ADMM updates from NanoQuant become solutions to Sylvester equations, while its discrete projection and deployment format remain unchanged. Because the curvature is local to a given set of weights, ShamAN-Q re-measures the input curvature statistic for each layer immediately before layer factorization, periodically refreshing all statistics on the partially quantized model. ShamAN-Q also redistributes the uniform rank from NanoQuant across layers at the same total number of bits. On Qwen3-Base, ShamAN-Q lowers WikiText-2 perplexity at ≈1 bpw from 27.56 to 22.96 (0.6B), 19.21 to 16.72 (1.7B), and 14.29 to 13.80 (4B) while matching or improving zero-shot accuracy on the Eleuther LM Evaluation Harness. On 0.6B, ShamAN-Q at ≈0.8 bpw matches the published perplexity of NanoQuant at ≈1.0 bpw.
ER -