RoLaQ: Joint Rotation and Range Calibration for 2-Bit KV Caches in Multi-Head Latent Attention
Abstract
Multi-head latent attention (MLA) compresses the content of a key-value cache into a shared latent per token, making low-bit storage attractive for long-context inference. At two bits, errors in this representation perturb both attention matching and value readout, with effects that depend on error direction and reconstruction scale. We introduce RoLaQ (Rotation-calibrated Latent Quantization), an offline method for joint rotation and range calibration. RoLaQ refines a shared rotation within subspaces defined by the frozen key and value projections, and jointly learns quantization ranges whose thresholds and reconstruction gains are coupled to control the shared signal's response. Prediction matching and attention-output supervision guide calibration with model weights frozen. Across three MLA models and six benchmarks, RoLaQ achieves the highest average accuracy among the compared 2-bit methods both without and with token protection. On DeepSeek-V2-Lite, an SGLang implementation provides 3.96 times the BF16 KV capacity under the same memory budget and 2.51 times its decode throughput with 256K input tokens.
Related
- In-context classification: Calibration and noise in linear attention with softmax readout
- Diagonalized Attention for Individualized Regression: Latent-Row Localization and Prediction
- Tensor Decomposition of Transformer Key-Value Caches: Spectral Structure and Format Comparison
- On the Sobolev stability threshold for 3D Navier-Stokes equations with rotation near the Couette flow
- Rotation groups, mediangle graphs, and periagroups: A unified point of view on Coxeter groups and graph products of groups
BibTeX
@misc{ji2026rolaq,
title = {{RoLaQ: Joint Rotation and Range Calibration for 2-Bit KV Caches in Multi-Head Latent Attention}},
author = {Shiyu Ji and Peng Ling and Chen Lin and Shupeng Li and Yixuan Wang and Qingfu Zhu and Wanxiang Che},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.20686},
eprint = {2610.20686},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.20686},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - RoLaQ: Joint Rotation and Range Calibration for 2-Bit KV Caches in Multi-Head Latent Attention
AU - Shiyu Ji
AU - Peng Ling
AU - Chen Lin
AU - Shupeng Li
AU - Yixuan Wang
AU - Qingfu Zhu
AU - Wanxiang Che
PY - 2026
DA - 2026/10/14
PB - arXiv
DO - 10.48550/arxiv.2610.20686
UR - https://arxiv.org/abs/2610.20686
AB - Multi-head latent attention (MLA) compresses the content of a key-value cache into a shared latent per token, making low-bit storage attractive for long-context inference. At two bits, errors in this representation perturb both attention matching and value readout, with effects that depend on error direction and reconstruction scale. We introduce RoLaQ (Rotation-calibrated Latent Quantization), an offline method for joint rotation and range calibration. RoLaQ refines a shared rotation within subspaces defined by the frozen key and value projections, and jointly learns quantization ranges whose thresholds and reconstruction gains are coupled to control the shared signal's response. Prediction matching and attention-output supervision guide calibration with model weights frozen. Across three MLA models and six benchmarks, RoLaQ achieves the highest average accuracy among the compared 2-bit methods both without and with token protection. On DeepSeek-V2-Lite, an SGLang implementation provides 3.96 times the BF16 KV capacity under the same memory budget and 2.51 times its decode throughput with 256K input tokens.
ER -