Product-Aware Deterministic Rounding for Quantized Matrix Multiplication
Abstract
Scalar rounding decisions interact through matrix multiplication. We study deterministic product-aware rounding after scales, clipping bounds, and grids are fixed, with each active scalar choosing between adjacent levels. For dynamic activation rounding, null-space reduction preserves the relaxed product while leaving at most r fractional decisions, where r is the rank of the active gap-weighted weight block. Conditional-expectation completion gives a deterministic polynomial-time algorithm with squared product error at most $ \mathrm{OPT}_{\mathrm{dyn}}+rν_{\max}^2/4$, where OPTdyn is the best admissible error and νmax is the largest row norm of that block. For reusable static weights, the exact expected product-loss metric is the uncentered input second moment with fixed output bias; free bias recalibration yields the centered covariance. Exact optimization is NP-hard even at rank one. In balanced blocks with K=1024 and r=16, conditional- expectation completion attains a dither-normalized median error of $0.010$, compared with $0.899$ for round-to-nearest. Clipping-aware initialization reduces median normalized error by a factor of $43.4$ at ten-percent clipping. Held-out Digits experiments show that retaining the input mean or correcting the output bias improves median product error over round-to-nearest in all four tested bit-width and calibration-size settings.
Related
- Information Complexity of Inner Product and Matrix Multiplication via Shannon-to-Min-Entropy Conversion
- Approximate Matrix Multiplication through Joint Leverage Score Sampling
- Four-Level Quantized Soft Information Aided Hard-Decision Decoder for Product-like Codes
- Unbiased Low-Rank Approximation for Repeated Matrix-Vector Multiplication: Stability and Optimal Sampling
- Tensor Degenerations for Matrix Multiplication
BibTeX
@misc{sao2026productaware,
title = {{Product-Aware Deterministic Rounding for Quantized Matrix Multiplication}},
author = {Piyush Sao and Narasinga Miniskar and Pedro Valero-Lara and Keita Teranishi and Sudip Seal},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.31641},
eprint = {2609.31641},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.31641},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Product-Aware Deterministic Rounding for Quantized Matrix Multiplication
AU - Piyush Sao
AU - Narasinga Miniskar
AU - Pedro Valero-Lara
AU - Keita Teranishi
AU - Sudip Seal
PY - 2026
DA - 2026/09/11
PB - arXiv
DO - 10.48550/arxiv.2609.31641
UR - https://arxiv.org/abs/2609.31641
AB - Scalar rounding decisions interact through matrix multiplication. We study deterministic product-aware rounding after scales, clipping bounds, and grids are fixed, with each active scalar choosing between adjacent levels. For dynamic activation rounding, null-space reduction preserves the relaxed product while leaving at most r fractional decisions, where r is the rank of the active gap-weighted weight block. Conditional-expectation completion gives a deterministic polynomial-time algorithm with squared product error at most $ \mathrm{OPT}_{\mathrm{dyn}}+rν_{\max}^2/4$, where OPT_dyn is the best admissible error and ν_max is the largest row norm of that block. For reusable static weights, the exact expected product-loss metric is the uncentered input second moment with fixed output bias; free bias recalibration yields the centered covariance. Exact optimization is NP-hard even at rank one. In balanced blocks with K=1024 and r=16, conditional- expectation completion attains a dither-normalized median error of $0.010$, compared with $0.899$ for round-to-nearest. Clipping-aware initialization reduces median normalized error by a factor of $43.4$ at ten-percent clipping. Held-out Digits experiments show that retaining the input mean or correcting the output bias improves median product error over round-to-nearest in all four tested bit-width and calibration-size settings.
ER -