Information-Theoretic Limits for Rectangular Spiked Matrix Models with Rotationally Invariant Noise
Abstract
We study the information-theoretic limits of inference in rectangular rank-one spiked matrix models with additive rotationally invariant noise, thereby going beyond the standard assumption of independent noise entries. Using the replica method from statistical physics, we derive a variational formula for the mutual information between the signal and the data, together with a system of saddle point equations predicting the minimum mean squared error in estimating the rank-one spike. We further show that, in the Bayes-optimal setting, these saddle point equations coincide with the fixed point equations of the state evolution of a recently proposed orthogonal approximate message passing algorithm with Bayes-optimal denoisers. This equivalence provides an algorithmic interpretation of the replica prediction and supports the Bayes-optimality of orthogonal approximate message passing whenever the relevant fixed point is unique. Numerical evaluations of the saddle point equations for several rotationally invariant noise ensembles and signal priors, together with finite-size simulations of the algorithm, confirm the asymptotic predictions. They illustrate both regimes with a unique informative fixed point, where orthogonal approximate message passing is predicted to attain the minimum mean squared error, and regimes with multiple fixed points, where a statistical-to-computational gap exists.
Related
- Spectral Dependence of Convex Regularization: Fundamental Limits under Right-Rotationally Invariant Designs
- Fundamental Limits of Information-Theoretic Secure Aggregation with User Dropouts in Federated Learning
- On the Information-Theoretic Limits of Latent-Space Watermarking Through Pretrained Generators
- Finite-rank multiplicative perturbations of rotationally invariant non-Hermitian random matrices
- Spectral properties of deterministic matrices multiplied by rotationally invariant random non-Hermitian ensembles
BibTeX
@misc{chen2026informationtheoretic,
title = {{Information-Theoretic Limits for Rectangular Spiked Matrix Models with Rotationally Invariant Noise}},
author = {Qun Chen and Rudy Skerk and Jean Barbier and Francesco Camilli and Junjie Ma and Haochuan Zhang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.16797},
eprint = {2610.16797},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.16797},
}
RIS
TY - UNPB
TI - Information-Theoretic Limits for Rectangular Spiked Matrix Models with Rotationally Invariant Noise
AU - Qun Chen
AU - Rudy Skerk
AU - Jean Barbier
AU - Francesco Camilli
AU - Junjie Ma
AU - Haochuan Zhang
PY - 2026
DA - 2026/10/12
PB - arXiv
DO - 10.48550/arxiv.2610.16797
UR - https://arxiv.org/abs/2610.16797
AB - We study the information-theoretic limits of inference in rectangular rank-one spiked matrix models with additive rotationally invariant noise, thereby going beyond the standard assumption of independent noise entries. Using the replica method from statistical physics, we derive a variational formula for the mutual information between the signal and the data, together with a system of saddle point equations predicting the minimum mean squared error in estimating the rank-one spike. We further show that, in the Bayes-optimal setting, these saddle point equations coincide with the fixed point equations of the state evolution of a recently proposed orthogonal approximate message passing algorithm with Bayes-optimal denoisers. This equivalence provides an algorithmic interpretation of the replica prediction and supports the Bayes-optimality of orthogonal approximate message passing whenever the relevant fixed point is unique. Numerical evaluations of the saddle point equations for several rotationally invariant noise ensembles and signal priors, together with finite-size simulations of the algorithm, confirm the asymptotic predictions. They illustrate both regimes with a unique informative fixed point, where orthogonal approximate message passing is predicted to attain the minimum mean squared error, and regimes with multiple fixed points, where a statistical-to-computational gap exists.
ER -