Blind Noise Equalization for Large Data Matrices under General Variance Profiles
Abstract
Low-rank approximation is a basic tool for inference from large and noisy data matrices. However, when noise variances differ across entries, a setting known as heteroskedasticity, the noise can distort the observed spectrum, producing spurious components and obscuring weak signals. To address these challenges, we develop BRINE (Blind Resolvent-based Iterative Noise Equalizer), a data-driven procedure that iteratively rescales rows and columns to equalize the average noise variance across both dimensions. Under suitable conditions, this procedure restores the Marchenko-Pastur law for the noise spectrum and can amplify signal components without prior knowledge of the signal or noise structure. A key idea underlying our approach is to equalize the diagonal entries of the data resolvent as a proxy for equalizing the unknown noise variances. At the population level, we identify a potential function that characterizes this equalization and motivates an iterative minimization procedure. We prove convergence of the iterations to the true equalizer for arbitrary positive variance profiles, with a geometric rate under mild conditions. The updates can be estimated directly from noisy data, and we establish probabilistic error bounds for the scaling factors. Simulations and experiments across diverse biological data modalities and preprocessing pipelines illustrate the effectiveness of our approach.
Related
- Nonparametric shrinkage estimation of high-dimensional variance component matrices
- Discrete total variation under the multinomial, Part 4: variance under a general distribution
- Statistics of entries of matrices from the general linear group over a finite field
- Minor-Order Exponent Profiles of Oscillatory Matrices
- Exponent Profiles of Oscillatory Matrices under Matrix and Inverse Sparsity
BibTeX
@misc{landa2026blind,
title = {{Blind Noise Equalization for Large Data Matrices under General Variance Profiles}},
author = {Boris Landa and Francesco Strino and Fabio Parisi and Ruiqi Li and Yuval Kluger},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.08784},
eprint = {2611.08784},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.08784},
}
RIS
TY - UNPB
TI - Blind Noise Equalization for Large Data Matrices under General Variance Profiles
AU - Boris Landa
AU - Francesco Strino
AU - Fabio Parisi
AU - Ruiqi Li
AU - Yuval Kluger
PY - 2026
DA - 2026/11/05
PB - arXiv
DO - 10.48550/arxiv.2611.08784
UR - https://arxiv.org/abs/2611.08784
AB - Low-rank approximation is a basic tool for inference from large and noisy data matrices. However, when noise variances differ across entries, a setting known as heteroskedasticity, the noise can distort the observed spectrum, producing spurious components and obscuring weak signals. To address these challenges, we develop BRINE (Blind Resolvent-based Iterative Noise Equalizer), a data-driven procedure that iteratively rescales rows and columns to equalize the average noise variance across both dimensions. Under suitable conditions, this procedure restores the Marchenko-Pastur law for the noise spectrum and can amplify signal components without prior knowledge of the signal or noise structure. A key idea underlying our approach is to equalize the diagonal entries of the data resolvent as a proxy for equalizing the unknown noise variances. At the population level, we identify a potential function that characterizes this equalization and motivates an iterative minimization procedure. We prove convergence of the iterations to the true equalizer for arbitrary positive variance profiles, with a geometric rate under mild conditions. The updates can be estimated directly from noisy data, and we establish probabilistic error bounds for the scaling factors. Simulations and experiments across diverse biological data modalities and preprocessing pipelines illustrate the effectiveness of our approach.
ER -