PreprintCC BY 4.0arXiv
The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression
Abstract
Over-parameterized linear regression has been widely studied over the last decade. However, most existing works assume that the covariates are independent and that their covariance matrices are non-degenerate. In this paper, we relax both assumptions and derive deterministic equivalents for the prediction risk in a vanishing-ridge regime. We show that degeneracy of the covariance matrices and dependence can lead to multiple descent, and characterize where the corresponding peaks can occur. Our proofs use a novel graph representation of the variance profile. We show that maximum matchings and the Dulmage--Mendelsohn decomposition of the associated bipartite graph identify the configurations at which the variance becomes singular.
Related
- Finite-Sample Performance of Gradient Descent in Logistic Regression with Gaussian Design
- A Gaussian Process Regression-Enhanced Two-Sweep Parameterized Power Method for Computing Extreme Eigenpairs
- Sharp Generalization for Nonparametric Regression in Interpolation Space by Shallow Over-Parameterized Neural Networks with Channel Attention
- Gradient descent reaches arbitrary accuracy in a fixed number of large steps on multiclass logistic regression
- New Multiple Descent Phenomena Induced by Heterogeneous Data
BibTeX
@misc{huang2026zero,
title = {{The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression}},
author = {Kevin Han Huang and Haoyu Ye and Somak Laha and Morgane Austern},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2607.24041},
eprint = {2607.24041},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.24041},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression
AU - Kevin Han Huang
AU - Haoyu Ye
AU - Somak Laha
AU - Morgane Austern
PY - 2026
DA - 2026/10/14
PB - arXiv
DO - 10.48550/arxiv.2607.24041
UR - https://arxiv.org/abs/2607.24041
AB - Over-parameterized linear regression has been widely studied over the last decade. However, most existing works assume that the covariates are independent and that their covariance matrices are non-degenerate. In this paper, we relax both assumptions and derive deterministic equivalents for the prediction risk in a vanishing-ridge regime. We show that degeneracy of the covariance matrices and dependence can lead to multiple descent, and characterize where the corresponding peaks can occur. Our proofs use a novel graph representation of the variance profile. We show that maximum matchings and the Dulmage--Mendelsohn decomposition of the associated bipartite graph identify the configurations at which the variance becomes singular.
ER -