Double Descent in High-dimensional Linear Discriminant Analysis
Abstract
The double-descent phenomenon observed in modern machine learning models has challenged the classical bias-variance trade-off by showing that prediction error can decrease again as model complexity exceeds the interpolation threshold. Although this phenomenon has been extensively studied for deep neural networks and other high-capacity models, its implications for classical statistical methods remain less well understood. In this paper, we investigate double descent in linear discriminant analysis (LDA), a fundamental classification method for binary classification. Using random matrix theory, we derive the asymptotic misclassification risk of LDA in the high-dimensional regime where the dimension (p) and sample size (n) satisfy (p/n \to γ). Specifically, we obtain an explicit asymptotic expression for the LDA risk with a general population covariance matrix when (γ\in (0,1)), and for the Moore-Penrose pseudo-inverse LDA classifier under an isotropic covariance model when (γ\in (1,\infty)). Together, these results provide a complete characterization of the double-descent risk curve of LDA across both the under- and over-parameterized regimes. Simulation studies and experiments on the ARCENE cancer classification dataset demonstrate close agreement with the theoretical predictions.
Related
- Linear maps preserving the matrix discriminant. An enhanced approach
- Optimal Linear Functional Estimation for Proportionally High-Dimensional Linear Models
- CP-factorization for high dimensional tensor time series and double projection iterations
- Honest Inference for High-Dimensional Linear Models
- High dimensional Riemann--Roch spaces in linear spaces with small squares
BibTeX
@misc{lu2026double,
title = {{Double Descent in High-dimensional Linear Discriminant Analysis}},
author = {Yonghe Lu and Han Lin Shang and Yanrong Yang and Kehan Zhao},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.19061},
eprint = {2609.19061},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.19061},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Double Descent in High-dimensional Linear Discriminant Analysis
AU - Yonghe Lu
AU - Han Lin Shang
AU - Yanrong Yang
AU - Kehan Zhao
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2609.19061
UR - https://arxiv.org/abs/2609.19061
AB - The double-descent phenomenon observed in modern machine learning models has challenged the classical bias-variance trade-off by showing that prediction error can decrease again as model complexity exceeds the interpolation threshold. Although this phenomenon has been extensively studied for deep neural networks and other high-capacity models, its implications for classical statistical methods remain less well understood. In this paper, we investigate double descent in linear discriminant analysis (LDA), a fundamental classification method for binary classification. Using random matrix theory, we derive the asymptotic misclassification risk of LDA in the high-dimensional regime where the dimension (p) and sample size (n) satisfy (p/n \to γ). Specifically, we obtain an explicit asymptotic expression for the LDA risk with a general population covariance matrix when (γ\in (0,1)), and for the Moore-Penrose pseudo-inverse LDA classifier under an isotropic covariance model when (γ\in (1,\infty)). Together, these results provide a complete characterization of the double-descent risk curve of LDA across both the under- and over-parameterized regimes. Simulation studies and experiments on the ARCENE cancer classification dataset demonstrate close agreement with the theoretical predictions.
ER -