Double Descent for Random Fourier Series Models
Abstract
We investigate the least squares linear regression problem with random partial Discrete Fourier Transform (DFT) matrices, providing a rigorous analysis of the model's generalization error. By leveraging tools from random matrix theory, we derive exact non-asymptotic bounds for the risk of the Moore-Penrose estimator, which hold for finite-dimensional problems and reveal the precise dependence on key parameters such as the sample size, dimension, and noise variance. Then we obtain a characterization of the double descent phenomenon in the linear regression context, demonstrating how the risk evolves when the number of parameters p and the number of samples n tend to infinity, with p/n fixed. The analysis relies on applications of the Stieltjes transform for random Fourier matrices, enabling a precise description of the spectral properties of these matrices and their impact on regression performance. To validate our theoretical findings, we present several numerical examples that illustrate the double descent curves. These simulations align closely with our derived bounds, confirming their predictive power in both under-parameterized and over-parameterized regimes.
Related
- Even-degree Hermitian Ikeda lifts via Fourier-Jacobi descent
- The double descent and Runge phenomena in overparametrized polynomial interpolation
- Fourier Ratio and Restricted Isometry via Entropic Descent
- Double Descent in High-dimensional Linear Discriminant Analysis
- Double descent is the principle of least action
BibTeX
@misc{xu2026double,
title = {{Double Descent for Random Fourier Series Models}},
author = {Hang Xu and Yi Shen and Yuzhong Zhao},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.30338},
eprint = {2609.30338},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.30338},
}
RIS
TY - UNPB
TI - Double Descent for Random Fourier Series Models
AU - Hang Xu
AU - Yi Shen
AU - Yuzhong Zhao
PY - 2026
DA - 2026/09/24
PB - arXiv
DO - 10.48550/arxiv.2609.30338
UR - https://arxiv.org/abs/2609.30338
AB - We investigate the least squares linear regression problem with random partial Discrete Fourier Transform (DFT) matrices, providing a rigorous analysis of the model's generalization error. By leveraging tools from random matrix theory, we derive exact non-asymptotic bounds for the risk of the Moore-Penrose estimator, which hold for finite-dimensional problems and reveal the precise dependence on key parameters such as the sample size, dimension, and noise variance. Then we obtain a characterization of the double descent phenomenon in the linear regression context, demonstrating how the risk evolves when the number of parameters p and the number of samples n tend to infinity, with p/n fixed. The analysis relies on applications of the Stieltjes transform for random Fourier matrices, enabling a precise description of the spectral properties of these matrices and their impact on regression performance. To validate our theoretical findings, we present several numerical examples that illustrate the double descent curves. These simulations align closely with our derived bounds, confirming their predictive power in both under-parameterized and over-parameterized regimes.
ER -