Exact Finite-Sample Risk of Minimum-Norm Interpolation: Proofs of the Double-Descent Curve for Isotropic Gaussian Least Squares, with Simulation Checks and the Ridge Comparison
- 1Former Officer, Chaitanya Godavari Grameena Bank (CGGB)
Abstract
For linear regression with isotropic Gaussian features, n samples, p parameters, signal energy r² and noise variance σ², I prove the exact finite-sample excess risk of the least-squares or minimum-norm interpolating estimator. It is σ²p/(n−p−1) for p ≤ n−2 and r²(1−n/p) + σ²n/(p−n−1) for p ≥ n+2. The proofs use the mean of the inverse Wishart distribution and the rotational invariance of random projections. The formulas reproduce the Hastie–Montanari–Rosset–Tibshirani asymptotics as n, p → ∞ and explain the risk spike at p ≈ n. Simulations with n = 100 and r²/σ² = 4 (400 replications per point) agree with the formulas to within two standard errors wherever |p − n| ≥ 5. Near p = n they become unstable, and I show this is predicted: the second moment of the risk is infinite when |p − n| ≤ 3. At p/n = 2 the min-norm risk is 0.75. That is below the null risk r² = 1 but about 17% above ridge regression at the Bayes-optimal penalty (0.65). In this model, "benign" overparameterisation is a statement about the min-norm estimator, not about the value of interpolation.
Related
- Overlap-free multi-material topology optimization for minimum compliance in two and three dimensions by level-set-based negative-mapping interpolation
- Minimum-Norm Interpolation in Weighted Variation Spaces: Direct and Inverse Problems
- Grokking through the Lens of Minimum-Norm Interpolation
- Interpolation of Gaussian Free Fields via Random Matrices
- Comparison of machine learning and spatial interpolation methods for hourly rainfall data imputation in the Oliwski Stream catchment
BibTeX
@misc{aguru2026exact,
title = {{Exact Finite-Sample Risk of Minimum-Norm Interpolation: Proofs of the Double-Descent Curve for Isotropic Gaussian Least Squares, with Simulation Checks and the Ridge Comparison}},
author = {Manoj Kumar Aguru},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23450083},
url = {https://zenodo.org/records/23450083},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Exact Finite-Sample Risk of Minimum-Norm Interpolation: Proofs of the Double-Descent Curve for Isotropic Gaussian Least Squares, with Simulation Checks and the Ridge Comparison
AU - Manoj Kumar Aguru
PY - 2026
DA - 2026/10/18
PB - Zenodo
DO - 10.5281/zenodo.23450083
UR - https://zenodo.org/records/23450083
AB - For linear regression with isotropic Gaussian features, n samples, p parameters, signal energy r² and noise variance σ², I prove the exact finite-sample excess risk of the least-squares or minimum-norm interpolating estimator. It is σ²p/(n−p−1) for p ≤ n−2 and r²(1−n/p) + σ²n/(p−n−1) for p ≥ n+2. The proofs use the mean of the inverse Wishart distribution and the rotational invariance of random projections. The formulas reproduce the Hastie–Montanari–Rosset–Tibshirani asymptotics as n, p → ∞ and explain the risk spike at p ≈ n. Simulations with n = 100 and r²/σ² = 4 (400 replications per point) agree with the formulas to within two standard errors wherever |p − n| ≥ 5. Near p = n they become unstable, and I show this is predicted: the second moment of the risk is infinite when |p − n| ≤ 3. At p/n = 2 the min-norm risk is 0.75. That is below the null risk r² = 1 but about 17% above ridge regression at the Bayes-optimal penalty (0.65). In this model, "benign" overparameterisation is a statement about the min-norm estimator, not about the value of interpolation.
ER -