Benign Overfitting Property of the Minimum ℓq -norm Interpolant Estimator in Regression Problem via Feature Space Decomposition
Abstract
This paper establishes high-probability, non-asymptotic upper bounds for the population excess risk of minimum ℓq-norm interpolating estimators in linear regression, valid for all 1<q<∞ under three scenarios: convex concentration, mild moment assumptions and log-concavity. Consequently, we derive sufficient conditions for these estimators to exhibit benign overfitting. Our analysis hinges on a Feature Space Decomposition (FSD) framework, which naturally highlights the self-regularizing properties of minimum norm interpolants as well as the role of alignment between the signal and the features for benign overfitting. From a technical perspective, our approach circumvents the traditionally used Convex Gaussian Min-max Theorem (CGMT) by instead leveraging the Dvoretzky-Milman theorem. We emphasize that the FSD method has the potential to refine standard uniform convergence approaches. Furthermore, the isomorphic Dvoretzky-Milman theorem established herein -- proven under the three scenarios mentioned above -- for the ℓq-norm is of independent mathematical interest.
Related
- Instance-Specific Sharp Convergence Rate of the Add-λ Estimator via Feature Space Decomposition
- OLS Testing with Predictors Scaled to Have Unit Sample Variance, and Similar Results for a Robust Regression Estimator
- A penalized logistic generalized regression estimator
- Existence and consistency of weighted maximum likelihood estimator for extreme quantile regression
- Integer decomposition property of three-dimensional smooth polytopes: a shorter proof
BibTeX
@misc{adamczak2026benign,
title = {{Benign Overfitting Property of the Minimum $\ell_q$ -norm Interpolant Estimator in Regression Problem via Feature Space Decomposition}},
author = {Radosław Adamczak and Guillaume Lecué and Zong Shang and Marta Strzelecka},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.24350},
eprint = {2610.24350},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.24350},
}
RIS
TY - UNPB
TI - Benign Overfitting Property of the Minimum ℓ_q -norm Interpolant Estimator in Regression Problem via Feature Space Decomposition
AU - Radosław Adamczak
AU - Guillaume Lecué
AU - Zong Shang
AU - Marta Strzelecka
PY - 2026
DA - 2026/10/17
PB - arXiv
DO - 10.48550/arxiv.2610.24350
UR - https://arxiv.org/abs/2610.24350
AB - This paper establishes high-probability, non-asymptotic upper bounds for the population excess risk of minimum ℓ_q-norm interpolating estimators in linear regression, valid for all 1<q<∞ under three scenarios: convex concentration, mild moment assumptions and log-concavity. Consequently, we derive sufficient conditions for these estimators to exhibit benign overfitting. Our analysis hinges on a Feature Space Decomposition (FSD) framework, which naturally highlights the self-regularizing properties of minimum norm interpolants as well as the role of alignment between the signal and the features for benign overfitting. From a technical perspective, our approach circumvents the traditionally used Convex Gaussian Min-max Theorem (CGMT) by instead leveraging the Dvoretzky-Milman theorem. We emphasize that the FSD method has the potential to refine standard uniform convergence approaches. Furthermore, the isomorphic Dvoretzky-Milman theorem established herein -- proven under the three scenarios mentioned above -- for the ℓ_q-norm is of independent mathematical interest.
ER -