User-friendly approximation theory for statistical regression
Abstract
The error of regression using noisy function values naturally decomposes into an approximation error from the choice of model space and stochastic error from the noise on the observations. Statistical theory is mostly concerned with the stochastic error while approximation theory focuses on the approximation error, often subject to the assumption of noiseless observations. We combine the tools from approximation theory and statistics into a framework for bounding both errors in generalized least squares regression. The results cover weighted sup-norms and L2(μ)-norms, real- and complex-valued functions, and correlated noise under a range of tail and moment assumptions. Weighted Lebesgue functions and constants control the amplification of the best approximation error, while concentration inequalities provide bounds on the stochastic error. We organize these results into reusable bounds with explicit constants and establish sharpness where possible. For the common strategy of randomized regression we prove concentration of the Lebesgue function and constant around those of the corresponding L2(μ)-projection. For Fourier and Chebyshev regression on standard sampling grids, we obtain explicit, asymptotically tight 𝒪(log m) bounds on the Lebesgue constants where m is the model dimension. This improves upon the previous 𝒪(√m) bound obtained for the regression setting and matches the bounds known for interpolation. A catalogue of approximation bounds and worked examples illustrates how to turn these results into explicit regression guarantees.
Related
- Statistical Inference for Persistence Diagrams via Landmark Embeddings: Minimax Theory and Finite Approximation
- Statistical Foundations for a Google Play User-Review Sentiment Index: Signal Fusion, Shrinkage, Distributional Validation, and Dynamic Smoothing
- Communication with Adversarial Users: Byzantine Models in Multi-User Information Theory
- Statistical inference for quantile correlation regression with bivariate outcomes
- Approximation Families and the Statistical Efficiency of Denoising Score Matching
BibTeX
@misc{benning2026userfriendly,
title = {{User-friendly approximation theory for statistical regression}},
author = {Felix Benning},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.01342},
eprint = {2610.01342},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.01342},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - User-friendly approximation theory for statistical regression
AU - Felix Benning
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2610.01342
UR - https://arxiv.org/abs/2610.01342
AB - The error of regression using noisy function values naturally decomposes into an approximation error from the choice of model space and stochastic error from the noise on the observations. Statistical theory is mostly concerned with the stochastic error while approximation theory focuses on the approximation error, often subject to the assumption of noiseless observations. We combine the tools from approximation theory and statistics into a framework for bounding both errors in generalized least squares regression. The results cover weighted sup-norms and L^2(μ)-norms, real- and complex-valued functions, and correlated noise under a range of tail and moment assumptions. Weighted Lebesgue functions and constants control the amplification of the best approximation error, while concentration inequalities provide bounds on the stochastic error. We organize these results into reusable bounds with explicit constants and establish sharpness where possible. For the common strategy of randomized regression we prove concentration of the Lebesgue function and constant around those of the corresponding L^2(μ)-projection. For Fourier and Chebyshev regression on standard sampling grids, we obtain explicit, asymptotically tight 𝒪(log m) bounds on the Lebesgue constants where m is the model dimension. This improves upon the previous 𝒪(√m) bound obtained for the regression setting and matches the bounds known for interpolation. A catalogue of approximation bounds and worked examples illustrates how to turn these results into explicit regression guarantees.
ER -