Optimal rates for aggregation of affine estimators
Abstract
Aggregation of estimation procedures has found important applications in econometrics, statistics, and machine learning. Classical statistical aggregation theory has mainly focused on a \emph{pure aggregation} setting, where the candidate estimators are either deterministic or constructed using a held-out sample independent of that used for aggregation. When the candidate estimators and the aggregation weights are estimated from the same dataset without sample splitting, Bellec [\emph{Ann. Statist.} \textbf{46} (2018), 30--59] established the optimal rate for model-selection aggregation of affine estimators. Beyond this regime, however, fundamental questions regarding the optimal aggregation rates and the construction of aggregation rules attaining these rates remain largely unresolved. In this paper, we consider the problem of aggregating a finite collection of affine estimators to learn an optimal convex combination of them. This framework encompasses a rich class of estimators widely used in statistics and machine learning, including least squares estimators, kernel ridge estimators, random feature regression estimators, and among many others. We establish the minimax rate for convex aggregation of affine estimators. In particular, we show that estimating the weights by minimizing a Mallows' Cp criterion attains the optimal rate. We further study the linear aggregation regime with unrestricted weights and establish matching minimax lower and upper bounds over suitable classes of affine estimators.
Related
- Eigenvector distributions and optimal shrinkage estimators for large covariance and precision matrices
- Optimal Two‐Phase Sampling Designs for Generalized Raking Estimators With Multiple Parameters of Interest
- Harmonizable Multifractional Stable Motion: estimators for the Hurst function and the stability parameter from discrete-time observations, their rates of convergence and the associated central limit theorems
- Birthday Estimators Are Uniformly Optimal for Renyi Entropy
- Minimax-Optimal Static Estimators Under Interference
BibTeX
@misc{peng2026optimal,
title = {{Optimal rates for aggregation of affine estimators}},
author = {Jingfu Peng},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.07166},
eprint = {2609.07166},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.07166},
}
RIS
TY - UNPB
TI - Optimal rates for aggregation of affine estimators
AU - Jingfu Peng
PY - 2026
DA - 2026/09/07
PB - arXiv
DO - 10.48550/arxiv.2609.07166
UR - https://arxiv.org/abs/2609.07166
AB - Aggregation of estimation procedures has found important applications in econometrics, statistics, and machine learning. Classical statistical aggregation theory has mainly focused on a \emph{pure aggregation} setting, where the candidate estimators are either deterministic or constructed using a held-out sample independent of that used for aggregation. When the candidate estimators and the aggregation weights are estimated from the same dataset without sample splitting, Bellec [\emph{Ann. Statist.} \textbf{46} (2018), 30--59] established the optimal rate for model-selection aggregation of affine estimators. Beyond this regime, however, fundamental questions regarding the optimal aggregation rates and the construction of aggregation rules attaining these rates remain largely unresolved. In this paper, we consider the problem of aggregating a finite collection of affine estimators to learn an optimal convex combination of them. This framework encompasses a rich class of estimators widely used in statistics and machine learning, including least squares estimators, kernel ridge estimators, random feature regression estimators, and among many others. We establish the minimax rate for convex aggregation of affine estimators. In particular, we show that estimating the weights by minimizing a Mallows' C_p criterion attains the optimal rate. We further study the linear aggregation regime with unrestricted weights and establish matching minimax lower and upper bounds over suitable classes of affine estimators.
ER -