Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation
Abstract
We study bounded regression under model misspecification, evaluating a function f by its L1(PX) distance to the conditional mean η(x)=𝔼[Y| X=x], rather than the standard L2(PX). This L1 metric is natural for downstream decision problems such as bandits and reinforcement learning. We analyze the min--max estimator of Baraud & Maillard (2025), which we call \emph{sign-betting regression}. Our main results show that sign-betting regression achieves high-probability L1 oracle inequalities under misspecification. Under realizability, our bound adapts to the average conditional variance and recovers the variance-adaptive rate of Li et al. (2026). We further characterize the optimal universal approximation factor for proper learning by showing that sign betting achieves factor 3, while no proper learner can satisfy a distribution-free, high-probability L1 oracle inequality with a universal approximation factor strictly below 3. We extend the upper-bound analysis beyond finite classes and complement the theory with experiments in synthetic and real-world model-selection tasks.
Related
- OLS Testing with Predictors Scaled to Have Unit Sample Variance, and Similar Results for a Robust Regression Estimator
- Variance estimation for generalized linear regression with binary outcome when using case-cohort data
- Improved Variance Estimation in Homoskedastic Nonparametric Random-Design Regression via a Two-Scale Approach
- Three-group variance-ratio tests for heteroscedasticity in linear regression: an exact null distribution and an outlier-resistant version
- User-friendly approximation theory for statistical regression
BibTeX
@misc{li2026sign,
title = {{Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation}},
author = {Yinan Li and Han Han and Kwang-Sung Jun},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.20903},
eprint = {2610.20903},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.20903},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation
AU - Yinan Li
AU - Han Han
AU - Kwang-Sung Jun
PY - 2026
DA - 2026/10/14
PB - arXiv
DO - 10.48550/arxiv.2610.20903
UR - https://arxiv.org/abs/2610.20903
AB - We study bounded regression under model misspecification, evaluating a function f by its L_1(P_X) distance to the conditional mean η(x)=𝔼[Y| X=x], rather than the standard L_2(P_X). This L_1 metric is natural for downstream decision problems such as bandits and reinforcement learning. We analyze the min--max estimator of Baraud & Maillard (2025), which we call \emph{sign-betting regression}. Our main results show that sign-betting regression achieves high-probability L_1 oracle inequalities under misspecification. Under realizability, our bound adapts to the average conditional variance and recovers the variance-adaptive rate of Li et al. (2026). We further characterize the optimal universal approximation factor for proper learning by showing that sign betting achieves factor 3, while no proper learner can satisfy a distribution-free, high-probability L_1 oracle inequality with a universal approximation factor strictly below 3. We extend the upper-bound analysis beyond finite classes and complement the theory with experiments in synthetic and real-world model-selection tasks.
ER -