Beyond Discrepancy: Oracle Drift in Nonstationary Weighted Risk Minimization
Abstract
We analyze distributional drift in nonstationary weighted risk minimization by decomposing the excess risk into estimation, drift, and approximation components. The drift error is commonly bounded through discrepancy-type quantities that control changes of the risk over classes of predictors. We instead relate the drift error directly to the movement of the best predictor within the model class and analyze its behavior in parametrized regression models. Our analysis shows that discrepancy-type quantities can be overly conservative: unlike our oracle drift, they may scale at a larger order than the resulting excess-risk contribution and can respond to changes in the risk surface that do not affect the optimal predictor. For linear regression, we characterize this gap sharply and combine the resulting drift bounds with estimation guarantees. We further extend the analysis to nonparametric regression.
Related
- On the Optimal Linear Minimization Oracle Complexity of Active-Hull Methods for Minimizing Convex Hölder Smooth Functions
- Deep Weighted Bellman Residual Minimization for Q* Estimation
- Weighted Empirical Risk Minimization for Machine Learning under Long-Range Dependence: Exact Pathwise Rates and Learning-Error Geometry
- Exact oracle complexity for function-value suboptimality under weighted initial conditions
- Robust low-rank tensor completion via factorized weighted tensor schatten-p norm minimization
BibTeX
@misc{brock2026beyond,
title = {{Beyond Discrepancy: Oracle Drift in Nonstationary Weighted Risk Minimization}},
author = {Tobias Brock and Thomas Nagler},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.13598},
eprint = {2610.13598},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.13598},
}
RIS
TY - UNPB
TI - Beyond Discrepancy: Oracle Drift in Nonstationary Weighted Risk Minimization
AU - Tobias Brock
AU - Thomas Nagler
PY - 2026
DA - 2026/10/09
PB - arXiv
DO - 10.48550/arxiv.2610.13598
UR - https://arxiv.org/abs/2610.13598
AB - We analyze distributional drift in nonstationary weighted risk minimization by decomposing the excess risk into estimation, drift, and approximation components. The drift error is commonly bounded through discrepancy-type quantities that control changes of the risk over classes of predictors. We instead relate the drift error directly to the movement of the best predictor within the model class and analyze its behavior in parametrized regression models. Our analysis shows that discrepancy-type quantities can be overly conservative: unlike our oracle drift, they may scale at a larger order than the resulting excess-risk contribution and can respond to changes in the risk surface that do not affect the optimal predictor. For linear regression, we characterize this gap sharply and combine the resulting drift bounds with estimation guarantees. We further extend the analysis to nonparametric regression.
ER -