Online Quantile Omniprediction for Proper Losses
Abstract
The task of constructing a predictor that is simultaneously accurate under multiple evaluation metrics has been popularized recently under the name omniprediction. Prior work in this area has largely focused on binary prediction tasks, and existing results for multiclass and continuous outcomes typically suffer from slow error rates. In this paper, we consider forecasts expressed as a set of quantiles over a discrete but otherwise arbitrary set of probability levels. We develop sample-efficient algorithms for constructing forecasts that are accurate simultaneously across all proper losses that satisfy mild regularity conditions, where proper losses are those minimized by predicting the true quantiles. Our algorithms are online and can operate in a dynamic environment, while simultaneously taking multiple losses into account, and their forecasts offer the potential to provide diverse end-users with rich information for evaluating risk and informing a range of practical decision-making tasks. As an example, we demonstrate how our procedure can be used to ensemble quantile forecasts of COVID-19 hospitalizations made by experts during the pandemic.
Related
- Omniprediction for All Mixable Proper Losses and Beyond
- Private Online Prediction from Experts with Small Losses
- CAQ: Clustered Asymptotically Anytime-Valid Quantile Treatment Effects for Online Experiments
- Finite-Sample Distribution Theory and Efficient Large-Scale Inference for Online Quantile Regression
- Online Inference for Quantile Distributional Reinforcement Learning
BibTeX
@misc{kim2026online,
title = {{Online Quantile Omniprediction for Proper Losses}},
author = {Hyunsuk Kim and Isaac Gibbs and Michael W. Mahoney and Ryan J. Tibshirani},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.21342},
eprint = {2610.21342},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.21342},
}
RIS
TY - UNPB
TI - Online Quantile Omniprediction for Proper Losses
AU - Hyunsuk Kim
AU - Isaac Gibbs
AU - Michael W. Mahoney
AU - Ryan J. Tibshirani
PY - 2026
DA - 2026/10/15
PB - arXiv
DO - 10.48550/arxiv.2610.21342
UR - https://arxiv.org/abs/2610.21342
AB - The task of constructing a predictor that is simultaneously accurate under multiple evaluation metrics has been popularized recently under the name omniprediction. Prior work in this area has largely focused on binary prediction tasks, and existing results for multiclass and continuous outcomes typically suffer from slow error rates. In this paper, we consider forecasts expressed as a set of quantiles over a discrete but otherwise arbitrary set of probability levels. We develop sample-efficient algorithms for constructing forecasts that are accurate simultaneously across all proper losses that satisfy mild regularity conditions, where proper losses are those minimized by predicting the true quantiles. Our algorithms are online and can operate in a dynamic environment, while simultaneously taking multiple losses into account, and their forecasts offer the potential to provide diverse end-users with rich information for evaluating risk and informing a range of practical decision-making tasks. As an example, we demonstrate how our procedure can be used to ensemble quantile forecasts of COVID-19 hospitalizations made by experts during the pandemic.
ER -