Practical Advantages of Online Conformal Prediction for Data Streams
Abstract
Full conformal prediction offers finite-sample marginal coverage guarantees under exchangeability, but exact computation can be costly, while its classical guarantees do not directly extend to non-stationary data streams. We compare four sequential conformal regression procedures with a weighted full-conformal benchmark in terms of empirical coverage, prediction interval width, and computational cost. The procedures combine ordinary or forgetting-factor least squares with uniform or exponentially weighted empirical residual quantiles, or with online gradient-based quantile tracking. Experiments cover an independent and identically distributed control, two synthetic distribution-shift scenarios, and four real-world data streams. Adapting the regression model as well as the conformal threshold substantially improves the coverage--width tradeoff under drift. The quantile-tracking procedure requires O(p2) time and memory per observation, independently of stream length, although its coverage can deteriorate after abrupt changes. These results illustrate the practical tradeoffs between predictor adaptation, uncertainty quantification, and computational resources.
Related
- Online Conformal Prediction for Non-Exchangeable Panel Data
- Efficient conformal prediction intervals for time series: Online PID-Expert aggregation
- Fitness advantages, prediction of resistances and protocol design: exploiting the synergy between mathematical modelling and statistics
- Sequential Probability Ratio Test using Z-Statistics (SPRT-z): A Practical Approach for Online Experimentation
- Conformal Annotation with Randomized Expert Review: Application to Online Cell-Type Labeling
BibTeX
@misc{chretien2026practical,
title = {{Practical Advantages of Online Conformal Prediction for Data Streams}},
author = {Stéphane Chrétien and Maryam Hoseini-Behbahani and Guillaume Metzler},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.27740},
eprint = {2610.27740},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.27740},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Practical Advantages of Online Conformal Prediction for Data Streams
AU - Stéphane Chrétien
AU - Maryam Hoseini-Behbahani
AU - Guillaume Metzler
PY - 2026
DA - 2026/10/20
PB - arXiv
DO - 10.48550/arxiv.2610.27740
UR - https://arxiv.org/abs/2610.27740
AB - Full conformal prediction offers finite-sample marginal coverage guarantees under exchangeability, but exact computation can be costly, while its classical guarantees do not directly extend to non-stationary data streams. We compare four sequential conformal regression procedures with a weighted full-conformal benchmark in terms of empirical coverage, prediction interval width, and computational cost. The procedures combine ordinary or forgetting-factor least squares with uniform or exponentially weighted empirical residual quantiles, or with online gradient-based quantile tracking. Experiments cover an independent and identically distributed control, two synthetic distribution-shift scenarios, and four real-world data streams. Adapting the regression model as well as the conformal threshold substantially improves the coverage--width tradeoff under drift. The quantile-tracking procedure requires O(p^2) time and memory per observation, independently of stream length, although its coverage can deteriorate after abrupt changes. These results illustrate the practical tradeoffs between predictor adaptation, uncertainty quantification, and computational resources.
ER -