Rolling Conformal Prediction in Sequential Model Training
Abstract
We introduce Rolling Conformal Prediction (rolling-CP), a distribution-free predictive inference method for the setting of sequential model training. Specifically, given a data stream (X1,Y1),(X2,Y2),…, at each time n the trained model may depend on the observed history {(Xi,Yi)}i<n. This setting arises naturally in modern sequential training, including one-pass training over massive datasets and continual fine-tuning or test-time adaptation of language models during deployment. Rolling-CP first calibrates each incoming observation against the current predictor and then rolls it into future training. In this way, we avoid the need for data splitting. Remarkably, although the models at times n=1,2,… may have entirely different properties and accuracy levels, for exchangeable data it is nonetheless possible to establish a guarantee of marginal coverage, with a familiar universal factor-two guarantee (a worst case guarantee of 1−2α coverage, as compared to the target level 1−α), without any assumptions of stability or any restrictions on the model training process. For i.i.d. data streams, we further prove high-probability training-conditional validity uniformly over time; under stability conditions, coverage guarantees sharpen towards 1−α. Numerical experiments on sequential regression, multiclass SGD, and one-pass neural-network training further demonstrate the practical effectiveness of rolling-CP.
Related
- COINS: Any-Stage-Valid and Utility-Oriented Sequential Conformal Prediction
- Conformal Prediction for Spatially Dependent Data via Sequential Whitening
- conformalForecast: An R package for sequential conformal prediction in time series
- Sharp training-conditional coverage for conformal prediction under covariate shift
- SLICE: Sequential Likelihood-Free Inference via Conformal E-Values
BibTeX
@misc{cheng2026rolling,
title = {{Rolling Conformal Prediction in Sequential Model Training}},
author = {Chen Cheng and Ruiting Liang and Rina Foygel Barber},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.26951},
eprint = {2609.26951},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.26951},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Rolling Conformal Prediction in Sequential Model Training
AU - Chen Cheng
AU - Ruiting Liang
AU - Rina Foygel Barber
PY - 2026
DA - 2026/09/22
PB - arXiv
DO - 10.48550/arxiv.2609.26951
UR - https://arxiv.org/abs/2609.26951
AB - We introduce Rolling Conformal Prediction (rolling-CP), a distribution-free predictive inference method for the setting of sequential model training. Specifically, given a data stream (X_1,Y_1),(X_2,Y_2),…, at each time n the trained model may depend on the observed history {(X_i,Y_i)}_i<n. This setting arises naturally in modern sequential training, including one-pass training over massive datasets and continual fine-tuning or test-time adaptation of language models during deployment. Rolling-CP first calibrates each incoming observation against the current predictor and then rolls it into future training. In this way, we avoid the need for data splitting. Remarkably, although the models at times n=1,2,… may have entirely different properties and accuracy levels, for exchangeable data it is nonetheless possible to establish a guarantee of marginal coverage, with a familiar universal factor-two guarantee (a worst case guarantee of 1−2α coverage, as compared to the target level 1−α), without any assumptions of stability or any restrictions on the model training process. For i.i.d. data streams, we further prove high-probability training-conditional validity uniformly over time; under stability conditions, coverage guarantees sharpen towards 1−α. Numerical experiments on sequential regression, multiclass SGD, and one-pass neural-network training further demonstrate the practical effectiveness of rolling-CP.
ER -