Adaptive Conformal Inference Under Delayed Feedback: Coverage Guarantees and a Delay-to-Memory Diagnostic
Abstract
Adaptive Conformal Inference (ACI) extends conformal prediction to non-exchangeable settings by adjusting the nominal miscoverage level online in response to recent coverage errors. When forecasts are issued with horizon τ, however, the outcome needed to evaluate a prediction is observed only τ steps later, so these adaptive updates must rely on delayed feedback. We study this setting and ask how the effect of delay depends on the persistence of the residual process. First, we show that the τ-delayed ACI recursion can be decomposed into τ interleaved ACI-like sequences. This representation yields a finite-sample bound on long-run empirical coverage with explicit dependence on τ. We also derive an approximate marginal coverage bound that relates coverage deviation to changes in the underlying environment across the forecast horizon and to the adaptation rate γ. We then introduce the delay-to-memory ratio r=τ/L, where L is the time scale over which the temporal signal driving non-exchangeability decays. Simulation results show that the usefulness of this ratio depends on the form of temporal dependence: it strongly organizes performance under AR(1) dependence, is less predictive of overall performance under GARCH(1,1) and Markov switching, but more clearly characterizes when scale normalization remains useful in those settings. Abrupt mean- and variance-shift experiments further show that the preferred adaptation rate depends on the residual dynamics. Overall, the results show that the effect of forecast delay is best understood relative to the time scale over which past residual information remains relevant.
Related
- Adaptive Conformal Inference through the Lens of Blackwell Approachability
- Dynamic Dual-Feedback Conformal Inference for Time Series Forecasting
- Conformal Uncertainty Quantification Guarantees for Neural Operators
- Quasi-Static Fault-Tolerant Feedback Control of a Quadrotor under Rotor Failure with Provable Safety Guarantees
- Conformal Policy Learning with Distribution-Free Safety Guarantees
BibTeX
@misc{halabi2026adaptive,
title = {{Adaptive Conformal Inference Under Delayed Feedback: Coverage Guarantees and a Delay-to-Memory Diagnostic}},
author = {Lama El Halabi and Adam Brandt},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.07251},
eprint = {2609.07251},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.07251},
}
RIS
TY - UNPB
TI - Adaptive Conformal Inference Under Delayed Feedback: Coverage Guarantees and a Delay-to-Memory Diagnostic
AU - Lama El Halabi
AU - Adam Brandt
PY - 2026
DA - 2026/09/07
PB - arXiv
DO - 10.48550/arxiv.2609.07251
UR - https://arxiv.org/abs/2609.07251
AB - Adaptive Conformal Inference (ACI) extends conformal prediction to non-exchangeable settings by adjusting the nominal miscoverage level online in response to recent coverage errors. When forecasts are issued with horizon τ, however, the outcome needed to evaluate a prediction is observed only τ steps later, so these adaptive updates must rely on delayed feedback. We study this setting and ask how the effect of delay depends on the persistence of the residual process. First, we show that the τ-delayed ACI recursion can be decomposed into τ interleaved ACI-like sequences. This representation yields a finite-sample bound on long-run empirical coverage with explicit dependence on τ. We also derive an approximate marginal coverage bound that relates coverage deviation to changes in the underlying environment across the forecast horizon and to the adaptation rate γ. We then introduce the delay-to-memory ratio r=τ/L, where L is the time scale over which the temporal signal driving non-exchangeability decays. Simulation results show that the usefulness of this ratio depends on the form of temporal dependence: it strongly organizes performance under AR(1) dependence, is less predictive of overall performance under GARCH(1,1) and Markov switching, but more clearly characterizes when scale normalization remains useful in those settings. Abrupt mean- and variance-shift experiments further show that the preferred adaptation rate depends on the residual dynamics. Overall, the results show that the effect of forecast delay is best understood relative to the time scale over which past residual information remains relevant.
ER -