The Premise Is the Problem: Exchangeability Failure in Self-Monitored Test-Time Adaptation
Abstract
Modern forecasting models are often updated after deployment so they can respond to changing data. These updates can also make predictions worse, so practical systems need a reliable monitor that can detect harmful changes and trigger protection. A natural design is to monitor the same prediction errors that guide the updates. This paper asks whether the statistical guarantee behind such a monitor remains valid when monitoring and adaptation use the same feedback. We study this question in multi-step time-series forecasting. We show that overlapping targets and dependence in forecast errors can break a key assumption required by the guarantee. The monitor may then raise alarms even when no harmful change has occurred, and its response can further damage prediction quality. We also find that adaptation can hide sustained changes from its own monitor, while the original frozen model retains a clearer signal. These results expose a basic failure mode in self-monitored adaptation. They show why reliable deployment requires checking the monitor's assumptions, comparing adaptation with the frozen model under realistic feedback, and limiting the effect of every protective response.
Related
- Pooling interval-monitored step-stress accelerated life tests for reliability prediction: A failure-rate-based meta-analysis for the proportional hazards family
- Measuring evidence against exchangeability and group invariance with E-values
- Independence of premise for intuitionistic Zermelo-Fraenkel set theory
- The relation between non-exchangeability and radial asymmetry of copulas
- Remarks on DAG exchangeability and generalized wreath products
BibTeX
@misc{han2026premise,
title = {{The Premise Is the Problem: Exchangeability Failure in Self-Monitored Test-Time Adaptation}},
author = {Weijia Han and Lisha Qu and Zhenda Li and Liying Liang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.07038},
eprint = {2610.07038},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.07038},
}
RIS
TY - UNPB
TI - The Premise Is the Problem: Exchangeability Failure in Self-Monitored Test-Time Adaptation
AU - Weijia Han
AU - Lisha Qu
AU - Zhenda Li
AU - Liying Liang
PY - 2026
DA - 2026/10/04
PB - arXiv
DO - 10.48550/arxiv.2610.07038
UR - https://arxiv.org/abs/2610.07038
AB - Modern forecasting models are often updated after deployment so they can respond to changing data. These updates can also make predictions worse, so practical systems need a reliable monitor that can detect harmful changes and trigger protection. A natural design is to monitor the same prediction errors that guide the updates. This paper asks whether the statistical guarantee behind such a monitor remains valid when monitoring and adaptation use the same feedback. We study this question in multi-step time-series forecasting. We show that overlapping targets and dependence in forecast errors can break a key assumption required by the guarantee. The monitor may then raise alarms even when no harmful change has occurred, and its response can further damage prediction quality. We also find that adaptation can hide sustained changes from its own monitor, while the original frozen model retains a clearer signal. These results expose a basic failure mode in self-monitored adaptation. They show why reliable deployment requires checking the monitor's assumptions, comparing adaptation with the frozen model under realistic feedback, and limiting the effect of every protective response.
ER -