Online Shift Detection and Conformal Adaptation for Deployed Safety Classifiers
Abstract
We study three failure modes of deployed safety classifiers: distribution shift, targeted evasion, and invalid scores from reasoning-model monitors. A reduced 800-cell subset of a pre-registered factorial (4 classifiers x 5 shifts x 20 seeds x 2 windows) gives 86.8% control-validated detection (alarm at or after onset, quiet paired no-shift stream; raw TPR 91.5%), mean latency 39.7 steps at w=100 and a classifier-by-shift crossover (η2 = 0.186 complete-case; 0.02-0.03 under a censoring sensitivity analysis) that motivates per-classifier monitoring. Shifted corpora are mostly harmful prompts, so this detects a constructed score change, not accuracy loss. The density-ratio step of weighted conformal adaptation collapses for 3/4 classifiers in high-dimensional embeddings; PCA to 32 dimensions raises temporal-shift coverage (+33pp, +20.5pp). A score-disagreement canary (an un-targeted classifier compared with the primary) flags 37/49 (75.5%) of the GCG inputs DeBERTa scores as safe (Llama Guard pair; 93.9% best single canary), leaving a residual that evades both. For one fixed attack objective we describe a local stall condition (gap = 1/(2λ); observed/predicted 0.86-1.00 at λ≥ 2, n=20; blocked rate does not replicate). Where both models judge content harmful there is no disagreement to detect (2%, 1/50). The best-value frontier LLM canary flags 83.7% (Wilson lower bound 71%) of 49 adversarial prompts with 7/1000 benign false positives. KS false-alarm rates inflate 5.4x under heterogeneous traffic; the two CUSUM runs there conflict. On o3, GCG-appended harmful inputs need 2.5x the token budget of simple benign inputs (T50 = 117 vs 46; length and content not separated), and at 16 tokens o3, o4-mini and gemini-2.5-pro all return 0/100 valid scores.
Related
- Practical Advantages of Online Conformal Prediction for Data Streams
- Online Conformal Prediction for Non-Exchangeable Panel Data
- Efficient conformal prediction intervals for time series: Online PID-Expert aggregation
- Conformal Annotation with Randomized Expert Review: Application to Online Cell-Type Labeling
- Conformal Novelty Detection via Data-enhanced Localization
BibTeX
@misc{leong2026online,
title = {{Online Shift Detection and Conformal Adaptation for Deployed Safety Classifiers}},
author = {Jun Wen Leong},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2606.11949},
eprint = {2606.11949},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.11949},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Online Shift Detection and Conformal Adaptation for Deployed Safety Classifiers
AU - Jun Wen Leong
PY - 2026
DA - 2026/10/25
PB - arXiv
DO - 10.48550/arxiv.2606.11949
UR - https://arxiv.org/abs/2606.11949
AB - We study three failure modes of deployed safety classifiers: distribution shift, targeted evasion, and invalid scores from reasoning-model monitors. A reduced 800-cell subset of a pre-registered factorial (4 classifiers x 5 shifts x 20 seeds x 2 windows) gives 86.8% control-validated detection (alarm at or after onset, quiet paired no-shift stream; raw TPR 91.5%), mean latency 39.7 steps at w=100 and a classifier-by-shift crossover (η^2 = 0.186 complete-case; 0.02-0.03 under a censoring sensitivity analysis) that motivates per-classifier monitoring. Shifted corpora are mostly harmful prompts, so this detects a constructed score change, not accuracy loss. The density-ratio step of weighted conformal adaptation collapses for 3/4 classifiers in high-dimensional embeddings; PCA to 32 dimensions raises temporal-shift coverage (+33pp, +20.5pp). A score-disagreement canary (an un-targeted classifier compared with the primary) flags 37/49 (75.5%) of the GCG inputs DeBERTa scores as safe (Llama Guard pair; 93.9% best single canary), leaving a residual that evades both. For one fixed attack objective we describe a local stall condition (gap = 1/(2λ); observed/predicted 0.86-1.00 at λ≥ 2, n=20; blocked rate does not replicate). Where both models judge content harmful there is no disagreement to detect (2%, 1/50). The best-value frontier LLM canary flags 83.7% (Wilson lower bound 71%) of 49 adversarial prompts with 7/1000 benign false positives. KS false-alarm rates inflate 5.4x under heterogeneous traffic; the two CUSUM runs there conflict. On o3, GCG-appended harmful inputs need 2.5x the token budget of simple benign inputs (T50 = 117 vs 46; length and content not separated), and at 16 tokens o3, o4-mini and gemini-2.5-pro all return 0/100 valid scores.
ER -