Verdict Instability of OOD Scores under Reference Resampling
Abstract
Post-hoc out-of-distribution detectors are fitted on a finite reference set, so every score they produce is an estimate. If we had chosen a different set, some verdicts would have moved. We measure that movement by resampling the reference set and recording the bootstrap standard deviation of the score, which we call verdict instability. It admits a closed form with no fitted parameters. The instability of a verdict is the within-class dispersion of the assigned class along the query's direction, divided by the square root of that class's reference count. That count is what separates verdict instability from the geometry of the score distribution, and it is identifiable only under class imbalance. Instability grows with the local dispersion. Far-OOD queries lie along the low-variance directions of an anisotropic embedding, so the distance-based scores in common use assign their highest values to the verdicts that are most reproducible. Only estimators of local dispersion carry the sign a practitioner expects. We give a rule that predicts this sign for any score from a single label-free correlation, and abstention driven by a wrong-signed score turns out worse than abstention at random on every dataset we test.
Related
- Stochastic Mirror-Prox Optimisation of Group Distributionally Robust Multimodal Logistic Regression for Multilingual Fact-Checking Verdict Prediction
- Improving the Predictive Performance of Bootstrap Aggregating by Dirichlet Resampling
- Differentiable Systematic Resampling for Variational Sequential Monte Carlo
- A Dirichlet-Quantile Bootstrap approach for the probabilistic resampling of time series
- Multiradial SLE with spiral: resampling property and boundary perturbation
BibTeX
@misc{lee2026verdict,
title = {{Verdict Instability of OOD Scores under Reference Resampling}},
author = {Donghoon Lee and Shinjin Kang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.00691},
eprint = {2609.00691},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.00691},
note = {CC BY-SA 4.0},
}
RIS
TY - UNPB
TI - Verdict Instability of OOD Scores under Reference Resampling
AU - Donghoon Lee
AU - Shinjin Kang
PY - 2026
DA - 2026/10/23
PB - arXiv
DO - 10.48550/arxiv.2609.00691
UR - https://arxiv.org/abs/2609.00691
AB - Post-hoc out-of-distribution detectors are fitted on a finite reference set, so every score they produce is an estimate. If we had chosen a different set, some verdicts would have moved. We measure that movement by resampling the reference set and recording the bootstrap standard deviation of the score, which we call verdict instability. It admits a closed form with no fitted parameters. The instability of a verdict is the within-class dispersion of the assigned class along the query's direction, divided by the square root of that class's reference count. That count is what separates verdict instability from the geometry of the score distribution, and it is identifiable only under class imbalance. Instability grows with the local dispersion. Far-OOD queries lie along the low-variance directions of an anisotropic embedding, so the distance-based scores in common use assign their highest values to the verdicts that are most reproducible. Only estimators of local dispersion carry the sign a practitioner expects. We give a rule that predicts this sign for any score from a single label-free correlation, and abstention driven by a wrong-signed score turns out worse than abstention at random on every dataset we test.
ER -