Calibration Count Reuse: Validity Does Not Determine Efficiency
Abstract
Calibration count reuse raises separate validity and efficiency questions. We give a validity criterion for general count-dependent nonconformity scores: transferring one count from another class to the scored class must not improve its conformity. A leave-self-out full conformal reference proves the criterion without requiring normalization or preservation of same-class score order. For a common separable transformation, universal exchangeable validity is equivalent to being nondecreasing in the count, provided Kα≥ 1; normalized multiplicative weights obey the complementary nonincreasing condition. Additive penalties are covered under the stated information restrictions. Efficiency has no parallel ordering: two iid constructions make the same valid rule improve or worsen expected size at unchanged coverage. An expanded 55-rule study finds no resolved advantage from selected live-count rules over uniform weights. Image studies identify undercoverage under iid resampling, including at numerical convergence. Separately, execution of the released Conf-OT pipeline on its DTD and Aircraft benchmark subsets produces near-nominal median coverage under fixed stratified counts. The native results are reported separately from the iid analyses, without treating a benchmark observation as a universal guarantee. The findings separate validity, classifier confidence, numerical convergence, and population-specific efficiency.
Related
- Reinforced-count simulation for decision calibration under over-dispersed multi-type service demand
- Cross-Item Pooled Calibration of Safety Stocks: Impossibility, Finite-Sample Coverage, and the Price of Per-Item Validity
- Likelihood-Free Schrödinger--Föllmer Sampling for Bayesian Inference with Fixed-Proposal Reuse
- Data Reuse in Non-Stationary Learning
- A Generalized McDiarmid Inequality: Sample Reuse and Non-Uniform Unbounded Differences
BibTeX
@misc{chopra2026calibration,
title = {{Calibration Count Reuse: Validity Does Not Determine Efficiency}},
author = {Rudra Chopra},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.25138},
eprint = {2609.25138},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.25138},
}
RIS
TY - UNPB
TI - Calibration Count Reuse: Validity Does Not Determine Efficiency
AU - Rudra Chopra
PY - 2026
DA - 2026/09/21
PB - arXiv
DO - 10.48550/arxiv.2609.25138
UR - https://arxiv.org/abs/2609.25138
AB - Calibration count reuse raises separate validity and efficiency questions. We give a validity criterion for general count-dependent nonconformity scores: transferring one count from another class to the scored class must not improve its conformity. A leave-self-out full conformal reference proves the criterion without requiring normalization or preservation of same-class score order. For a common separable transformation, universal exchangeable validity is equivalent to being nondecreasing in the count, provided Kα≥ 1; normalized multiplicative weights obey the complementary nonincreasing condition. Additive penalties are covered under the stated information restrictions. Efficiency has no parallel ordering: two iid constructions make the same valid rule improve or worsen expected size at unchanged coverage. An expanded 55-rule study finds no resolved advantage from selected live-count rules over uniform weights. Image studies identify undercoverage under iid resampling, including at numerical convergence. Separately, execution of the released Conf-OT pipeline on its DTD and Aircraft benchmark subsets produces near-nominal median coverage under fixed stratified counts. The native results are reported separately from the iid analyses, without treating a benchmark observation as a universal guarantee. The findings separate validity, classifier confidence, numerical convergence, and population-specific efficiency.
ER -