Cross-domain confidence reliability and remappability of frozen single-cell representations
- 1Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA 90069, USA
Abstract
Single-cell foundation models are increasingly adopted for downstream applications such as cell-type prediction. However, these predictions are often utilized without assessing their reliability, or by relying on a simple cutoff applied to the maximum softmax probability (MSP) derived from the classifier. This raises a critical question: can raw MSP be trusted as a reliable measure of confidence on unseen data (target) exhibiting diverse technical and biological variations? To address this, we introduce an audit framework to systematically evaluate the confidence estimates derived from the training set (source) against the realities of the target test set. We demonstrate that raw MSP fundamentally fails to represent true confidence when applied to target datasets. By decomposing the confidence gap between source and target data, we reveal that these discrepancies stem from a combination of rank-ordering errors and systemic probability drift. We further show that the drift can be successfully recalibrated by leveraging a small subset of labeled target data. While this recalibration improves the reliability of automated acceptance, it inherently introduces a trade-off by increasing the volume of cells requiring manual review. Ultimately, our framework establishes that the safe deployment of these models necessitates target-specific confidence adjustment using representative local labels.
Related
- Same stimuli, different spaces: how task, observer and amount of data influence geometry and reliability of mental representations
- Isolating domain-specific and domain-general contributions to global confidence
- Benchmarking interface confidence as a selection criterion for computational nanobody design against survivin's BIR-domain groove
- Quantifying the reliability gap in cross-domain plant disease classification: benchmarking the limited efficacy of standard mitigation techniques under controlled-to-field shift
- A Frozen Plant DNA Language Model for Computational Prioritization and Functional Annotation of Melon (Cucumis melo) Variants
BibTeX
@misc{weng2026crossdomain,
title = {{Cross-domain confidence reliability and remappability of frozen single-cell representations}},
author = {G. Weng and D. Zhu and Y. Zhao and P. C. Martin and H. Kim and J. Jung and G.-H. Nam and K. J. Won},
howpublished = {bioRxiv},
year = {2026},
month = aug,
doi = {10.64898/2026.08.31.748446},
url = {https://www.biorxiv.org/content/10.64898/2026.08.31.748446v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Cross-domain confidence reliability and remappability of frozen single-cell representations
AU - G. Weng
AU - D. Zhu
AU - Y. Zhao
AU - P. C. Martin
AU - H. Kim
AU - J. Jung
AU - G.-H. Nam
AU - K. J. Won
PY - 2026
DA - 2026/08/31
PB - bioRxiv
DO - 10.64898/2026.08.31.748446
UR - https://www.biorxiv.org/content/10.64898/2026.08.31.748446v1
AB - Single-cell foundation models are increasingly adopted for downstream applications such as cell-type prediction. However, these predictions are often utilized without assessing their reliability, or by relying on a simple cutoff applied to the maximum softmax probability (MSP) derived from the classifier. This raises a critical question: can raw MSP be trusted as a reliable measure of confidence on unseen data (target) exhibiting diverse technical and biological variations? To address this, we introduce an audit framework to systematically evaluate the confidence estimates derived from the training set (source) against the realities of the target test set. We demonstrate that raw MSP fundamentally fails to represent true confidence when applied to target datasets. By decomposing the confidence gap between source and target data, we reveal that these discrepancies stem from a combination of rank-ordering errors and systemic probability drift. We further show that the drift can be successfully recalibrated by leveraging a small subset of labeled target data. While this recalibration improves the reliability of automated acceptance, it inherently introduces a trade-off by increasing the volume of cells requiring manual review. Ultimately, our framework establishes that the safe deployment of these models necessitates target-specific confidence adjustment using representative local labels.
ER -