The information in diagnostic tests
- 1Division of AI, Emergency Medicine, Mass General Brigham; Laboratory of Computer Science, Massachusetts General Hospital; Harvard Medical School; Master of Scie
Abstract
BackgroundDiagnostic accuracy measures describe test performance, but expected learning also depends on the probability of disease before testing. We explain diagnostic information as expected uncertainty reduction and construct an empirical reference for its interpretation.
MethodsWe reanalysed structured study-level diagnostic accuracy data from Cochrane reviews, public repositories and PubMed Central review tables. A common continuity-corrected bivariate random-effects model pooled sensitivity and specificity for defined diagnostic profiles. We calculated mutual information in bits and as the percentage of starting uncertainty resolved at stated disease probabilities. Profiles contributed equally to the reference distribution. Sensitivity analyses examined alternative estimators, equal-review weighting and whole-review resampling.
ResultsThe reference included 273 pooled profiles from 210 reviews, comprising 4104 study-result appearances. Median uncertainty resolved was 23.3%, 29.2% and 31.4% at starting probabilities of 5%, 20% and 50%, respectively. At 20%, the median represented 0.211 of 0.722 bits, with a profile-bootstrap 95% confidence interval of 27.6% to 32.7% for the percentage resolved. Equal-review weighting gave 30.0%; whole-review resampling gave an interval of 27.5% to 33.5%. Two components of the head impulse, nystagmus and test of skew examination with nearly equal Youden indices resolved 18.7% and 28.1% at a 5% starting probability. Carcinoembryonic-antigen thresholds illustrated probability-dependent information ordering; the Ottawa ankle rule distinguished expected learning from uncertainty change after one result.
ConclusionsDiagnostic information makes expected learning explicit alongside conventional accuracy measures. Reporting bits and percentage of starting uncertainty resolved, with the starting probability and clinical context, supports interpretation and comparison of diagnostic evidence.
Related
- Evaluation of the clinical performance of two new onchocerciasis rapid diagnostic tests in districts targeted for elimination in Ghana
- Diagnostic Accuracy of Rapid Diagnostic Tests Versus Enzyme-Linked Immunosorbent Assay (ELISA) Across Major Infectious Diseases: A Systematic Review and Meta-Analysis
- Molecular and Conventional Tests in Onychomycosis: A Systematic Review of Paired Diagnostic Discordance and Clinical Interpretation
- Identifying subgroups with decreased performance on malaria rapid diagnostic tests in Malawi
- Differentiating hypothalamic from pituitary adrenal insufficiency: underlying causes and diagnostic utility of the corticotropin-releasing hormone and growth hormone-releasing peptide-2 stimulation tests
BibTeX
@misc{he2026information,
title = {{The information in diagnostic tests}},
author = {S. He and B. W. Locke and J. W. Joseph and D. M. Liebovitz and C. Rohlfsen and A. Goff and P. Safari and C. Kabrhel and A. Mohamed and A. S. Raja and J. N. Goldstein},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.13.26362573},
url = {https://www.medrxiv.org/content/10.64898/2026.09.13.26362573v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - The information in diagnostic tests
AU - S. He
AU - B. W. Locke
AU - J. W. Joseph
AU - D. M. Liebovitz
AU - C. Rohlfsen
AU - A. Goff
AU - P. Safari
AU - C. Kabrhel
AU - A. Mohamed
AU - A. S. Raja
AU - J. N. Goldstein
PY - 2026
DA - 2026/09/13
PB - medRxiv
DO - 10.64898/2026.09.13.26362573
UR - https://www.medrxiv.org/content/10.64898/2026.09.13.26362573v1
AB - BackgroundDiagnostic accuracy measures describe test performance, but expected learning also depends on the probability of disease before testing. We explain diagnostic information as expected uncertainty reduction and construct an empirical reference for its interpretation. MethodsWe reanalysed structured study-level diagnostic accuracy data from Cochrane reviews, public repositories and PubMed Central review tables. A common continuity-corrected bivariate random-effects model pooled sensitivity and specificity for defined diagnostic profiles. We calculated mutual information in bits and as the percentage of starting uncertainty resolved at stated disease probabilities. Profiles contributed equally to the reference distribution. Sensitivity analyses examined alternative estimators, equal-review weighting and whole-review resampling. ResultsThe reference included 273 pooled profiles from 210 reviews, comprising 4104 study-result appearances. Median uncertainty resolved was 23.3%, 29.2% and 31.4% at starting probabilities of 5%, 20% and 50%, respectively. At 20%, the median represented 0.211 of 0.722 bits, with a profile-bootstrap 95% confidence interval of 27.6% to 32.7% for the percentage resolved. Equal-review weighting gave 30.0%; whole-review resampling gave an interval of 27.5% to 33.5%. Two components of the head impulse, nystagmus and test of skew examination with nearly equal Youden indices resolved 18.7% and 28.1% at a 5% starting probability. Carcinoembryonic-antigen thresholds illustrated probability-dependent information ordering; the Ottawa ankle rule distinguished expected learning from uncertainty change after one result. ConclusionsDiagnostic information makes expected learning explicit alongside conventional accuracy measures. Reporting bits and percentage of starting uncertainty resolved, with the starting probability and clinical context, supports interpretation and comparison of diagnostic evidence.
ER -