Planted Errors Cannot Measure What a Clinician Misses
- 1ConcertAI
Abstract
Distribution-free risk control certifies that the rate of defects that would change clinical management, among the answers an AI system releases, stays below a stated level, calibrated on cases a human reviewer has checked. The reviewer is an imperfect reference standard whose errors run one way: a missed defect enters calibration as a safe label. This one-sided misclassification lowers the risk on record at every release threshold, so the certificate is anti-conservative by a factor set by the reviewers miss rate on the systems own defects. We show that seeded probes, which plant known defects and count the catches, do not identify this miss rate. A reviewers chance of missing a defect depends on how hard that defect is to spot, its detectability. A probe samples defects at detectabilities its designer chose. The systems own defects fall at detectabilities nothing chose, and converting the probes miss rate into theirs requires the very rate being sought. This holds whenever detectability is not fully captured by an observed score, since a score that captured it would let probes covering its range identify the rate. Cytology field validation builds probe sets whose miss rate is a floor on the true one. Even then the probe bounds the rate from below only, so it can reject a certificate and cannot grant one. The finite-sample label-noise corrections assume noise independent of the case given its label, but a reviewer misses the defects that read as plausible, and the condition they need fails in all four simulated reviewer profiles we examine. Testing that condition on an independent sample takes 185 adjudicated cases for a reviewer missing 85% of defects and 147,938 for one missing 3%. A second reader who adjudicates the cases the careful reviewer passed finds a miss with 95% confidence within 498, so looking for the noise directly costs far less than testing a model of it. In a public corpus of model answers graded by physicians, a physician passed 26.3% of the defects the other graders agreed were there (95% interval 21.6% to 31.7%, resampling whole conversations), on criteria about seeking context and stating uncertainty, and individual rates ran from 8.0% to 54.5%.
Related
- Large language models detect methodological errors that human peer review misses in clinical trials
- Managing What You Cannot Measure: Anesthetic Implications of Takayasu’s Arteritis in Hypertensive Intracranial Hemorrhage
- Drivers of banana re-infection in newly planted gardens in banana bunchy top disease endemic zones: Implications for the recovery of banana production
- Loss and damage as mental health policy: Valuing what cannot be replaced
- From Custom Devices to Engineered Hearts: A Clinician’s Guide to 3D Printing and Bioprinting
BibTeX
@misc{kotabhattara2026planted,
title = {{Planted Errors Cannot Measure What a Clinician Misses}},
author = {K. C. Kotabhattara and A. Siroha and P. Pal and S. K and V. S. Dhara and P. Prabhakar and N. Honkote and M. Marreddy and N. Namita and A. K. K and P. Saha and J. M. Agris},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.01.26364465},
url = {https://www.medrxiv.org/content/10.64898/2026.10.01.26364465v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Planted Errors Cannot Measure What a Clinician Misses
AU - K. C. Kotabhattara
AU - A. Siroha
AU - P. Pal
AU - S. K
AU - V. S. Dhara
AU - P. Prabhakar
AU - N. Honkote
AU - M. Marreddy
AU - N. Namita
AU - A. K. K
AU - P. Saha
AU - J. M. Agris
PY - 2026
DA - 2026/10/01
PB - medRxiv
DO - 10.64898/2026.10.01.26364465
UR - https://www.medrxiv.org/content/10.64898/2026.10.01.26364465v1
AB - Distribution-free risk control certifies that the rate of defects that would change clinical management, among the answers an AI system releases, stays below a stated level, calibrated on cases a human reviewer has checked. The reviewer is an imperfect reference standard whose errors run one way: a missed defect enters calibration as a safe label. This one-sided misclassification lowers the risk on record at every release threshold, so the certificate is anti-conservative by a factor set by the reviewers miss rate on the systems own defects. We show that seeded probes, which plant known defects and count the catches, do not identify this miss rate. A reviewers chance of missing a defect depends on how hard that defect is to spot, its detectability. A probe samples defects at detectabilities its designer chose. The systems own defects fall at detectabilities nothing chose, and converting the probes miss rate into theirs requires the very rate being sought. This holds whenever detectability is not fully captured by an observed score, since a score that captured it would let probes covering its range identify the rate. Cytology field validation builds probe sets whose miss rate is a floor on the true one. Even then the probe bounds the rate from below only, so it can reject a certificate and cannot grant one. The finite-sample label-noise corrections assume noise independent of the case given its label, but a reviewer misses the defects that read as plausible, and the condition they need fails in all four simulated reviewer profiles we examine. Testing that condition on an independent sample takes 185 adjudicated cases for a reviewer missing 85% of defects and 147,938 for one missing 3%. A second reader who adjudicates the cases the careful reviewer passed finds a miss with 95% confidence within 498, so looking for the noise directly costs far less than testing a model of it. In a public corpus of model answers graded by physicians, a physician passed 26.3% of the defects the other graders agreed were there (95% interval 21.6% to 31.7%, resampling whole conversations), on criteria about seeking context and stating uncertainty, and individual rates ran from 8.0% to 54.5%.
ER -