{
  "abstract": "Distribution-free risk control certifies that the rate of defects that would change clinical management, among the answers an AI system releases, stays below a stated level, calibrated on cases a human reviewer has checked. The reviewer is an imperfect reference standard whose errors run one way: a missed defect enters calibration as a safe label. This one-sided misclassification lowers the risk on record at every release threshold, so the certificate is anti-conservative by a factor set by the reviewers miss rate on the systems own defects. We show that seeded probes, which plant known defects and count the catches, do not identify this miss rate. A reviewers chance of missing a defect depends on how hard that defect is to spot, its detectability. A probe samples defects at detectabilities its designer chose. The systems own defects fall at detectabilities nothing chose, and converting the probes miss rate into theirs requires the very rate being sought. This holds whenever detectability is not fully captured by an observed score, since a score that captured it would let probes covering its range identify the rate. Cytology field validation builds probe sets whose miss rate is a floor on the true one. Even then the probe bounds the rate from below only, so it can reject a certificate and cannot grant one. The finite-sample label-noise corrections assume noise independent of the case given its label, but a reviewer misses the defects that read as plausible, and the condition they need fails in all four simulated reviewer profiles we examine. Testing that condition on an independent sample takes 185 adjudicated cases for a reviewer missing 85% of defects and 147,938 for one missing 3%. A second reader who adjudicates the cases the careful reviewer passed finds a miss with 95% confidence within 498, so looking for the noise directly costs far less than testing a model of it. In a public corpus of model answers graded by physicians, a physician passed 26.3% of the defects the other graders agreed were there (95% interval 21.6% to 31.7%, resampling whole conversations), on criteria about seeking context and stating uncertainty, and individual rates ran from 8.0% to 54.5%.",
  "abstract_shown": true,
  "affiliations": [
    "ConcertAI"
  ],
  "article_type": "PUBLISHAHEADOFPRINT",
  "arxiv": null,
  "author_total": 12,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "K. C. Kotabhattara"
    },
    {
      "name": "A. Siroha"
    },
    {
      "name": "P. Pal"
    },
    {
      "name": "S. K"
    },
    {
      "name": "V. S. Dhara"
    },
    {
      "name": "P. Prabhakar"
    },
    {
      "name": "N. Honkote"
    },
    {
      "name": "M. Marreddy"
    },
    {
      "name": "N. Namita"
    },
    {
      "name": "A. K. K"
    },
    {
      "name": "P. Saha"
    },
    {
      "name": "J. M. Agris"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/5enp1rq4fvpj5.bib",
  "cited_by": 0,
  "doi": "10.64898/2026.10.01.26364465",
  "fulltext_url": "https://www.medrxiv.org/content/10.64898/2026.10.01.26364465v1.full",
  "id": "5enp1rq4fvpj5",
  "issns": [],
  "landing_url": "https://www.medrxiv.org/content/10.64898/2026.10.01.26364465v1",
  "language": null,
  "licence": "CC-BY-NC-ND",
  "licence_label": "CC BY-NC-ND",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc-nd/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "medrxiv",
      "health informatics"
    ],
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7216292385",
  "pdf_url": "https://www.medrxiv.org/content/10.64898/2026.10.01.26364465v1.full.pdf",
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-01",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/5enp1rq4fvpj5.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:19:56Z",
      "native_id": "10.64898/2026.10.01.26364465",
      "source": "medrxiv",
      "url": "https://www.medrxiv.org/content/10.64898/2026.10.01.26364465v3"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "medrxiv:health informatics",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Planted Errors Cannot Measure What a Clinician Misses",
  "title_text": "Planted Errors Cannot Measure What a Clinician Misses",
  "type": "preprint",
  "updated_at": "2026-10-30T12:21:21Z",
  "url": "https://research.draconitic.com/paper/5enp1rq4fvpj5",
  "venue": "medRxiv",
  "versions": []
}