{
  "abstract": "ObjectiveAmbient AI scribes evaluated using frequency-based metrics such as word error rate (WER), which do not represent clinical consequence. We tested whether variation in these metrics tracks consequential transcription errors.\n\nMethodsWe constructed a multilingual corpus from five clinical dictation scripts spanning a complexity gradient, translated into 99 languages, rendered to synthetic speech under three acoustic conditions, and transcribed by a production ambient scribe. Six frequency metrics were computed. Three independent large language model raters from external providers assessed clinically meaningful error patterns in context using a Severity x Likelihood framework informed by UK digital clinical-safety-risk-management principles.\n\nResultsAcross 59,819 genuine transcription-error occurrences, 58,329 (97.5%) were LOW risk and 251 (0.42%) CRITICAL or HIGH. None of six frequency metrics showed a statistically detectable association with serious clinical risk across languages; correlations were small (absolute Spearman {rho}<0.16). A Severity x Likelihood sum remained strongly correlated with WER ({rho}=0.80), showing that the aggregate remained dominated by benign errors. At complexity level 3, low-resource languages had worse WER than high-resource languages ({beta}=+0.078, 95% CI +0.045 to +0.111; p<0.0001), without a detectable difference in CRITICAL/HIGH risk (OR 1.21, 95% CI 0.43 to 3.43; p=0.72).\n\nConsultation complexity was the principal predictor of serious risk (OR 3.06 per level, p<0.0001).\n\nConclusionAcross this controlled multilingual corpus, aggregate transcription-frequency metrics did not reliably track the sparse severe tail of clinically consequential errors. WER remains appropriate for transcription quality, but these data do not support its use alone as a proxy for clinical safety.\n\nContext-aware assessment of error consequence provides complementary information that frequency measures can dilute.\n\nHighlightsO_LIWER did not track serious clinical-risks rate across 77 languages C_LIO_LIFive related frequency metrics showed the same dissociation C_LIO_LISeverity-weighting remained dominated by common low-risk errors C_LIO_LIQuality tracked language resource; serious risk tracked complexity C_LIO_LIWe provide a reusable context-aware clinical-risk instrument C_LI",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": "PUBLISHAHEADOFPRINT",
  "arxiv": null,
  "author_total": 4,
  "authors": [
    {
      "name": "H. Bergman"
    },
    {
      "name": "V. Liu"
    },
    {
      "name": "B. Austin"
    },
    {
      "name": "R. Sanghera"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/85acn7bg3jg3v.bib",
  "cited_by": 0,
  "doi": "10.64898/2026.09.10.26362715",
  "fulltext_url": "https://www.medrxiv.org/content/10.64898/2026.09.10.26362715v1.full",
  "id": "85acn7bg3jg3v",
  "issns": [],
  "landing_url": "https://www.medrxiv.org/content/10.64898/2026.09.10.26362715v1",
  "language": null,
  "licence": "CC-BY-NC",
  "licence_label": "CC BY-NC",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "medrxiv",
      "health informatics"
    ],
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7213301711",
  "pdf_url": "https://www.medrxiv.org/content/10.64898/2026.09.10.26362715v1.full.pdf",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-10",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/85acn7bg3jg3v.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:18:35Z",
      "native_id": "10.64898/2026.09.10.26362715",
      "source": "medrxiv",
      "url": "https://www.medrxiv.org/content/10.64898/2026.09.10.26362715v1"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "medrxiv:health informatics",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Beyond word error rate: clinical risk as the necessary standard for ambient AI scribe evaluation: evidence from 77 global languages",
  "title_text": "Beyond word error rate: clinical risk as the necessary standard for ambient AI scribe evaluation: evidence from 77 global languages",
  "type": "preprint",
  "updated_at": "2026-10-30T12:18:35Z",
  "url": "https://research.draconitic.com/paper/85acn7bg3jg3v",
  "venue": "medRxiv",
  "versions": []
}