Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records
Abstract
Objective:To characterize the kinds of internal documentation inconsistencies a general-domain large language model (LLM) can surface from real-world discharge summaries, and to identify recurring failure modes that limit reliability at scale. Materials and Methods: We applied a two-stage LLM pipeline---open-ended candidate identification (Gemini 2.5 Pro) followed by context-grounded verification (Gemini 2.5 Flash)---to 3000 MIMIC-IV-Note discharge summaries drawn as three contiguous 1000-note batches from distinct positions in the corpus. Two clinicians independently reviewed a domain-stratified sample of 268 retained candidates; a third adjudicated disagreements. Results: Stage~1 proposed 11333 candidates; 3461 were retained after Stage~2 verification---machine-generated candidates---with at least one in 69.7% of admissions. Medication (43.2%) and diagnosis or clinical-finding (25.3%) candidates predominated. In the reviewed sample, reviewers agreed on 91.0\% of candidates, and 253 of 268 (94.4\%) were judged valid after adjudication. Representative examples from all seven domains carried direct implications for clinical reasoning or patient safety. Review also revealed recurring failure modes involving temporal reasoning, evolving diagnoses, and outpatient-prescribing conventions. Discussion: Detection is highly context-dependent: judging a flagged pair requires locating each statement and assessing whether the conflict reflects a true contradiction or missing context. We propose a graded ontology spanning strict contradiction, ambiguity, and gap, with a schema characterizing each flagged case by type, section, scope, domain, attribute, and inconsistency axis. Conclusion: This formative study establishes a methodological foundation and conceptual framework to guide subsequent validated, large-scale EHR-inconsistency analysis.
Related
- Enhancing Spectral Embedding through Robust and Flexible Knowledge Transfer in Electronic Health Records
- Early life factors documented in electronic health records predict recurrent acute otitis media
- Diagnosing Inconsistencies in 4d N=1 Gauge Theories with Explainable AI
- FPScan: An Automated Constraint-Based Analyzer for Floating-Point Anomaly Detection
- Automated Detection of Match Phases in Football from Spatio-Temporal Tracking Data Using Graph Neural Networks
BibTeX
@misc{lu2026toward,
title = {{Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records}},
author = {Jian Lu and Panyu Chen and Miriam Treggiari and Robert Blessing and Danyang Zhuo and Chunhua Weng and William W. Stead and Anru R. Zhang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2607.22954},
eprint = {2607.22954},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.22954},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records
AU - Jian Lu
AU - Panyu Chen
AU - Miriam Treggiari
AU - Robert Blessing
AU - Danyang Zhuo
AU - Chunhua Weng
AU - William W. Stead
AU - Anru R. Zhang
PY - 2026
DA - 2026/10/18
PB - arXiv
DO - 10.48550/arxiv.2607.22954
UR - https://arxiv.org/abs/2607.22954
AB - Objective:To characterize the kinds of internal documentation inconsistencies a general-domain large language model (LLM) can surface from real-world discharge summaries, and to identify recurring failure modes that limit reliability at scale. Materials and Methods: We applied a two-stage LLM pipeline---open-ended candidate identification (Gemini 2.5 Pro) followed by context-grounded verification (Gemini 2.5 Flash)---to 3000 MIMIC-IV-Note discharge summaries drawn as three contiguous 1000-note batches from distinct positions in the corpus. Two clinicians independently reviewed a domain-stratified sample of 268 retained candidates; a third adjudicated disagreements. Results: Stage~1 proposed 11333 candidates; 3461 were retained after Stage~2 verification---machine-generated candidates---with at least one in 69.7% of admissions. Medication (43.2%) and diagnosis or clinical-finding (25.3%) candidates predominated. In the reviewed sample, reviewers agreed on 91.0\% of candidates, and 253 of 268 (94.4\%) were judged valid after adjudication. Representative examples from all seven domains carried direct implications for clinical reasoning or patient safety. Review also revealed recurring failure modes involving temporal reasoning, evolving diagnoses, and outpatient-prescribing conventions. Discussion: Detection is highly context-dependent: judging a flagged pair requires locating each statement and assessing whether the conflict reflects a true contradiction or missing context. We propose a graded ontology spanning strict contradiction, ambiguity, and gap, with a schema characterizing each flagged case by type, section, scope, domain, attribute, and inconsistency axis. Conclusion: This formative study establishes a methodological foundation and conceptual framework to guide subsequent validated, large-scale EHR-inconsistency analysis.
ER -