Language model-assisted label refinement for accurate sepsis detection from electronic health records
- 1Saïd Business School, University of Oxford
Abstract
Sepsis is a leading cause of hospital mortality, yet timely recognition is hampered by nonspecific presentations and label noise in code-based case definitions. We developed STRIDE, a machine-learning framework for sepsis detection across seven hospitals with a scalable approach to label quality. We refined a pragmatic operational definition using a large language model applied to discharge summaries, with an independent physician-adjudicated cohort as the gold standard. We compared 8-, 24-, and 48-hour observation windows and benchmarked against SOFA, SIRS, and Epic, assessing calibration and discrimination. Among 356,610 encounters, the 8-hour model achieved an AUC of 0.960 in derivation and 0.878 in physician-adjudicated validation, matching or outperforming longer-window models. STRIDE outperformed SOFA and Epic on discrimination and showed favorable calibration by Brier score, while retaining strong discrimination among SIRS-positive non-septic encounters and reaching 78.2% specificity at 80% sensitivity in validation. These findings support accurate sepsis surveillance while limiting unnecessary alerts and requiring minimal prior history.
Related
- GCTRNet: global context-gated temporal refinement for short-window ECG-based premature ventricular contraction detection.
- Multimodal deep learning with particle swarm guided feature refinement for automated pain detection
- Characterizing cognitive concerns in subjective cognitive decline using large language models and electronic health record notes
- Effect of guideline-assisted prompting on large language model performance in medication-related osteonecrosis of the jaw–related endodontic decision-making
- ASC-YOLOv8n: Enhanced multi-scale feature fusion for accurate detection of four cigar appearance defects
BibTeX
@misc{kalyvianakis2026language,
title = {{Language model-assisted label refinement for accurate sepsis detection from electronic health records}},
author = {I. Kitsos Kalyvianakis and C. Perez de Amezaga and E. S. Lee and A. Palmer and J. Garner and U. S. Wu and A. Orfanoudaki},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.08.26362428},
url = {https://www.medrxiv.org/content/10.64898/2026.09.08.26362428v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Language model-assisted label refinement for accurate sepsis detection from electronic health records
AU - I. Kitsos Kalyvianakis
AU - C. Perez de Amezaga
AU - E. S. Lee
AU - A. Palmer
AU - J. Garner
AU - U. S. Wu
AU - A. Orfanoudaki
PY - 2026
DA - 2026/09/08
PB - medRxiv
DO - 10.64898/2026.09.08.26362428
UR - https://www.medrxiv.org/content/10.64898/2026.09.08.26362428v1
AB - Sepsis is a leading cause of hospital mortality, yet timely recognition is hampered by nonspecific presentations and label noise in code-based case definitions. We developed STRIDE, a machine-learning framework for sepsis detection across seven hospitals with a scalable approach to label quality. We refined a pragmatic operational definition using a large language model applied to discharge summaries, with an independent physician-adjudicated cohort as the gold standard. We compared 8-, 24-, and 48-hour observation windows and benchmarked against SOFA, SIRS, and Epic, assessing calibration and discrimination. Among 356,610 encounters, the 8-hour model achieved an AUC of 0.960 in derivation and 0.878 in physician-adjudicated validation, matching or outperforming longer-window models. STRIDE outperformed SOFA and Epic on discrimination and showed favorable calibration by Brier score, while retaining strong discrimination among SIRS-positive non-septic encounters and reaching 78.2% specificity at 80% sensitivity in validation. These findings support accurate sepsis surveillance while limiting unnecessary alerts and requiring minimal prior history.
ER -