Performance of GPT-5.5 thinking for Manchester Triage System-based emergency department triage: agreement with an expert reference standard and prediction of short-term outcomes.
- 1Department of Emergency Medicine, University of Health Sciences, Haseki Training and Research Hospital, Ugur Mumcu, Belediye Str. No:7 Sultangazi, Istanbul, Türkiye, Turkey. [email protected].
- 2Department of Emergency Medicine, University of Health Sciences, Haseki Training and Research Hospital, Ugur Mumcu, Belediye Str. No:7 Sultangazi, Istanbul, Türkiye, Turkey.
- 3Department of Emergency Medicine, University of Health Sciences, Kanuni Sultan Suleyman Training and Research Hospital, Istanbul, Türkiye, Turkey.
Abstract
Objective
To evaluate the agreement between Manchester Triage System (MTS) category assignments generated by GPT-5.5 Thinking and an expert-derived reference standard in a selected cohort of emergency department (ED) patients, and to assess the model's ability to predict hospital admission and 72-hour mortality.Methods
This single-center observational study prospectively sampled consecutive ED presentations during a prespecified continuous 72-hour window, with triage-time variables and secondary outcomes obtained from hospital records. Patients classified as MTS Category 5 (Blue; nonurgent) by the expert reference standard were excluded from the analytic cohort. GPT-5.5 Thinking was prompted in a standardized single-shot format using structured clinical vignettes derived from triage data. The primary endpoint was agreement between model-assigned MTS categories and an expert reference standard established by two independent blinded emergency medicine specialist rater teams, with disagreements resolved through senior adjudication. Exact agreement, within-1-level agreement, overtriage, undertriage, and dangerous undertriage were calculated. Secondary analyses evaluated prediction of hospital admission and 72-hour mortality using diagnostic performance metrics.Results
A total of 1,841 patients were included. Overall agreement between GPT-5.5 Thinking and the expert reference standard was substantial (κ = 0.721; 95% CI 0.691-0.740). Exact agreement rate was 62.0%, and within-1-level agreement rate was 99.0%. Overtriage occurred in 35.3% of cases, undertriage in 2.7%, and dangerous undertriage in 0.2%. Exact agreement rate was significantly lower in patients aged ≥ 65 years than in younger adults (57.1% vs. 64.2%, p = 0.013). For hospital admission, accuracy rate was 79.9%, sensitivity 83.8%, positive predictive value (PPV) 45.2%, and negative predictive value (NPV) 95.9%. For 72-hour mortality, accuracy rate was 86.5%, sensitivity 72.3%, PPV 12.6%, and NPV 99.1%.Conclusion
Within this selected ED cohort, GPT-5.5 Thinking showed substantial agreement with expert MTS-based ED triage and generally preserved the ordinal hierarchy of clinical urgency. Its performance was characterized by frequent adjacent-level agreement, predominant overtriage, and very rare but nonzero dangerous undertriage. The model also showed stronger rule-out than rule-in characteristics for hospital admission and 72-hour mortality.
Related
- Comparative performance of pre- and post-endoscopic risk scores for outcome prediction in endoscopy-confirmed upper gastrointestinal bleeding in the emergency department : A retrospective single-center study
- Artificial Intelligence Echocardiographic Interpretation in the Emergency Department: Agreement, Interpretation Yield and Reporting Scope
- On-Device ECG and Vital-Sign Prediction of 24-Hour Mortality and Unplanned ICU Admission in Emergency Department Patients: A Retrospective Evaluation of MEDI-AI
- Performance of the Heatwave ED Forecast Tool in the Emergency Department Network of Ticino Canton, Switzerland
- Implementation and Performance of a Clinical Debriefing Programme in a Paediatric Emergency Department: A Descriptive Qualitative Review
BibTeX
@article{fettahoglu2026performance,
title = {{Performance of GPT-5.5 thinking for Manchester Triage System-based emergency department triage: agreement with an expert reference standard and prediction of short-term outcomes.}},
author = {Sureyya Tuba Fettahoglu and Adem Az and Ozgur Sogut and Tarik Akdemir and Muhammed Furkan Ozden and Salih Fettahoglu and Mucahide Samli Helvacı and Enes Sisman and Mehmet Ozcomlekci and Emir Bilici and Nurbaki Kilic and Tugay Saricicek and Ayse Feyza Celik and Zekeriya Colak and Zulkuf Eralp},
journal = {BMC emergency medicine},
year = {2026},
month = jul,
doi = {10.1186/s12873-026-01704-7},
url = {https://europepmc.org/article/MED/42902219},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - Performance of GPT-5.5 thinking for Manchester Triage System-based emergency department triage: agreement with an expert reference standard and prediction of short-term outcomes.
AU - Sureyya Tuba Fettahoglu
AU - Adem Az
AU - Ozgur Sogut
AU - Tarik Akdemir
AU - Muhammed Furkan Ozden
AU - Salih Fettahoglu
AU - Mucahide Samli Helvacı
AU - Enes Sisman
AU - Mehmet Ozcomlekci
AU - Emir Bilici
AU - Nurbaki Kilic
AU - Tugay Saricicek
AU - Ayse Feyza Celik
AU - Zekeriya Colak
AU - Zulkuf Eralp
PY - 2026
DA - 2026/07/01
JO - BMC emergency medicine
DO - 10.1186/s12873-026-01704-7
UR - https://europepmc.org/article/MED/42902219
AB - Objective To evaluate the agreement between Manchester Triage System (MTS) category assignments generated by GPT-5.5 Thinking and an expert-derived reference standard in a selected cohort of emergency department (ED) patients, and to assess the model's ability to predict hospital admission and 72-hour mortality.Methods This single-center observational study prospectively sampled consecutive ED presentations during a prespecified continuous 72-hour window, with triage-time variables and secondary outcomes obtained from hospital records. Patients classified as MTS Category 5 (Blue; nonurgent) by the expert reference standard were excluded from the analytic cohort. GPT-5.5 Thinking was prompted in a standardized single-shot format using structured clinical vignettes derived from triage data. The primary endpoint was agreement between model-assigned MTS categories and an expert reference standard established by two independent blinded emergency medicine specialist rater teams, with disagreements resolved through senior adjudication. Exact agreement, within-1-level agreement, overtriage, undertriage, and dangerous undertriage were calculated. Secondary analyses evaluated prediction of hospital admission and 72-hour mortality using diagnostic performance metrics.Results A total of 1,841 patients were included. Overall agreement between GPT-5.5 Thinking and the expert reference standard was substantial (κ = 0.721; 95% CI 0.691-0.740). Exact agreement rate was 62.0%, and within-1-level agreement rate was 99.0%. Overtriage occurred in 35.3% of cases, undertriage in 2.7%, and dangerous undertriage in 0.2%. Exact agreement rate was significantly lower in patients aged ≥ 65 years than in younger adults (57.1% vs. 64.2%, p = 0.013). For hospital admission, accuracy rate was 79.9%, sensitivity 83.8%, positive predictive value (PPV) 45.2%, and negative predictive value (NPV) 95.9%. For 72-hour mortality, accuracy rate was 86.5%, sensitivity 72.3%, PPV 12.6%, and NPV 99.1%.Conclusion Within this selected ED cohort, GPT-5.5 Thinking showed substantial agreement with expert MTS-based ED triage and generally preserved the ordinal hierarchy of clinical urgency. Its performance was characterized by frequent adjacent-level agreement, predominant overtriage, and very rare but nonzero dangerous undertriage. The model also showed stronger rule-out than rule-in characteristics for hospital admission and 72-hour mortality.
SN - 1471-227X
ER -