Large Language Model-Enhanced Electronic Frailty Assessment and Prognostic Performance in Hospitalized Adults: A Retrospective Cohort Study
- 1Brigham and Women's Hospital
Abstract
Background: Electronic frailty indices (eFIs) derived from structured electronic health record (EHR) data may incompletely capture functional deficits. Large language models (LLMs) can extract frailty-related information from clinical notes, but whether improved frailty ascertainment improves prognostic performance is uncertain. Objective: To compare frailty ascertainment and prognostic performance between standard and LLM-enhanced eFIs among hospitalized adults. Design: Retrospective cohort study Participants: Adults aged 50 years or older enrolled in two studies at an academic medical center and affiliated community hospital between February 2022 and April 2024. Main Measures: A validated 35-deficit diagnosis-based eFI and an LLM-enhanced eFI incorporating deficits extracted from physical therapy (PT) documentation using GPT-4o. Frailty was defined using an index threshold of 0.20 or more. The primary outcome was a composite of discharge to a facility, readmission, or death within 90 days. Multivariable logistic regression models assessed associations with the outcome, and discrimination was compared using area under the receiver operating characteristic curve (AUROC). Secondary analyses examined threshold-based classification and PT-documentation availability; prespecified sensitivity analyses evaluated population and outcome restrictions. Results: Among 616 hospitalizations, mean (SD) age was 70.9 (10.2) years, 329 (53.4%) had PT documentation, and 320 (51.9%) experienced the composite outcome. Standard and LLM-enhanced eFIs classified 216 (35.1%) and 362 (58.8%) hospitalizations as frail, respectively. Both were independently associated with the outcome (standard eFI: adjusted OR, 1.98 [95% CI, 1.39-2.83]; LLM-enhanced eFI: adjusted OR, 2.46 [95% CI, 1.75-3.47]). Discrimination did not differ significantly (AUROC, 0.636 vs 0.660; difference, 0.024 [95% CI, -0.006 to 0.053]; p=0.12). LLM enhancement increased sensitivity (42.5% to 69.4%) but decreased specificity (73.0% to 52.7%). Conclusions: LLM enhancement substantially increased frailty ascertainment and sensitivity for adverse outcomes but did not significantly improve prognostic discrimination. Thus, LLM-enhanced frailty assessment may depend on whether the intended use prioritizes broad identification of functional vulnerability or prediction of adverse post-hospital outcomes.
Related
- Interest of the multidimensional prognostic index (MPI) as an assessment tool in hospitalized patients in geriatrics
- CO038 | PROGNOSTIC PERFORMANCE OF BLEEDING SEVERITY CLASSIFICATIONS IN HOSPITALIZED PATIENTS WITH GASTROINTESTINAL BLEEDING ON DIRECT ORAL ANTICOAGULANTS
- Development and Internal Validation of an Electronic Medical Record-Based Machine Learning Model for Frailty Prediction in Hospitalized Patients: Retrospective Cohort Study.
- Prognostic performance of the 3-h COBRA score for ICU admission in comatose poisoned patients: a retrospective study.
- LDAR shows more consistent prognostic performance than other albumin-derived indices for 28-day ICU mortality in critically ill patients with urosepsis: evidence from a dual-cohort retrospective study
BibTeX
@misc{dalal2026large,
title = {{Large Language Model-Enhanced Electronic Frailty Assessment and Prognostic Performance in Hospitalized Adults: A Retrospective Cohort Study}},
author = {A. K. Dalal and J. Lundstedt and N. Ernecoff and S. Plombon and S. Streiter and M. Leeson and J. Plasek and S. Mueller and J. Fiskio and L. Zhou and S. Lipsitz and A. Orkaby and J. Loewenthal},
howpublished = {medRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.04.26367272},
url = {https://www.medrxiv.org/content/10.64898/2026.11.04.26367272v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Large Language Model-Enhanced Electronic Frailty Assessment and Prognostic Performance in Hospitalized Adults: A Retrospective Cohort Study
AU - A. K. Dalal
AU - J. Lundstedt
AU - N. Ernecoff
AU - S. Plombon
AU - S. Streiter
AU - M. Leeson
AU - J. Plasek
AU - S. Mueller
AU - J. Fiskio
AU - L. Zhou
AU - S. Lipsitz
AU - A. Orkaby
AU - J. Loewenthal
PY - 2026
DA - 2026/11/04
PB - medRxiv
DO - 10.64898/2026.11.04.26367272
UR - https://www.medrxiv.org/content/10.64898/2026.11.04.26367272v1
AB - Background: Electronic frailty indices (eFIs) derived from structured electronic health record (EHR) data may incompletely capture functional deficits. Large language models (LLMs) can extract frailty-related information from clinical notes, but whether improved frailty ascertainment improves prognostic performance is uncertain. Objective: To compare frailty ascertainment and prognostic performance between standard and LLM-enhanced eFIs among hospitalized adults. Design: Retrospective cohort study Participants: Adults aged 50 years or older enrolled in two studies at an academic medical center and affiliated community hospital between February 2022 and April 2024. Main Measures: A validated 35-deficit diagnosis-based eFI and an LLM-enhanced eFI incorporating deficits extracted from physical therapy (PT) documentation using GPT-4o. Frailty was defined using an index threshold of 0.20 or more. The primary outcome was a composite of discharge to a facility, readmission, or death within 90 days. Multivariable logistic regression models assessed associations with the outcome, and discrimination was compared using area under the receiver operating characteristic curve (AUROC). Secondary analyses examined threshold-based classification and PT-documentation availability; prespecified sensitivity analyses evaluated population and outcome restrictions. Results: Among 616 hospitalizations, mean (SD) age was 70.9 (10.2) years, 329 (53.4%) had PT documentation, and 320 (51.9%) experienced the composite outcome. Standard and LLM-enhanced eFIs classified 216 (35.1%) and 362 (58.8%) hospitalizations as frail, respectively. Both were independently associated with the outcome (standard eFI: adjusted OR, 1.98 [95% CI, 1.39-2.83]; LLM-enhanced eFI: adjusted OR, 2.46 [95% CI, 1.75-3.47]). Discrimination did not differ significantly (AUROC, 0.636 vs 0.660; difference, 0.024 [95% CI, -0.006 to 0.053]; p=0.12). LLM enhancement increased sensitivity (42.5% to 69.4%) but decreased specificity (73.0% to 52.7%). Conclusions: LLM enhancement substantially increased frailty ascertainment and sensitivity for adverse outcomes but did not significantly improve prognostic discrimination. Thus, LLM-enhanced frailty assessment may depend on whether the intended use prioritizes broad identification of functional vulnerability or prediction of adverse post-hospital outcomes.
ER -