A local outpatient practice-level prediction model for short-term psychiatric emergency presentation
Abstract
Objectives: To determine whether statewide health information exchange (HIE) data can support practice-level prediction of short-term psychiatric emergency presentation, and to compare a locally trained model with general-purpose clinical risk scores.
Materials and Methods: Retrospective cohort of patients at 1 Montana psychiatry practice with records in the Big Sky Care Connect statewide HIE. Structured HIE data were linked to zip code-level Social Vulnerability Index and Area Deprivation Index measures. Each observation was a patient snapshot. Logistic regression, random forest, and LightGBM models were tuned with 5 × 5 nested cross-validation and evaluated on a temporally held-out test set. The primary outcome was psychiatric or substance-related emergency department (ED) presentation within 30 days. Discrimination was compared with a parsimonious clinical baseline and the LACE and Elixhauser indices.
Results: The test set contained 16 469 snapshots, of which 343 (2.1%) preceded a qualifying ED presentation (102 visits among 68 patients). A random forest model achieved area under the receiver-operator characteristic curve 0.85 (95% CI, 0.75-0.91). At a threshold targeting one-third recall, precision was 0.29 (0.12-0.44) and number needed to evaluate 3.4 (2.3-8.1). Discrimination exceeded the clinical baseline (Δ area under the curve 0.10), LACE (0.19), and Elixhauser (0.27); all P values less than .001.
Discussion: Structured HIE data, without clinical notes, supported a practice-specific model that outperformed general-purpose scores such as LACE and Elixhauser. Wide CIs reflect a small event count and constrain point-estimate claims.
Conclusion: health information exchange infrastructure may allow participating practices to build locally calibrated psychiatric risk models. Prospective study of clinical utility is needed.
Related
- Bridging the Gap: A Quality Improvement Project Improving Communication Between Community-Based Inpatient and Outpatient Psychiatric Providers
- Yield of Low Serum Vitamin B12 Results in General Pediatric Outpatient Practice: Age-Specific Patterns and Associations with Testing Indications and Recorded Clinical Features
- Ethical approach assessment in emergency psychiatric care and its association with malpractice risk awareness : A methodological and cross-sectional study
- Prenatal cannabis use disorder and postpartum psychiatric emergency department visits: A retrospective cohort study of trends and associations in California, 2016–2021
- Recognising Mild Cognitive Impairment and Early Alzheimer's Disease in General and Psychiatric Practice: Clinical Clues and Appropriate Referral
BibTeX
@article{havlik2026local,
title = {{A local outpatient practice-level prediction model for short-term psychiatric emergency presentation}},
author = {John L Havlik and Ben Tyrrell and Jeanette Polaschek and Nathan Bell and Eric R Arzubi},
journal = {JAMIA open},
year = {2026},
month = oct,
doi = {10.1093/jamiaopen/ooag215},
url = {https://europepmc.org/article/PMC/PMC13725626},
note = {CC BY},
}
RIS
TY - JOUR
TI - A local outpatient practice-level prediction model for short-term psychiatric emergency presentation
AU - John L Havlik
AU - Ben Tyrrell
AU - Jeanette Polaschek
AU - Nathan Bell
AU - Eric R Arzubi
PY - 2026
DA - 2026/10/01
JO - JAMIA open
DO - 10.1093/jamiaopen/ooag215
UR - https://europepmc.org/article/PMC/PMC13725626
AB - Objectives: To determine whether statewide health information exchange (HIE) data can support practice-level prediction of short-term psychiatric emergency presentation, and to compare a locally trained model with general-purpose clinical risk scores. Materials and Methods: Retrospective cohort of patients at 1 Montana psychiatry practice with records in the Big Sky Care Connect statewide HIE. Structured HIE data were linked to zip code-level Social Vulnerability Index and Area Deprivation Index measures. Each observation was a patient snapshot. Logistic regression, random forest, and LightGBM models were tuned with 5 × 5 nested cross-validation and evaluated on a temporally held-out test set. The primary outcome was psychiatric or substance-related emergency department (ED) presentation within 30 days. Discrimination was compared with a parsimonious clinical baseline and the LACE and Elixhauser indices. Results: The test set contained 16 469 snapshots, of which 343 (2.1%) preceded a qualifying ED presentation (102 visits among 68 patients). A random forest model achieved area under the receiver-operator characteristic curve 0.85 (95% CI, 0.75-0.91). At a threshold targeting one-third recall, precision was 0.29 (0.12-0.44) and number needed to evaluate 3.4 (2.3-8.1). Discrimination exceeded the clinical baseline (Δ area under the curve 0.10), LACE (0.19), and Elixhauser (0.27); all P values less than .001. Discussion: Structured HIE data, without clinical notes, supported a practice-specific model that outperformed general-purpose scores such as LACE and Elixhauser. Wide CIs reflect a small event count and constrain point-estimate claims. Conclusion: health information exchange infrastructure may allow participating practices to build locally calibrated psychiatric risk models. Prospective study of clinical utility is needed.
SN - 2574-2531
ER -