Development and internal validation of machine learning models for identifying suicidal behavior in inpatients with mood disorders.
- 1The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University & the Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing, China.
- 2Department of Clinical Psychology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Shandong, China.
Abstract
Background
Suicidal behavior (SB) is a major concern in mood disorders. Models developed from retrospective records are vulnerable to ambiguous targets, data leakage, and inappropriate handling of class imbalance. We developed and internally validated a model to identify the risk of suicidal behavior in patients with mood disorders upon admission.Methods
The dataset included 1, 099 inpatients with ICD-10 mood disorders admitted to Beijing Anding Hospital in 2022. Exclusion of 20 records with more than 20% row-level missingness and one record without an outcome left 1, 078 patients, including 126 with SB. Current SB comprised documented suicidal ideation, planning, or attempt within the three months before index admission, including the admission assessment. Previous SB and previous suicide attempt (SA) referred to events more than three months before admission. Elastic Net selected 14 predictors from 65 candidates in the development set. We compared 13 classifiers from six model families. All preprocessing and resampling were confined to training folds, and evaluation used only observed patients. Assessment included repeated fivefold cross-validation, probability calibration, bootstrap confidence intervals, precision-recall and decision-curve analyses, clinical baselines, and sensitivity analyses.Results
The analytic cohort comprised 952 SB-negative and 126 SB-positive patients (11.69%). The development set contained 864 patients (97 SB-positive), and the observed test set contained 214 patients (29 SB-positive). Calibrated linear discriminant analysis (LDA) provided the most consistent balance of discrimination and calibration. In repeated cross-validation, mean AUC was 0.721 ± 0.052, mean AUPRC was 0.268 ± 0.060, and mean Brier score was 0.095. In the observed test set, AUC was 0.704 (0.597-0.803), AUPRC was 0.283 (0.198-0.453), and Brier score was 0.111. At the training-derived threshold of 0.111, sensitivity was 0.552 and specificity was 0.659. The leading SHAP contributions were onset-polarity category, previous SA, age, and previous SB, but all attributions were associational.Conclusions
Model performance was acceptable. These internally validated findings are exploratory and require prospective assessment, standardized outcome measurement, and external validation before clinical use.
Related
- Refining E-detectioN Tools for Emerging mental disoRders (ENTER) to identify young people at clinical high risk for psychosis in the community: model development and internal validation
- Development and Internal Validation of an Interpretable Machine Learning Model for Identifying Past-Year Nonsuicidal Self-Injury Among Adolescents With Depression: Retrospective Study.
- Development and Internal Validation of a Preoperative Risk Stratification Model for Difficult Laparoscopic Cholecystectomy in Acute Cholecystitis
- Development and internal validation of a clinical risk score for early delirium detection in older emergency department patients
- Development and internal validation of a nomogram for predicting bacterial meningitis in preterm infants with late-onset sepsis
BibTeX
@article{liang2026development,
title = {{Development and internal validation of machine learning models for identifying suicidal behavior in inpatients with mood disorders.}},
author = {Sixiang Liang and Qitong Jiang and Jinhe Zhang and Yannan Deng and Yuxuan Xu and Guangwei Zhao and Xinyu Liu and Yujie Xing and Hongye Yu and Chen Hu and Sha Sha},
journal = {Frontiers in psychiatry},
year = {2026},
month = oct,
doi = {10.3389/fpsyt.2026.1950186},
url = {https://europepmc.org/article/MED/42898953},
note = {CC BY},
}
RIS
TY - JOUR
TI - Development and internal validation of machine learning models for identifying suicidal behavior in inpatients with mood disorders.
AU - Sixiang Liang
AU - Qitong Jiang
AU - Jinhe Zhang
AU - Yannan Deng
AU - Yuxuan Xu
AU - Guangwei Zhao
AU - Xinyu Liu
AU - Yujie Xing
AU - Hongye Yu
AU - Chen Hu
AU - Sha Sha
PY - 2026
DA - 2026/10/05
JO - Frontiers in psychiatry
DO - 10.3389/fpsyt.2026.1950186
UR - https://europepmc.org/article/MED/42898953
AB - Background Suicidal behavior (SB) is a major concern in mood disorders. Models developed from retrospective records are vulnerable to ambiguous targets, data leakage, and inappropriate handling of class imbalance. We developed and internally validated a model to identify the risk of suicidal behavior in patients with mood disorders upon admission.Methods The dataset included 1, 099 inpatients with ICD-10 mood disorders admitted to Beijing Anding Hospital in 2022. Exclusion of 20 records with more than 20% row-level missingness and one record without an outcome left 1, 078 patients, including 126 with SB. Current SB comprised documented suicidal ideation, planning, or attempt within the three months before index admission, including the admission assessment. Previous SB and previous suicide attempt (SA) referred to events more than three months before admission. Elastic Net selected 14 predictors from 65 candidates in the development set. We compared 13 classifiers from six model families. All preprocessing and resampling were confined to training folds, and evaluation used only observed patients. Assessment included repeated fivefold cross-validation, probability calibration, bootstrap confidence intervals, precision-recall and decision-curve analyses, clinical baselines, and sensitivity analyses.Results The analytic cohort comprised 952 SB-negative and 126 SB-positive patients (11.69%). The development set contained 864 patients (97 SB-positive), and the observed test set contained 214 patients (29 SB-positive). Calibrated linear discriminant analysis (LDA) provided the most consistent balance of discrimination and calibration. In repeated cross-validation, mean AUC was 0.721 ± 0.052, mean AUPRC was 0.268 ± 0.060, and mean Brier score was 0.095. In the observed test set, AUC was 0.704 (0.597-0.803), AUPRC was 0.283 (0.198-0.453), and Brier score was 0.111. At the training-derived threshold of 0.111, sensitivity was 0.552 and specificity was 0.659. The leading SHAP contributions were onset-polarity category, previous SA, age, and previous SB, but all attributions were associational.Conclusions Model performance was acceptable. These internally validated findings are exploratory and require prospective assessment, standardized outcome measurement, and external validation before clinical use.
SN - 1664-0640
ER -