Integrating longitudinal mixed-effects modeling and XGBoost for chronic disease progression prediction
Abstract
Objective Accurate prediction of chronic disease progression is essential for improving clinical decision-making, personalized treatment, and healthcare resource allocation. However, longitudinal healthcare data are characterized by temporal dependencies, patient-specific heterogeneity, and complex nonlinear relationships that often limit the effectiveness of conventional predictive models. To address these challenges, this study proposes a hybrid Linear Mixed Model (LMM)–XGBoost framework for predicting chronic disease progression using simulated longitudinal healthcare data. Methods A large-scale synthetic dataset comprising 10,000 patients observed over 10 follow-up visits under slow, moderate, and rapid disease progression scenarios was generated through Monte Carlo simulation, yielding 300,000 longitudinal observations. Missing Completely at Random (MCAR) and Missing at Random (MAR) mechanisms were incorporated and handled using Multiple Imputation by Chained Equations (MICE). The proposed framework integrates LMM’s ability to capture longitudinal and patient-specific effects with Extreme Gradient Boosting (XGBoost)’s nonlinear predictive capability. Its performance was compared with Generalized Estimating Equations (GEE), Random Forest (RF), Light Gradient Boosting Machine (LightGBM) and XGBoost models. Results demonstrated that the proposed Hybrid LMM–XGBoost framework achieved the best predictive performance among all competing models, yielding an Root Mean Squared Error (RMSE) of 2.84, Mean Absolute Error (MAE) of 2.00, and coefficient of determination (R²) of 0.980. For disease-stage classification, the model attained an accuracy of 0.93, precision of 0.92, recall of 0.92, F1-score of 0.92, and Area Under the Curve (AUC) of 0.981. SHapley Additive exPlanations (SHAP) analysis identified previous disease severity (DS_Lag), Body Mass Index (BMI), glucose level (GLU), and blood pressure (BP) as the most influential predictors. Monte Carlo validation (B = 1000) confirmed the model’s robustness, with a mean RMSE of 2.85 (95% CI: 2.67–3.03). Conclusion The findings demonstrate the effectiveness of integrating longitudinal statistical modeling, machine learning, and explainable artificial intelligence for chronic disease progression prediction.
Related
- A machine learning-driven and explainable prediction model for in-hospital mortality in patients with diabetes mellitus and congestive heart failure: integrating XGBoost and SHAP approaches.
- Prediction of Parkinson disease progression from sparse longitudinal trajectories using multiclass likelihood contrast learning
- Effects of occupational noise exposure on menstrual abnormalities among female workers: a path analysis based on XGBoost and structural equation modeling
- Toward Transportable Acute Kidney Injury Prediction: An Explainable XGBoost Model with Temporal Validation Using MIMIC-IV
- Forecasting Alzheimer's disease progression using non-imaging clinical features and XGBoost
BibTeX
@article{parvin2026integrating,
title = {{Integrating longitudinal mixed-effects modeling and XGBoost for chronic disease progression prediction}},
author = {Rehana Parvin},
journal = {Digital Health},
year = {2026},
month = oct,
doi = {10.1177/20552076261501912},
url = {https://doi.org/10.1177/20552076261501912},
note = {CC BY},
}
RIS
TY - JOUR
TI - Integrating longitudinal mixed-effects modeling and XGBoost for chronic disease progression prediction
AU - Rehana Parvin
PY - 2026
DA - 2026/10/23
JO - Digital Health
DO - 10.1177/20552076261501912
UR - https://doi.org/10.1177/20552076261501912
AB - Objective Accurate prediction of chronic disease progression is essential for improving clinical decision-making, personalized treatment, and healthcare resource allocation. However, longitudinal healthcare data are characterized by temporal dependencies, patient-specific heterogeneity, and complex nonlinear relationships that often limit the effectiveness of conventional predictive models. To address these challenges, this study proposes a hybrid Linear Mixed Model (LMM)–XGBoost framework for predicting chronic disease progression using simulated longitudinal healthcare data. Methods A large-scale synthetic dataset comprising 10,000 patients observed over 10 follow-up visits under slow, moderate, and rapid disease progression scenarios was generated through Monte Carlo simulation, yielding 300,000 longitudinal observations. Missing Completely at Random (MCAR) and Missing at Random (MAR) mechanisms were incorporated and handled using Multiple Imputation by Chained Equations (MICE). The proposed framework integrates LMM’s ability to capture longitudinal and patient-specific effects with Extreme Gradient Boosting (XGBoost)’s nonlinear predictive capability. Its performance was compared with Generalized Estimating Equations (GEE), Random Forest (RF), Light Gradient Boosting Machine (LightGBM) and XGBoost models. Results demonstrated that the proposed Hybrid LMM–XGBoost framework achieved the best predictive performance among all competing models, yielding an Root Mean Squared Error (RMSE) of 2.84, Mean Absolute Error (MAE) of 2.00, and coefficient of determination (R²) of 0.980. For disease-stage classification, the model attained an accuracy of 0.93, precision of 0.92, recall of 0.92, F1-score of 0.92, and Area Under the Curve (AUC) of 0.981. SHapley Additive exPlanations (SHAP) analysis identified previous disease severity (DS_Lag), Body Mass Index (BMI), glucose level (GLU), and blood pressure (BP) as the most influential predictors. Monte Carlo validation (B = 1000) confirmed the model’s robustness, with a mean RMSE of 2.85 (95% CI: 2.67–3.03). Conclusion The findings demonstrate the effectiveness of integrating longitudinal statistical modeling, machine learning, and explainable artificial intelligence for chronic disease progression prediction.
SN - 2055-2076
ER -