Handwriting-based Alzheimer’s disease classification on the DARWIN dataset: Repeated nested validation, task-level reduction, and cross-validated explainability
- 1Department of Computing, School of Electrical Engineering and Computer Science, National University of Sciences and Technology (NUST), Islamabad, Pakistan
- 2School of Nursing, Shandong Xiehe University, Jinan, Shandong, China
Abstract
Online handwriting provides a low-burden source of cognitive–motor measurements, but small datasets can yield optimistic results when preprocessing, task selection, or explanation is not separated from evaluation. We evaluated eight classifiers on the DARWIN dataset (174 participants; 25 tasks; 450 predictors) using ten repetitions of 5-outer/3-inner-fold nested stratified cross-validation. All preprocessing, hyperparameter selection, and calibration were restricted to outer training data. We added a training-derived operating point, fully nested seven-task selection, cross-model subset transfer, foldwise SHAP analysis, and an all-feature SHAP sensitivity analysis. Random Forest had the highest descriptive ROC-AUC (0.938 ± 0.033 across 50 held-out folds). The Friedman omnibus test indicated model differences (p < 0.001), but no one of 28 paired comparisons remained significant after Holm correction. Training-derived thresholds (median 0.461) yielded held-out sensitivity 0.869 and specificity 0.831; these are exploratory operating characteristics, not a clinical cutoff. Nested seven-task selection achieved AUC 0.916 ± 0.037, compared with 0.956 ± 0.030 for all 25 tasks, and selected subsets were unstable (mean Jaccard 0.195). Cross-validated SHAP emphasized temporal variables, although an all-450-feature sensitivity analysis showed that individual ranks depend partly on ANOVA preselection. The study contributes an integrated, leakage-controlled evaluation rather than a new classifier or a validated diagnostic instrument; external prospective validation remains necessary.
Related
- CBAM-enhanced MaxViT-Tiny for gastrointestinal endoscopy image classification: Robustness, explainability, and external validation
- MEMOIR-VLM—a multimodal vision-language model for Alzheimer's disease classification and question answering
- Evolution of deep learning for structural magnetic resonance imaging analysis in Alzheimer’s disease: from slice-level classification to brain structure modeling
- Frequency-Resolved Connectivity Graphs for EEG-Based Classification of Alzheimer and Frontotemporal Dementia
- Unveiling the neuroinflammatory potential of aspartame in Alzheimer’s disease: A comprehensive network toxicology and experimental validation approach
BibTeX
@article{fatima2026handwritingbased,
title = {{Handwriting-based Alzheimer’s disease classification on the DARWIN dataset: Repeated nested validation, task-level reduction, and cross-validated explainability}},
author = {Tehreem Fatima and Tahira Anwar Lashari and Momina Moetesum and Sana Anwar Lashari},
journal = {PLOS One},
year = {2026},
month = oct,
doi = {10.1371/journal.pone.0361117},
url = {https://doi.org/10.1371/journal.pone.0361117},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Handwriting-based Alzheimer’s disease classification on the DARWIN dataset: Repeated nested validation, task-level reduction, and cross-validated explainability
AU - Tehreem Fatima
AU - Tahira Anwar Lashari
AU - Momina Moetesum
AU - Sana Anwar Lashari
PY - 2026
DA - 2026/10/26
JO - PLOS One
DO - 10.1371/journal.pone.0361117
UR - https://doi.org/10.1371/journal.pone.0361117
AB - Online handwriting provides a low-burden source of cognitive–motor measurements, but small datasets can yield optimistic results when preprocessing, task selection, or explanation is not separated from evaluation. We evaluated eight classifiers on the DARWIN dataset (174 participants; 25 tasks; 450 predictors) using ten repetitions of 5-outer/3-inner-fold nested stratified cross-validation. All preprocessing, hyperparameter selection, and calibration were restricted to outer training data. We added a training-derived operating point, fully nested seven-task selection, cross-model subset transfer, foldwise SHAP analysis, and an all-feature SHAP sensitivity analysis. Random Forest had the highest descriptive ROC-AUC (0.938 ± 0.033 across 50 held-out folds). The Friedman omnibus test indicated model differences (p < 0.001), but no one of 28 paired comparisons remained significant after Holm correction. Training-derived thresholds (median 0.461) yielded held-out sensitivity 0.869 and specificity 0.831; these are exploratory operating characteristics, not a clinical cutoff. Nested seven-task selection achieved AUC 0.916 ± 0.037, compared with 0.956 ± 0.030 for all 25 tasks, and selected subsets were unstable (mean Jaccard 0.195). Cross-validated SHAP emphasized temporal variables, although an all-450-feature sensitivity analysis showed that individual ranks depend partly on ANOVA preselection. The study contributes an integrated, leakage-controlled evaluation rather than a new classifier or a validated diagnostic instrument; external prospective validation remains necessary.
SN - 1932-6203
ER -