Feature-Engineering Strategies for EEG-Based ADHD Classification in Children: A Controlled Benchmark of Expert, LLM-Guided and Full-Space Comparators
- 1Uskudar University, Istanbul, Turkey; Brain and Trauma Foundation Grisons, Landquart, Switzerland
Abstract
EEG-based machine-learning studies of attention-deficit/hyperactivity disorder (ADHD) are vulnerable to optimistic performance estimates when feature selection or model choice is informed by data outside the training fold. We compared expert-reconstructed, large language model (LLM)-guided and full-space feature strategies under leakage-controlled subject-level validation in a frozen cohort of 77 children (45 ADHD, 32 controls) represented by 1,185 gamma-free EEG features. Four classifiers (SVM, GMM, random forest [RF] and XGBoost) were evaluated with nested leave-one-subject-out cross-validation (LOOCV), with all data-dependent operations restricted to outer-training subjects. Across 100 repetitions of stratified 10-fold cross-validation, RF/expert achieved mean accuracy 62.8% (SD 3.5%; empirical 95% interval 55.8%-69.5%), mean balanced accuracy 62.3% and mean AUC-ROC 0.618. The corresponding LOOCV point estimate was 71.4% accuracy and 71.0% balanced accuracy (AUC-ROC 0.673). GMM/expert achieved the highest AUC-ROC (0.699), indicating metric-dependent model ranking. The expert-reconstructed strategy achieved the highest accuracy and balanced accuracy for every classifier, whereas the LLM-guided strategy showed no consistent advantage over the full feature space. Pairwise classifier differences were not significant after Holm correction. A pre-specified 200 {micro}V artifact gate reduced RF/expert accuracy to approximately 55.8%, below the 58.4% no-information rate; this drop may reflect removal of artifact-related signal and/or reduced epoch counts leading to noisier feature estimates. These findings support cautious benchmarking and external validation rather than claims of algorithmic or LLM superiority.
Related
- Timing and Context Features in Machine Learning Classification of Inter-Patient ECG Heartbeats on the MIT-BIH Benchmark
- SiNFluD: A novel benchmark dataset for sindhi figurative language classification
- Context-Dependent FHIR Serialisation Strategies for Clinical LLM Deployment: A Multi-Layer Benchmark on UK Core Data
- Injury risk classification from feature grouped tabular athlete data using deep neural networks: a cross sectional benchmark study.
- Multisystem Outcomes of GLP-1 Receptor Agonists Versus Active Comparators in Type 2 Diabetes: A Multicenter Target Trial Emulation
BibTeX
@misc{rohani2026featureengineering,
title = {{Feature-Engineering Strategies for EEG-Based ADHD Classification in Children: A Controlled Benchmark of Expert, LLM-Guided and Full-Space Comparators}},
author = {F. Rohani and H. Alhatab and M. Jarrud and N. Salous and S. Kasimoglu and A. Mueller},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.06.26364825},
url = {https://www.medrxiv.org/content/10.64898/2026.10.06.26364825v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Feature-Engineering Strategies for EEG-Based ADHD Classification in Children: A Controlled Benchmark of Expert, LLM-Guided and Full-Space Comparators
AU - F. Rohani
AU - H. Alhatab
AU - M. Jarrud
AU - N. Salous
AU - S. Kasimoglu
AU - A. Mueller
PY - 2026
DA - 2026/10/06
PB - medRxiv
DO - 10.64898/2026.10.06.26364825
UR - https://www.medrxiv.org/content/10.64898/2026.10.06.26364825v1
AB - EEG-based machine-learning studies of attention-deficit/hyperactivity disorder (ADHD) are vulnerable to optimistic performance estimates when feature selection or model choice is informed by data outside the training fold. We compared expert-reconstructed, large language model (LLM)-guided and full-space feature strategies under leakage-controlled subject-level validation in a frozen cohort of 77 children (45 ADHD, 32 controls) represented by 1,185 gamma-free EEG features. Four classifiers (SVM, GMM, random forest [RF] and XGBoost) were evaluated with nested leave-one-subject-out cross-validation (LOOCV), with all data-dependent operations restricted to outer-training subjects. Across 100 repetitions of stratified 10-fold cross-validation, RF/expert achieved mean accuracy 62.8% (SD 3.5%; empirical 95% interval 55.8%-69.5%), mean balanced accuracy 62.3% and mean AUC-ROC 0.618. The corresponding LOOCV point estimate was 71.4% accuracy and 71.0% balanced accuracy (AUC-ROC 0.673). GMM/expert achieved the highest AUC-ROC (0.699), indicating metric-dependent model ranking. The expert-reconstructed strategy achieved the highest accuracy and balanced accuracy for every classifier, whereas the LLM-guided strategy showed no consistent advantage over the full feature space. Pairwise classifier differences were not significant after Holm correction. A pre-specified 200 {micro}V artifact gate reduced RF/expert accuracy to approximately 55.8%, below the 58.4% no-information rate; this drop may reflect removal of artifact-related signal and/or reduced epoch counts leading to noisier feature estimates. These findings support cautious benchmarking and external validation rather than claims of algorithmic or LLM superiority.
ER -