Cross-cohort validation of EEG biomarkers using a pre-registered three-criterion framework: a demonstration in amyotrophic lateral sclerosis
- 1Department of Pharmacy, College of Health Sciences and Comprehensive Specialized Hospital, Axum University, Axum, Ethiopia
Abstract
Background. Despite decades of EEG biomarker development for amyotrophic lateral sclerosis (ALS), no candidate has progressed to clinical deployment, partly because cross-cohort validation is rare. We aimed to develop and demonstrate a pre-registered three-criterion screening framework to identify which candidate EEG features show the combined properties required for biomarker translation: cross-sectional discrimination, within-subject reliability, and cross-cohort effect consistency. Methods. We pre-registered a validation framework adapted from FDA-NIH BEST principles and applied it to 18 EEG features across three independent cohorts: EEGET-ALS (six ALS patients, 170 healthy controls; home recordings over 17 days), UCL/Penn State (eight ALS patients; clinic recordings over 1-2 months), and a healthy adult test-retest reference (60 participants, three sessions over one month). Features were required to meet three pre-registered criteria: (1) cross-sectional discrimination (Cohen's d > 0.5), (2) within-subject reliability (ICC(3,1) > 0.6 in both ALS cohorts independently), and (3) cross-cohort effect direction consistency. Cross-cohort permutation testing (1000 iterations) and subject-level bootstrap resampling (500 iterations) were applied to assess robustness. Results. Three features met all three pre-registered criteria: gamma power, theta power, and spectral entropy. Cross-cohort permutation testing showed that no permutation, out of 1000, produced any feature in the three-cohort reliability intersection (Monte Carlo p = 0.001). Bootstrap resampling indicated that the composition of the selected set was sensitive to sample composition (median count 1, 95% CI [0, 4]), with gamma power (67% inclusion), spectral entropy (43%), and theta power (32%) the most stable features at a relaxed reliability threshold (ICC > 0.5); at the primary pre-registered threshold (ICC > 0.6), the corresponding inclusion probabilities are lower (57%, 27%, and 19%, respectively). A principal methodological finding was that cross-cohort differences in feature reliability were large relative to the apparent disease-associated signal, most clearly demonstrated by relative beta power (ICC = 0.09 in a home-recorded cohort versus 0.93 in a clinic-recorded cohort); because recording protocol, hardware, setting, population, and preprocessing all differ jointly between cohorts in this dataset, this is evidence of cohort-associated variability that plausibly includes a protocol contribution, not a separately identified, isolated protocol effect. Conclusions. This study provides a pre-registered, multi-cohort EEG biomarker screening framework for ALS with explicit bootstrap-quantified uncertainty. Gamma power, spectral entropy, and theta power are identified as priority candidates for prospective validation. A recurring methodological observation is that protocol-related cohort differences appear to contribute more to cross-cohort variability than disease-related effects within this dataset; because protocol is confounded with cohort, equipment, and preprocessing, this pattern is descriptive rather than causal. The observation nonetheless has direct implications for the design of multi-site EEG biomarker studies in neural engineering.
Related
- Cerebrospinal fluid amyloid and neurodegeneration biomarkers in the diagnosis and prognosis of amyotrophic lateral sclerosis: a multicohort observational study
- Toward a National Tuberculosis Lateral-Flow Urine Lipoarabinomannan (LF-LAM) External Quality Assessment Program in Thailand: Feasibility Demonstration Through Panel Development, Validation, and Proficiency Testing, March 2025–April 2026
- Rethinking Digital Outcome Measures for Amyotrophic Lateral Sclerosis Trials: Consensus Priorities for Actigraphy Harmonization, Validation, and Regulatory Readiness
- Core needle biopsy washout biomarkers combined with ultrasound for preoperative axillary risk stratification in breast cancer: a retrospective nomogram study with temporal validation
- Identification of candidate selenium-associated transcriptomic biomarkers in chronic obstructive pulmonary disease: an integrated bioinformatics and expression validation study
BibTeX
@misc{gebregergis2026crosscohort,
title = {{Cross-cohort validation of EEG biomarkers using a pre-registered three-criterion framework: a demonstration in amyotrophic lateral sclerosis}},
author = {B. T. Gebregergis and H. G. Yhdego and T. Teklu},
howpublished = {medRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.10.26367741},
url = {https://www.medrxiv.org/content/10.64898/2026.11.10.26367741v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Cross-cohort validation of EEG biomarkers using a pre-registered three-criterion framework: a demonstration in amyotrophic lateral sclerosis
AU - B. T. Gebregergis
AU - H. G. Yhdego
AU - T. Teklu
PY - 2026
DA - 2026/11/10
PB - medRxiv
DO - 10.64898/2026.11.10.26367741
UR - https://www.medrxiv.org/content/10.64898/2026.11.10.26367741v1
AB - Background. Despite decades of EEG biomarker development for amyotrophic lateral sclerosis (ALS), no candidate has progressed to clinical deployment, partly because cross-cohort validation is rare. We aimed to develop and demonstrate a pre-registered three-criterion screening framework to identify which candidate EEG features show the combined properties required for biomarker translation: cross-sectional discrimination, within-subject reliability, and cross-cohort effect consistency. Methods. We pre-registered a validation framework adapted from FDA-NIH BEST principles and applied it to 18 EEG features across three independent cohorts: EEGET-ALS (six ALS patients, 170 healthy controls; home recordings over 17 days), UCL/Penn State (eight ALS patients; clinic recordings over 1-2 months), and a healthy adult test-retest reference (60 participants, three sessions over one month). Features were required to meet three pre-registered criteria: (1) cross-sectional discrimination (Cohen's d > 0.5), (2) within-subject reliability (ICC(3,1) > 0.6 in both ALS cohorts independently), and (3) cross-cohort effect direction consistency. Cross-cohort permutation testing (1000 iterations) and subject-level bootstrap resampling (500 iterations) were applied to assess robustness. Results. Three features met all three pre-registered criteria: gamma power, theta power, and spectral entropy. Cross-cohort permutation testing showed that no permutation, out of 1000, produced any feature in the three-cohort reliability intersection (Monte Carlo p = 0.001). Bootstrap resampling indicated that the composition of the selected set was sensitive to sample composition (median count 1, 95% CI [0, 4]), with gamma power (67% inclusion), spectral entropy (43%), and theta power (32%) the most stable features at a relaxed reliability threshold (ICC > 0.5); at the primary pre-registered threshold (ICC > 0.6), the corresponding inclusion probabilities are lower (57%, 27%, and 19%, respectively). A principal methodological finding was that cross-cohort differences in feature reliability were large relative to the apparent disease-associated signal, most clearly demonstrated by relative beta power (ICC = 0.09 in a home-recorded cohort versus 0.93 in a clinic-recorded cohort); because recording protocol, hardware, setting, population, and preprocessing all differ jointly between cohorts in this dataset, this is evidence of cohort-associated variability that plausibly includes a protocol contribution, not a separately identified, isolated protocol effect. Conclusions. This study provides a pre-registered, multi-cohort EEG biomarker screening framework for ALS with explicit bootstrap-quantified uncertainty. Gamma power, spectral entropy, and theta power are identified as priority candidates for prospective validation. A recurring methodological observation is that protocol-related cohort differences appear to contribute more to cross-cohort variability than disease-related effects within this dataset; because protocol is confounded with cohort, equipment, and preprocessing, this pattern is descriptive rather than causal. The observation nonetheless has direct implications for the design of multi-site EEG biomarker studies in neural engineering.
ER -