How large an effect would we have seen? Positive controls, detection floors and dataset pitfalls in scalp-EEG speech decoding
Abstract
Objective. Studies that decode imagined speech from scalp electroencephalography (EEG) commonly validate their pipelines with a positive control, usually overt speech, and report null results without stating what effect size the pipeline could have detected. Approach. Using ds003626 ("Thinking out loud"), KaraOne and Track 3 of the 2020 International Brain-Computer Interface Competition, we audited decoding of overt speech, articulator preparation and imagined speech with artifact-channel subsets, band-limited decoding and trial selection, under a protocol that enforces split integrity by assertion. We injected class-dependent effects of known amplitude into the real recordings to measure the detection floor of five decoder families, and we checked the ds003626 labels against the dataset's acquisition protocol. Main results. On ds003626, two lip electrodes decoded overt speech at 0.7430 and the eight external channels at 0.7068, against 0.4231 for the 128-channel scalp montage, and no decodable signal survived in the 8-30 Hz band. On the other two corpora, within-subject accuracy rose monotonically with frequency into the range dominated by muscle activity. Band-power logistic regression detected an injected 8-30 Hz desynchronization of 0.54 uV under a criterion fixed in advance, whereas a compact convolutional network (EEGNet) detected nothing within subject up to 4.31 uV. In ds003626, the condition codes are the reverse of the intuitive reading, the class encoding changed between public releases, and condition is a deterministic function of time within each session. Significance. In these corpora the overt-speech positive control is dominated by myogenic signal and cannot validate a decoding pipeline. We recommend that speech-decoding studies report band-limited and artifact-channel accuracy, the direction of accuracy across frequency, and a measured detection floor.
Related
- Decoding how RNase E membrane targeting controls RNA degradosome activity in Pseudomonas aeruginosa
- Many Brains, One Geometry: A Shared Visual-Semantic Space for Cross-Dataset fMRI Decoding
- AVID: A 5T fMRI Dataset for Benchmarking Auditory-induced Visual Mental Imagery Decoding
- A novel positive feedback regulatory loop, TGF-β1/SMAD4/miR-184, controls the follicular development in sows
- Cross-dataset few-shot deep learning for landmark detection in biological morphometrics
BibTeX
@misc{adeika2026large,
title = {{How large an effect would we have seen? Positive controls, detection floors and dataset pitfalls in scalp-EEG speech decoding}},
author = {Blessing Isoyiza Adeika and Onyema Osuagwu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.20942},
eprint = {2610.20942},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.20942},
}
RIS
TY - UNPB
TI - How large an effect would we have seen? Positive controls, detection floors and dataset pitfalls in scalp-EEG speech decoding
AU - Blessing Isoyiza Adeika
AU - Onyema Osuagwu
PY - 2026
DA - 2026/10/14
PB - arXiv
DO - 10.48550/arxiv.2610.20942
UR - https://arxiv.org/abs/2610.20942
AB - Objective. Studies that decode imagined speech from scalp electroencephalography (EEG) commonly validate their pipelines with a positive control, usually overt speech, and report null results without stating what effect size the pipeline could have detected. Approach. Using ds003626 ("Thinking out loud"), KaraOne and Track 3 of the 2020 International Brain-Computer Interface Competition, we audited decoding of overt speech, articulator preparation and imagined speech with artifact-channel subsets, band-limited decoding and trial selection, under a protocol that enforces split integrity by assertion. We injected class-dependent effects of known amplitude into the real recordings to measure the detection floor of five decoder families, and we checked the ds003626 labels against the dataset's acquisition protocol. Main results. On ds003626, two lip electrodes decoded overt speech at 0.7430 and the eight external channels at 0.7068, against 0.4231 for the 128-channel scalp montage, and no decodable signal survived in the 8-30 Hz band. On the other two corpora, within-subject accuracy rose monotonically with frequency into the range dominated by muscle activity. Band-power logistic regression detected an injected 8-30 Hz desynchronization of 0.54 uV under a criterion fixed in advance, whereas a compact convolutional network (EEGNet) detected nothing within subject up to 4.31 uV. In ds003626, the condition codes are the reverse of the intuitive reading, the class encoding changed between public releases, and condition is a deterministic function of time within each session. Significance. In these corpora the overt-speech positive control is dominated by myogenic signal and cannot validate a decoding pipeline. We recommend that speech-decoding studies report band-limited and artifact-channel accuracy, the direction of accuracy across frequency, and a measured detection floor.
ER -