An adaptive transformer-based model with entropy-driven knowledge distillation for single-channel EEG REM sleep detection
Abstract
Accurate detection of rapid eye movement (REM) sleep is critical for neurophysiological monitoring and early identification of sleep-related disorders. An entropy-driven adaptive knowledge distillation framework integrated with a lightweight Transformer was developed to support robust REM sleep detection from single-channel electroencephalogram (EEG) signals. By dynamically adjusting temperature scaling and distillation weighting based on teacher–student entropy, the proposed approach explicitly models uncertainty inherent in REM sleep. When evaluated under subject-independent cross-validation on three public datasets (Sleep-EDF, ISRUC-Sleep, and MASS), the proposed model outperformed CNN, LSTM, and standard Transformer baselines, achieving an average accuracy of 93.2% and an F1 score of 92.8%. Moreover, the distilled student model reduced training time and memory usage by approximately 30%, indicating potential suitability for future optimization toward real-time and wearable EEG applications. These results suggest that uncertainty-aware adaptive distillation may support accurate and computationally efficient REM sleep detection from single-channel EEG under subject-independent experimental conditions.
Related
- A temperature-adaptive logit-standardized knowledge distillation for lightweight network intrusion detection
- Improved vision transformer for sports image classification via transfer learning and knowledge distillation
- Learning from Adaptive Oscillatory Modes: An Interpretable Synchrosqueezing Transformer for Patient-Independent Seizure Detection
- Relational Knowledge Distillation Brings DNN Representations Close Enough to Humans to Be Aligned Without Supervision
- Hybrid local–global vision transformer framework for robust plant disease detection in precision agriculture
BibTeX
@article{huang2026adaptive,
title = {{An adaptive transformer-based model with entropy-driven knowledge distillation for single-channel EEG REM sleep detection}},
author = {Ji-Jer Huang and Shih-Chung Chen and Irawan Dwiwahyono},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-62389-3},
url = {https://www.nature.com/articles/s41598-026-62389-3},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - An adaptive transformer-based model with entropy-driven knowledge distillation for single-channel EEG REM sleep detection
AU - Ji-Jer Huang
AU - Shih-Chung Chen
AU - Irawan Dwiwahyono
PY - 2026
DA - 2026/10/24
JO - Scientific Reports
DO - 10.1038/s41598-026-62389-3
UR - https://www.nature.com/articles/s41598-026-62389-3
AB - Accurate detection of rapid eye movement (REM) sleep is critical for neurophysiological monitoring and early identification of sleep-related disorders. An entropy-driven adaptive knowledge distillation framework integrated with a lightweight Transformer was developed to support robust REM sleep detection from single-channel electroencephalogram (EEG) signals. By dynamically adjusting temperature scaling and distillation weighting based on teacher–student entropy, the proposed approach explicitly models uncertainty inherent in REM sleep. When evaluated under subject-independent cross-validation on three public datasets (Sleep-EDF, ISRUC-Sleep, and MASS), the proposed model outperformed CNN, LSTM, and standard Transformer baselines, achieving an average accuracy of 93.2% and an F1 score of 92.8%. Moreover, the distilled student model reduced training time and memory usage by approximately 30%, indicating potential suitability for future optimization toward real-time and wearable EEG applications. These results suggest that uncertainty-aware adaptive distillation may support accurate and computationally efficient REM sleep detection from single-channel EEG under subject-independent experimental conditions.
SN - 2045-2322
ER -