Multimodal Physiological Decoding Reveals Individualized Arousal Dynamics in Closed-Loop Neurofeedback
Abstract
Most arousal-based brain-computer interfaces (BCIs) use EEG to decode cognitive state. But arousal is an autonomic process. We examined if peripheral physiological signals give a better decode of task-related arousal. We used a public dataset from a difficult boundary-avoidance flight task. In this task, participants received EEG-based BCI neurofeedback, sham feedback, or no feedback. We trained a multimodal deep learning decoder on heart rate, heart rate variability (HRV), respiration, electrodermal activity, and pupil diameter. We compared it with an EEG-only decoder and with the original filter-bank common spatial pattern (FBCSP) decoder. All analyses were offline. The peripheral decoder had a within-subject AUC of 93.0%. The EEG-only decoder had 85.2% and the FBCSP decoder had 79.8%. Only the peripheral decoder showed the Yerkes-Dodson inverted-U relation between decoded arousal and performance. The feedback conditions did not change the arousal trajectories. Thus, the BCI benefit possibly comes from regulation at critical moments in the task, not from a continuous shift. We calculated a time-varying optimal arousal trajectory from control trials. Per-trial deviations from this trajectory predicted performance (r = -0.28 to -0.24, p < 0.01). Baseline HRV and gamma power changed the arousal-performance curve of each subject. We used these values to divide subjects into arousal-sensitive and arousal-tolerant groups. Control bands specific to each group increased trial separation (Cohen's d from 1.21 to 1.32 and from 0.85 to 1.10). These results show that peripheral signals are better than EEG for continuous arousal decoding. They also give a specification for individualized closed-loop arousal regulation. A prospective study must test this specification, for example with personalized transcutaneous vagus nerve stimulation.
Related
- Generative-augmented multimodal neuroaffective decoding for product evaluation: toward neurofeedback-driven human-in-the-loop co-design
- Individualized brain-body signatures of positive emotions revealed by large-scale multimodal neurophysiological recording
- Replacing EEG with peripheral physiological signals for driving behavior recognition: a physiology-informed multimodal approach
- Simultaneous speech and gesture decoding for multimodal communication in paralysis
- Circadian multimodal physiological, ocular and behavioural data from a 40-hour forced-desynchrony protocol
BibTeX
@misc{natarajan2026multimodal,
title = {{Multimodal Physiological Decoding Reveals Individualized Arousal Dynamics in Closed-Loop Neurofeedback}},
author = {Anirudh Natarajan and Paul Sajda},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.04113},
eprint = {2610.04113},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.04113},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Multimodal Physiological Decoding Reveals Individualized Arousal Dynamics in Closed-Loop Neurofeedback
AU - Anirudh Natarajan
AU - Paul Sajda
PY - 2026
DA - 2026/10/02
PB - arXiv
DO - 10.48550/arxiv.2610.04113
UR - https://arxiv.org/abs/2610.04113
AB - Most arousal-based brain-computer interfaces (BCIs) use EEG to decode cognitive state. But arousal is an autonomic process. We examined if peripheral physiological signals give a better decode of task-related arousal. We used a public dataset from a difficult boundary-avoidance flight task. In this task, participants received EEG-based BCI neurofeedback, sham feedback, or no feedback. We trained a multimodal deep learning decoder on heart rate, heart rate variability (HRV), respiration, electrodermal activity, and pupil diameter. We compared it with an EEG-only decoder and with the original filter-bank common spatial pattern (FBCSP) decoder. All analyses were offline. The peripheral decoder had a within-subject AUC of 93.0%. The EEG-only decoder had 85.2% and the FBCSP decoder had 79.8%. Only the peripheral decoder showed the Yerkes-Dodson inverted-U relation between decoded arousal and performance. The feedback conditions did not change the arousal trajectories. Thus, the BCI benefit possibly comes from regulation at critical moments in the task, not from a continuous shift. We calculated a time-varying optimal arousal trajectory from control trials. Per-trial deviations from this trajectory predicted performance (r = -0.28 to -0.24, p < 0.01). Baseline HRV and gamma power changed the arousal-performance curve of each subject. We used these values to divide subjects into arousal-sensitive and arousal-tolerant groups. Control bands specific to each group increased trial separation (Cohen's d from 1.21 to 1.32 and from 0.85 to 1.10). These results show that peripheral signals are better than EEG for continuous arousal decoding. They also give a specification for individualized closed-loop arousal regulation. A prospective study must test this specification, for example with personalized transcutaneous vagus nerve stimulation.
ER -