MAESTRO: a Multimodal Auditory-attention Egocentric Speech-TRacking Open corpus
Abstract
Humans rely on gaze, head movements, and visual cues to attend to speakers in noisy environments, yet auditory attention decoding (AAD) has been studied primarily using electroencephalography (EEG). We introduce the Multimodal Auditory-attention Egocentric Speech-TRacking Open (MAESTRO) corpus, the first AAD dataset to simultaneously record EEG, eye gaze, pupillometry, egocentric video, and head inertial measurement unit (IMU) data. MAESTRO includes four competing speakers and background noise across multiple signal-to-noise ratio (SNR) conditions, enabling attention decoding under realistic listening scenarios. Through a four-speaker attention decoding benchmark, we show that combining behavioral and physiological signals improves decoding performance over EEG-only approaches, enabling future advances in multimodal auditory attention decoding. These findings open the door to new applications, analyses, and methodological advances in multimodal AAD. The complete dataset is publicly available at https://huggingface.co/datasets/aspire-osu/maestro-eeg-dataset . The official code repository is available at https://github.com/ASPIRE-OSU/MAESTRO .
Related
- Menstrual cycle and hormonal contraceptive intake alter the electrophysiological signature of selective auditory spatial attention
- Auditory tracking magnitude, temporal alignment, and anatomical distribution during wakefulness and propofol-based anesthesia in human sEEG
- Endogenous auditory and motor brain rhythms predict individual speech tracking
- Dopamine modulation of motor and implicit learning contributions to auditory temporal attention
- Neural dynamics of auditory attention: Interactions between top-down and bottom-up processes
BibTeX
@misc{hassan2026maestro,
title = {{MAESTRO: a Multimodal Auditory-attention Egocentric Speech-TRacking Open corpus}},
author = {K M Naimul Hassan and Ali Alavi and Donald S. Williamson},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.31898},
eprint = {2609.31898},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.31898},
}
RIS
TY - UNPB
TI - MAESTRO: a Multimodal Auditory-attention Egocentric Speech-TRacking Open corpus
AU - K M Naimul Hassan
AU - Ali Alavi
AU - Donald S. Williamson
PY - 2026
DA - 2026/09/25
PB - arXiv
DO - 10.48550/arxiv.2609.31898
UR - https://arxiv.org/abs/2609.31898
AB - Humans rely on gaze, head movements, and visual cues to attend to speakers in noisy environments, yet auditory attention decoding (AAD) has been studied primarily using electroencephalography (EEG). We introduce the Multimodal Auditory-attention Egocentric Speech-TRacking Open (MAESTRO) corpus, the first AAD dataset to simultaneously record EEG, eye gaze, pupillometry, egocentric video, and head inertial measurement unit (IMU) data. MAESTRO includes four competing speakers and background noise across multiple signal-to-noise ratio (SNR) conditions, enabling attention decoding under realistic listening scenarios. Through a four-speaker attention decoding benchmark, we show that combining behavioral and physiological signals improves decoding performance over EEG-only approaches, enabling future advances in multimodal auditory attention decoding. These findings open the door to new applications, analyses, and methodological advances in multimodal AAD. The complete dataset is publicly available at https://huggingface.co/datasets/aspire-osu/maestro-eeg-dataset . The official code repository is available at https://github.com/ASPIRE-OSU/MAESTRO .
ER -