Brain2voice 2.0: High-performance voice synthesis brain-computer interface
- 1University of California, Davis
Abstract
Brain-computer interfaces (BCIs) offer a promising solution to speech loss due to neurological injury by decoding intended speech directly from brain activity. While recent BCIs have restored high-accuracy text-based communication, they fail to provide instantaneous voice output essential for the natural flow of conversation. Brain-to-voice BCIs address this gap by decoding voice directly from neural signals. However, even the state-of-the-art (SOTA) BCI-synthesized voice is not yet intelligible enough for real-world adoption. We introduce brain2voice 2.0, a new multimodal Transformer-based BCI decoder architecture capable of synthesizing highly intelligible voice from intracortical neural signals in real-time. Brain2voice 2.0 is trained on continuous and custom-tokenized acoustic targets and phoneme targets, leveraging their complementary speech information. We use self-supervised and adversarial training objectives that enhance acoustic feature quality and improve synthesis intelligibility. At each 10 ms timestep, the model causally outputs continuous and tokenized acoustic features for real-time voice synthesis as well as time-aligned phoneme predictions (raw phoneme error rate: 7%, comparable to the latest brain-to-text models). We evaluated this new approach on our prior intracortical brain-to-voice benchmark dataset (Wairagkar et al. 2025). Naive human listeners transcribed brain2voice 2.0 synthesized voice with a word error rate of 5.24%--an 8x improvement in intelligibility over previous SOTA results (43.75%). Brain2voice 2.0 demonstrates that highly intelligible real-time voice synthesis from neural signals is achievable, for the first time crossing the intelligibility threshold necessary for clinically viable brain-to-voice BCIs for people with paralysis.
Related
- Brain-writing-BERT for generalizable handwriting brain-computer interface
- Self-paced silent speech brain-computer interface for device control
- The Effect of Visual Impairment on the Controllability of a Brain-Computer Interface Using Visual Spatial Attention
- Quantifying Event-Related (De)Synchronization Variability for Brain-Computer Interface: A Unified and Interpretable Framework
- Mapping Brain-Computer Interface Research in Depression (1995-2026): A Bibliometric Analysis.
BibTeX
@misc{wairagkar2026brain2voice,
title = {{Brain2voice 2.0: High-performance voice synthesis brain-computer interface}},
author = {M. Wairagkar and A. Srinivasan and N. S. Card and T. Singer-Clark and X. Hou and C. Iacobacci and L. M. Miller and L. R. Hochberg and D. M. Brandman and S. D. Stavisky},
howpublished = {bioRxiv},
year = {2026},
month = jun,
doi = {10.64898/2026.06.30.735633},
url = {https://www.biorxiv.org/content/10.64898/2026.06.30.735633v2},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Brain2voice 2.0: High-performance voice synthesis brain-computer interface
AU - M. Wairagkar
AU - A. Srinivasan
AU - N. S. Card
AU - T. Singer-Clark
AU - X. Hou
AU - C. Iacobacci
AU - L. M. Miller
AU - L. R. Hochberg
AU - D. M. Brandman
AU - S. D. Stavisky
PY - 2026
DA - 2026/06/30
PB - bioRxiv
DO - 10.64898/2026.06.30.735633
UR - https://www.biorxiv.org/content/10.64898/2026.06.30.735633v2
AB - Brain-computer interfaces (BCIs) offer a promising solution to speech loss due to neurological injury by decoding intended speech directly from brain activity. While recent BCIs have restored high-accuracy text-based communication, they fail to provide instantaneous voice output essential for the natural flow of conversation. Brain-to-voice BCIs address this gap by decoding voice directly from neural signals. However, even the state-of-the-art (SOTA) BCI-synthesized voice is not yet intelligible enough for real-world adoption. We introduce brain2voice 2.0, a new multimodal Transformer-based BCI decoder architecture capable of synthesizing highly intelligible voice from intracortical neural signals in real-time. Brain2voice 2.0 is trained on continuous and custom-tokenized acoustic targets and phoneme targets, leveraging their complementary speech information. We use self-supervised and adversarial training objectives that enhance acoustic feature quality and improve synthesis intelligibility. At each 10 ms timestep, the model causally outputs continuous and tokenized acoustic features for real-time voice synthesis as well as time-aligned phoneme predictions (raw phoneme error rate: 7%, comparable to the latest brain-to-text models). We evaluated this new approach on our prior intracortical brain-to-voice benchmark dataset (Wairagkar et al. 2025). Naive human listeners transcribed brain2voice 2.0 synthesized voice with a word error rate of 5.24%--an 8x improvement in intelligibility over previous SOTA results (43.75%). Brain2voice 2.0 demonstrates that highly intelligible real-time voice synthesis from neural signals is achievable, for the first time crossing the intelligibility threshold necessary for clinically viable brain-to-voice BCIs for people with paralysis.
ER -