Social vision expands into auditory cortices of people born deaf: evidence from naturalistic film and encoding models
- 1University of Warsaw, Johns Hopkins University
Abstract
Comparing deaf and hearing adults provides insight into the contribution of experience to function of auditory cortical networks. We used naturalistic fMRI and voxel-wise encoding to measure information processing in auditory networks of deaf (n=24) and hearing (n=22) adults who watched a 35-minute silent animated film. Unlike simpler stimuli, naturalistic films are rich in a broad range of perceptual, affective, and social features, including motion, dynamic social interactions, and Theory of Mind. Machine vision algorithms and human coding were used to densely annotate visual and social film features. In both groups we observed a right-lateralized processing hierarchy with encoding of low-level visual features in early visual areas, social visual features along the superior temporal sulcus (STS) and Theory of Mind in temporoparietal junction. In deaf participants, auditory cortices showed enhanced encoding of a broad range of visual information, from low level motion to Theory of Mind. In higher order auditory networks, social interactions emerging as the strongest predictor of neural activity in deaf people and showed the strongest enhancement in deafness. Auditory networks of deaf adults showed systematic anatomical organization within and across hemispheres. Within the right hemisphere, higher-order auditory regions neighboring the STS preferentially responded to social visual features whereas lower-level visual features were represented broadly across the auditory hierarchy. These results suggest that proximity and connectivity principles govern anatomical expansion of information encoding in sensory loss.
Related
- Neural encoding of grasp and object properties in the posterior parietal and motor cortices of the cortical grasping network in tetraplegic humans
- Temporal Progression of Prior Precision, Surprisal, Model Updating and Tone Adaptation in Human Auditory Cortices
- EEG Oscillations in Guided Mindfulness versus Mind-Wandering in Young Adults: Effects of Auditory Task Instruction and Naturalistic Video
- Human perception of avian biodiversity in naturalistic auditory scenes
- Aging effects on retrospective and naturalistic encoding of time and their relationship with memory.
BibTeX
@misc{zimmermann2026social,
title = {{Social vision expands into auditory cortices of people born deaf: evidence from naturalistic film and encoding models}},
author = {M. Zimmermann and M. Szwed and L. Isik and M. Bedny},
howpublished = {bioRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.04.763721},
url = {https://www.biorxiv.org/content/10.64898/2026.11.04.763721v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Social vision expands into auditory cortices of people born deaf: evidence from naturalistic film and encoding models
AU - M. Zimmermann
AU - M. Szwed
AU - L. Isik
AU - M. Bedny
PY - 2026
DA - 2026/11/04
PB - bioRxiv
DO - 10.64898/2026.11.04.763721
UR - https://www.biorxiv.org/content/10.64898/2026.11.04.763721v1
AB - Comparing deaf and hearing adults provides insight into the contribution of experience to function of auditory cortical networks. We used naturalistic fMRI and voxel-wise encoding to measure information processing in auditory networks of deaf (n=24) and hearing (n=22) adults who watched a 35-minute silent animated film. Unlike simpler stimuli, naturalistic films are rich in a broad range of perceptual, affective, and social features, including motion, dynamic social interactions, and Theory of Mind. Machine vision algorithms and human coding were used to densely annotate visual and social film features. In both groups we observed a right-lateralized processing hierarchy with encoding of low-level visual features in early visual areas, social visual features along the superior temporal sulcus (STS) and Theory of Mind in temporoparietal junction. In deaf participants, auditory cortices showed enhanced encoding of a broad range of visual information, from low level motion to Theory of Mind. In higher order auditory networks, social interactions emerging as the strongest predictor of neural activity in deaf people and showed the strongest enhancement in deafness. Auditory networks of deaf adults showed systematic anatomical organization within and across hemispheres. Within the right hemisphere, higher-order auditory regions neighboring the STS preferentially responded to social visual features whereas lower-level visual features were represented broadly across the auditory hierarchy. These results suggest that proximity and connectivity principles govern anatomical expansion of information encoding in sensory loss.
ER -