The human hippocampus supports auditory coding and multisensory integration
- 1Department of Psychology, Yale University, New Haven, Connecticut, United States of America
- 2Wu Tsai Institute, Yale University, New Haven, Connecticut, United States of America
- 3Department of Biomedical Engineering, Duke University, Durham, North Carolina, United States of America
Abstract
The hippocampus receives convergent input from multiple sensory systems, yet in humans it has been studied almost exclusively through vision. Here we examine how the hippocampus contributes to sensory processing beyond the visual modality and to the multisensory integration of visual information with these other modalities. Participants were exposed to multiple repetitions of short naturalistic movie clips, each presented in four formats: auditory-only, visual-only, congruent audiovisual, and incongruent audiovisual (audio and video from different movies). Using high-resolution fMRI, we measured univariate activation and multivariate representations across the longitudinal axis and subfields of the human hippocampus. Whereas univariate analyses detected only visual responses across hippocampal subregions, with no activation for auditory stimuli and no benefit of congruent stimuli, multivariate analyses revealed robust representations of both auditory and visual scenes. The posterior hippocampus showed enhanced pattern similarity for congruent stimuli relative to unisensory stimuli, especially in the left hemisphere, demonstrating multisensory facilitation. The anterior hippocampus showed crossmodal decoding between auditory and visual versions of the same clip, driven by the right hemisphere, suggesting a more abstract representation. Finally, whole-brain searchlight analyses revealed parallel effects in cortical regions known to support multisensory integration. These findings advance understanding of auditory and multisensory coding in the human hippocampus, revealing specialization and lateralization along its longitudinal axis.
How does the hippocampus integrate fragmented sights and sounds into a coherent experience? This study reveals auditory and multisensory coding in the human hippocampus, including a functional dissociation along its longitudinal axis, with multisensory facilitation in the posterior and crossmodal generalization in the anterior hippocampus.
Related
- Perceptual integration of multisensory haptic, visual, and auditory feedback for roughness discrimination in augmented reality
- High-dimensional hippocampus representation facilitates learning of low-dimensional abstract representation and supports generalization
- Distinct patterns of multisensory integration underly the expression of flavor preferences and aversions in rats
- Olfactory combinatorial coding supports risk-reward decision-making in Caenorhabditis elegans.
- Large cranial windows distort auditory cortical maps and degrade feature integration in secondary auditory cortex
BibTeX
@article{raccah2026human,
title = {{The human hippocampus supports auditory coding and multisensory integration}},
author = {Omri Raccah and Aryan Agarwal and Yannan Zhu and Nicholas B. Turk-Browne},
journal = {PLOS Biology},
year = {2026},
month = nov,
doi = {10.1371/journal.pbio.3004012},
url = {https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3004012},
note = {CC BY},
}
RIS
TY - JOUR
TI - The human hippocampus supports auditory coding and multisensory integration
AU - Omri Raccah
AU - Aryan Agarwal
AU - Yannan Zhu
AU - Nicholas B. Turk-Browne
PY - 2026
DA - 2026/11/01
JO - PLOS Biology
DO - 10.1371/journal.pbio.3004012
UR - https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3004012
AB - The hippocampus receives convergent input from multiple sensory systems, yet in humans it has been studied almost exclusively through vision. Here we examine how the hippocampus contributes to sensory processing beyond the visual modality and to the multisensory integration of visual information with these other modalities. Participants were exposed to multiple repetitions of short naturalistic movie clips, each presented in four formats: auditory-only, visual-only, congruent audiovisual, and incongruent audiovisual (audio and video from different movies). Using high-resolution fMRI, we measured univariate activation and multivariate representations across the longitudinal axis and subfields of the human hippocampus. Whereas univariate analyses detected only visual responses across hippocampal subregions, with no activation for auditory stimuli and no benefit of congruent stimuli, multivariate analyses revealed robust representations of both auditory and visual scenes. The posterior hippocampus showed enhanced pattern similarity for congruent stimuli relative to unisensory stimuli, especially in the left hemisphere, demonstrating multisensory facilitation. The anterior hippocampus showed crossmodal decoding between auditory and visual versions of the same clip, driven by the right hemisphere, suggesting a more abstract representation. Finally, whole-brain searchlight analyses revealed parallel effects in cortical regions known to support multisensory integration. These findings advance understanding of auditory and multisensory coding in the human hippocampus, revealing specialization and lateralization along its longitudinal axis. How does the hippocampus integrate fragmented sights and sounds into a coherent experience? This study reveals auditory and multisensory coding in the human hippocampus, including a functional dissociation along its longitudinal axis, with multisensory facilitation in the posterior and crossmodal generalization in the anterior hippocampus.
SN - 1545-7885
SN - 1544-9173
ER -