Symbolic categorization of novel multisensory stimuli in the human brain
Abstract
When primates (both human and non-human) learn to categorize simple visual or acoustic stimuli by means of non-verbal matching tasks, two types of changes occur in their brain: early sensory cortices increase the precision with which they encode sensory information, and parietal and lateral prefrontal cortices develop a categorical response to the stimuli. Contrary to non-human animals, however, our species mostly constructs categories using linguistic labels. Moreover, we naturally tend to define categories by means of multiple sensory features of the stimuli. Here we trained adult subjects to parse a novel audiovisual stimulus space into 4 orthogonal categories, by associating each category to a specific symbol. We then used multi-voxel pattern analysis (MVPA) to show that during a cross-format category repetition detection task three neural representational changes were detectable. First, visual and acoustic cortices increased both precision and selectivity to their preferred sensory feature, displaying increased sensory segregation. Second, a frontoparietal network developed a multisensory object-specific response. Third, the right hippocampus and, at least to some extent, the left angular gyrus, developed a shared representational code common to symbols and objects. In particular, the right hippocampus displayed the highest level of abstraction and generalization from a format to the other, and also predicted symbolic categorization performance outside the scanner. Taken together, these results indicate that when humans categorize multisensory objects by means of language the set of changes occurring in the brain only partially overlaps with that described by classical models of non-verbal unisensory categorization in primates.
Related
- Adversarial robustness does not entail human-aligned categorization
- The human hippocampus supports auditory coding and multisensory integration
- Facilitation and Attenuation of Learning to Novel and Familiar Stimuli Through Contextual Mismatch: Experimental Evidence and a Computational Model
- Unveiling the functional specialisation of human white matter circuits with naturalistic stimuli
- A Novel Method for Synchronized Rhythmic Production of Finger Movements and the Effects of Auditory Stimuli and Feedback
BibTeX
@article{vigano2026symbolic,
title = {{Symbolic categorization of novel multisensory stimuli in the human brain}},
author = {Simone Viganò and Valentina Borghesani and Manuela Piazza},
journal = {NeuroImage},
year = {2026},
month = oct,
doi = {10.1016/j.neuroimage.2021.118016},
url = {https://hal.science/hal-05776634v1},
note = {CC BY},
}
RIS
TY - JOUR
TI - Symbolic categorization of novel multisensory stimuli in the human brain
AU - Simone Viganò
AU - Valentina Borghesani
AU - Manuela Piazza
PY - 2026
DA - 2026/10/03
JO - NeuroImage
DO - 10.1016/j.neuroimage.2021.118016
UR - https://hal.science/hal-05776634v1
AB - When primates (both human and non-human) learn to categorize simple visual or acoustic stimuli by means of non-verbal matching tasks, two types of changes occur in their brain: early sensory cortices increase the precision with which they encode sensory information, and parietal and lateral prefrontal cortices develop a categorical response to the stimuli. Contrary to non-human animals, however, our species mostly constructs categories using linguistic labels. Moreover, we naturally tend to define categories by means of multiple sensory features of the stimuli. Here we trained adult subjects to parse a novel audiovisual stimulus space into 4 orthogonal categories, by associating each category to a specific symbol. We then used multi-voxel pattern analysis (MVPA) to show that during a cross-format category repetition detection task three neural representational changes were detectable. First, visual and acoustic cortices increased both precision and selectivity to their preferred sensory feature, displaying increased sensory segregation. Second, a frontoparietal network developed a multisensory object-specific response. Third, the right hippocampus and, at least to some extent, the left angular gyrus, developed a shared representational code common to symbols and objects. In particular, the right hippocampus displayed the highest level of abstraction and generalization from a format to the other, and also predicted symbolic categorization performance outside the scanner. Taken together, these results indicate that when humans categorize multisensory objects by means of language the set of changes occurring in the brain only partially overlaps with that described by classical models of non-verbal unisensory categorization in primates.
SN - 1053-8119
SN - 1095-9572
ER -