Action features dominate cortical representation during natural vision
- 1Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755
- 2Independent Researcher, Sunnyvale, CA 94085
- 3Department of Psychology, Center for Computational Language Sciences, University of Southern California, Los Angeles, CA 90089
Abstract
Cortical resources are allocated into systems specialized for processing ecologically relevant features of the world. These features are typically studied in isolation using controlled experimental stimuli, making it difficult to assess the relative importance of different kinds of features that tend to overlap during natural vision. In the current study, we evaluated the relative contributions of action, agent, and scene features in predicting cortical activity while participants viewed a 1-h nature documentary. We tested four sets of model features: semantic vectors capturing the observed action, agent, and scene features derived from an annotation of the stimulus, as well as low-level visual motion-energy features. We used banded ridge regression to fit vertex-wise encoding models from the combination of all four sets of features. While each set of features predicted neural activity in the expected areas, the action features predicted more widespread activity. A variance partitioning analysis revealed that the action features captured the most unique variance in neural activity across a much larger extent of cortex than the agent or scene features. Our findings suggest that cortical activity during dynamic, natural vision is dominated by features for understanding the actions of others.
Related
- Functional ultrasound imaging in free-moving marmosets reveals that intra-call prosodic features modulate large-scale cortical dynamics
- FlatClip: Reusing Image Foundation Models for fMRI Representation Learning via Cortical Flatmaps
- Representation learning of human cortical folding to reveal long lasting neurodevelopmental signatures
- Cortical reinstatement of causally related events sparks narrative insights by updating neural representation patterns
- Developmental gene expression patterns driving species-specific cortical features
BibTeX
@article{jung2026action,
title = {{Action features dominate cortical representation during natural vision}},
author = {Heejung Jung and Hunar Jain and Xiaochun Han and Ma Feilong and Jane Han and Deepanshi Shokeen and Cara Van Uden and Isabella Hanson and Andrew C. Connolly and Tor D. Wager and James V. Haxby and Samuel A. Nastase},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
year = {2026},
month = oct,
doi = {10.1073/pnas.2625018123},
url = {https://doi.org/10.1073/pnas.2625018123},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - JOUR
TI - Action features dominate cortical representation during natural vision
AU - Heejung Jung
AU - Hunar Jain
AU - Xiaochun Han
AU - Ma Feilong
AU - Jane Han
AU - Deepanshi Shokeen
AU - Cara Van Uden
AU - Isabella Hanson
AU - Andrew C. Connolly
AU - Tor D. Wager
AU - James V. Haxby
AU - Samuel A. Nastase
PY - 2026
DA - 2026/10/23
JO - Proceedings of the National Academy of Sciences of the United States of America
DO - 10.1073/pnas.2625018123
UR - https://doi.org/10.1073/pnas.2625018123
AB - Cortical resources are allocated into systems specialized for processing ecologically relevant features of the world. These features are typically studied in isolation using controlled experimental stimuli, making it difficult to assess the relative importance of different kinds of features that tend to overlap during natural vision. In the current study, we evaluated the relative contributions of action, agent, and scene features in predicting cortical activity while participants viewed a 1-h nature documentary. We tested four sets of model features: semantic vectors capturing the observed action, agent, and scene features derived from an annotation of the stimulus, as well as low-level visual motion-energy features. We used banded ridge regression to fit vertex-wise encoding models from the combination of all four sets of features. While each set of features predicted neural activity in the expected areas, the action features predicted more widespread activity. A variance partitioning analysis revealed that the action features captured the most unique variance in neural activity across a much larger extent of cortex than the agent or scene features. Our findings suggest that cortical activity during dynamic, natural vision is dominated by features for understanding the actions of others.
SN - 0027-8424
SN - 1091-6490
ER -