CortexLens: Systematically Decoding Interpretable Visual Patterns from Human Visual Cortex

Abstract

Understanding how the human brain encodes visual information remains a fundamental challenge in cognitive neuroscience. Sparse dictionary learning (SDL) has recently been applied to fMRI to discover visual features represented in the visual cortex. Existing approaches, however, depend on a vision-language model to identify the feature at discovery, and report limited discovery of the low-level visual features processed in early retinotopic visual regions that play an essential role in human visual cognition. We propose an automatic framework that systematically discovers visual features encoded in the human visual cortex, encompassing both high-level and low-level visual features. The framework decomposes fMRI responses across the full visual hierarchy with Matryoshka sparse autoencoders applied per region of interest, interprets the resulting fMRI activity patterns with a vision-language model, and introduces a set of feature channels to recover the low-level visual features that language-based interpretation fails to capture. On the Natural Scenes Dataset (NSD) and on the Brain Encoder Generated Dataset (BEGD), a separate set of synthesized responses from a transformer brain encoder, the low-level stream recovers thousands of interpretable low-level visual features from early retinotopic visual regions, where prior language-based pipelines report limited discovery. The high-level stream recovers interpretable semantic visual features on both datasets and generalizes to BOLD5000. Our analysis further reveals a systematic confound in language-based interpretation: fMRI activity patterns whose true representation is a low-level visual feature can be misidentified as a semantic visual feature when that feature co-varies with a salient semantic visual feature.

High-dimensional population codes reveal interpretable and diverse features underlying visual perception
H. Issa, S. Liu, J. Balle +1
bioRxiv5 Sep 2026PreprintAll rights reservedFull textNeurosciencebioRxiv
Interpretable Decoding of Frequency-Resolved Functional Connectivity
S. Ruuskanen, E. Saarro, C. M. Caivano +2
bioRxiv20 Aug 2026PreprintCC BYFull textNeurosciencebioRxiv
FloREN: Decoding Immune Regulatory Networks through Interpretable Graph Transformer Patient Representations
Chloé Quéré, Valérie Devauchelle-Pensec, Philippe Thuillier +23
European Journal of Nuclear Medicine and Molecular Imaging20 Jul 2026PreprintCC BY-NC-NDImmunologyHAL
Decoding Extrahepatic Targeting of Lipid Nanoparticles with Interpretable Machine Learning
Asal Mehradfar, Mohammad Shahab Sepehri, Owen Antholine +4
arXiv15 Sep 2026PreprintarXiv licenceFull textBioinformatics & computational biologyarXiv
Img2EEG: A Scalable and Interpretable Encoding Framework for Simulating Human EEG Responses to Visual Inputs
Z. Lu, J. D. Golomb
bioRxiv16 Sep 2026PreprintCC BY-NC-NDFull textNeurosciencebioRxiv

Cite

BibTeX RIS

BibTeX

@misc{qi2026cortexlens,
  title = {{CortexLens: Systematically Decoding Interpretable Visual Patterns from Human Visual Cortex}},
  author = {Qinglin Qi and Yiming Tang and Lu Mi and Juan Zhou and Zhenguang Garry Cai and Xufeng Duan and Dianbo Liu},
  howpublished = {PsyArXiv},
  year = {2026},
  month = oct,
  doi = {10.31234/osf.io/df3w5_v1},
  url = {https://osf.io/preprints/psyarxiv/df3w5_v1/},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - CortexLens: Systematically Decoding Interpretable Visual Patterns from Human Visual Cortex
AU  - Qinglin Qi
AU  - Yiming Tang
AU  - Lu Mi
AU  - Juan Zhou
AU  - Zhenguang Garry Cai
AU  - Xufeng Duan
AU  - Dianbo Liu
PY  - 2026
DA  - 2026/10/23
PB  - PsyArXiv
DO  - 10.31234/osf.io/df3w5_v1
UR  - https://osf.io/preprints/psyarxiv/df3w5_v1/
AB  - Understanding how the human brain encodes visual information remains a fundamental challenge in cognitive neuroscience. Sparse dictionary learning (SDL) has recently been applied to fMRI to discover visual features represented in the visual cortex. Existing approaches, however, depend on a vision-language model to identify the feature at discovery, and report limited discovery of the low-level visual features processed in early retinotopic visual regions that play an essential role in human visual cognition. We propose an automatic framework that systematically discovers visual features encoded in the human visual cortex, encompassing both high-level and low-level visual features. The framework decomposes fMRI responses across the full visual hierarchy with Matryoshka sparse autoencoders applied per region of interest, interprets the resulting fMRI activity patterns with a vision-language model, and introduces a set of feature channels to recover the low-level visual features that language-based interpretation fails to capture. On the Natural Scenes Dataset (NSD) and on the Brain Encoder Generated Dataset (BEGD), a separate set of synthesized responses from a transformer brain encoder, the low-level stream recovers thousands of interpretable low-level visual features from early retinotopic visual regions, where prior language-based pipelines report limited discovery. The high-level stream recovers interpretable semantic visual features on both datasets and generalizes to BOLD5000. Our analysis further reveals a systematic confound in language-based interpretation: fMRI activity patterns whose true representation is a low-level visual feature can be misidentified as a semantic visual feature when that feature co-varies with a salient semantic visual feature.
ER  -

Abstract

Related

Cite