Modeling Visual Cortex with Layerwise Multiscale Manifold Capacity Maximization
Abstract
Task-optimized deep neural networks have risen to prominence as the most predictive phenomenological models of responses in primate visual cortex, but leave much to be desired from the perspective of biological plausibility. One such limitation is the reliance on precise credit assignment through global backpropagation of error signals. Recent work has shown that this weakness can be circumvented by requiring each subsequent stage to solve a distinct and increasingly complex task, allowing for layerwise local learning signals. We propose a novel strategy for crafting such intermediate losses that uses an efficient coding framework formulated in terms of manifold capacity, and which can be computed using a sequence of {canonical cortical computations}. In particular, we leverage the relationship between the multiscale nature of visual signals and the dilation of receptive field sizes in cascaded visual representations to modulate complexity, allowing for the reapplication of these common loss computations at each stage of the hierarchy. We evaluate our approach on its ability to predict neural datasets spanning three areas of the ventral stream hierarchy in macaques, as well as human psychophysical data on an object classification task. We find that our unsupervised layerwise model matches or exceeds the performance of competitive architecture-matched baselines on all evaluations considered.
Related
- Multiscale predictive cellular modeling: integrating hypothesis grammars, digital twins, and multi-omics for In silico oncology and precision theranostics
- Data-driven multiscale modeling deciphers MOI-dependent dual antiviral mechanisms of OP7
- Multiscale modeling of host-pathogen interactions and mucociliary clearance during non-tuberculous mycobacterial pulmonary infection
- Multiscale MIGHT: Modeling Local and Systemic Particle Transport in the Cardiovascular System
- Multiscale modeling reveals non-linear dynamics of AKT-mediated NO-ROS crosstalk in shear stress-induced endothelial phenotype
BibTeX
@misc{yerxa2026modeling,
title = {{Modeling Visual Cortex with Layerwise Multiscale Manifold Capacity Maximization}},
author = {Thomas Yerxa and SueYeon Chung and Eero P. Simoncelli},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.02245},
eprint = {2611.02245},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.02245},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Modeling Visual Cortex with Layerwise Multiscale Manifold Capacity Maximization
AU - Thomas Yerxa
AU - SueYeon Chung
AU - Eero P. Simoncelli
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2611.02245
UR - https://arxiv.org/abs/2611.02245
AB - Task-optimized deep neural networks have risen to prominence as the most predictive phenomenological models of responses in primate visual cortex, but leave much to be desired from the perspective of biological plausibility. One such limitation is the reliance on precise credit assignment through global backpropagation of error signals. Recent work has shown that this weakness can be circumvented by requiring each subsequent stage to solve a distinct and increasingly complex task, allowing for layerwise local learning signals. We propose a novel strategy for crafting such intermediate losses that uses an efficient coding framework formulated in terms of manifold capacity, and which can be computed using a sequence of {canonical cortical computations}. In particular, we leverage the relationship between the multiscale nature of visual signals and the dilation of receptive field sizes in cascaded visual representations to modulate complexity, allowing for the reapplication of these common loss computations at each stage of the hierarchy. We evaluate our approach on its ability to predict neural datasets spanning three areas of the ventral stream hierarchy in macaques, as well as human psychophysical data on an object classification task. We find that our unsupervised layerwise model matches or exceeds the performance of competitive architecture-matched baselines on all evaluations considered.
ER -