Foveation as self-supervision signal: Blur masks beat blank masks for learning robust visual representations
- 1Zuckerman Institute, Columbia University
Abstract
The primate retina is strikingly non-uniform, with receptor density falling off sharply from fovea to periphery. Every eye-movement brings a previously low-acuity peripheral area to high-acuity foveal processing. We hypothesize that this process provides a natural self-supervision signal, enabling representation learning by predicting fine details in the periphery. We implement this in a SimMIM-style vision transformer, replacing its blank masks (ViT-Blank) with a uniform blur mask (ViT-Blur) and further with retinal masks in which a foveal patch is randomly selected, and all other patches are progressively blurred based on eccentricity from the foveal patch (ViT-Retina). ViTs were pre-trained to reconstruct the full-resolution image from the corrupted input and then fine-tuned on clean images for classification. ViT-Blur and ViT-Retina outperform ViT-Blank on the pre-training reconstruction task while controlling for retained information from the input across corruption types. Critically, they also outperform ViT-Blank on downstream classification, showing that they learn representations that generalize better. The ViT-Blur and ViT-Retina are also more robust in impoverishment conditions such as higher mask ratios, fewer pre-training epochs, smaller training sets, and test time image corruptions. Using additive band-limited noise, we discovered that ViT-Blank relies on higher spatial frequencies compared to ViT-Blur and ViT-Retina, and we show this to be the likely cause of poorer generalization: the centroid of a model's spatial frequency tuning is negatively correlated with its classification accuracy (r = −0.73). We further show that a closed-form linear-ridge encoder derived from the same reconstruction objective reproduces this tuning (r = 0.88). Overall, our experiments show the promise of a biologically-inspired self-supervision objective for learning robust visual representations.
Related
- Relational Knowledge Distillation Brings DNN Representations Close Enough to Humans to Be Aligned Without Supervision
- GRASP: Graph Representation Learning with Assay Supervision for Molecular Properties
- BiomECG: Learning Transferable ECG Representations from Small Cohorts
- Intrinsic dimensionality of deep learning representations reveals cell death-associated heterogeneity in Parkinsons disease iPSC-derived neurons
- Dissociable norepinephrine and dopamine representations during strategic decision-making and reward learning
BibTeX
@misc{maruya2026foveation,
title = {{Foveation as self-supervision signal: Blur masks beat blank masks for learning robust visual representations}},
author = {A. Maruya and H. Adeli Jelodar and T. Zheng and N. Kriegeskorte and N. Qian},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.26.754698},
url = {https://www.biorxiv.org/content/10.64898/2026.09.26.754698v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Foveation as self-supervision signal: Blur masks beat blank masks for learning robust visual representations
AU - A. Maruya
AU - H. Adeli Jelodar
AU - T. Zheng
AU - N. Kriegeskorte
AU - N. Qian
PY - 2026
DA - 2026/09/26
PB - bioRxiv
DO - 10.64898/2026.09.26.754698
UR - https://www.biorxiv.org/content/10.64898/2026.09.26.754698v1
AB - The primate retina is strikingly non-uniform, with receptor density falling off sharply from fovea to periphery. Every eye-movement brings a previously low-acuity peripheral area to high-acuity foveal processing. We hypothesize that this process provides a natural self-supervision signal, enabling representation learning by predicting fine details in the periphery. We implement this in a SimMIM-style vision transformer, replacing its blank masks (ViT-Blank) with a uniform blur mask (ViT-Blur) and further with retinal masks in which a foveal patch is randomly selected, and all other patches are progressively blurred based on eccentricity from the foveal patch (ViT-Retina). ViTs were pre-trained to reconstruct the full-resolution image from the corrupted input and then fine-tuned on clean images for classification. ViT-Blur and ViT-Retina outperform ViT-Blank on the pre-training reconstruction task while controlling for retained information from the input across corruption types. Critically, they also outperform ViT-Blank on downstream classification, showing that they learn representations that generalize better. The ViT-Blur and ViT-Retina are also more robust in impoverishment conditions such as higher mask ratios, fewer pre-training epochs, smaller training sets, and test time image corruptions. Using additive band-limited noise, we discovered that ViT-Blank relies on higher spatial frequencies compared to ViT-Blur and ViT-Retina, and we show this to be the likely cause of poorer generalization: the centroid of a model's spatial frequency tuning is negatively correlated with its classification accuracy (r = −0.73). We further show that a closed-form linear-ridge encoder derived from the same reconstruction objective reproduces this tuning (r = 0.88). Overall, our experiments show the promise of a biologically-inspired self-supervision objective for learning robust visual representations.
ER -