PreprintarXiv licencearXiv
Shared Is Not Enough: Predictor Width and Factor Readability Under Masked Latent Prediction
Abstract
Masked latent prediction methods such as I-JEPA aim to learn predictable features, but what determines which information they retain is less clear. We built a simulator in which every factor behind an image is known, so a probe can measure how readable each factor remains after training. We trained small encoders (0.7M parameters) with I-JEPA, SimCLR and BYOL on the same pairs of views and probed a factor that both views share. I-JEPA had a predictor with half the encoder's width and depth, the proportions it uses for ViT-B. Its median score was about 40% of SimCLR's. Widening the predictor to the encoder's width reversed this ordering at every training seed, against both other methods. We release the simulator and the trained models.
Related
- Finite is not enough: Triviality bounds in nonlocal field theories
- Energy Is Not Enough: Leakage-Aware Ansatz Criterion for Variational Quantum Eigensolvers
- Transformer-inspired architectures make machine-learned electron densities accurate and transferable enough to bypass self-consistency
- Is Your AI Fast Enough to Run a Fusion Reactor?
- Qubits Are Not Enough: Deflation and Measurement Limits in Variational Quantum Modal Analysis
BibTeX
@misc{kakati2026shared,
title = {{Shared Is Not Enough: Predictor Width and Factor Readability Under Masked Latent Prediction}},
author = {Nilotpal Kakati},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.32562},
eprint = {2610.32562},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.32562},
}
RIS
TY - UNPB
TI - Shared Is Not Enough: Predictor Width and Factor Readability Under Masked Latent Prediction
AU - Nilotpal Kakati
PY - 2026
DA - 2026/10/23
PB - arXiv
DO - 10.48550/arxiv.2610.32562
UR - https://arxiv.org/abs/2610.32562
AB - Masked latent prediction methods such as I-JEPA aim to learn predictable features, but what determines which information they retain is less clear. We built a simulator in which every factor behind an image is known, so a probe can measure how readable each factor remains after training. We trained small encoders (0.7M parameters) with I-JEPA, SimCLR and BYOL on the same pairs of views and probed a factor that both views share. I-JEPA had a predictor with half the encoder's width and depth, the proportions it uses for ViT-B. Its median score was about 40% of SimCLR's. Widening the predictor to the encoder's width reversed this ordering at every training seed, against both other methods. We release the simulator and the trained models.
ER -