The microscope is the mask: privileged views and labels from a cryo-ET forward model
Abstract
We explore the use of simulated data for training a model for protein annotation in crowded cryo-electron tomography volumes reconstructed from images collected at limited tilt angles and severely corrupted by the measurement operator. Firstly, we leverage the corruptions imposed by the forward model to generate domain-specific augmented paired views of the exact same scene for an invariance objective integrated into the LeJEPA self-supervised training framework. Secondly, we use additional information from the simulation pipeline such as the positions and identity of proteins in the simulated volumes to inform the architecture of the model and the loss function, so that semantic information is localised at protein positions in the resulting dense feature volume. The resulting model, CARNIVAL, is evaluated without finetuning on classification and detection tasks in real tomograms, using a benchmark dataset containing multiple protein types and two tomogram processing types. We show that CARNIVAL outperforms a state-of-the-art model trained using a contrastive objective on simulated data but without forward model-based paired views or privileged information.
Related
- Privileged SL3-local systems are motivic
- A q-microscope for mock-theta denominator patterns
- Quantum Fluctuations and Wigner Dynamics of a Carbon Nanotube Oscillator Driven by an Atomic Force Microscope
- A detectability floor for treatment-response labels in rheumatology transcriptomics
- Finite-Mask Gaussian-Process Reconstruction on a Periodic Sensor Ring: Mask-Geometry Dependence, Fourier-Mode Coupling, and Posterior Trace
BibTeX
@misc{toader2026microscope,
title = {{The microscope is the mask: privileged views and labels from a cryo-ET forward model}},
author = {Bogdan Toader and Kiarash Jamali and Tanmay A. M. Bharat and Sjors H. W. Scheres},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.04325},
eprint = {2609.04325},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.04325},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The microscope is the mask: privileged views and labels from a cryo-ET forward model
AU - Bogdan Toader
AU - Kiarash Jamali
AU - Tanmay A. M. Bharat
AU - Sjors H. W. Scheres
PY - 2026
DA - 2026/09/03
PB - arXiv
DO - 10.48550/arxiv.2609.04325
UR - https://arxiv.org/abs/2609.04325
AB - We explore the use of simulated data for training a model for protein annotation in crowded cryo-electron tomography volumes reconstructed from images collected at limited tilt angles and severely corrupted by the measurement operator. Firstly, we leverage the corruptions imposed by the forward model to generate domain-specific augmented paired views of the exact same scene for an invariance objective integrated into the LeJEPA self-supervised training framework. Secondly, we use additional information from the simulation pipeline such as the positions and identity of proteins in the simulated volumes to inform the architecture of the model and the loss function, so that semantic information is localised at protein positions in the resulting dense feature volume. The resulting model, CARNIVAL, is evaluated without finetuning on classification and detection tasks in real tomograms, using a benchmark dataset containing multiple protein types and two tomogram processing types. We show that CARNIVAL outperforms a state-of-the-art model trained using a contrastive objective on simulated data but without forward model-based paired views or privileged information.
ER -