Skip to content
DraconiticResearch
Search Sources API
203,203 studies31 sourcesUpdated 58 min ago
PreprintCC BY 4.0arXiv

SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics

  • Dylan Sheldon Rankin
  • Ho Fung Tsoi
Open at sourcePDFDOICite

Abstract

Foundation models often obtain powerful results through very large unlabeled training datasets that dwarf available labeled datasets in size, but high energy physics already provides large samples of high-fidelity labeled simulation. We therefore ask whether self-supervised objectives improve over supervised pretraining alone when large labeled samples already exist. We introduce SuperJEPA, a joint-embedding predictive architec- ture augmented with a supervised objective, and find improved downstream jet-tagging performance across multiple tasks, model sizes, and pretraining dataset sizes. The gains persist for both frozen and fine-tuned encoders, motivating predictive self-supervision in future large-scale particle-physics foundation models.

Related

  1. Effects of Deep Learning Architecture and Pretraining on Automated Vocal Fold Segmentation in Ex Vivo High-Speed Phonation Images
    Varun Vinayak Kalaiarasan, Charles Farbos de Luzan, Khalid Malik +2
    arXivPreprintarXiv licenceFull textFluid dynamicsarXiv
  2. AI-Augmented Inquiry and Regulation in Hybrid Systems: A Control Allocation Architecture for Preserving Epistemic Agency in Hybrid Human-AI Cognition
    Jochen Kuhn, Peter Gerjets, Ulrich Trautwein +4
    arXivPreprintCC BY 4.0Full textGeneral physicsarXiv
  3. Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems
    Kasper Helverskov Petersen, Rasmus Hannibal Tirsgaard, François R J Cornet +2
    arXivPreprintCC BY 4.0Full textComputational physicsarXiv
  4. Pretrain finite element method: A pretraining and warm-start framework for PDEs via physics-informed neural operators
    Yizheng Wang, Zhongkai Hao, Mohammad Sadegh Eshaghi +4
    Journal of the Mechanics and Physics of SolidsPeer-reviewedLicence not statedFull text7 citedStatistical & nonlinear physicsOpenAlex
  5. How Does Distribution Shift Shape Pretraining Gains in Neural PDE Surrogates?
    Pochinapeddi Sai Bhargav, Nithin Somasekharan, Rohit Sunil Kanchi +2
    arXivPreprintarXiv licenceFull textComputational physicsFluid dynamicsarXiv

Cite

BibTeXRIS
BibTeX
@misc{rankin2026superjepa,
  title = {{SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics}},
  author = {Dylan Sheldon Rankin and Ho Fung Tsoi},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.36121},
  eprint = {2610.36121},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.36121},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics
AU  - Dylan Sheldon Rankin
AU  - Ho Fung Tsoi
PY  - 2026
DA  - 2026/10/26
PB  - arXiv
DO  - 10.48550/arxiv.2610.36121
UR  - https://arxiv.org/abs/2610.36121
AB  - Foundation models often obtain powerful results through very large unlabeled training datasets that dwarf available labeled datasets in size, but high energy physics already provides large samples of high-fidelity labeled simulation. We therefore ask whether self-supervised objectives improve over supervised pretraining alone when large labeled samples already exist. We introduce SuperJEPA, a joint-embedding predictive architec- ture augmented with a supervised objective, and find improved downstream jet-tagging performance across multiple tasks, model sizes, and pretraining dataset sizes. The gains persist for both frozen and fine-tuned encoders, motivating predictive self-supervision in future large-scale particle-physics foundation models.
ER  -