Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems

Abstract

Large-scale self-supervised pretraining has reshaped modern machine learning, substantially advancing the ability of language and vision models to generalize across downstream tasks. While deep learning has driven considerable progress in modeling atomistic systems in recent years, self-supervised pretraining in this domain has not yet achieved comparable downstream generalization. To address this, we introduce Atom-JEPA, a self-supervised pretraining framework that learns latent representations from unlabeled 3D structures through complementary atom-level and substructure-level objectives inspired by joint-embedding predictive architectures. We pretrain Atom-JEPA on large-scale molecular and crystalline datasets and evaluate its transfer performance by fine-tuning on a diverse set of downstream property prediction tasks. Atom-JEPA achieves state-of-the-art performance on molecular ADMET and quantum-chemical property prediction tasks, and is highly competitive in predicting the physical properties of crystalline materials. These results demonstrate the potential of latent-space predictive pretraining to support broad downstream generalization from structural data alone. Code and pretrained model checkpoints are publicly available at https://github.com/khelverskovp/atom-jepa

SuperJEPA: Pretraining with a Supervision-Augmented Joint-Embedding Predictive Architecture for High Energy Physics
Dylan Sheldon Rankin, Ho Fung Tsoi
arXiv26 Oct 2026PreprintCC BY 4.0Full textHigh-energy & particle physicsarXiv
Exploring Joint-Embedding Predictive Architectures for Self-Supervised Representation Learning of Powder X-ray Diffraction Patterns
Ming-Chiang Chang, Brian DeCost
arXiv23 Oct 2026PreprintCC0Full textCondensed matterarXiv
Ferroelectric Switching in ZnO/Zn1-xMgxO Heterostructures: Atomistic Insights into Interfacial Coupling and Layer Architecture
Alireza Sepehrinezhad, Ali Mohammadi Dinani, Ece Gunay +4
arXiv25 Sep 2026PreprintCC BY 4.0Full textPhysical chemistry Condensed matterarXiv
Train for Accuracy, Execute at Scale: Architecture-Preserving Inference for Equivariant Atomistic Foundation Models
Lei Fu, Zihui Feng, Yongheng Li +8
arXiv1 Oct 2026PreprintarXiv licenceFull textCondensed matterarXiv
A quantitative model for embedding quantum-secure communications and evaluating communication autonomy in industrial control systems
Konstantinos Gkouliaras, Vasileios Theos, True Miller +6
npj Quantum Information12 Oct 2026Peer-reviewedCC BY-NC-ND 4.0Full textQuantum physics & quantum informationCrossref

Cite

BibTeX RIS

BibTeX

@misc{petersen2026atomjepa,
  title = {{Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems}},
  author = {Kasper Helverskov Petersen and Rasmus Hannibal Tirsgaard and François R J Cornet and Mikkel Jordahn and Mikkel N. Schmidt},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.08400},
  eprint = {2610.08400},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.08400},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems
AU  - Kasper Helverskov Petersen
AU  - Rasmus Hannibal Tirsgaard
AU  - François R J Cornet
AU  - Mikkel Jordahn
AU  - Mikkel N. Schmidt
PY  - 2026
DA  - 2026/10/06
PB  - arXiv
DO  - 10.48550/arxiv.2610.08400
UR  - https://arxiv.org/abs/2610.08400
AB  - Large-scale self-supervised pretraining has reshaped modern machine learning, substantially advancing the ability of language and vision models to generalize across downstream tasks. While deep learning has driven considerable progress in modeling atomistic systems in recent years, self-supervised pretraining in this domain has not yet achieved comparable downstream generalization. To address this, we introduce Atom-JEPA, a self-supervised pretraining framework that learns latent representations from unlabeled 3D structures through complementary atom-level and substructure-level objectives inspired by joint-embedding predictive architectures. We pretrain Atom-JEPA on large-scale molecular and crystalline datasets and evaluate its transfer performance by fine-tuning on a diverse set of downstream property prediction tasks. Atom-JEPA achieves state-of-the-art performance on molecular ADMET and quantum-chemical property prediction tasks, and is highly competitive in predicting the physical properties of crystalline materials. These results demonstrate the potential of latent-space predictive pretraining to support broad downstream generalization from structural data alone. Code and pretrained model checkpoints are publicly available at https://github.com/khelverskovp/atom-jepa
ER  -

Abstract

Related

Cite