How good are universal MLIPs for alloys? A benchmark is only as trustworthy as its controls

Abstract

Universal machine-learned interatomic potentials (uMLIPs) are ranked by their formation-energy error against a DFT database, as if that database were ground truth, but independent DFT databases disagree. We benchmark fourteen uMLIPs on 1.45 million ordered intermetallics against AFLOW, which none trained on, and compare every energy and stability error with that disagreement. For half the structures, AFLOW, Alexandria, and OQMD differ by less than 6-8 meV/atom, but a minority differ by far more, raising the average to 16-30. That disagreement sets a floor on any model's measurable error. Re-referenced, the best uMLIP agrees with AFLOW as closely as Alexandria does, and the next four sit 2-7 meV/atom above the floor. Once broken elemental references are screened, the energy and stability rankings agree almost everywhere. Three apparent model failures shrink or vanish under the right control: poorly correlated shear moduli, thousands of "bad" DFT references, and the leader's inflated lead.

CrystalREPA: Transferring Physical Priors from Universal MLIPs to Crystal Generative Models
Chengqian Zhang, Yucheng Jin, Duo Zhang +2
arXiv26 Oct 2026PreprintCC BY 4.0Full textPhysical chemistry Computational physicsarXiv
Can Universal Machine-Learning Interatomic Potentials Capture Small Polarons? A Geometry-Based Benchmark in Transition-Metal Oxides
Vladislav Korostelev, Pjotrs Žguns, Konstantin Klyukin
arXiv12 Oct 2026PreprintarXiv licenceFull textPhysical chemistry Computational physicsarXiv
Pressure-transferable universal machine-learning potentials: the WBM-MatPES benchmark and Tuga-FF
Pierre-Paul De Breuck, Thalis H. B. da Silva, Antoine Loew +2
arXiv10 Oct 2026PreprintCC BY 4.0Full textCondensed matterarXiv
uMOF: A Universal Database, Benchmark, and Machine Learning Interatomic Potentials for Metal-Organic Frameworks
Théo Jaffrelot Inizan, Prathami Divakar Kamath, Alin Marin Elena +1
arXiv8 Sep 2026PreprintCC BY 4.0Full textCondensed matterarXiv
Bayesian Kolmogorov-Arnold networks for trustworthy Reynolds-stress closures
Ali Eidi, Kherlen Jigjid, Tyler Buchanan +1
arXiv5 Nov 2026PreprintCC BY 4.0Full textFluid dynamicsarXiv

Cite

BibTeX RIS

BibTeX

@misc{hart2026good,
  title = {{How good are universal MLIPs for alloys? A benchmark is only as trustworthy as its controls}},
  author = {Gus L. W. Hart and Jacob N. Pursley},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.30641},
  eprint = {2610.30641},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.30641},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - How good are universal MLIPs for alloys? A benchmark is only as trustworthy as its controls
AU  - Gus L. W. Hart
AU  - Jacob N. Pursley
PY  - 2026
DA  - 2026/10/22
PB  - arXiv
DO  - 10.48550/arxiv.2610.30641
UR  - https://arxiv.org/abs/2610.30641
AB  - Universal machine-learned interatomic potentials (uMLIPs) are ranked by their formation-energy error against a DFT database, as if that database were ground truth, but independent DFT databases disagree. We benchmark fourteen uMLIPs on 1.45 million ordered intermetallics against AFLOW, which none trained on, and compare every energy and stability error with that disagreement. For half the structures, AFLOW, Alexandria, and OQMD differ by less than 6-8 meV/atom, but a minority differ by far more, raising the average to 16-30. That disagreement sets a floor on any model's measurable error. Re-referenced, the best uMLIP agrees with AFLOW as closely as Alexandria does, and the next four sit 2-7 meV/atom above the floor. Once broken elemental references are screened, the energy and stability rankings agree almost everywhere. Three apparent model failures shrink or vanish under the right control: poorly correlated shear moduli, thousands of "bad" DFT references, and the leader's inflated lead.
ER  -

Abstract

Related

Cite