Understanding multi-fidelity training of machine-learned force-fields
- 1University of Oxford
- 2Cambridge CB1 2FB
Abstract
This study systematically investigates two multi-fidelity strategies used to train machine-learned force fields (MLFFs)—pre-training/fine-tuning and multi-headed training—and elucidates the mechanisms underpinning their success. For pre-training and fine-tuning, we uncover a log–log linear relationship between pre-trained and fine-tuned accuracies that holds across model architectures, model sizes, and quantum-chemical methods. The success of this approach hinges on the quantity and quality of available pre-training data, and, critically, the inclusion of force labels. We demonstrate that pre-trained representations are inherently method-specific, requiring adaptation of the model backbone during fine-tuning. In contrast, multi-headed models learn method-independent backbone representations, where again the heads’ accuracies are log–log linearly related. Relative to pre-training and fine-tuning, these shared representations marginally reduce model performance in most cases. However, this trade-off is offset by practical advantages: multi-headed training extends naturally to multiple labelling methods and enables partial replacement of expensive labels with cheaper alternatives, paving the way towards cost-efficient universal MLFFs.
Related
- Bridging Atomistic and Mesoscale Lithium Transport via Machine-Learned Force Fields and Markov State Models
- Towards Out-of-Distribution Uncertainty Quantification for Machine-Learned Interatomic Potentials via Pre-Training
- Selectivity- and Activity-Aware Catalyst Descriptors for CO2 Hydrogenation on Alloy Nanocatalysts using Machine-Learned Force Fields
- Accelerating Mechanistic Understanding and Prediction of Chemical Selectivity with Foundation Machine-Learned Interatomic Potentials
- Full-data accuracy with fewer labels for training and fine-tuning machine-learning force fields
BibTeX
@article{gardner2026understanding,
title = {{Understanding multi-fidelity training of machine-learned force-fields}},
author = {John L. A. Gardner and Hannes Schulz and Jean Hélie and Lixin Sun and Gregor N. C. Simm},
journal = {Machine Learning Science and Technology},
year = {2026},
month = jul,
doi = {10.1088/2632-2153/ae8e2e},
url = {https://doi.org/10.1088/2632-2153/ae8e2e},
note = {CC BY},
}
RIS
TY - JOUR
TI - Understanding multi-fidelity training of machine-learned force-fields
AU - John L. A. Gardner
AU - Hannes Schulz
AU - Jean Hélie
AU - Lixin Sun
AU - Gregor N. C. Simm
PY - 2026
DA - 2026/07/21
JO - Machine Learning Science and Technology
DO - 10.1088/2632-2153/ae8e2e
UR - https://doi.org/10.1088/2632-2153/ae8e2e
AB - This study systematically investigates two multi-fidelity strategies used to train machine-learned force fields (MLFFs)—pre-training/fine-tuning and multi-headed training—and elucidates the mechanisms underpinning their success. For pre-training and fine-tuning, we uncover a log–log linear relationship between pre-trained and fine-tuned accuracies that holds across model architectures, model sizes, and quantum-chemical methods. The success of this approach hinges on the quantity and quality of available pre-training data, and, critically, the inclusion of force labels. We demonstrate that pre-trained representations are inherently method-specific, requiring adaptation of the model backbone during fine-tuning. In contrast, multi-headed models learn method-independent backbone representations, where again the heads’ accuracies are log–log linearly related. Relative to pre-training and fine-tuning, these shared representations marginally reduce model performance in most cases. However, this trade-off is offset by practical advantages: multi-headed training extends naturally to multiple labelling methods and enables partial replacement of expensive labels with cheaper alternatives, paving the way towards cost-efficient universal MLFFs.
SN - 2632-2153
ER -