Equilibrium Training of Energy-Based Models with Parallel Trajectory Tempering
Abstract
Energy-Based Models (EBMs) provide an interpretable framework for generative modeling of scientific data, but poor Markov Chain Monte Carlo mixing often limits their reliability. We introduce a training algorithm based on Parallel Trajectory Tempering (PTT), which exploits the continuity of the optimization path to maintain equilibrium sampling throughout learning. This enables stable and fast training on highly multimodal and data-scarce scientific datasets. Combined with reservoir sampling and adaptive optimization, PTT has a computational cost comparable to Persistent Contrastive Divergence, making it a practical replacement for standard training methods. It also provides direct estimates of thermalization times, equilibrium samples from trained models, and accurate log-likelihoods at essentially no additional cost. Experiments on Restricted Boltzmann Machines show that PTT consistently outperforms existing EBM training approaches. On discrete tabular data, it also surpasses state-of-the-art deep generative models, yielding higher-quality samples and greater robustness to overfitting and limited data. Our results make equilibrium maximum-likelihood training of EBMs practical and computationally efficient.
Related
- Fast free-energy estimation for complex systems with nonreversible simulated tempering
- Real-time Tokamak Equilibrium Reconstruction Under Limited Experimental Data via Physics-Grounded Synthetic Pre-training
- Cramer-Rao Inequality Generalizes the Equilibrium Energy Fluctuation-Response Relation to Nonequilibrium Steady States
- Smite: A quasiclassical trajectory (QCT) program for bimolecular collisions and unimolecular dynamics on ab-initio, and machine-learned potential energy surfaces
- Optimal Intermediate Hamiltonians for Non-Equilibrium Free Energy Calculations: A Numerical Study of Markov Models
BibTeX
@misc{bereux2026equilibrium,
title = {{Equilibrium Training of Energy-Based Models with Parallel Trajectory Tempering}},
author = {Nicolas Béreux and Aurélien Decelle and Cyril Furtlehner and Beatriz Seoane},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.27077},
eprint = {2607.27077},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.27077},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Equilibrium Training of Energy-Based Models with Parallel Trajectory Tempering
AU - Nicolas Béreux
AU - Aurélien Decelle
AU - Cyril Furtlehner
AU - Beatriz Seoane
PY - 2026
DA - 2026/09/07
PB - arXiv
DO - 10.48550/arxiv.2607.27077
UR - https://arxiv.org/abs/2607.27077
AB - Energy-Based Models (EBMs) provide an interpretable framework for generative modeling of scientific data, but poor Markov Chain Monte Carlo mixing often limits their reliability. We introduce a training algorithm based on Parallel Trajectory Tempering (PTT), which exploits the continuity of the optimization path to maintain equilibrium sampling throughout learning. This enables stable and fast training on highly multimodal and data-scarce scientific datasets. Combined with reservoir sampling and adaptive optimization, PTT has a computational cost comparable to Persistent Contrastive Divergence, making it a practical replacement for standard training methods. It also provides direct estimates of thermalization times, equilibrium samples from trained models, and accurate log-likelihoods at essentially no additional cost. Experiments on Restricted Boltzmann Machines show that PTT consistently outperforms existing EBM training approaches. On discrete tabular data, it also surpasses state-of-the-art deep generative models, yielding higher-quality samples and greater robustness to overfitting and limited data. Our results make equilibrium maximum-likelihood training of EBMs practical and computationally efficient.
ER -