Evolutionary stochastic gradient Langevin dynamics: overcoming local traps in large-scale Bayesian sampling.
- 1Department of Statistics, Purdue University, West Lafayette, IN 47907 USA.
Abstract
The scalability of stochastic gradient MCMC algorithms has made them increasingly important for Bayesian analysis of big data. However, these algorithms are prone to getting trapped in local energy minima. We introduce a novel algorithm-evolutionary stochastic gradient Langevin dynamics (ESGLD)- that is particularly effective at mitigating local-trap behavior when simulating posterior distributions in big data scenarios. ESGLD operates with a population of Markov chains, each at a different temperature, and these chains interact through a snooker operator. The snooker operator performs a short run of SGLD along the line connecting the current sample to a selected anchor point. As long as a mode is located by other chains in the population, the target chain can transition to it by running the snooker operator, particularly when the mode is sharp and well-separated from others. The snooker operator enables the Markov chain to effectively escape local traps while maintaining invariance with respect to the target distribution. This significantly accelerates convergence, especially when the energy landscape is rugged. We provide theoretical guarantees for the convergence of the ESGLD algorithm in the big data regime and demonstrate its numerical effectiveness through simulated and real data examples, with the latter modeled by neural networks. The online version contains supplementary material available at https://doi.org/10.1007/s11222-026-11020-6.
Related
- Theoretical guarantees for stochastic gradient Langevin dynamics
- On the computational cost of Stochastic Gradient Langevin Dynamics
- Statistical and Privacy Guarantees for Deep M-Regression via Stochastic Gradient Langevin Dynamics
- Local Algebraic Stability of Wasserstein Gradient Descent-Ascent and Mean Field Langevin Dynamics without Spectral Gap
- Autonomous self-harmonic drift in Langevin dynamics causes a compaction of underlying domain for the density of Stochastic Localization
BibTeX
@article{huang2026evolutionary,
title = {{Evolutionary stochastic gradient Langevin dynamics: overcoming local traps in large-scale Bayesian sampling.}},
author = {Yushu Huang and Faming Liang},
journal = {Statistics and computing},
year = {2026},
month = nov,
doi = {10.1007/s11222-026-11020-6},
url = {https://europepmc.org/article/MED/42996770},
note = {CC BY},
}
RIS
TY - JOUR
TI - Evolutionary stochastic gradient Langevin dynamics: overcoming local traps in large-scale Bayesian sampling.
AU - Yushu Huang
AU - Faming Liang
PY - 2026
DA - 2026/11/05
JO - Statistics and computing
DO - 10.1007/s11222-026-11020-6
UR - https://europepmc.org/article/MED/42996770
AB - The scalability of stochastic gradient MCMC algorithms has made them increasingly important for Bayesian analysis of big data. However, these algorithms are prone to getting trapped in local energy minima. We introduce a novel algorithm-evolutionary stochastic gradient Langevin dynamics (ESGLD)- that is particularly effective at mitigating local-trap behavior when simulating posterior distributions in big data scenarios. ESGLD operates with a population of Markov chains, each at a different temperature, and these chains interact through a snooker operator. The snooker operator performs a short run of SGLD along the line connecting the current sample to a selected anchor point. As long as a mode is located by other chains in the population, the target chain can transition to it by running the snooker operator, particularly when the mode is sharp and well-separated from others. The snooker operator enables the Markov chain to effectively escape local traps while maintaining invariance with respect to the target distribution. This significantly accelerates convergence, especially when the energy landscape is rugged. We provide theoretical guarantees for the convergence of the ESGLD algorithm in the big data regime and demonstrate its numerical effectiveness through simulated and real data examples, with the latter modeled by neural networks. The online version contains supplementary material available at https://doi.org/10.1007/s11222-026-11020-6.
SN - 0960-3174
ER -