Global Convergence of Third-Order Langevin Dynamics for Non-Convex Optimization via Simulated Annealing
Abstract
We study global convergence guarantees of third-order Langevin dynamics for non-convex optimization via simulated annealing with fixed friction and decreasing noise. An explicit three-block distorted entropy transfers dissipation from the noisy auxiliary variable to the full state. Under dissipativity, regularity, and low-temperature functional-inequality assumptions, logarithmic cooling drives the objective values to the global minimum in probability at the barrier-controlled kinetic rate. For the exact-force-integral and midpoint three-stage discretizations, polynomially decreasing steps preserve this rate on the physical time scale. The cubic local endpoint estimate gives a less restrictive sufficient step-size condition than the available frozen-force kinetic result. A comparison with the one-gradient UBU integrator shows how its centered stochastic local error leads, under the same strong-coupling analysis, to a smaller sufficient iteration exponent. Numerical experiments are conducted to illustrate our theory. For a double well objective, third-order Langevin terminal-success point estimates are higher than UBU at both a common horizon and an equal gradient budget. For a high-dimensional nonconvex neural-network objective using synthetic data, independently tuned UBU and third-order Langevin schemes both outperform overdamped Langevin dynamics; the third-order Langevin point estimate is higher. For the same neural-network objective on real data, we show the same point-estimate ordering for best-basin probability and post-quench test accuracy. Numerical code and associated experiment results are publicly available at https://github.com/gagawjbytw/simulated-annealing-third-order-langevin.
Related
- Reversible Stratonovich perturbations for accelerating convergence to equilibrium of Langevin dynamics
- Initialization Stability and Convergence of Projected Langevin Dynamics
- The Fenchel Game of Underdamped Langevin Dynamics: Insights into Accelerated Convergence
- Langevin dynamics for high-dimensional optimization: the case of multi-spiked tensor PCA
- Accelerating Langevin Dynamics with Weighted Laplacian Flow: Global Well-Posedness and Exponential Convergence
BibTeX
@misc{wang2026global,
title = {{Global Convergence of Third-Order Langevin Dynamics for Non-Convex Optimization via Simulated Annealing}},
author = {Yingli Wang and Lingjiong Zhu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.28611},
eprint = {2609.28611},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.28611},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Global Convergence of Third-Order Langevin Dynamics for Non-Convex Optimization via Simulated Annealing
AU - Yingli Wang
AU - Lingjiong Zhu
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2609.28611
UR - https://arxiv.org/abs/2609.28611
AB - We study global convergence guarantees of third-order Langevin dynamics for non-convex optimization via simulated annealing with fixed friction and decreasing noise. An explicit three-block distorted entropy transfers dissipation from the noisy auxiliary variable to the full state. Under dissipativity, regularity, and low-temperature functional-inequality assumptions, logarithmic cooling drives the objective values to the global minimum in probability at the barrier-controlled kinetic rate. For the exact-force-integral and midpoint three-stage discretizations, polynomially decreasing steps preserve this rate on the physical time scale. The cubic local endpoint estimate gives a less restrictive sufficient step-size condition than the available frozen-force kinetic result. A comparison with the one-gradient UBU integrator shows how its centered stochastic local error leads, under the same strong-coupling analysis, to a smaller sufficient iteration exponent. Numerical experiments are conducted to illustrate our theory. For a double well objective, third-order Langevin terminal-success point estimates are higher than UBU at both a common horizon and an equal gradient budget. For a high-dimensional nonconvex neural-network objective using synthetic data, independently tuned UBU and third-order Langevin schemes both outperform overdamped Langevin dynamics; the third-order Langevin point estimate is higher. For the same neural-network objective on real data, we show the same point-estimate ordering for best-basin probability and post-quench test accuracy. Numerical code and associated experiment results are publicly available at https://github.com/gagawjbytw/simulated-annealing-third-order-langevin.
ER -