SULA: Robust Langevin Sampling via Drift-Diffusion Saturation
Abstract
The unadjusted Langevin algorithm (ULA) and its stochastic-gradient variant, SGLD, are workhorses of modern Bayesian machine learning, but can be fragile when faced with a target distribution possessing a non-Lipschitz drift. In such settings, the expected squared jump distance can grow rapidly, leading to numerical instability or the need for prohibitively small step-sizes. We propose the saturated unadjusted Langevin algorithm (SULA) that pairs a saturated gradient drift with a gradient-informed state-dependent diffusion, while preserving first-order consistency with the Langevin diffusion. The drift and diffusion terms are chosen to guarantee that the one-step squared update is independent of the gradient. Consequently, SULA is robust to the choice of step-size and preserves estimation and prediction quality. We provide sufficient conditions under which SULA with a step-size h converges weakly to the Langevin diffusion at the standard O(h) rate, the SULA chain is geometrically ergodic, and that the bias of its invariant measure relative to the target is O(h). Further, we develop SG-SULA, a stochastic-gradient version of SULA that rescales the stochastic gradient using a Gaussian CDF correction under Gaussianity of the gradient noise. Numerical experiments on various examples demonstrate SULA's robustness to tuning and SG-SULA's improved predictive performance.
Related
- Drift-diffusion equations with saturation on ℝd
- Kinetic Langevin Meets Split Gibbs: Accelerated Posterior Sampling for Imaging Inverse Problems with Diffusion Priors
- Autonomous self-harmonic drift in Langevin dynamics causes a compaction of underlying domain for the density of Stochastic Localization
- Nonlinear Assimilation via Score-Based Sequential Langevin Sampling
- Neural Actor--Critic Learning with Minimax Critics and Reflected Langevin Sampling
BibTeX
@misc{soni2026sula,
title = {{SULA: Robust Langevin Sampling via Drift-Diffusion Saturation}},
author = {Ritik Soni and Dootika Vats},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.19670},
eprint = {2610.19670},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.19670},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - SULA: Robust Langevin Sampling via Drift-Diffusion Saturation
AU - Ritik Soni
AU - Dootika Vats
PY - 2026
DA - 2026/10/14
PB - arXiv
DO - 10.48550/arxiv.2610.19670
UR - https://arxiv.org/abs/2610.19670
AB - The unadjusted Langevin algorithm (ULA) and its stochastic-gradient variant, SGLD, are workhorses of modern Bayesian machine learning, but can be fragile when faced with a target distribution possessing a non-Lipschitz drift. In such settings, the expected squared jump distance can grow rapidly, leading to numerical instability or the need for prohibitively small step-sizes. We propose the saturated unadjusted Langevin algorithm (SULA) that pairs a saturated gradient drift with a gradient-informed state-dependent diffusion, while preserving first-order consistency with the Langevin diffusion. The drift and diffusion terms are chosen to guarantee that the one-step squared update is independent of the gradient. Consequently, SULA is robust to the choice of step-size and preserves estimation and prediction quality. We provide sufficient conditions under which SULA with a step-size h converges weakly to the Langevin diffusion at the standard O(h) rate, the SULA chain is geometrically ergodic, and that the bias of its invariant measure relative to the target is O(h). Further, we develop SG-SULA, a stochastic-gradient version of SULA that rescales the stochastic gradient using a Gaussian CDF correction under Gaussianity of the gradient noise. Numerical experiments on various examples demonstrate SULA's robustness to tuning and SG-SULA's improved predictive performance.
ER -