Convergence of Denoising Diffusion Probabilistic Models: A Rényi Divergence Perspective
Abstract
Denoising diffusion probabilistic models (DDPMs) have emerged as a powerful framework for generative modeling by transforming standard Gaussian inputs into outputs that approximate a target distribution. While DDPMs have been proved to converge in total variation, KL divergence, and Wasserstein distance, these discrepancy measures share a fundamental limitation of being insensitive to distributional mismatch in low-probability regions. To address this, we adopt the stronger Rényi divergence as the discrepancy measure. Leveraging a Girsanov-type change-of-measure argument, we establish the first Rényi characterization of DDPMs to the best of our knowledge, showing that the total error decomposes into a mixing error, a discretization error, and a score estimation error. We assume the target distribution is sub-Gaussian and characterize the score error via the cumulant generating function, relaxing both the log-Sobolev-inequality and L∞-score-error conditions used in prior Rényi divergence analyses of inexact Langevin Monte Carlo, a score-based sampling algorithm closely related to DDPM. Our results also establish an iteration complexity for achieving ε-accuracy in α-Rényi divergence that is asymptotically proportional to α3/ε. Finally, by extending our analysis, we establish the first Rényi divergence convergence guarantee for critically damped DDPMs, a kinetic variant governed by critically damped Langevin dynamics. Similar to existing characterizations of critically damped DDPMs, this result shares the structural scaling of standard DDPMs, differing by at most a logarithmic factor in the mixing error exponent, and supports the conjecture that critically damped DDPMs do not yield a theoretical improvement.
Related
- Connections between the Föllmer process and the denoising diffusion probabilistic model
- Spatio-temporal Latent Denoising Diffusion Probabilistic Models for Reduced-order Modeling of Parametrized Dynamical Systems
- RDDMPI: Residual Denoising Diffusion Model for Probabilistic Multivariate Time Series Imputation
- Exact Global MCMC with Denoising Diffusion
- Denoising as Projection: Constrained Optimization with Gradient-Guided Diffusion
BibTeX
@misc{liu2026convergence,
title = {{Convergence of Denoising Diffusion Probabilistic Models: A Rényi Divergence Perspective}},
author = {Wenliang Liu and Santiago Mazuelas and Guanding Yu and Renjie Liao and Lele Wang and Yuan Shen},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.24984},
eprint = {2610.24984},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.24984},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Convergence of Denoising Diffusion Probabilistic Models: A Rényi Divergence Perspective
AU - Wenliang Liu
AU - Santiago Mazuelas
AU - Guanding Yu
AU - Renjie Liao
AU - Lele Wang
AU - Yuan Shen
PY - 2026
DA - 2026/10/18
PB - arXiv
DO - 10.48550/arxiv.2610.24984
UR - https://arxiv.org/abs/2610.24984
AB - Denoising diffusion probabilistic models (DDPMs) have emerged as a powerful framework for generative modeling by transforming standard Gaussian inputs into outputs that approximate a target distribution. While DDPMs have been proved to converge in total variation, KL divergence, and Wasserstein distance, these discrepancy measures share a fundamental limitation of being insensitive to distributional mismatch in low-probability regions. To address this, we adopt the stronger Rényi divergence as the discrepancy measure. Leveraging a Girsanov-type change-of-measure argument, we establish the first Rényi characterization of DDPMs to the best of our knowledge, showing that the total error decomposes into a mixing error, a discretization error, and a score estimation error. We assume the target distribution is sub-Gaussian and characterize the score error via the cumulant generating function, relaxing both the log-Sobolev-inequality and L^∞-score-error conditions used in prior Rényi divergence analyses of inexact Langevin Monte Carlo, a score-based sampling algorithm closely related to DDPM. Our results also establish an iteration complexity for achieving ε-accuracy in α-Rényi divergence that is asymptotically proportional to α^3/ε. Finally, by extending our analysis, we establish the first Rényi divergence convergence guarantee for critically damped DDPMs, a kinetic variant governed by critically damped Langevin dynamics. Similar to existing characterizations of critically damped DDPMs, this result shares the structural scaling of standard DDPMs, differing by at most a logarithmic factor in the mixing error exponent, and supports the conjecture that critically damped DDPMs do not yield a theoretical improvement.
ER -