An ODE-based Sampler for Discrete Diffusion Models
Abstract
The marginals of a continuous diffusion model can be reproduced by a deterministic ordinary differential equation (ODE), known as the probability flow ODE, which provides a foundation for accelerated sampling and consistency distillation. In this paper we address the question whether an analogous formulation exists for discrete diffusion models and answer it affirmatively by introducing exponential residual flows (ERFs). Our framework augments the discrete state with a continuous residual variable for each token and position. These residuals evolve deterministically according to piecewise-defined ODEs whose vector fields depend on the current discrete state and the forward and backward rate matrices of the prescribed diffusion model. The discrete state changes whenever a residual reaches zero. We prove that this construction preserves the marginals of the original diffusion. We derive a practical discretization of our sampler and specialize it to masked and uniform diffusion. The deterministic trajectories of ERFs support consistency distillation and yield empirical improvements over previous approaches across a variety of benchmarks including code and text generation tasks.
Related
- Detection and Resolution of Periodic Artifacts in OpenDP's Discrete Laplace Sampler
- Local Geometric Mixing via Dobrushin Contraction with Applications to Diffusion Path Monte Carlo and the Proximal Sampler
- How Many Directions Must a Truncated Diffusion Sampler Retain? Matching Bounds Under Power-Law Spectra
- A stabilized scheme satisfying the discrete maximum principle for a time fractional convection-diffusion-reaction equation
- Gradient flow formulation and discrete approximation of reaction-cross-diffusion systems with size exclusion
BibTeX
@misc{gourevitch2026odebased,
title = {{An ODE-based Sampler for Discrete Diffusion Models}},
author = {Samson Gourevitch and Yazid Janati and Eric Moulines and Eric P. Xing and Alain Durmus and Giacomo Zanella},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.40618},
eprint = {2610.40618},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.40618},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - An ODE-based Sampler for Discrete Diffusion Models
AU - Samson Gourevitch
AU - Yazid Janati
AU - Eric Moulines
AU - Eric P. Xing
AU - Alain Durmus
AU - Giacomo Zanella
PY - 2026
DA - 2026/10/29
PB - arXiv
DO - 10.48550/arxiv.2610.40618
UR - https://arxiv.org/abs/2610.40618
AB - The marginals of a continuous diffusion model can be reproduced by a deterministic ordinary differential equation (ODE), known as the probability flow ODE, which provides a foundation for accelerated sampling and consistency distillation. In this paper we address the question whether an analogous formulation exists for discrete diffusion models and answer it affirmatively by introducing exponential residual flows (ERFs). Our framework augments the discrete state with a continuous residual variable for each token and position. These residuals evolve deterministically according to piecewise-defined ODEs whose vector fields depend on the current discrete state and the forward and backward rate matrices of the prescribed diffusion model. The discrete state changes whenever a residual reaches zero. We prove that this construction preserves the marginals of the original diffusion. We derive a practical discretization of our sampler and specialize it to masked and uniform diffusion. The deterministic trajectories of ERFs support consistency distillation and yield empirical improvements over previous approaches across a variety of benchmarks including code and text generation tasks.
ER -