Discrete Diffusion Models via Evolving Variational Autoregressive Networks
Abstract
Conventional score-based diffusion models learn scores without representing normalized densities, whereas tractable normalized models support both sampling and direct likelihood evaluation. A recent tensor-network approach provides such a representation but is largely restricted to low-dimensional lattices. Here we introduce a discrete diffusion model that parameterizes normalized probability distributions using variational autoregressive networks. Explicit Markov jump operators govern the forward noising and reverse denoising dynamics, extending discrete diffusion models with normalized distributions to spin systems on higher-dimensional lattices. We apply this framework to the two- and three-dimensional Ising models across ordered, critical, and disordered regimes, accurately computing thermodynamic quantities including free energy, energy, and magnetization. We further integrate the framework with Monte Carlo sampling, using adaptive diffusion steps to maintain high acceptance rates even at low temperatures while enhancing sample diversity. These results establish a neural-network framework for the discrete diffusion model with normalized probability distributions.
Related
- ADAPTIVE MULTIRESOLUTION DIFFUSION OPERATORS: A VARIATIONAL THEORY ON EVOLVING MULTIRESOLUTION SPACES
- Synchronization in Probability of Reaction–Diffusion Neural Networks Under Stochastic Sampling Governed by Infinite Discrete Distributions
- Scalable Sparse Autoregressive Variational Inference for Non-Gaussian Spatial Fields
- Online Change-Point Detection in Linear Regression with Autoregressive Errors via Streaming Variational Bayes
- HIGH-ORDER DISCRETE DIFFERENTIAL AND INTEGRAL CALCULUS AND GALERKIN VARIATIONAL INTEGRATORS
BibTeX
@misc{pan2026discrete,
title = {{Discrete Diffusion Models via Evolving Variational Autoregressive Networks}},
author = {Kewen Pan and Ying Tang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.27306},
eprint = {2609.27306},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.27306},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Discrete Diffusion Models via Evolving Variational Autoregressive Networks
AU - Kewen Pan
AU - Ying Tang
PY - 2026
DA - 2026/09/23
PB - arXiv
DO - 10.48550/arxiv.2609.27306
UR - https://arxiv.org/abs/2609.27306
AB - Conventional score-based diffusion models learn scores without representing normalized densities, whereas tractable normalized models support both sampling and direct likelihood evaluation. A recent tensor-network approach provides such a representation but is largely restricted to low-dimensional lattices. Here we introduce a discrete diffusion model that parameterizes normalized probability distributions using variational autoregressive networks. Explicit Markov jump operators govern the forward noising and reverse denoising dynamics, extending discrete diffusion models with normalized distributions to spin systems on higher-dimensional lattices. We apply this framework to the two- and three-dimensional Ising models across ordered, critical, and disordered regimes, accurately computing thermodynamic quantities including free energy, energy, and magnetization. We further integrate the framework with Monte Carlo sampling, using adaptive diffusion steps to maintain high acceptance rates even at low temperatures while enhancing sample diversity. These results establish a neural-network framework for the discrete diffusion model with normalized probability distributions.
ER -