Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion
Abstract
We introduce sticky jump diffusions (SJDs), continuous-time Markov processes on ℝd whose discrete anchors are token embeddings. In forward time, anchors release their mass at a hazard rate and the released mass diffuses in the continuous ambient space; time reversal couples a score-driven SDE with a sticky jump kernel whose rate and destination are fixed by flux balance with the forward law. We estimate the score and the per-anchor reverse hazards from a single denoising classifier via denoising hazard matching, the hazard analogue of denoising score matching, with simulation-free cross-entropy training. SJD recovers masked diffusion, continuous diffusion, and hybrid diffusion as limits. Its reversal explains features that each family treats as given: the mask of masked diffusion carries no evidence about the source token because the unsticking kernel of every anchor collapses to the same absorbing point; the terminal projection of continuous diffusion is required due to the absence of atoms in its forward marginal, without which flux balance yields no reverse jumps; and the update rules of hybrid diffusion (commit rate, destination, and drift) all follow from flux balance rather than from separate design. Beyond these limits, the unsticking kernel becomes a design space: a cross-position blending corrupts each position toward a blend of its neighbors' clean values or embeddings, turning dependency structure such as spatial locality or a constraint graph into an inductive bias of the corruption itself, and improves over the identity-kernel hybrid on CIFAR-10, Text8, and Sudoku.
Related
- Limit theorems for one-dimensional diffusions: reflected, skew, sticky, skew-sticky, and snapping-out limits
- Sticky-threshold diffusions, local time approximation and parameter estimation
- Jump limits of continuous multiscale diffusions: Homogenization under heavy-tailed activity bursts
- Structured Masked Diffusion for Shared-Codebook Unsourced Random Access Decoding
- Representation-based Masked Diffusion Model
BibTeX
@misc{jutrasdube2026sticky,
title = {{Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion}},
author = {Pascal Jutras-Dubé and Patrick Pynadath and Jeremy Lu and Yuan Gao and Ruqi Zhang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2607.10951},
eprint = {2607.10951},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.10951},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion
AU - Pascal Jutras-Dubé
AU - Patrick Pynadath
AU - Jeremy Lu
AU - Yuan Gao
AU - Ruqi Zhang
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2607.10951
UR - https://arxiv.org/abs/2607.10951
AB - We introduce sticky jump diffusions (SJDs), continuous-time Markov processes on ℝ^d whose discrete anchors are token embeddings. In forward time, anchors release their mass at a hazard rate and the released mass diffuses in the continuous ambient space; time reversal couples a score-driven SDE with a sticky jump kernel whose rate and destination are fixed by flux balance with the forward law. We estimate the score and the per-anchor reverse hazards from a single denoising classifier via denoising hazard matching, the hazard analogue of denoising score matching, with simulation-free cross-entropy training. SJD recovers masked diffusion, continuous diffusion, and hybrid diffusion as limits. Its reversal explains features that each family treats as given: the mask of masked diffusion carries no evidence about the source token because the unsticking kernel of every anchor collapses to the same absorbing point; the terminal projection of continuous diffusion is required due to the absence of atoms in its forward marginal, without which flux balance yields no reverse jumps; and the update rules of hybrid diffusion (commit rate, destination, and drift) all follow from flux balance rather than from separate design. Beyond these limits, the unsticking kernel becomes a design space: a cross-position blending corrupts each position toward a blend of its neighbors' clean values or embeddings, turning dependency structure such as spatial locality or a constraint graph into an inductive bias of the corruption itself, and improves over the identity-kernel hybrid on CIFAR-10, Text8, and Sudoku.
ER -