Are discrete diffusion models with few data just optimised copy-paste generative methods ?
Abstract
Diffusion models achieve strong generative performance in many applications, but remain difficult to interpret. This issue is particularly important in medical imaging, where datasets are often small and the novelty of generated images must be carefully assessed. Motivated by the generation of breast anatomy images from limited training data, we study what continuoustime discrete diffusion models learn and generate when trained on few images. Focusing on convolutional architectures, we model their inductive biases by constraining the score function to be local and translation-equivariant, and derive the optimal score under the empirical training distribution for a uniform corruption process. We prove that the corresponding reverse diffusion process can generate only images that satisfy the patch mosaic rule of local consistency. Our analysis builds on the seminal work of Kamb and Ganguli for continuous state-space Gaussian diffusion models and establishes a discrete state-space counterpart under a uniform corruption process. We further establish a connection between the resulting reverse diffusion procedure and classical exemplar-based image synthesis methods. In particular, we show that Tweedie τ -leaping with the optimal constrained score and a modified version of the optimization-based texture synthesis algorithm of Kwatra et al. rely on closely related patch-copying operations. Experiments on 2D breast anatomy images show that practical convolutional score networks produce samples close to those generated by the theoretical optimal score when using identical noise seeds. Finally, we quantify the sample novelty of these convolutional discrete diffusion models, across several training settings.
Related
- Generative SDEs with Trainable Diffusion and Density-Based Losses
- Discrete hypocoercive estimates for discontinuous Galerkin methods: application to the Vlasov-Poisson-Fokker-Planck system
- A stabilized scheme satisfying the discrete maximum principle for a time fractional convection-diffusion-reaction equation
- Gradient flow formulation and discrete approximation of reaction-cross-diffusion systems with size exclusion
- Jumping up and down: Denoiser diffusion models for discrete ordinal data
BibTeX
@misc{duval2026discrete,
title = {{Are discrete diffusion models with few data just optimised copy-paste generative methods ?}},
author = {Denis Duval and Agnès Desolneux},
howpublished = {HAL},
year = {2026},
month = sep,
url = {https://hal.science/hal-05757652v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Are discrete diffusion models with few data just optimised copy-paste generative methods ?
AU - Denis Duval
AU - Agnès Desolneux
PY - 2026
DA - 2026/09/21
PB - HAL
UR - https://hal.science/hal-05757652v1
AB - Diffusion models achieve strong generative performance in many applications, but remain difficult to interpret. This issue is particularly important in medical imaging, where datasets are often small and the novelty of generated images must be carefully assessed. Motivated by the generation of breast anatomy images from limited training data, we study what continuoustime discrete diffusion models learn and generate when trained on few images. Focusing on convolutional architectures, we model their inductive biases by constraining the score function to be local and translation-equivariant, and derive the optimal score under the empirical training distribution for a uniform corruption process. We prove that the corresponding reverse diffusion process can generate only images that satisfy the patch mosaic rule of local consistency. Our analysis builds on the seminal work of Kamb and Ganguli for continuous state-space Gaussian diffusion models and establishes a discrete state-space counterpart under a uniform corruption process. We further establish a connection between the resulting reverse diffusion procedure and classical exemplar-based image synthesis methods. In particular, we show that Tweedie τ -leaping with the optimal constrained score and a modified version of the optimization-based texture synthesis algorithm of Kwatra et al. rely on closely related patch-copying operations. Experiments on 2D breast anatomy images show that practical convolutional score networks produce samples close to those generated by the theoretical optimal score when using identical noise seeds. Finally, we quantify the sample novelty of these convolutional discrete diffusion models, across several training settings.
ER -