Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion
Abstract
Discrete diffusion models have become a powerful framework for modeling complex discrete distributions across a range of applications. While theoretical convergence guarantees depend on consistency between the target and training distributions, such a case rarely holds in practice, making score mismatch a central challenge in discrete diffusion sampling. This is particularly relevant to zero-shot conditional generation, where a model trained on the unconditional distribution is directly reused at inference time to sample from a conditional target distribution. In this paper, we provide the first theoretical analysis for score-mismatched discrete diffusion models and derive a general KL divergence upper bound that clearly identifies the additional bias caused by such a mismatch. Our analysis covers partial-observation constraints, including text infilling, sentence completion, and more general coordinatewise set constraints. Specifically, we analyze the classical Conditional SEDD sampler and show that it can incur a bias term that scales with the dimension, vocabulary size, and compatibility. Based on this analysis, we propose a Randomized Hard Clamping (RHC) sampler for coordinatewise set constraints. We show that the assignment probabilities can be expressed exactly by the unconditional scores with absorbing rate. Finally, numerical and real-data experiments demonstrate the effectiveness of the proposed RHC sampler.
Related
- Direct Conditional Transition Sampling for Diffusion Inverse Problems
- Finite-Sample Guarantees for Fully Discrete Conditional Diffusion Models
- Synchronization in Probability of Reaction–Diffusion Neural Networks Under Stochastic Sampling Governed by Infinite Discrete Distributions
- Conditional Decomposition Discrepancy: Separating Conditional Bias from Sampling Bias in Small-Sample Cross-Domain Cost Forecasting
- Local Anticoncentration for Gaussian Boson Sampling via Conditional Wishart Geometry
BibTeX
@misc{huang2026score,
title = {{Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion}},
author = {Renxiang Huang and Yuchen Liang and Yingbin Liang and Ness Shroff and Lifeng Lai},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.21061},
eprint = {2610.21061},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.21061},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Score Mismatch and Set-Constrained Sampling in Zero-Shot Conditional Discrete Diffusion
AU - Renxiang Huang
AU - Yuchen Liang
AU - Yingbin Liang
AU - Ness Shroff
AU - Lifeng Lai
PY - 2026
DA - 2026/10/15
PB - arXiv
DO - 10.48550/arxiv.2610.21061
UR - https://arxiv.org/abs/2610.21061
AB - Discrete diffusion models have become a powerful framework for modeling complex discrete distributions across a range of applications. While theoretical convergence guarantees depend on consistency between the target and training distributions, such a case rarely holds in practice, making score mismatch a central challenge in discrete diffusion sampling. This is particularly relevant to zero-shot conditional generation, where a model trained on the unconditional distribution is directly reused at inference time to sample from a conditional target distribution. In this paper, we provide the first theoretical analysis for score-mismatched discrete diffusion models and derive a general KL divergence upper bound that clearly identifies the additional bias caused by such a mismatch. Our analysis covers partial-observation constraints, including text infilling, sentence completion, and more general coordinatewise set constraints. Specifically, we analyze the classical Conditional SEDD sampler and show that it can incur a bias term that scales with the dimension, vocabulary size, and compatibility. Based on this analysis, we propose a Randomized Hard Clamping (RHC) sampler for coordinatewise set constraints. We show that the assignment probabilities can be expressed exactly by the unconditional scores with absorbing rate. Finally, numerical and real-data experiments demonstrate the effectiveness of the proposed RHC sampler.
ER -