FluxLite: Inference-Time Proposal Control for Discrete Diffusion Models
Abstract
Many inference-time tasks for pretrained discrete diffusion models and diffusion language models reduce to drawing samples from a tilted version of the pretrained distribution. Feynman-Kac sequential Monte Carlo (SMC) makes this correction exact in principle, but its prescribed weights routinely degenerate when the proposal dynamics are misaligned with the tilt, capping the practical gains from additional particles. We introduce FluxLite, a lightweight, training-free proposal-control framework for discrete diffusion. On the sparse directed graph of pretrained reverse rates, any sparse jump-rate perturbation can be exactly compensated by a qt-weighted graph-divergence term in the Feynman-Kac potential; the target path is therefore preserved while the residual reweighting variance becomes a local convex objective. We instantiate this principle as two practical samplers: a one-hop local reallocation rule (HEU) and a small nonnegative quadratic program over pretrained-rate bases (D-VCG). We further prove population stability under the standard score-entropy training loss, identifying a tilted-path coverage factor that governs robustness to score error, together with finite-particle convergence for a fixed controlled Feynman-Kac recursion. Empirically, FluxLite improves over standard Feynman-Kac SMC baselines by up to two orders of magnitude in terminal KL on an analytically tractable finite-state CTMC benchmark, and reduces row-correlation MSE on 2D Ising sampling by 5-7x in geometric mean and up to 55x at peak.
Related
- Likelihood-Free Schrödinger--Föllmer Sampling for Bayesian Inference with Fixed-Proposal Reuse
- Risk-Calibrated Proposal Transport for Finite-Particle Diffusion Steering
- A stabilized scheme satisfying the discrete maximum principle for a time fractional convection-diffusion-reaction equation
- Gradient flow formulation and discrete approximation of reaction-cross-diffusion systems with size exclusion
- Jumping up and down: Denoiser diffusion models for discrete ordinal data
BibTeX
@misc{ren2026fluxlite,
title = {{FluxLite: Inference-Time Proposal Control for Discrete Diffusion Models}},
author = {Yinuo Ren and Haoxuan Chen and Grant M. Rotskoff and Jiequn Han and Lexing Ying},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.35947},
eprint = {2609.35947},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.35947},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - FluxLite: Inference-Time Proposal Control for Discrete Diffusion Models
AU - Yinuo Ren
AU - Haoxuan Chen
AU - Grant M. Rotskoff
AU - Jiequn Han
AU - Lexing Ying
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2609.35947
UR - https://arxiv.org/abs/2609.35947
AB - Many inference-time tasks for pretrained discrete diffusion models and diffusion language models reduce to drawing samples from a tilted version of the pretrained distribution. Feynman-Kac sequential Monte Carlo (SMC) makes this correction exact in principle, but its prescribed weights routinely degenerate when the proposal dynamics are misaligned with the tilt, capping the practical gains from additional particles. We introduce FluxLite, a lightweight, training-free proposal-control framework for discrete diffusion. On the sparse directed graph of pretrained reverse rates, any sparse jump-rate perturbation can be exactly compensated by a q_t-weighted graph-divergence term in the Feynman-Kac potential; the target path is therefore preserved while the residual reweighting variance becomes a local convex objective. We instantiate this principle as two practical samplers: a one-hop local reallocation rule (HEU) and a small nonnegative quadratic program over pretrained-rate bases (D-VCG). We further prove population stability under the standard score-entropy training loss, identifying a tilted-path coverage factor that governs robustness to score error, together with finite-particle convergence for a fixed controlled Feynman-Kac recursion. Empirically, FluxLite improves over standard Feynman-Kac SMC baselines by up to two orders of magnitude in terminal KL on an analytically tractable finite-state CTMC benchmark, and reduces row-correlation MSE on 2D Ising sampling by 5-7x in geometric mean and up to 55x at peak.
ER -