Internal Guidance as a Sampling Method for Tilted Diffusion Models
Abstract
Reward guidance methods sample an exponential tilt of a base density using a diffusion model trained with only samples from the base distribution. Existing methods use what we call external guidance: additive terms in the drift or multiplicative terms on the transition probabilities. We propose internal guidance which leaves the output of the base denoiser unmodified and instead changes the point at which it is evaluated. For discrete distributions embedded in the hypercube {−1,1}d, the denoiser of any linearly tilted density is exactly the base denoiser evaluated at a shifted and rescaled input. In practice the base denoiser is a neural network trained on noised samples of the base distribution. The required evaluation points for sampling the tilted density would be out-of-distribution for basic noising of the base distribution. We modify the training algorithm with random diagonal rescalings of the data, so that the denoiser network learns both in- and out-of-distribution evaluations of the denoiser relative to the base distribution. A single denoiser network samples binary reward tilts, class labels, and infilling with no fine-tuning and no external guidance term.
Related
- Interacting particle guidance for sampling reward-tilted generative priors
- The Tilted Sampling Problem: Optimism, Posterior Sampling, and Logarithmic Regret
- Direct Intermediate Initialization for Tilted Diffusion Samplers
- Steepest Guidance: A Practical and Principled Approach to Inference-Time Alignment of Flow and Diffusion-based Models
- Iterative Exact Discrete Guidance for Energy-Based Sampling
BibTeX
@misc{mcdonald2026internal,
title = {{Internal Guidance as a Sampling Method for Tilted Diffusion Models}},
author = {Curtis McDonald and Peter Bartlett},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.29086},
eprint = {2610.29086},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.29086},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Internal Guidance as a Sampling Method for Tilted Diffusion Models
AU - Curtis McDonald
AU - Peter Bartlett
PY - 2026
DA - 2026/10/21
PB - arXiv
DO - 10.48550/arxiv.2610.29086
UR - https://arxiv.org/abs/2610.29086
AB - Reward guidance methods sample an exponential tilt of a base density using a diffusion model trained with only samples from the base distribution. Existing methods use what we call external guidance: additive terms in the drift or multiplicative terms on the transition probabilities. We propose internal guidance which leaves the output of the base denoiser unmodified and instead changes the point at which it is evaluated. For discrete distributions embedded in the hypercube {−1,1}^d, the denoiser of any linearly tilted density is exactly the base denoiser evaluated at a shifted and rescaled input. In practice the base denoiser is a neural network trained on noised samples of the base distribution. The required evaluation points for sampling the tilted density would be out-of-distribution for basic noising of the base distribution. We modify the training algorithm with random diagonal rescalings of the data, so that the denoiser network learns both in- and out-of-distribution evaluations of the denoiser relative to the base distribution. A single denoiser network samples binary reward tilts, class labels, and infilling with no fine-tuning and no external guidance term.
ER -