Interacting particle guidance for sampling reward-tilted generative priors
Abstract
Inference-time steering adapts pretrained diffusion and flow-based models to new tasks, e.g., to generate samples from a conditional distribution or samples with desired properties, without retraining. This can be formalized as sampling from a reward-tilted generative prior. As exact sampling from this distribution is intractable, guidance-based methods rely on approximations producing biased samples, and sequential Monte Carlo (SMC) methods correct for this bias using importance weights. However, while exact in the large particle limit, SMC suffers from weight degeneracy and particle collapse in practice. We propose interacting particle guidance (IPG), which replaces reweighting with transport. The particles interact through an additional drift, derived from the Feynman--Kac PDE to cancel the reweighting term, and remain unweighted. Choosing the drift in a reproducing kernel Hilbert space yields a closed-form solution that is cheap to compute, with negligible overhead compared to SMC. We demonstrate the method on Gaussian mixtures with known posteriors, and on high-dimensional image inpainting and protein structure inference tasks.
Related
- VGAS: Variance-Reduced Guidance and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusion
- How to post-train on a surrogate: Envelope sampling mitigates reward hacking
- Two-way Guidance: Simultaneous Learning of Summary Statistics for Generative Bayes
- Encoding Designers’ Aesthetic Judgment into Fast Generative Interior Design: Expert-Calibrated Reward Alignment with ARIA-Diff
- Internal Guidance as a Sampling Method for Tilted Diffusion Models
BibTeX
@misc{kalaivanan2026interacting,
title = {{Interacting particle guidance for sampling reward-tilted generative priors}},
author = {Adhithyan Kalaivanan and Zheng Zhao and Jens Sjölund and Fredrik Lindsten},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.37227},
eprint = {2609.37227},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.37227},
}
RIS
TY - UNPB
TI - Interacting particle guidance for sampling reward-tilted generative priors
AU - Adhithyan Kalaivanan
AU - Zheng Zhao
AU - Jens Sjölund
AU - Fredrik Lindsten
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.37227
UR - https://arxiv.org/abs/2609.37227
AB - Inference-time steering adapts pretrained diffusion and flow-based models to new tasks, e.g., to generate samples from a conditional distribution or samples with desired properties, without retraining. This can be formalized as sampling from a reward-tilted generative prior. As exact sampling from this distribution is intractable, guidance-based methods rely on approximations producing biased samples, and sequential Monte Carlo (SMC) methods correct for this bias using importance weights. However, while exact in the large particle limit, SMC suffers from weight degeneracy and particle collapse in practice. We propose interacting particle guidance (IPG), which replaces reweighting with transport. The particles interact through an additional drift, derived from the Feynman--Kac PDE to cancel the reweighting term, and remain unweighted. Choosing the drift in a reproducing kernel Hilbert space yields a closed-form solution that is cheap to compute, with negligible overhead compared to SMC. We demonstrate the method on Gaussian mixtures with known posteriors, and on high-dimensional image inpainting and protein structure inference tasks.
ER -