Weighted Adjoint Matching: Finetuning Flow Matching Models with Time-Dependent Control Cost
Abstract
We introduce Weighted Adjoint Matching, a method to finetune a pretrained diffusion or flow matching model given a differentiable reward. We derive it from Adjoint Matching, which reformulates finetuning as a stochastic optimal control problem with a quadratic control cost. Building on the observation that these models generate coarse features before fine details, we place a time-dependent weight on the control cost. A high cost early in the generative process preserves balance and diversity in the coarse features, which carry most of the semantics, while a low cost at the end lets the model increase the reward by refining details. We prove that our method is equivalent to maximizing the reward under time-varying Kullback--Leibler regularization on the generative paths. Experiments on Gaussian mixtures and on MNIST show a better tradeoff between reward and mode balance than Adjoint Matching.
Related
- LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL
- Bias-Aware Adjoint Calibration and Optimal Control of Stochastic Multiscale Bioprocess Digital Twins
- Sink polynomials of line graphs and weighted matching polynomials
- Scalable Pontryagin-Guided Adjoint-to-Control Recovery for Constrained Dynamic Portfolio Choice
- Weighted Score Matching on Bounded Domains: Nonparametric Density Estimation and Interaction Graph Recovery
BibTeX
@misc{beyler2026weighted,
title = {{Weighted Adjoint Matching: Finetuning Flow Matching Models with Time-Dependent Control Cost}},
author = {Eliot Beyler and Francis Bach},
howpublished = {HAL},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.20035},
eprint = {2610.20035},
archivePrefix = {arXiv},
url = {https://hal.science/hal-05795740v1},
}
RIS
TY - UNPB
TI - Weighted Adjoint Matching: Finetuning Flow Matching Models with Time-Dependent Control Cost
AU - Eliot Beyler
AU - Francis Bach
PY - 2026
DA - 2026/10/12
PB - HAL
DO - 10.48550/arxiv.2610.20035
UR - https://hal.science/hal-05795740v1
AB - We introduce Weighted Adjoint Matching, a method to finetune a pretrained diffusion or flow matching model given a differentiable reward. We derive it from Adjoint Matching, which reformulates finetuning as a stochastic optimal control problem with a quadratic control cost. Building on the observation that these models generate coarse features before fine details, we place a time-dependent weight on the control cost. A high cost early in the generative process preserves balance and diversity in the coarse features, which carry most of the semantics, while a low cost at the end lets the model increase the reward by refining details. We prove that our method is equivalent to maximizing the reward under time-varying Kullback--Leibler regularization on the generative paths. Experiments on Gaussian mixtures and on MNIST show a better tradeoff between reward and mode balance than Adjoint Matching.
ER -