Same Flow, Different Paths: Variance Reduction in Flow Matching
Abstract
In flow matching (FM), a velocity model vθ is trained using a predefined path gt that connects data and noise samples (e.g., gt(x0, x1) = (1 − t) x0 + t x1). In this work, we study the choice of this path from an optimization perspective by analyzing the variance of stochastic gradients. We consider the class G(pt,v⋆t) of paths that induce the same marginal distributions pt and marginal velocity field v⋆t, and therefore the same FM objective. Our main finding is that the choice of path gt can fundamentally change the convergence rate of SGD, even when the FM objective remains exactly the same. (i) For a linear velocity model and one-dimensional Gaussian data, we derive a tight bound on the SGD iteration complexity up to logarithmic factors and find an analytically optimal path that minimizes this bound among linear paths inducing the same FM problem. (ii) We then extend the variance analysis to general FM problems and formulate path selection at a fixed θ as the variance-minimization problem PathOptθ, constrained to gt∈ G(pt,v⋆t). We show that this constraint is essential: reducing variance without it can lead to slower convergence. (iii) Since the constraint gt ∈ G(pt,v⋆t) cannot generally be verified directly, we derive an equivalent formulation with constraints that can be estimated from samples, allowing paths to be found numerically. Our theoretical results are supported by experiments with Gaussian data, Gaussian mixture models, and real datasets.
Related
- Same stimuli, different spaces: how task, observer and amount of data influence geometry and reliability of mental representations
- Variance Bounds for Minimum Perfect Matching Problem by Random Walk on Multinomial Distribution
- Generalized Posterior Calibration via Infinitesimal-Jackknife Variance Matching
- Same path, different walk: a mechanistic comparison of looped and stacked transformer encoders on 12-lead ECG
- Different representation learning objectives recover distinct latent structures from the same psychometric data
BibTeX
@misc{tyurin2026same,
title = {{Same Flow, Different Paths: Variance Reduction in Flow Matching}},
author = {Alexander Tyurin},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.17287},
eprint = {2609.17287},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.17287},
}
RIS
TY - UNPB
TI - Same Flow, Different Paths: Variance Reduction in Flow Matching
AU - Alexander Tyurin
PY - 2026
DA - 2026/09/15
PB - arXiv
DO - 10.48550/arxiv.2609.17287
UR - https://arxiv.org/abs/2609.17287
AB - In flow matching (FM), a velocity model v_θ is trained using a predefined path g_t that connects data and noise samples (e.g., g_t(x_0, x_1) = (1 − t) x_0 + t x_1). In this work, we study the choice of this path from an optimization perspective by analyzing the variance of stochastic gradients. We consider the class G(p_t,v^⋆_t) of paths that induce the same marginal distributions p_t and marginal velocity field v^⋆_t, and therefore the same FM objective. Our main finding is that the choice of path g_t can fundamentally change the convergence rate of SGD, even when the FM objective remains exactly the same. (i) For a linear velocity model and one-dimensional Gaussian data, we derive a tight bound on the SGD iteration complexity up to logarithmic factors and find an analytically optimal path that minimizes this bound among linear paths inducing the same FM problem. (ii) We then extend the variance analysis to general FM problems and formulate path selection at a fixed θ as the variance-minimization problem PathOpt_θ, constrained to g_t∈ G(p_t,v^⋆_t). We show that this constraint is essential: reducing variance without it can lead to slower convergence. (iii) Since the constraint g_t ∈ G(p_t,v^⋆_t) cannot generally be verified directly, we derive an equivalent formulation with constraints that can be estimated from samples, allowing paths to be found numerically. Our theoretical results are supported by experiments with Gaussian data, Gaussian mixture models, and real datasets.
ER -