Flow-Transformed Implicit Processes for Function-Space Variational Inference
Abstract
Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space modelling. However, performing posterior inference with such priors is challenging because their induced function-space distributions are typically not available in closed form. One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples. Existing approaches commonly place a Gaussian variational distribution over the combination weights. While tractable, this choice limits the shapes of posterior uncertainty that can be represented, especially when the true posterior is asymmetric, heavy-tailed, or multimodal. We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive. Instead of using a Gaussian distribution over the combination weights, FTIP uses a normalizing flow to define a richer variational distribution. This induces a flexible posterior distribution over functions while preserving tractable optimization. We train the model using a Black-Box α objective, allowing us to compare mass-covering and mode-seeking variational behaviour. Experiments show that FTIP captures asymmetric and multimodal posterior structure in function space that Gaussian coefficient approximations tend to smooth or collapse.
Related
- Denoising Implicit Variational Inference
- Residual Feedback for Transformed-State Equilibrium Seeking via General Variational Inequalities
- INI-VPINN: A Variational Physics-Informed Neural Network with Implicit Neumann and Interface Handling for Multi-Material Domains with Geometric Singularities
- Implicit Neural Representations for Variational Problems on Graphons
- An invariant variational proof for canonical processes with independent symmetric log-concave tails
BibTeX
@misc{ortega2026flowtransformed,
title = {{Flow-Transformed Implicit Processes for Function-Space Variational Inference}},
author = {Luis A. Ortega and Andrés R. Masegosa and Thomas D. Nielsen},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2606.01954},
eprint = {2606.01954},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.01954},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Flow-Transformed Implicit Processes for Function-Space Variational Inference
AU - Luis A. Ortega
AU - Andrés R. Masegosa
AU - Thomas D. Nielsen
PY - 2026
DA - 2026/10/06
PB - arXiv
DO - 10.48550/arxiv.2606.01954
UR - https://arxiv.org/abs/2606.01954
AB - Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space modelling. However, performing posterior inference with such priors is challenging because their induced function-space distributions are typically not available in closed form. One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples. Existing approaches commonly place a Gaussian variational distribution over the combination weights. While tractable, this choice limits the shapes of posterior uncertainty that can be represented, especially when the true posterior is asymmetric, heavy-tailed, or multimodal. We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive. Instead of using a Gaussian distribution over the combination weights, FTIP uses a normalizing flow to define a richer variational distribution. This induces a flexible posterior distribution over functions while preserving tractable optimization. We train the model using a Black-Box α objective, allowing us to compare mass-covering and mode-seeking variational behaviour. Experiments show that FTIP captures asymmetric and multimodal posterior structure in function space that Gaussian coefficient approximations tend to smooth or collapse.
ER -