Universal Approximation of Measure-to-Measure Operators by Pushforwards
Abstract
Many learning tasks map an input distribution to an output distribution. A natural way to model such an operator is to transform each input sample using a continuous function that may depend on the entire input distribution, and then take the distribution of the transformed samples. This defines a measure-dependent pushforward model and includes measure-theoretic formulations of transformers. We ask when such models can approximate arbitrary continuous operators between spaces of probability measures. We first show that universal approximation fails when atomic inputs are allowed: some continuous measure-to-measure operators that split or redistribute atomic mass cannot be approximated arbitrarily well by deterministic pushforward models. We then introduce the uniform level set condition, which requires a continuous measure-dependent scalarization whose shrinking level set neighborhoods carry uniformly vanishing mass over the input family. This condition is satisfied, in particular, by compact families of absolutely continuous measures. On every compact family satisfying this condition, we prove that any continuous measure-to-measure operator with outputs of finite p-th moment can be uniformly approximated, in the p-Wasserstein distance, by continuous measure-dependent pushforwards. Combining our theorem with existing approximation results for measure-dependent in-context maps yields universal approximation by measure-theoretic transformers. We also extend the framework to continuously-varying source measures, yielding a corresponding universality result for a class of pushforward models that are closely aligned with cross-attention architectures.
Related
- Universal Approximation of Nonlinear Operators and Their Derivatives
- The universal measure of probabilistically nonrandom objects
- Finite-Support Approximation in Variable-Coefficient Sobolev Spaces on Nonhomogeneous Flow-Measure Trees
- Generalized Harmonic Measures: Synchronized Approximation, Representation and Quantitative Measure Recovery
- Quantitative decomposition and approximation for quasi-local operators
BibTeX
@misc{furuya2026universal,
title = {{Universal Approximation of Measure-to-Measure Operators by Pushforwards}},
author = {Takashi Furuya and Nicholas H. Nelsen and Frank Cole},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.35483},
eprint = {2609.35483},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.35483},
}
RIS
TY - UNPB
TI - Universal Approximation of Measure-to-Measure Operators by Pushforwards
AU - Takashi Furuya
AU - Nicholas H. Nelsen
AU - Frank Cole
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2609.35483
UR - https://arxiv.org/abs/2609.35483
AB - Many learning tasks map an input distribution to an output distribution. A natural way to model such an operator is to transform each input sample using a continuous function that may depend on the entire input distribution, and then take the distribution of the transformed samples. This defines a measure-dependent pushforward model and includes measure-theoretic formulations of transformers. We ask when such models can approximate arbitrary continuous operators between spaces of probability measures. We first show that universal approximation fails when atomic inputs are allowed: some continuous measure-to-measure operators that split or redistribute atomic mass cannot be approximated arbitrarily well by deterministic pushforward models. We then introduce the uniform level set condition, which requires a continuous measure-dependent scalarization whose shrinking level set neighborhoods carry uniformly vanishing mass over the input family. This condition is satisfied, in particular, by compact families of absolutely continuous measures. On every compact family satisfying this condition, we prove that any continuous measure-to-measure operator with outputs of finite p-th moment can be uniformly approximated, in the p-Wasserstein distance, by continuous measure-dependent pushforwards. Combining our theorem with existing approximation results for measure-dependent in-context maps yields universal approximation by measure-theoretic transformers. We also extend the framework to continuously-varying source measures, yielding a corresponding universality result for a class of pushforward models that are closely aligned with cross-attention architectures.
ER -