Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence
Abstract
Randomly initialised neural networks of width m converge to Gaussian processes as m → ∞, with two types of quantitative bounds characterising this convergence. Finite-sample bounds compare the network with its Gaussian limit at a fixed finite sample set, on an arbitrary domain, in the quadratic Wasserstein distance 𝒲2, achieving a convergence rate of m−1/2 with explicit constants. However, these constants grow with the number of samples, so the bound becomes uninformative as the sample set is refined. Functional bounds remove that dependence by comparing the two as random functions, at the price of spherical input, metrics weaker than 𝒲2, or slower rates. In this paper, we therefore lift a finite-sample bound to a functional one. The lifting refines the sample set until it is dense in the domain, so the bound must first be made independent of the number of samples. We achieve this by measuring the error per sample, which makes the constant uniform. For a polynomial activation and any compact input set, this gives a rate of m−1/2 in the supremum norm, with an explicit constant.
Related
- Constructive Universal Approximation by Infinite-Dimensional Neural Networks via Gaussian-Weighted Ridgelet Transform
- Low regularity maps on Lipschitz manifolds: lifting, pullback of forms, and approximation
- Exact Convex Reformulations of Linear Neural Networks via Completely Positive Lifting
- Numerical approximation of nonlinear fractional dynamical systems for coastal wave modeling: rigorous analysis of convergence and computational efficiency
- Convergence of a relaxation method for the approximation of time dependent viscoplastic flow
BibTeX
@misc{wiederhold2026lifting,
title = {{Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence}},
author = {Luis Wiederhold and Nil Ayday and Debarghya Ghoshdastidar},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.18166},
eprint = {2610.18166},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.18166},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence
AU - Luis Wiederhold
AU - Nil Ayday
AU - Debarghya Ghoshdastidar
PY - 2026
DA - 2026/10/13
PB - arXiv
DO - 10.48550/arxiv.2610.18166
UR - https://arxiv.org/abs/2610.18166
AB - Randomly initialised neural networks of width m converge to Gaussian processes as m → ∞, with two types of quantitative bounds characterising this convergence. Finite-sample bounds compare the network with its Gaussian limit at a fixed finite sample set, on an arbitrary domain, in the quadratic Wasserstein distance 𝒲_2, achieving a convergence rate of m^−1/2 with explicit constants. However, these constants grow with the number of samples, so the bound becomes uninformative as the sample set is refined. Functional bounds remove that dependence by comparing the two as random functions, at the price of spherical input, metrics weaker than 𝒲_2, or slower rates. In this paper, we therefore lift a finite-sample bound to a functional one. The lifting refines the sample set until it is dense in the domain, so the bound must first be made independent of the number of samples. We achieve this by measuring the error per sample, which makes the constant uniform. For a polynomial activation and any compact input set, this gives a rate of m^−1/2 in the supremum norm, with an explicit constant.
ER -