Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization
Abstract
Inspired by the function-space theory of neural networks, we formulate and analyze a variation space for nonlinear operators between Hilbert spaces, defined through vector-valued Borel measures of bounded variation. We characterize its unit ball as the closed convex hull of a vector-valued single-neuron dictionary in Bochner spaces. For the ReLU activation, the bounded linear operators in this space are precisely the Schatten-$1$ operators, with equivalent norms. For operators in this space, we establish encoder--decoder approximation bounds in the Bochner Lq-norm, where the error decomposes into input and output encoding errors and a finite-width term of order N−1/2. Under sub-Gaussian assumptions on the input and noise, we further derive high-probability generalization bounds for empirical least squares over path-norm-constrained encoder--decoder networks; the finite-sample contribution to the squared prediction error is of order K−1/2 up to logarithmic factors. The finite-width and finite-sample constants are independent of the encoding dimensions and bases, with the latter also independent of the network width. When the encoding errors decay algebraically, these bounds yield algebraic approximation and learning rates, in contrast to the complexity barriers for Lipschitz and Fréchet differentiable operator classes.
Related
- Resolution-Independent Analysis of Encoder--Decoder Operator Learning via Limiting Kernels
- Generalization of Toeplitz + Hankel operators in the polydisc
- A generalization of the Dirichlet approximation theorem for affine actions on the real line
- Quantitative decomposition and approximation for quasi-local operators
- Characterizing variation bounding in discrete-time Hankel operators
BibTeX
@misc{yang2026variation,
title = {{Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization}},
author = {Jia-Qi Yang and Lei Shi},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.01244},
eprint = {2606.01244},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.01244},
}
RIS
TY - UNPB
TI - Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization
AU - Jia-Qi Yang
AU - Lei Shi
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2606.01244
UR - https://arxiv.org/abs/2606.01244
AB - Inspired by the function-space theory of neural networks, we formulate and analyze a variation space for nonlinear operators between Hilbert spaces, defined through vector-valued Borel measures of bounded variation. We characterize its unit ball as the closed convex hull of a vector-valued single-neuron dictionary in Bochner spaces. For the ReLU activation, the bounded linear operators in this space are precisely the Schatten-$1$ operators, with equivalent norms. For operators in this space, we establish encoder--decoder approximation bounds in the Bochner L^q-norm, where the error decomposes into input and output encoding errors and a finite-width term of order N^−1/2. Under sub-Gaussian assumptions on the input and noise, we further derive high-probability generalization bounds for empirical least squares over path-norm-constrained encoder--decoder networks; the finite-sample contribution to the squared prediction error is of order K^−1/2 up to logarithmic factors. The finite-width and finite-sample constants are independent of the encoding dimensions and bases, with the latter also independent of the network width. When the encoding errors decay algebraically, these bounds yield algebraic approximation and learning rates, in contrast to the complexity barriers for Lipschitz and Fréchet differentiable operator classes.
ER -