Approximating Smooth Functionals with ReLU Networks
Abstract
We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by ReLU neural networks. A key feature in deep learning for functional data is the varying importance of different coordinates/dimensions. Representing the functional input in a basis expansion, we quantify the importance of each coordinate through both the magnitude of its corresponding basis score and the directional sensitivity of the target functional. Our analysis combines coordinate truncation, anisotropic partitioning, local Taylor approximation, and ReLU network realization, while allowing unrestricted interactions among the retained coordinates. We establish a general nonasymptotic upper bound for the uniform approximation error and a complementary pseudo-dimension-based lower bound for the worst-case approximation error. Under generalized exponential coordinate decay wdsdasympexp(−cdρ), with ρ>0, the upper and lower bounds match at the leading order, which is stretched-exponential in the logarithm of the network size budget, and thus yield the nearly optimal approximation rate. This is the first work to characterize neural network approximation error for infinite-dimensional functional inputs explicitly through the joint dimensional decay of coordinate magnitudes and directional sensitivities.
Related
- Optimal approximation of smooth functions by ReLU neural networks
- AKAPINN: Adaptive Kolmogorov-Arnold Physics-Informed Neural Networks for approximating solutions to quasilinear partial differential equations
- Deep ReLU Networks Can Fail the Best Approximation Property for Continuous Target Functions
- Distributional Regression for Arbitrary Response Variables Using ReLU Neural Networks
- Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks
BibTeX
@misc{jiao2026approximating,
title = {{Approximating Smooth Functionals with ReLU Networks}},
author = {Shuhao Jiao},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.15355},
eprint = {2609.15355},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.15355},
}
RIS
TY - UNPB
TI - Approximating Smooth Functionals with ReLU Networks
AU - Shuhao Jiao
PY - 2026
DA - 2026/09/20
PB - arXiv
DO - 10.48550/arxiv.2609.15355
UR - https://arxiv.org/abs/2609.15355
AB - We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by ReLU neural networks. A key feature in deep learning for functional data is the varying importance of different coordinates/dimensions. Representing the functional input in a basis expansion, we quantify the importance of each coordinate through both the magnitude of its corresponding basis score and the directional sensitivity of the target functional. Our analysis combines coordinate truncation, anisotropic partitioning, local Taylor approximation, and ReLU network realization, while allowing unrestricted interactions among the retained coordinates. We establish a general nonasymptotic upper bound for the uniform approximation error and a complementary pseudo-dimension-based lower bound for the worst-case approximation error. Under generalized exponential coordinate decay w_ds_dasympexp(−cd^ρ), with ρ>0, the upper and lower bounds match at the leading order, which is stretched-exponential in the logarithm of the network size budget, and thus yield the nearly optimal approximation rate. This is the first work to characterize neural network approximation error for infinite-dimensional functional inputs explicitly through the joint dimensional decay of coordinate magnitudes and directional sensitivities.
ER -