Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks

Abstract

Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights. We extend these results from uniform approximation to approximation in Sobolev norm. Specifically, we analyze how well Sobolev functions in W^n,p can be approximated by neural networks with width W, depth L and path norm bounded by K, when the approximation error is measured in the W^1,p-norm. For shallow networks with depth L=1, we derive the approximation error bound 𝒪(max{W^−(n−1)/d, K^{−(n−1)/(s−n)}}), when the smoothness index satisfies n<s=(d+3)/2 and the input is d-dimensional. For deep networks, we remove the restriction on the smoothness by showing that the approximation bound 𝒪(K^{−(n−1)/(d+d/p+1)}) holds if the width W and depth L are sufficiently large.

Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds
Jia-He Yao
arXiv1 Oct 2026PreprintarXiv licenceFull textNumerical analysis & scientific computingarXiv
Para-Gaussian Approximations of Random Deep Neural Networks
Mihai Nica, Janosch Ortmann
arXiv27 Oct 2026PreprintarXiv licenceFull textProbabilityarXiv
Fully Deterministic Shallow Neural Networks with Localized Activations. Part I: Construction and Optimal Sobolev Approximation
Ran Bi, Weibing Deng
arXiv7 Oct 2026PreprintarXiv licenceFull textNumerical analysis & scientific computingarXiv
Shortcomings and capacities of real-constrained neural networks in complex spaces
Andrew Gracyk
arXiv2 Sep 2026PreprintarXiv licenceFull textProbability Condensed matterarXiv
Monotone Physics-Constrained Graph Neural Networks for Image-Based Structural Damage Assessment: Theory and Leakage-Free Evaluation
Tao Zhang
Mathematics1 Oct 2026Peer-reviewedCC BY 4.0General mathematicsCrossref

Cite

BibTeX RIS

BibTeX

@misc{li2026error,
  title = {{Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks}},
  author = {Xianjun Li and Yunfei Yang},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2609.19937},
  eprint = {2609.19937},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.19937},
}

RIS

TY  - UNPB
TI  - Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks
AU  - Xianjun Li
AU  - Yunfei Yang
PY  - 2026
DA  - 2026/09/17
PB  - arXiv
DO  - 10.48550/arxiv.2609.19937
UR  - https://arxiv.org/abs/2609.19937
AB  - Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights. We extend these results from uniform approximation to approximation in Sobolev norm. Specifically, we analyze how well Sobolev functions in W^n,p can be approximated by neural networks with width W, depth L and path norm bounded by K, when the approximation error is measured in the W^1,p-norm. For shallow networks with depth L=1, we derive the approximation error bound 𝒪(max{W^−(n−1)/d, K^−(n−1)/(s−n)}), when the smoothness index satisfies n<s=(d+3)/2 and the input is d-dimensional. For deep networks, we remove the restriction on the smoothness by showing that the approximation bound 𝒪(K^−(n−1)/(d+d/p+1)) holds if the width W and depth L are sufficiently large.
ER  -

Abstract

Related

Cite