Shallow neural network approximation in mixed Sobolev spaces
Abstract
We investigate the best L2 approximation of mixed Sobolev spaces by shallow neural networks with n neurons and general activation functions. We first establish an activation-independent Fourier-block principle: if an activation has univariate approximation order ρ in the sense of the Fourier-block property, then the global approximation rate has algebraic order min{α,ρ} for target functions of mixed smoothness α, up to explicit logarithmic factors. To verify this property for concrete activations, we introduce a structured univariate approximation condition that implies the Fourier-block property with explicit parameters. For ReLUk, a matching algebraic lower bound identifies min{α,k+1} as the optimal algebraic approximation exponent in any dimension, up to logarithmic factors in the upper bound. The framework also yields the exponent min{α,k+1} for cardinal B-splines and soft-ReLUk, and the full mixed-smoothness exponent α for ELU and cosine activations, again up to logarithmic~factors.
Related
- Fully Deterministic Shallow Neural Networks with Localized Activations. Part I: Construction and Optimal Sobolev Approximation
- Harmonic analysis and circuit approximation of Sobolev functions on quantum input spaces
- Finite-Support Approximation in Variable-Coefficient Sobolev Spaces on Nonhomogeneous Flow-Measure Trees
- Sharp Sobolev Sandwich and Approximation Rates of Radon-Domain Lp Ridge Integral Spaces for ReLUk Networks
- Divergence-Free Approximation in Sobolev and Lebesgue Spaces on General Unbounded Domains with Applications to Energy Equality in Fluid Dynamics
BibTeX
@misc{li2026shallow,
title = {{Shallow neural network approximation in mixed Sobolev spaces}},
author = {Yuwen Li and Guozhi Zhang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.05263},
eprint = {2609.05263},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.05263},
}
RIS
TY - UNPB
TI - Shallow neural network approximation in mixed Sobolev spaces
AU - Yuwen Li
AU - Guozhi Zhang
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2609.05263
UR - https://arxiv.org/abs/2609.05263
AB - We investigate the best L_2 approximation of mixed Sobolev spaces by shallow neural networks with n neurons and general activation functions. We first establish an activation-independent Fourier-block principle: if an activation has univariate approximation order ρ in the sense of the Fourier-block property, then the global approximation rate has algebraic order min{α,ρ} for target functions of mixed smoothness α, up to explicit logarithmic factors. To verify this property for concrete activations, we introduce a structured univariate approximation condition that implies the Fourier-block property with explicit parameters. For ReLU^k, a matching algebraic lower bound identifies min{α,k+1} as the optimal algebraic approximation exponent in any dimension, up to logarithmic factors in the upper bound. The framework also yields the exponent min{α,k+1} for cardinal B-splines and soft-ReLU^k, and the full mixed-smoothness exponent α for ELU and cosine activations, again up to logarithmic~factors.
ER -