Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity
Abstract
We study the minimum number of hidden neurons required for arbitrary-accuracy approximation of multivariate Hölder-continuous functions on [0,1]d and the associated encoding complexity. For d≥ 2, we construct a fixed, explicitly defined activation function for which a closed-form network with two hidden layers of widths d and $1$ achieves arbitrary accuracy in the uniform norm. We prove that d+1 is the exact minimum total number of hidden neurons among standard feedforward networks with locally integrable activations and affine outputs. We further give a simpler construction using a single elementary activation that combines the floor and exponential functions. This construction requires three hidden layers of widths d, $1$, and $2$, only two neurons above the minimum. If a skip connection is allowed, widths d, $1$, and $1$ suffice. These constructions use explicit grid addressing and integer encoding of quantized function values. For a bounded α-Hölder class, they require O(ε−d/αlog(1/ε)) bits, matching the metric-entropy lower bound up to a logarithmic factor.
Related
- Optimal approximation of smooth functions by ReLU neural networks
- Neural Network Approximation and Structural Complexity of Semilinear PDEs with Lower Bounds
- Fully Deterministic Shallow Neural Networks with Localized Activations. Part I: Construction and Optimal Sobolev Approximation
- Optimal Tradeoffs Between Network Size and Parameter Magnitude in Neural Approximation and Minimax Regression
- On the sample complexity of the generalized hidden shift problem over arbitrary finite groups
BibTeX
@misc{cheng2026arbitraryaccuracy,
title = {{Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity}},
author = {Zilan Cheng and Li-Lian Wang and Zhongjian Wang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.35628},
eprint = {2609.35628},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.35628},
note = {CC BY-SA 4.0},
}
RIS
TY - UNPB
TI - Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity
AU - Zilan Cheng
AU - Li-Lian Wang
AU - Zhongjian Wang
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2609.35628
UR - https://arxiv.org/abs/2609.35628
AB - We study the minimum number of hidden neurons required for arbitrary-accuracy approximation of multivariate Hölder-continuous functions on [0,1]^d and the associated encoding complexity. For d≥ 2, we construct a fixed, explicitly defined activation function for which a closed-form network with two hidden layers of widths d and $1$ achieves arbitrary accuracy in the uniform norm. We prove that d+1 is the exact minimum total number of hidden neurons among standard feedforward networks with locally integrable activations and affine outputs. We further give a simpler construction using a single elementary activation that combines the floor and exponential functions. This construction requires three hidden layers of widths d, $1$, and $2$, only two neurons above the minimum. If a skip connection is allowed, widths d, $1$, and $1$ suffice. These constructions use explicit grid addressing and integer encoding of quantized function values. For a bounded α-Hölder class, they require O(ε^−d/αlog(1/ε)) bits, matching the metric-entropy lower bound up to a logarithmic factor.
ER -