PreprintarXiv licencearXiv
Deep ReLU Networks Can Fail the Best Approximation Property for Continuous Target Functions
Abstract
We construct a continuous function on [−1,1]2 for which the infimum of the L2-approximation error over the class of ReLU networks with two hidden layers and architecture $(2,2,1,1)$ is not attained. This counterexample shows that deep ReLU networks do not satisfy the best approximation property in general, even for continuous target functions. Our findings contrast with existing results for shallow neural networks, where ReLU is one of the few activation functions that guarantee the existence of global minima in typical regression tasks. The proof is based on the fact that the breaklines for neurons in the second hidden layer can bend, which breaks the symmetry arguments used in the one-hidden-layer case.
Related
- The ball fixed point property for continuous functions on locally compact spaces
- A survey on the Asplund property for spaces C(X) of continuous functions
- Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds
- Optimal approximation of smooth functions by ReLU neural networks
- Approximation theory of tree tensor networks: tensorized multivariate functions
BibTeX
@misc{kassing2026deep,
title = {{Deep ReLU Networks Can Fail the Best Approximation Property for Continuous Target Functions}},
author = {Sebastian Kassing},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.31264},
eprint = {2610.31264},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.31264},
}
RIS
TY - UNPB
TI - Deep ReLU Networks Can Fail the Best Approximation Property for Continuous Target Functions
AU - Sebastian Kassing
PY - 2026
DA - 2026/10/22
PB - arXiv
DO - 10.48550/arxiv.2610.31264
UR - https://arxiv.org/abs/2610.31264
AB - We construct a continuous function on [−1,1]^2 for which the infimum of the L^2-approximation error over the class of ReLU networks with two hidden layers and architecture $(2,2,1,1)$ is not attained. This counterexample shows that deep ReLU networks do not satisfy the best approximation property in general, even for continuous target functions. Our findings contrast with existing results for shallow neural networks, where ReLU is one of the few activation functions that guarantee the existence of global minima in typical regression tasks. The proof is based on the fact that the breaklines for neurons in the second hidden layer can bend, which breaks the symmetry arguments used in the one-hidden-layer case.
ER -