Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds
Abstract
The universal approximation property of dropout neural networks does not by itself describe the network size required for an accurate random realization. In this work, we study approximation of the unit ball of Wn,∞([0,1]d) by ReLU networks whose edges are retained independently with probability p. The approximation error is measured uniformly over the input domain, and the guarantee holds with probability at least 1−δ for a single sampled network. We construct networks of constant depth and size Õn,d(p−9ε−max{d/n,2} log(1/δ)). The construction combines bounded local subnetworks, localization on a successful approximation event, and a multiscale Taylor decomposition. Conversely, Sobolev capacity imposes a lower bound on the number of surviving edges, while approximation of a fixed affine function requires an output-layer cost of order ((1−p)/p)ε−2log(1/δ) at sufficiently high confidence. For fixed p∈(0,1) and δ<min{1/2,1−p}, the upper and lower bounds match in the accuracy exponent under a fixed or logarithmic depth budget. When d≤2n, they also match in confidence up to logarithms of accuracy. We extend the lower bounds to Wn,r targets with Ls error, and distinguish this extension from the upper bound for Wn,∞. The optimal retention dependence and logarithmic factors remain open.
Related
- Sharp Sobolev Sandwich and Approximation Rates of Radon-Domain Lp Ridge Integral Spaces for ReLUk Networks
- Fully Deterministic Shallow Neural Networks with Localized Activations. Part I: Construction and Optimal Sobolev Approximation
- Deep ReLU Networks Can Fail the Best Approximation Property for Continuous Target Functions
- Beyond Constant Truncations: The Bounded Sobolev Approximation Gap
- Harmonic analysis and circuit approximation of Sobolev functions on quantum input spaces
BibTeX
@misc{yao2026approximation,
title = {{Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds}},
author = {Jia-He Yao},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.02253},
eprint = {2610.02253},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.02253},
}
RIS
TY - UNPB
TI - Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds
AU - Jia-He Yao
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2610.02253
UR - https://arxiv.org/abs/2610.02253
AB - The universal approximation property of dropout neural networks does not by itself describe the network size required for an accurate random realization. In this work, we study approximation of the unit ball of W^n,∞([0,1]^d) by ReLU networks whose edges are retained independently with probability p. The approximation error is measured uniformly over the input domain, and the guarantee holds with probability at least 1−δ for a single sampled network. We construct networks of constant depth and size Õ_n,d(p^−9ε^−max{d/n,2} log(1/δ)). The construction combines bounded local subnetworks, localization on a successful approximation event, and a multiscale Taylor decomposition. Conversely, Sobolev capacity imposes a lower bound on the number of surviving edges, while approximation of a fixed affine function requires an output-layer cost of order ((1−p)/p)ε^−2log(1/δ) at sufficiently high confidence. For fixed p∈(0,1) and δ<min{1/2,1−p}, the upper and lower bounds match in the accuracy exponent under a fixed or logarithmic depth budget. When d≤2n, they also match in confidence up to logarithms of accuracy. We extend the lower bounds to W^n,r targets with L^s error, and distinguish this extension from the upper bound for W^n,∞. The optimal retention dependence and logarithmic factors remain open.
ER -