Skip to content
DraconiticResearch
Search Sources API
203,203 studies31 sourcesUpdated 18 min ago
PreprintarXiv licencearXiv

Depth Separation for Neural Networks under Gaussian Measure

  • Cynthia Bortolotto
  • Francisco Ganacim
  • Roberto I. Oliveira
Open at sourcePDFDOICite

Abstract

We revisit the problem of depth separation for neural networks, as studied by Eldan and Shamir, Daniely, and others. Specifically, we consider the natural setting of L2 approximation of radial functions under the standard Gaussian measure in d dimensions. We show that there is a radial function in L2 under the standard Gaussian measure that can be approximated to a fixed sufficiently small L2 error by a depth-3 ReLU neural network of polynomial size in d but can only be approximated by depth-2 networks with subexponentially bounded weights if their size is at least exponential in d. For that, we prove a general result showing that if a radial function has substantial mass in the high frequencies of its Hermite expansion, then it cannot be well approximated by a depth-2 neural network with small width and bounded weights. All theorems, lemmas, propositions, and corollaries are verified in Lean~4.

Related

  1. Mixed-Frequency Time Series Forecasting via Depth-Separable Neural Networks
    Yize Wang, Qianqian Zhu, Guodong Li
    arXivPreprintarXiv licenceFull textStatisticsarXiv
  2. Radon Measure Representations for Infinite-Width Neural Networks with Singular Activations
    Mathias Dus
    HALPreprintCC BY-NC-ND 4.0Full textAnalysis & PDEsHALarXiv
  3. Para-Gaussian Approximations of Random Deep Neural Networks
    Mihai Nica, Janosch Ortmann
    arXivPreprintarXiv licenceFull textProbabilityarXiv
  4. Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence
    Luis Wiederhold, Nil Ayday, Debarghya Ghoshdastidar
    arXivPreprintCC BY 4.0Full textStatisticsarXiv
  5. Constructive Universal Approximation by Infinite-Dimensional Neural Networks via Gaussian-Weighted Ridgelet Transform
    Sho Sonoda, Luca Galimberti, Yuanchao Xu +2
    arXivPreprintCC BY 4.0Full textAnalysis & PDEsStatisticsarXiv

Cite

BibTeXRIS
BibTeX
@misc{bortolotto2026depth,
  title = {{Depth Separation for Neural Networks under Gaussian Measure}},
  author = {Cynthia Bortolotto and Francisco Ganacim and Roberto I. Oliveira},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.28800},
  eprint = {2610.28800},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.28800},
}
RIS
TY  - UNPB
TI  - Depth Separation for Neural Networks under Gaussian Measure
AU  - Cynthia Bortolotto
AU  - Francisco Ganacim
AU  - Roberto I. Oliveira
PY  - 2026
DA  - 2026/10/20
PB  - arXiv
DO  - 10.48550/arxiv.2610.28800
UR  - https://arxiv.org/abs/2610.28800
AB  - We revisit the problem of depth separation for neural networks, as studied by Eldan and Shamir, Daniely, and others. Specifically, we consider the natural setting of L^2 approximation of radial functions under the standard Gaussian measure in d dimensions. We show that there is a radial function in L^2 under the standard Gaussian measure that can be approximated to a fixed sufficiently small L^2 error by a depth-3 ReLU neural network of polynomial size in d but can only be approximated by depth-2 networks with subexponentially bounded weights if their size is at least exponential in d. For that, we prove a general result showing that if a radial function has substantial mass in the high frequencies of its Hermite expansion, then it cannot be well approximated by a depth-2 neural network with small width and bounded weights. All theorems, lemmas, propositions, and corollaries are verified in Lean~4.
ER  -