Diffusion Score Estimation with Shallow Neural Networks: Regularity, Approximation, and Finite-Sample Rates
Abstract
We study the sample complexity of estimating Ornstein--Uhlenbeck diffusion scores in ℝd using denoising score matching (DSM) with finite-width shallow ReLU networks, and its implications for reverse-process sampling. Under weak differentiability and polynomial-growth conditions on normalized derivatives of the initial density, we combine complementary estimates from the Gaussian transition kernel and the initial density to obtain explicit Sobolev bounds that are uniform in time on each fixed ball. Combining these bounds with existing shallow-network approximation theory gives explicit control of the approximation error and a sufficient path-norm budget. We then use truncation and localized Rademacher complexity to establish high-probability risk bounds for path-norm-constrained empirical DSM minimizers with unbounded inputs and labels. Under additional bounded-density and sub-Gaussian assumptions, suitable choices of network width and path-norm budget yield squared L2(Pt) score risk Õ(n−(d+3)/(2d+3)) at each fixed time t>0, where n is the number of independent training samples and Pt is the forward marginal. The nonasymptotic bounds retain explicit dependence on diffusion time and the regularity and tail parameters of the initial distribution. Aggregating the score errors over a time grid and applying existing sampling theory together with an early-stopping bound yields a high-probability total-variation error Õ(n−(d+3)/(4d+6)) between the generated and original data distributions under suitable sampling parameters. Here Õ suppresses logarithmic factors for fixed dimension, confidence level, and initial distribution.
Related
- Fully Deterministic Shallow Neural Networks with Localized Activations. Part I: Construction and Optimal Sobolev Approximation
- Sharp Approximation on the Sphere by Linearized Shallow Networks with Analytic Activations
- Shallow neural network approximation in mixed Sobolev spaces
- Boundary Geometry Determines Optimal Shallow ReLU Approximation under Weighted Coefficient Constraints
- Low regularity maps on Lipschitz manifolds: lifting, pullback of forms, and approximation
BibTeX
@misc{liu2026diffusion,
title = {{Diffusion Score Estimation with Shallow Neural Networks: Regularity, Approximation, and Finite-Sample Rates}},
author = {Fan Liu and Yahong Yang and Wei Zhu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.27206},
eprint = {2610.27206},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.27206},
}
RIS
TY - UNPB
TI - Diffusion Score Estimation with Shallow Neural Networks: Regularity, Approximation, and Finite-Sample Rates
AU - Fan Liu
AU - Yahong Yang
AU - Wei Zhu
PY - 2026
DA - 2026/10/19
PB - arXiv
DO - 10.48550/arxiv.2610.27206
UR - https://arxiv.org/abs/2610.27206
AB - We study the sample complexity of estimating Ornstein--Uhlenbeck diffusion scores in ℝ^d using denoising score matching (DSM) with finite-width shallow ReLU networks, and its implications for reverse-process sampling. Under weak differentiability and polynomial-growth conditions on normalized derivatives of the initial density, we combine complementary estimates from the Gaussian transition kernel and the initial density to obtain explicit Sobolev bounds that are uniform in time on each fixed ball. Combining these bounds with existing shallow-network approximation theory gives explicit control of the approximation error and a sufficient path-norm budget. We then use truncation and localized Rademacher complexity to establish high-probability risk bounds for path-norm-constrained empirical DSM minimizers with unbounded inputs and labels. Under additional bounded-density and sub-Gaussian assumptions, suitable choices of network width and path-norm budget yield squared L^2(P_t) score risk Õ(n^−(d+3)/(2d+3)) at each fixed time t>0, where n is the number of independent training samples and P_t is the forward marginal. The nonasymptotic bounds retain explicit dependence on diffusion time and the regularity and tail parameters of the initial distribution. Aggregating the score errors over a time grid and applying existing sampling theory together with an early-stopping bound yields a high-probability total-variation error Õ(n^−(d+3)/(4d+6)) between the generated and original data distributions under suitable sampling parameters. Here Õ suppresses logarithmic factors for fixed dimension, confidence level, and initial distribution.
ER -