Approximation Families and the Statistical Efficiency of Denoising Score Matching
Abstract
We study score matching over finite-dimensional approximation spaces generated by gradients of eigenfunctions of the infinitesimal generator of a forward diffusion. We compare polynomial approximation under one-dimensional Ornstein-Uhlenbeck flow with trigonometric-polynomial approximation under heat flow on the torus. In the Ornstein-Uhlenbeck setting, we show that for every non-Gaussian initial distribution uniformly comparable to the standard Gaussian, the asymptotic score-estimation variance grows exponentially along a subsequence of polynomial degrees. Averaging the score-matching loss under the diffusion semigroup, which we call operator-informed score matching (OISM), reduces the variance at each fixed degree but does not eliminate this exponential instability at sufficiently small times. A local minimax lower bound shows that the same high-degree dependence arises when estimating the polynomial projection of the score, rather than being an artifact of OISM. In contrast, under heat flow on the torus, OISM over trigonometric polynomials admits finite-sample control of the score error along the forward diffusion and yields the classical minimax rate for estimating sufficiently smooth densities under expected Kullback-Leibler loss.
Related
- A statistical theory for blind denoising: minimax estimation of the noise level from a single sample
- Improved Subexponential Upper Bounds for $3$-Restricted Matching Vector Families
- Numerical approximation of nonlinear fractional dynamical systems for coastal wave modeling: rigorous analysis of convergence and computational efficiency
- A geometrically linear approximation of min-max optimal matching
- Probabilistic Geodesic Flow Matching on Location-Scale Families
BibTeX
@misc{miyatake2026approximation,
title = {{Approximation Families and the Statistical Efficiency of Denoising Score Matching}},
author = {Brian Miyatake},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.36205},
eprint = {2610.36205},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.36205},
}
RIS
TY - UNPB
TI - Approximation Families and the Statistical Efficiency of Denoising Score Matching
AU - Brian Miyatake
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.36205
UR - https://arxiv.org/abs/2610.36205
AB - We study score matching over finite-dimensional approximation spaces generated by gradients of eigenfunctions of the infinitesimal generator of a forward diffusion. We compare polynomial approximation under one-dimensional Ornstein-Uhlenbeck flow with trigonometric-polynomial approximation under heat flow on the torus. In the Ornstein-Uhlenbeck setting, we show that for every non-Gaussian initial distribution uniformly comparable to the standard Gaussian, the asymptotic score-estimation variance grows exponentially along a subsequence of polynomial degrees. Averaging the score-matching loss under the diffusion semigroup, which we call operator-informed score matching (OISM), reduces the variance at each fixed degree but does not eliminate this exponential instability at sufficiently small times. A local minimax lower bound shows that the same high-degree dependence arises when estimating the polynomial projection of the score, rather than being an artifact of OISM. In contrast, under heat flow on the torus, OISM over trigonometric polynomials admits finite-sample control of the score error along the forward diffusion and yields the classical minimax rate for estimating sufficiently smooth densities under expected Kullback-Leibler loss.
ER -