Gaussian Approximation of Besov Functions for Logistic Classification on Manifolds
Abstract
We study Gaussian kernel approximation and regularized logistic classification on a connected compact smooth embedded manifold X⊂ℝd without boundary and of intrinsic dimension ϱ. Two main challenges are to derive a fast oracle inequality without assuming a bounded Bayes function and to construct Gaussian approximants that exploit both the intrinsic dimension and the target's regularity without saturation. We address the first challenge through a Bernstein-type inequality whose constant depends linearly on the supremum norm of the predictor. Combined with localized complexity estimates, this yields an oracle inequality over separable normed function spaces. To address the second challenge, we establish a uniform Sobolev operator expansion and cancel its lower-order bias terms using a signed combination of Gaussian kernel integral operators at different bandwidths. Real interpolation then gives an L2(X) approximation error of order γα for functions in Bα2,∞(X), where γ is the bandwidth. By constructing the approximants directly on the manifold without ambient extension, we obtain a squared Gaussian RKHS norm bound of order γ−ϱ. This construction thus overcomes second-order saturation while retaining norm control governed by the intrinsic dimension. When the marginal distribution has a density g∈ Lp(X), these results yield the logistic excess-risk rate ((log n)ϱ+1/n)4α(1−1/p)/(4α(1−1/p)+3ϱ). We further show that clipping predictions yields the faster clipped logistic excess-risk rate ((log n)ϱ+2/n)2α(1−1/p)/(2α(1−1/p)+ϱ). For bounded densities, its polynomial exponent matches that of the minimax optimal rate for squared-loss regression, and its corresponding excess misclassification risk also matches the minimax optimal rate.
Related
- Continuous logistic Gaussian random measure fields for spatial distributional modelling
- Logistic Gaussian process density regression: a generalized Bayesian approach
- Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling
- Finite-Rank Logistic Gaussian Processes with Exact Likelihood for Conditional Density Estimation
- PU classification under Non-SCAR: clustering-assisted logistic model with oversampling enhancement
BibTeX
@misc{lei2026gaussian,
title = {{Gaussian Approximation of Besov Functions for Logistic Classification on Manifolds}},
author = {Guanhang Lei and Lei Shi and Xiaoming Yuan},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.27022},
eprint = {2610.27022},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.27022},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Gaussian Approximation of Besov Functions for Logistic Classification on Manifolds
AU - Guanhang Lei
AU - Lei Shi
AU - Xiaoming Yuan
PY - 2026
DA - 2026/10/19
PB - arXiv
DO - 10.48550/arxiv.2610.27022
UR - https://arxiv.org/abs/2610.27022
AB - We study Gaussian kernel approximation and regularized logistic classification on a connected compact smooth embedded manifold X⊂ℝ^d without boundary and of intrinsic dimension ϱ. Two main challenges are to derive a fast oracle inequality without assuming a bounded Bayes function and to construct Gaussian approximants that exploit both the intrinsic dimension and the target's regularity without saturation. We address the first challenge through a Bernstein-type inequality whose constant depends linearly on the supremum norm of the predictor. Combined with localized complexity estimates, this yields an oracle inequality over separable normed function spaces. To address the second challenge, we establish a uniform Sobolev operator expansion and cancel its lower-order bias terms using a signed combination of Gaussian kernel integral operators at different bandwidths. Real interpolation then gives an L_2(X) approximation error of order γ^α for functions in B^α_2,∞(X), where γ is the bandwidth. By constructing the approximants directly on the manifold without ambient extension, we obtain a squared Gaussian RKHS norm bound of order γ^−ϱ. This construction thus overcomes second-order saturation while retaining norm control governed by the intrinsic dimension. When the marginal distribution has a density g∈ L_p(X), these results yield the logistic excess-risk rate ((log n)^ϱ+1/n)^4α(1−1/p)/(4α(1−1/p)+3ϱ). We further show that clipping predictions yields the faster clipped logistic excess-risk rate ((log n)^ϱ+2/n)^2α(1−1/p)/(2α(1−1/p)+ϱ). For bounded densities, its polynomial exponent matches that of the minimax optimal rate for squared-loss regression, and its corresponding excess misclassification risk also matches the minimax optimal rate.
ER -