A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network…

Abstract

Several classical machine-learning methods, such as KRRs and SVRs, are both computationally and analytically tractable since their estimators either admit closed-form expressions or are obtained by minimizing convex training objectives; neither feature is generally available for deep neural networks. We address this by introducing a simple closed-form ``two-stage'' compositional formula f̂ for reconstructing an unknown Lipschitz function f:𝒳→ ℝ on a metric space (𝒳,ρ) from N i.i.d. noisy observations. Our main result is a high-probability uniform (L^∞) recovery guarantee that jointly controls approximation and statistical errors while enjoying an optimization error of zero; in particular, we do not assume oracle access to an approximate ERM. Our secondary main results establish the optimality of our formula in three complementary senses. 1) Function space: On Ahlfors-regular metric spaces, the hypothesis class parameterized by our formula attains the optimal fat-shattering dimension. 2) Parameter space: Its dependence on the parameters is maximally numerically stable, in the sense that a smaller approximation error cannot be achieved with a smaller Lipschitz dependence on the model parameters. 3) Forward pass: Its dependence on the input is maximally regular, matching the Lipschitz constant of the target function f. When 𝒳=[0,1]^d is equipped with the ℓ^∞ norm, f̂ admits algorithmic ReLU-MLP and exact ReLU-multi-head transformer realizations of depth 𝒪(log(N)) with 𝒪(N) nonzero parameters.

Risk Equivalence between RKHS Regression and Sequence Models for Lipschitz Spectral Algorithms
Yicheng Li, Yuqian Cheng, Zhuo Chen +1
arXiv8 Sep 2026PreprintCC BY-NC-SA 4.0Full textStatisticsarXiv
PINN-LPCDD: Physics-Informed Neural Network with Learnable and Physically Consistent Domain Decomposition for Compressible Multi-Medium Flows
Weiheng Zeng, Hao Wang, Jiaxin Zhu +3
arXiv16 Oct 2026PreprintarXiv licenceFull textNumerical analysis & scientific computing Computational physicsarXiv
Substochastic Path Operators for Refinement-Consistent Network Risks
Luca Grilli, Domenico Santoro, Giacinto Angelo Sgarro
Axioms16 Oct 2026Peer-reviewedCC BY 4.0General mathematicsCrossref
Linear regression with probabilistic networks for network-linked data
Jingnan Zhang, Shiwen Ye, Danyang Huang
arXiv13 Oct 2026PreprintCC BY 4.0Full textStatisticsarXiv
Popularity Regression-Based Latent Space Model for Large-Scale Sparse Network Analysis
Zhan Gao, Danyang Huang, Rui Pan +1
arXiv29 Sep 2026PreprintarXiv licenceFull textStatisticsarXiv

Cite

BibTeX RIS

BibTeX

@misc{hong2026closedform,
  title = {{A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations}},
  author = {Ruiyang Hong and Hrad Ghoukasian and Anastasis Kratsios},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2609.03129},
  eprint = {2609.03129},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.03129},
}

RIS

TY  - UNPB
TI  - A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations
AU  - Ruiyang Hong
AU  - Hrad Ghoukasian
AU  - Anastasis Kratsios
PY  - 2026
DA  - 2026/09/02
PB  - arXiv
DO  - 10.48550/arxiv.2609.03129
UR  - https://arxiv.org/abs/2609.03129
AB  - Several classical machine-learning methods, such as KRRs and SVRs, are both computationally and analytically tractable since their estimators either admit closed-form expressions or are obtained by minimizing convex training objectives; neither feature is generally available for deep neural networks. We address this by introducing a simple closed-form ``two-stage'' compositional formula f̂ for reconstructing an unknown Lipschitz function f:𝒳→ ℝ on a metric space (𝒳,ρ) from N i.i.d. noisy observations. Our main result is a high-probability uniform (L^∞) recovery guarantee that jointly controls approximation and statistical errors while enjoying an optimization error of zero; in particular, we do not assume oracle access to an approximate ERM. Our secondary main results establish the optimality of our formula in three complementary senses. 1) Function space: On Ahlfors-regular metric spaces, the hypothesis class parameterized by our formula attains the optimal fat-shattering dimension. 2) Parameter space: Its dependence on the parameters is maximally numerically stable, in the sense that a smaller approximation error cannot be achieved with a smaller Lipschitz dependence on the model parameters. 3) Forward pass: Its dependence on the input is maximally regular, matching the Lipschitz constant of the target function f. When 𝒳=[0,1]^d is equipped with the ℓ^∞ norm, f̂ admits algorithmic ReLU-MLP and exact ReLU-multi-head transformer realizations of depth 𝒪(log(N)) with 𝒪(N) nonzero parameters.
ER  -

Abstract

Related

Cite