Risk Equivalence between RKHS Regression and Sequence Models for Lipschitz Spectral Algorithms
Abstract
Kernel spectral algorithms are often summarized by convergence rates, which hide how their risk depends jointly on regularization, noise, the population spectrum, target coefficients, and the chosen filter. Gaussian sequence models arise as a simplified but characteristic setting for studying the interplay of these factors, where the kernel spectral algorithm corresponds to a coordinatewise shrinkage estimator. Under mild assumptions, we show that the risk of a kernel spectral estimator is asymptotically equivalent to that of the corresponding Gaussian sequence model estimator with the same filter. The explicit sequence model risk then yields a full characterization of the risk of kernel spectral algorithms in terms of the population spectrum, target coefficients, and filter. We establish this equivalence for a broad class of spectral filters, covering kernel ridge regression, generalized ridge regression, iterated kernel ridge regression, gradient flow, stable gradient descent, smoothed spectral cutoff, spectral clipping, and Pinsker shrinkage. Our risk equivalence not only holds in the classical fixed-dimensional regime but also applies to the high dimensional regime where the input dimension scales with the sample size. As applications, our risk equivalence recovers the minimax upper rates, and establishes the exact Pinsker constant in RKHS regression.
Related
- A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations
- On the Spectral Synthesis of Lipschitz Vectorizations of Persistence Diagrams
- On the bias distribution of pseudo-linear regression algorithms for recursive identification
- Trace-Class Results for MCMC Algorithms for Student-t Regression Models
- Benchmarking Goodness-of-Fit and Calibration Algorithms for Logistic Regression Classifiers: A Large-Scale Simulation Study under Sparse Data
BibTeX
@misc{li2026risk,
title = {{Risk Equivalence between RKHS Regression and Sequence Models for Lipschitz Spectral Algorithms}},
author = {Yicheng Li and Yuqian Cheng and Zhuo Chen and Qian Lin},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.08817},
eprint = {2609.08817},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.08817},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Risk Equivalence between RKHS Regression and Sequence Models for Lipschitz Spectral Algorithms
AU - Yicheng Li
AU - Yuqian Cheng
AU - Zhuo Chen
AU - Qian Lin
PY - 2026
DA - 2026/09/08
PB - arXiv
DO - 10.48550/arxiv.2609.08817
UR - https://arxiv.org/abs/2609.08817
AB - Kernel spectral algorithms are often summarized by convergence rates, which hide how their risk depends jointly on regularization, noise, the population spectrum, target coefficients, and the chosen filter. Gaussian sequence models arise as a simplified but characteristic setting for studying the interplay of these factors, where the kernel spectral algorithm corresponds to a coordinatewise shrinkage estimator. Under mild assumptions, we show that the risk of a kernel spectral estimator is asymptotically equivalent to that of the corresponding Gaussian sequence model estimator with the same filter. The explicit sequence model risk then yields a full characterization of the risk of kernel spectral algorithms in terms of the population spectrum, target coefficients, and filter. We establish this equivalence for a broad class of spectral filters, covering kernel ridge regression, generalized ridge regression, iterated kernel ridge regression, gradient flow, stable gradient descent, smoothed spectral cutoff, spectral clipping, and Pinsker shrinkage. Our risk equivalence not only holds in the classical fixed-dimensional regime but also applies to the high dimensional regime where the input dimension scales with the sample size. As applications, our risk equivalence recovers the minimax upper rates, and establishes the exact Pinsker constant in RKHS regression.
ER -