Learning between the peaks: sharp asymptotics for kernel ridge regression under power-law anisotropy
Abstract
We study kernel ridge regression under anisotropic Gaussian data, where the input covariance decays as a power law with exponent α≥ 0 for polynomial inner-product kernels. We derive asymptotically sharp expressions for the kernel spectrum and the generalization error in the polynomial high-dimensional regime n=Θ(dκ), revealing how anisotropy reshapes the learning curves. For weak anisotropy (0<α<1), the problem remains effectively high-dimensional and retains some features of the isotropic case, while departing from it in others: the variance still peaks at integer sample complexities κ∈ℕ, but these peaks are progressively damped as α grows; meanwhile, for targets strongly aligned with the data's principal directions, the bias drops at fractional sample complexities, decoupling the bias transitions from the interpolation peaks. For strong anisotropy (α> 1), the effective dimension of the problem is constant, and the variance stops depending on sample size altogether, plateauing under ridgeless interpolation or vanishing at an explicit rate under fixed ridge penalty. The bias undergoes a sharp transition governed by the target's decay rate: below a threshold, learning is abrupt rather than gradual; above it, the bias decays as a power law that recovers the classical source and capacity rates. We finally specialize these results to single-index targets, showing how the alignment of the index with the data's principal directions determines the effect of anisotropy on learning. Together, our results clarify how the input geometry shapes the kernel features and fundamentally impacts its generalization properties.
Related
- Koopman operator learning for predictive control via Khatri-Rao kernel regression
- Learning Closure of Dynamical Systems with Kernel Ridge Regression
- Nonparametric In-Context Learning and Inference via Measure-Indexed Kernel Ridge Regression
- Gaussian heat kernel asymptotics for conditioned random walks
- Bergman Kernel Asymptotics for Semipositive Line Bundles near Curvature-Degenerate Points
BibTeX
@misc{rizzi2026learning,
title = {{Learning between the peaks: sharp asymptotics for kernel ridge regression under power-law anisotropy}},
author = {Lorenzo Rizzi and Arie Wortsman Zurich and Bruno Loureiro},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.28564},
eprint = {2608.28564},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.28564},
}
RIS
TY - UNPB
TI - Learning between the peaks: sharp asymptotics for kernel ridge regression under power-law anisotropy
AU - Lorenzo Rizzi
AU - Arie Wortsman Zurich
AU - Bruno Loureiro
PY - 2026
DA - 2026/08/28
PB - arXiv
DO - 10.48550/arxiv.2608.28564
UR - https://arxiv.org/abs/2608.28564
AB - We study kernel ridge regression under anisotropic Gaussian data, where the input covariance decays as a power law with exponent α≥ 0 for polynomial inner-product kernels. We derive asymptotically sharp expressions for the kernel spectrum and the generalization error in the polynomial high-dimensional regime n=Θ(d^κ), revealing how anisotropy reshapes the learning curves. For weak anisotropy (0<α<1), the problem remains effectively high-dimensional and retains some features of the isotropic case, while departing from it in others: the variance still peaks at integer sample complexities κ∈ℕ, but these peaks are progressively damped as α grows; meanwhile, for targets strongly aligned with the data's principal directions, the bias drops at fractional sample complexities, decoupling the bias transitions from the interpolation peaks. For strong anisotropy (α> 1), the effective dimension of the problem is constant, and the variance stops depending on sample size altogether, plateauing under ridgeless interpolation or vanishing at an explicit rate under fixed ridge penalty. The bias undergoes a sharp transition governed by the target's decay rate: below a threshold, learning is abrupt rather than gradual; above it, the bias decays as a power law that recovers the classical source and capacity rates. We finally specialize these results to single-index targets, showing how the alignment of the index with the data's principal directions determines the effect of anisotropy on learning. Together, our results clarify how the input geometry shapes the kernel features and fundamentally impacts its generalization properties.
ER -