Generalization Bounds via Vector-Valued Lp Correlation Vα
Abstract
Understanding generalization of randomized learning algorithms remains a central challenge. Mutual-information and total-variation bounds provide complementary ways to quantify data--hypothesis dependence, but they can behave quite differently in high-dimensional regimes. We study a one-parameter family of dependence measures, the vector-valued Lp-norm correlation Vα, and derive generalization bounds whose dependence terms interpolate between total variation (TV) at α=1 and variance and χ2 related behavior at α=2. For α>2, the Hölder-conjugate loss moment is below two, yielding heavy-tailed specializations that can remain finite even when the loss variance is infinite. A particularly interpretable specialization occurs at α=2, where the bound can be expressed in terms of the variability of the algorithm's output density across datasets, ∫ √(VarS[p(w| S)]) dw. We show that the entire Vα family is invariant under smooth bijective reparameterizations of the hypothesis space. We also introduce an adaptive density stability (ADS) condition that implies explicit control of V2, and we verify it for Gibbs-type posteriors under bounded losses (yielding rates that depend on the inverse temperature and sample size). To address scaling, we extend the framework to a conditional formulation using supersamples, analogous in spirit to conditional mutual information (CMI), and discuss evaluations in the output space as a practical surrogate for estimating dependence. Simulations on Bayesian linear regression illustrate how tuning α can change the tightness of the resulting bounds across regimes. Additional supersample experiments on noisy MNIST and on a pretrained ResNet-50 for CIFAR-10 show similar behavior of the projected loss-space bounds beyond the original clean-MNIST setting.
Related
- Robust Error Bounds for Vector-Valued Kernel Ridge Regression
- Time-uniform self-normalized concentration for vector-valued processes
- Concentration inequalities for vector-valued martingales
- Convergence of a vector-valued Allen-Cahn system to Brakke's multiphase mean curvature flow
- Vector Valued Katok-Sarnak Formulas
BibTeX
@misc{hamaneh2026generalization,
title = {{Generalization Bounds via Vector-Valued $L_p$ Correlation $V_α$}},
author = {Mohammad Hossein Momeni Hamaneh and Mohammad Mahdi Mojahedian and Mohammad Hossein Yassaee},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.29425},
eprint = {2610.29425},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.29425},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Generalization Bounds via Vector-Valued L_p Correlation V_α
AU - Mohammad Hossein Momeni Hamaneh
AU - Mohammad Mahdi Mojahedian
AU - Mohammad Hossein Yassaee
PY - 2026
DA - 2026/10/21
PB - arXiv
DO - 10.48550/arxiv.2610.29425
UR - https://arxiv.org/abs/2610.29425
AB - Understanding generalization of randomized learning algorithms remains a central challenge. Mutual-information and total-variation bounds provide complementary ways to quantify data--hypothesis dependence, but they can behave quite differently in high-dimensional regimes. We study a one-parameter family of dependence measures, the vector-valued L_p-norm correlation V_α, and derive generalization bounds whose dependence terms interpolate between total variation (TV) at α=1 and variance and χ^2 related behavior at α=2. For α>2, the Hölder-conjugate loss moment is below two, yielding heavy-tailed specializations that can remain finite even when the loss variance is infinite. A particularly interpretable specialization occurs at α=2, where the bound can be expressed in terms of the variability of the algorithm's output density across datasets, ∫ √(Var_S[p(w| S)]) dw. We show that the entire V_α family is invariant under smooth bijective reparameterizations of the hypothesis space. We also introduce an adaptive density stability (ADS) condition that implies explicit control of V_2, and we verify it for Gibbs-type posteriors under bounded losses (yielding rates that depend on the inverse temperature and sample size). To address scaling, we extend the framework to a conditional formulation using supersamples, analogous in spirit to conditional mutual information (CMI), and discuss evaluations in the output space as a practical surrogate for estimating dependence. Simulations on Bayesian linear regression illustrate how tuning α can change the tightness of the resulting bounds across regimes. Additional supersample experiments on noisy MNIST and on a pretrained ResNet-50 for CIFAR-10 show similar behavior of the projected loss-space bounds beyond the original clean-MNIST setting.
ER -