Is Spectral Self-Supervised Learning a Minimax Optimal Method of Density Ratio Estimation?
Abstract
Spectral self-supervised learning is a method of density ratio estimation for the setting in which only samples drawn from the joint density of the numerator are available. The method has extensively been studied in terms of the computational properties, such as optimal solutions and connections to representation learning. However, theoretical understanding of its statistical property is still insufficient. In this work, we study three questions, namely, the minimax optimality, the applicability of the theory to regularized algorithms, and the statistical merits of density ratio estimation with data augmentation mechanisms that have recently been considered in the context of self-supervised representation learning. We first show that for general classes of Hölder continuous density ratios, there are regularized spectral self-supervised learning algorithms that yield estimators attaining the minimax optimality, up to logarithmic factors. We then prove that for subclasses endowed with a data augmentation mechanism, we obtain faster convergence rates that are also minimax optimal (up to logarithmic factors) in a weaker sense. We also present the key proof ideas tailored for self-supervised density ratio estimation.
Related
- Minimax estimation of the pseudo-spectral gap of continuous Markov chains, with applications to empirical concentration bounds
- Bias-Corrected Subspace Intersection: Minimax-Optimal Shared Subspace Estimation in Multi-View Data
- Minimax rates for learning spectral Barron functions by deep ReLU neural networks
- Minimax-optimal Scaling Laws in Ridge Regression via Transfer Learning
- Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers
BibTeX
@misc{waida2026spectral,
title = {{Is Spectral Self-Supervised Learning a Minimax Optimal Method of Density Ratio Estimation?}},
author = {Hiroki Waida and Takafumi Kanamori},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.31603},
eprint = {2610.31603},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.31603},
}
RIS
TY - UNPB
TI - Is Spectral Self-Supervised Learning a Minimax Optimal Method of Density Ratio Estimation?
AU - Hiroki Waida
AU - Takafumi Kanamori
PY - 2026
DA - 2026/10/22
PB - arXiv
DO - 10.48550/arxiv.2610.31603
UR - https://arxiv.org/abs/2610.31603
AB - Spectral self-supervised learning is a method of density ratio estimation for the setting in which only samples drawn from the joint density of the numerator are available. The method has extensively been studied in terms of the computational properties, such as optimal solutions and connections to representation learning. However, theoretical understanding of its statistical property is still insufficient. In this work, we study three questions, namely, the minimax optimality, the applicability of the theory to regularized algorithms, and the statistical merits of density ratio estimation with data augmentation mechanisms that have recently been considered in the context of self-supervised representation learning. We first show that for general classes of Hölder continuous density ratios, there are regularized spectral self-supervised learning algorithms that yield estimators attaining the minimax optimality, up to logarithmic factors. We then prove that for subclasses endowed with a data augmentation mechanism, we obtain faster convergence rates that are also minimax optimal (up to logarithmic factors) in a weaker sense. We also present the key proof ideas tailored for self-supervised density ratio estimation.
ER -