Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression
Abstract
We study density ratio estimation and importance-weighted regression under target shift with continuous outputs. Under target shift, the conditional distribution of the inputs given the outputs remains invariant across the training and test distributions, while the output marginal distribution may change. Although this problem has been extensively studied for discrete outputs, the continuous setting is substantially less understood: the importance weights are determined by an unknown density ratio function, for which existing estimation methods lack explicit finite-sample convergence rates. We propose a spectral regularization method in a reproducing kernel Hilbert space (RKHS) for estimating the continuous density ratio from labeled training samples and unlabeled test inputs. Under a source condition with regularity parameter ι>0, we establish high-probability finite-sample guarantees and show that the estimator achieves the capacity-independent minimax-optimal RKHS-norm rate O(nη−ι/(2ι+2)). We then incorporate the estimated density ratio into importance-weighted regression and characterize the propagation of density-ratio estimation error to the final predictor. When sufficiently many samples are available for density ratio estimation, the resulting regression estimator attains the minimax-optimal rates of standard kernel regression. These results establish a finite-sample theory for continuous density ratio estimation and importance-weighted learning under target shift.
Related
- Spatio-Temporal Weighted Regression model with fractional-colored noise: Parameter estimation and consistency
- Optimal estimation for Functional Linear Regression with Noisy Discretized Data
- Optimal Neural Estimation for High-Dimensional Nonparametric Regression under Multisource Covariate Shift
- Deep Barycentric Regression for Optimal Transport Map Estimation and its Statistical Optimality
- Minimax-optimal Scaling Laws in Ridge Regression via Transfer Learning
BibTeX
@misc{liu2026learning,
title = {{Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression}},
author = {Ren-Rui Liu and Zheng-Chu Guo},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.15785},
eprint = {2609.15785},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.15785},
}
RIS
TY - UNPB
TI - Learning under Target Shift: Optimal Density Ratio Estimation and Importance-Weighted Regression
AU - Ren-Rui Liu
AU - Zheng-Chu Guo
PY - 2026
DA - 2026/09/14
PB - arXiv
DO - 10.48550/arxiv.2609.15785
UR - https://arxiv.org/abs/2609.15785
AB - We study density ratio estimation and importance-weighted regression under target shift with continuous outputs. Under target shift, the conditional distribution of the inputs given the outputs remains invariant across the training and test distributions, while the output marginal distribution may change. Although this problem has been extensively studied for discrete outputs, the continuous setting is substantially less understood: the importance weights are determined by an unknown density ratio function, for which existing estimation methods lack explicit finite-sample convergence rates. We propose a spectral regularization method in a reproducing kernel Hilbert space (RKHS) for estimating the continuous density ratio from labeled training samples and unlabeled test inputs. Under a source condition with regularity parameter ι>0, we establish high-probability finite-sample guarantees and show that the estimator achieves the capacity-independent minimax-optimal RKHS-norm rate O(n_η^−ι/(2ι+2)). We then incorporate the estimated density ratio into importance-weighted regression and characterize the propagation of density-ratio estimation error to the final predictor. When sufficiently many samples are available for density ratio estimation, the resulting regression estimator attains the minimax-optimal rates of standard kernel regression. These results establish a finite-sample theory for continuous density ratio estimation and importance-weighted learning under target shift.
ER -