Optimal Neural Estimation for High-Dimensional Nonparametric Regression under Multisource Covariate Shift
Abstract
We study high-dimensional nonparametric regression under multisource covariate shift, where J sources of training data share a common regression function but differ in their covariate distributions, sample sizes, and noise variances. Our goal is to combine these sources to estimate the regression function accurately under a target covariate distribution. We propose an oracle weighted neural estimator and establish its minimax optimality up to logarithmic factors through matching upper and lower bounds. The resulting rate is neff−2p/(2p+K), where p and K denote the effective smoothness and dimension of the regression function, and the effective sample size is neff =∑j=1J nj/((1+σρ,j)σε,j2). nonumber Here, nj is the sample size of source j, σε,j2 is its noise variance, and σρ,j controls the deviation between the covariate distribution of source j and the target covariate distribution. We also develop an adaptive procedure that requires no knowledge of source noise variances or covariate-shift parameters. Under suitable conditions, this procedure attains the oracle minimax rate up to logarithmic factors.
Related
- Covariate-dependent Nonparametric g-modeling for regression via infinite Mixture-of-Expertizing class
- Nonparametric Intensity Estimation for Covariate-Driven Poisson Processes Using Deep Learning
- Scalable Nonparametric Change-Plane Regression with Random Features: Estimation and Inference
- Improved Variance Estimation in Homoskedastic Nonparametric Random-Design Regression via a Two-Scale Approach
- Nonparametric shrinkage estimation of high-dimensional variance component matrices
BibTeX
@misc{xu2026optimal,
title = {{Optimal Neural Estimation for High-Dimensional Nonparametric Regression under Multisource Covariate Shift}},
author = {Chenghao Xu and Haotian Xu and Shivam Kumar and Daren Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.40331},
eprint = {2610.40331},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.40331},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Optimal Neural Estimation for High-Dimensional Nonparametric Regression under Multisource Covariate Shift
AU - Chenghao Xu
AU - Haotian Xu
AU - Shivam Kumar
AU - Daren Wang
PY - 2026
DA - 2026/10/29
PB - arXiv
DO - 10.48550/arxiv.2610.40331
UR - https://arxiv.org/abs/2610.40331
AB - We study high-dimensional nonparametric regression under multisource covariate shift, where J sources of training data share a common regression function but differ in their covariate distributions, sample sizes, and noise variances. Our goal is to combine these sources to estimate the regression function accurately under a target covariate distribution. We propose an oracle weighted neural estimator and establish its minimax optimality up to logarithmic factors through matching upper and lower bounds. The resulting rate is n_eff^−2p/(2p+K), where p and K denote the effective smoothness and dimension of the regression function, and the effective sample size is n_eff =∑_j=1^J n_j/((1+σ_ρ,j)σ_ε,j^2). nonumber Here, n_j is the sample size of source j, σ_ε,j^2 is its noise variance, and σ_ρ,j controls the deviation between the covariate distribution of source j and the target covariate distribution. We also develop an adaptive procedure that requires no knowledge of source noise variances or covariate-shift parameters. Under suitable conditions, this procedure attains the oracle minimax rate up to logarithmic factors.
ER -