Transfer Learning with Heterogeneous Feature Spaces in Linear Regression
Abstract
Transfer learning improves target-task performance by leveraging related source data. Most methods assume shared feature spaces, yet in many applications, each source observes only a subset of target covariates. Classical imputation fails here due to block missingness, and standard imputation matrices are not optimized for target parameter estimation. We study low- and high-dimensional linear regression and propose Heterogeneous Importance Weighting (HIW). Our method aligns feature spaces via projection-based imputation and transfers information through sample-selected importance weighting. This framework accommodates diverse projection matrices to construct target-oriented imputation. We develop a classification-based procedure with pseudo-responses to estimate conditional error densities for the weights. We establish entry-wise and global convergence rates for the estimator, with numerical and real-data studies demonstrating its effectiveness.
Related
- Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devices
- Beyond Source-Level Transfer: Sample-Level Learning for High-Dimensional Quantile Regression
- Augmented transfer regression learning for completely missing covariates
- Minimax-optimal Scaling Laws in Ridge Regression via Transfer Learning
- Joint-Sparse Transfer Learning for High-Dimensional Multi-Output Regression
BibTeX
@misc{zheng2026transfer,
title = {{Transfer Learning with Heterogeneous Feature Spaces in Linear Regression}},
author = {Zejing Zheng and Rui Huang and Junlong Zhao},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.08526},
eprint = {2609.08526},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.08526},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Transfer Learning with Heterogeneous Feature Spaces in Linear Regression
AU - Zejing Zheng
AU - Rui Huang
AU - Junlong Zhao
PY - 2026
DA - 2026/09/08
PB - arXiv
DO - 10.48550/arxiv.2609.08526
UR - https://arxiv.org/abs/2609.08526
AB - Transfer learning improves target-task performance by leveraging related source data. Most methods assume shared feature spaces, yet in many applications, each source observes only a subset of target covariates. Classical imputation fails here due to block missingness, and standard imputation matrices are not optimized for target parameter estimation. We study low- and high-dimensional linear regression and propose Heterogeneous Importance Weighting (HIW). Our method aligns feature spaces via projection-based imputation and transfers information through sample-selected importance weighting. This framework accommodates diverse projection matrices to construct target-oriented imputation. We develop a classification-based procedure with pseudo-responses to estimate conditional error densities for the weights. We establish entry-wise and global convergence rates for the estimator, with numerical and real-data studies demonstrating its effectiveness.
ER -