Joint-Sparse Transfer Learning for High-Dimensional Multi-Output Regression
Abstract
Multitask linear models can improve estimation and prediction by exploiting structure shared across responses, bridging taskwise fitting and complete pooling. In many applications, however, the objective is estimation in a data-limited target domain, while data-rich but heterogeneous source domains are available. Borrowing from these sources can improve efficiency but introduce bias. We develop a joint-sparse transfer-learning framework for high-dimensional multi-output regression that combines shared predictor structure across responses with source-target similarity. The framework yields two complementary estimators: a fused estimator that aggregates jointly fitted domain-specific coefficients and a target-based debiased estimator that adjusts for source-induced shifts. Our error bounds show how transfer increases the available information and sharing predictors across responses reduces selection costs. They also reveal a tradeoff: the fused estimator benefits from larger sources but may retain bias if source shifts point in similar directions, whereas debiasing trades this bias for additional estimation error governed by the smaller target sample. Comparison with a minimax lower bound identifies regimes where the bounds match up to logarithmic factors, where matching remains unresolved, and where projection onto a target-based convex set closes the gap. Simulations and an analysis of single-cell RNA and surface-protein profiles across cell types support the theory.
Related
- Beyond Source-Level Transfer: Sample-Level Learning for High-Dimensional Quantile Regression
- Univariate-guided sparse regression for Biobank-scale high-dimensional omics data
- Minimax and Adaptive Transfer Learning for Sparse Canonical Correlation Analysis
- Transfer Learning with Heterogeneous Feature Spaces in Linear Regression
- Augmented transfer regression learning for completely missing covariates
BibTeX
@misc{lim2026jointsparse,
title = {{Joint-Sparse Transfer Learning for High-Dimensional Multi-Output Regression}},
author = {Sunwoo Lim and Mladen Kolar},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.30879},
eprint = {2609.30879},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.30879},
}
RIS
TY - UNPB
TI - Joint-Sparse Transfer Learning for High-Dimensional Multi-Output Regression
AU - Sunwoo Lim
AU - Mladen Kolar
PY - 2026
DA - 2026/09/25
PB - arXiv
DO - 10.48550/arxiv.2609.30879
UR - https://arxiv.org/abs/2609.30879
AB - Multitask linear models can improve estimation and prediction by exploiting structure shared across responses, bridging taskwise fitting and complete pooling. In many applications, however, the objective is estimation in a data-limited target domain, while data-rich but heterogeneous source domains are available. Borrowing from these sources can improve efficiency but introduce bias. We develop a joint-sparse transfer-learning framework for high-dimensional multi-output regression that combines shared predictor structure across responses with source-target similarity. The framework yields two complementary estimators: a fused estimator that aggregates jointly fitted domain-specific coefficients and a target-based debiased estimator that adjusts for source-induced shifts. Our error bounds show how transfer increases the available information and sharing predictors across responses reduces selection costs. They also reveal a tradeoff: the fused estimator benefits from larger sources but may retain bias if source shifts point in similar directions, whereas debiasing trades this bias for additional estimation error governed by the smaller target sample. Comparison with a minimax lower bound identifies regimes where the bounds match up to logarithmic factors, where matching remains unresolved, and where projection onto a target-based convex set closes the gap. Simulations and an analysis of single-cell RNA and surface-protein profiles across cell types support the theory.
ER -