Structure-based Transfer Learning
Abstract
Transfer learning improves estimation in a target study using information from related sources. Classical transfer learning is typically data-based, requiring access to the source data or to a model fitted on them. Neither is available in many modern studies, as both are often proprietary or unreported. What can be transferred instead is structure: summarized information derived from a source, such as the supports of predictors, which is available and interpretable without access to the source data or model. The prevalence of the Large Language Model (LLM) provides a rich source of such structure information. However, data-based transfer learning can not well utilize it as accessing the data and model of the LLM is often infeasible. In this paper, we propose a novel transfer learning framework based on structure. In particular, we focus on linear model and aim to use supports generated by multiple agents, such as LLM, to improve estimation in the target study. Our procedure maps each support into the target parameter space, aggregates the results with data-driven weights, and applies a sparse correction against inaccurate supports. We establish nonasymptotic guarantees: the estimator improves on the target-only Lasso when the supports are informative and never does worse otherwise. Simulations and a gene-expression application confirm both properties.
Related
- Transfer learning in bayesian optimization for aircraft design
- Transfer Learning with Heterogeneous Feature Spaces in Linear Regression
- Direct Knowledge Transfer for Offline Q-Learning
- Beyond Source-Level Transfer: Sample-Level Learning for High-Dimensional Quantile Regression
- Augmented transfer regression learning for completely missing covariates
BibTeX
@misc{ho2026structurebased,
title = {{Structure-based Transfer Learning}},
author = {Ho Yi Alexis Ho and Xinzhou Guo and Shuoxun Xu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.08487},
eprint = {2609.08487},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.08487},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Structure-based Transfer Learning
AU - Ho Yi Alexis Ho
AU - Xinzhou Guo
AU - Shuoxun Xu
PY - 2026
DA - 2026/09/08
PB - arXiv
DO - 10.48550/arxiv.2609.08487
UR - https://arxiv.org/abs/2609.08487
AB - Transfer learning improves estimation in a target study using information from related sources. Classical transfer learning is typically data-based, requiring access to the source data or to a model fitted on them. Neither is available in many modern studies, as both are often proprietary or unreported. What can be transferred instead is structure: summarized information derived from a source, such as the supports of predictors, which is available and interpretable without access to the source data or model. The prevalence of the Large Language Model (LLM) provides a rich source of such structure information. However, data-based transfer learning can not well utilize it as accessing the data and model of the LLM is often infeasible. In this paper, we propose a novel transfer learning framework based on structure. In particular, we focus on linear model and aim to use supports generated by multiple agents, such as LLM, to improve estimation in the target study. Our procedure maps each support into the target parameter space, aggregates the results with data-driven weights, and applies a sparse correction against inaccurate supports. We establish nonasymptotic guarantees: the estimator improves on the target-only Lasso when the supports are informative and never does worse otherwise. Simulations and a gene-expression application confirm both properties.
ER -