Language-Induced Priors for Domain Adaptation
Abstract
Domain adaptation faces a fundamental paradox in the cold-start regime. When target data is scarce, statistical methods fail to distinguish relevant source domains from irrelevant ones, which often leads to negative transfer. In this paper, we address this challenge by leveraging expert textual descriptions of the target domain, a resource that is often available but overlooked. We propose a probabilistic framework that translates these semantic descriptions into a choice model, namely a Language-Induced Prior (LIP), that learns the preferences from a pretrained Large Language Model. The LIP is then integrated into an Expectation-Maximization algorithm to identify source relevance. Methodologically, this framework is compatible with any parametric model where a likelihood is available. It allows the LIP to guide the selection of sources when target signals are weak, while gradually refining these choices as samples accumulate. Theoretically, we prove that the estimator roughly matches an oracle cold-start mean squared error under a correct prior, while remaining asymptotically consistent regardless of the quality of the LIP. Empirically, we validated the framework on a descriptive (Gaussian estimation), a predictive (C-MAPSS dataset), and a prescriptive task (MuJoCo Hopper).
Related
- Training-Free Multi-Agent Cooperative Embodied Search from Uncertain Vision-Language Priors
- Generative Priors Conditioned on Natural Language for Bayesian Inversion in PDEs
- Retrieval-Based In-Context Learning: A Domain Adaptation Framework
- A Domain-specific adaptation network for elliptic interface problems
- Joint Contrastive-Adversarial Representation Learning for Unsupervised Domain Adaptation
BibTeX
@misc{chen2026languageinduced,
title = {{Language-Induced Priors for Domain Adaptation}},
author = {Qiyuan Chen and Jiayu Zhou and Raed Al Kontar},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.14301},
eprint = {2605.14301},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.14301},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Language-Induced Priors for Domain Adaptation
AU - Qiyuan Chen
AU - Jiayu Zhou
AU - Raed Al Kontar
PY - 2026
DA - 2026/10/28
PB - arXiv
DO - 10.48550/arxiv.2605.14301
UR - https://arxiv.org/abs/2605.14301
AB - Domain adaptation faces a fundamental paradox in the cold-start regime. When target data is scarce, statistical methods fail to distinguish relevant source domains from irrelevant ones, which often leads to negative transfer. In this paper, we address this challenge by leveraging expert textual descriptions of the target domain, a resource that is often available but overlooked. We propose a probabilistic framework that translates these semantic descriptions into a choice model, namely a Language-Induced Prior (LIP), that learns the preferences from a pretrained Large Language Model. The LIP is then integrated into an Expectation-Maximization algorithm to identify source relevance. Methodologically, this framework is compatible with any parametric model where a likelihood is available. It allows the LIP to guide the selection of sources when target signals are weak, while gradually refining these choices as samples accumulate. Theoretically, we prove that the estimator roughly matches an oracle cold-start mean squared error under a correct prior, while remaining asymptotically consistent regardless of the quality of the LIP. Empirically, we validated the framework on a descriptive (Gaussian estimation), a predictive (C-MAPSS dataset), and a prescriptive task (MuJoCo Hopper).
ER -