PreprintCC BY 4.0arXiv
Retrieval-Based In-Context Learning: A Domain Adaptation Framework
Abstract
In-context retrieval (ICR) is a retrieval-based form of in-context learning (ICL) in which demonstrations are retrieved from a source database based on similarity to the query, rather than sampled independently. In this work, we formulate ICR as a type of domain adaptation problem, where the source distribution P of the database may differ from the target distribution Q of the test query-label pair. We investigate the performance of ICR under a flexible class of distributional shifts that substantially extends prior work \citep{li2024fine,guo2025retrieval}, and establish theoretical guarantees that quantify the benefits and pitfalls of this learning paradigm. Our theory is verified by experiments on synthetic and language tasks.
Related
- Joint Contrastive-Adversarial Representation Learning for Unsupervised Domain Adaptation
- Synergistic Learning in Domain Adaptation with a Known Link: A Minimax Understanding
- EntroPrefill: Renyi-Guided Context Pruning with Conditional Stability Guarantees for Retrieval-Augmented Generation
- Targeted Retrieval, Compact Representations: How CoT Reasoning Improves Long-Context Counting
- A Domain-specific adaptation network for elliptic interface problems
BibTeX
@misc{zhu2026retrievalbased,
title = {{Retrieval-Based In-Context Learning: A Domain Adaptation Framework}},
author = {Yilun Zhu and Naihao Deng and Yingcong Li and Naichen Shi and Clayton Scott},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.05717},
eprint = {2610.05717},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.05717},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Retrieval-Based In-Context Learning: A Domain Adaptation Framework
AU - Yilun Zhu
AU - Naihao Deng
AU - Yingcong Li
AU - Naichen Shi
AU - Clayton Scott
PY - 2026
DA - 2026/10/12
PB - arXiv
DO - 10.48550/arxiv.2610.05717
UR - https://arxiv.org/abs/2610.05717
AB - In-context retrieval (ICR) is a retrieval-based form of in-context learning (ICL) in which demonstrations are retrieved from a source database based on similarity to the query, rather than sampled independently. In this work, we formulate ICR as a type of domain adaptation problem, where the source distribution P of the database may differ from the target distribution Q of the test query-label pair. We investigate the performance of ICR under a flexible class of distributional shifts that substantially extends prior work \citep{li2024fine,guo2025retrieval}, and establish theoretical guarantees that quantify the benefits and pitfalls of this learning paradigm. Our theory is verified by experiments on synthetic and language tasks.
ER -