Limits of Single-Pass Retrieval-Augmented Generation for AI-Powered Cancer Care Navigation: A Comparison of…

Abstract

BackgroundPeople affected by cancer often face difficulty finding relevant clinical, psychological, and practical support services. AI-powered navigation assistants may improve access to these resources, but their retrieval performance must be reliable.

ObjectiveTo develop a single-pass retrieval-augmented generation assistant for cancer-care navigation and compare the retrieval strategies, including their robustness to reworded questions.

MethodsWe created a database of 853 cancer-support resources reviewed by librarians, clinicians, researchers, and patient partners. We evaluated the system using 100 questions derived from questions submitted by patients. We compared keyword-based, semantic, and hybrid retrieval using Precision@K, Hit@K, and nDCG@K. The best-performing configuration was then tested using semantically equivalent rewordings of the original questions.

ResultsKeyword-based retrieval performed poorly, achieving a P@1 of 25.0% and Hit@5 of 43.0%. Semantic retrieval improved these results to 58.0% and 86.0%, respectively. The best hybrid configuration achieved a P@1 of 64.0%, Hit@5 of 90.0%, and nDCG@5 of 51.0%. Performance remained relatively stable when the questions were reworded, with a P@1 of 61.0%, Hit@5 of 88.0%, and nDCG@5 of 46.1%.

ConclusionsHybrid retrieval performed best and remained relatively stable when questions were reworded. However, its limited ability to rank a relevant resource first highlights the limitations of single-pass retrieval for patient-facing cancer navigation. Future work will explore metadata filtering and a multi-agent architecture to improve retrieval reliability.

Retrieval-augmented generation in biomedicine: a survey of technologies, datasets, and clinical applications
Jiawei He, Boya Zhang, Hossein Rouhizadeh +6
Artificial Intelligence Review23 Oct 2026Peer-reviewedCC BY27 citedEpidemiology & public healthOpenAlex
Retrieval-augmented generation in medicine: A scoping review of technical implementations, clinical applications, and ethical considerations
Rui Yang, Matthew Yu Heng Wong, Huitao Li +13
Cell Reports Medicine1 Jul 2026Peer-reviewedCC BY-NC-ND18 citedEpidemiology & public healthOpenAlex
IPF-RAG: temporal-aware retrieval-augmented generation for idiopathic pulmonary fibrosis care.
Xuejia Zheng, Xiang Rui, Teng Long +8
Frontiers in digital health14 Oct 2026Peer-reviewedCC BYFull textEpidemiology & public healthEurope PMCPMC
Integrating knowledge graphs and retrieval-augmented generation to identify clinical pharmacology gaps in infectious disease trial protocols
Shreyas Waikar, Murali Ramanathan
Frontiers in Pharmacology14 Oct 2026Peer-reviewedCC BY 4.0Full textEpidemiology & public healthCrossrefEurope PMCPMC
AFP assistant: a retrieval-augmented generation and large language model-powered multilingual polio chatbot for low-resource language communities
Gelane Biru, Honey Gemechu, Firanol Teshome +8
npj Digital Medicine1 Jun 2026Peer-reviewedCC BYFull text1 citedEpidemiology & public healthDOAJ

Cite

BibTeX RIS

BibTeX

@misc{hasan2026limits,
  title = {{Limits of Single-Pass Retrieval-Augmented Generation for AI-Powered Cancer Care Navigation: A Comparison of Retrieval Strategies}},
  author = {E. Hasan and Y. Zhang and O. Cook and A. Loe and M. Sha and L. T'ien and M. Ng and C. Rauscher and S. Raman and J. L. Bender and R. T. Ng and A. Bates and J.-J. Nunez},
  howpublished = {medRxiv},
  year = {2026},
  month = aug,
  doi = {10.64898/2026.08.31.26361774},
  url = {https://www.medrxiv.org/content/10.64898/2026.08.31.26361774v1},
  note = {CC BY-NC-ND},
}

RIS

TY  - UNPB
TI  - Limits of Single-Pass Retrieval-Augmented Generation for AI-Powered Cancer Care Navigation: A Comparison of Retrieval Strategies
AU  - E. Hasan
AU  - Y. Zhang
AU  - O. Cook
AU  - A. Loe
AU  - M. Sha
AU  - L. T'ien
AU  - M. Ng
AU  - C. Rauscher
AU  - S. Raman
AU  - J. L. Bender
AU  - R. T. Ng
AU  - A. Bates
AU  - J.-J. Nunez
PY  - 2026
DA  - 2026/08/31
PB  - medRxiv
DO  - 10.64898/2026.08.31.26361774
UR  - https://www.medrxiv.org/content/10.64898/2026.08.31.26361774v1
AB  - BackgroundPeople affected by cancer often face difficulty finding relevant clinical, psychological, and practical support services. AI-powered navigation assistants may improve access to these resources, but their retrieval performance must be reliable.  ObjectiveTo develop a single-pass retrieval-augmented generation assistant for cancer-care navigation and compare the retrieval strategies, including their robustness to reworded questions.  MethodsWe created a database of 853 cancer-support resources reviewed by librarians, clinicians, researchers, and patient partners. We evaluated the system using 100 questions derived from questions submitted by patients. We compared keyword-based, semantic, and hybrid retrieval using Precision@K, Hit@K, and nDCG@K. The best-performing configuration was then tested using semantically equivalent rewordings of the original questions.  ResultsKeyword-based retrieval performed poorly, achieving a P@1 of 25.0% and Hit@5 of 43.0%. Semantic retrieval improved these results to 58.0% and 86.0%, respectively. The best hybrid configuration achieved a P@1 of 64.0%, Hit@5 of 90.0%, and nDCG@5 of 51.0%. Performance remained relatively stable when the questions were reworded, with a P@1 of 61.0%, Hit@5 of 88.0%, and nDCG@5 of 46.1%.  ConclusionsHybrid retrieval performed best and remained relatively stable when questions were reworded. However, its limited ability to rank a relevant resource first highlights the limitations of single-pass retrieval for patient-facing cancer navigation. Future work will explore metadata filtering and a multi-agent architecture to improve retrieval reliability.
ER  -

Abstract

Related

Cite