Medical Students' Perceptions of Retrieval-Augmented Generation and Large Language Models: Dyadic Interview and Exploratory Survey Study.
- 1Bristol Medical School, University of Bristol, 5 Tyndall Avenue, Bristol, BS8 1UD, United Kingdom, 44 117 331 1831.
Abstract
Background
There is evidence of rapid adoption of large language models (LLMs) in undergraduate medical education, with medical students using them to support their learning. This raises concerns about the accuracy of LLM-generated information that students are learning from. Retrieval-augmented generation (RAG), which grounds an LLM in curated materials, may be a strategy to improve the accuracy of this information.Objective
This study explored medical students' perceptions after using an online tool, NotebookLM, that uses RAG.Methods
A cohort of 284 second-year medical students at Bristol Medical School were encouraged to use NotebookLM over a period of 4 months (March 2025 to June 2025) to help them learn course content. Following this, volunteers were recruited to share their perspectives through interviews and a survey that was open for 4 months (June 2025 to September 2025). A total of 9 individuals participated: 2 through interviews and 7 through the survey. Qualitative data were analyzed using reflexive thematic analysis and supplemented by Likert scale data.Results
NotebookLM was perceived as more reliable than standard LLM responses by interview participants, mainly due to the transparency of and control over information sources. Reliability ratings from survey participants were more mixed, but none rated unreliable. Students described it as an accessible, on-demand learning resource, which they perceived as improving efficiency and supporting self-directed study. They reported continued use of strategies to cross-check generated information. Participants demonstrated some awareness of ethical concerns, but it was unclear whether this altered their practice.Conclusions
In this small, exploratory study, students described NotebookLM as improving trust in LLM outputs while maintaining their flexibility. The findings suggest that RAG may be a strategy to support students who use LLMs. However, further research is needed to formally evaluate RAG vs standard LLMs, including the impact of RAG on accuracy, curricular alignment, and the ability of students to meet learning outcomes.
Related
- Retrieval-augmented generation in biomedicine: a survey of technologies, datasets, and clinical applications
- Retrieval-augmented generation in medicine: A scoping review of technical implementations, clinical applications, and ethical considerations
- IPF-RAG: temporal-aware retrieval-augmented generation for idiopathic pulmonary fibrosis care.
- Limits of Single-Pass Retrieval-Augmented Generation for AI-Powered Cancer Care Navigation: A Comparison of Retrieval Strategies
- Integrating knowledge graphs and retrieval-augmented generation to identify clinical pharmacology gaps in infectious disease trial protocols
BibTeX
@article{athavale2026medical,
title = {{Medical Students' Perceptions of Retrieval-Augmented Generation and Large Language Models: Dyadic Interview and Exploratory Survey Study.}},
author = {Rohin Athavale and Alexander Cresswell and Alice Huffman},
journal = {JMIR formative research},
year = {2026},
month = oct,
doi = {10.2196/106157},
url = {https://europepmc.org/article/MED/42962066},
note = {CC BY},
}
RIS
TY - JOUR
TI - Medical Students' Perceptions of Retrieval-Augmented Generation and Large Language Models: Dyadic Interview and Exploratory Survey Study.
AU - Rohin Athavale
AU - Alexander Cresswell
AU - Alice Huffman
PY - 2026
DA - 2026/10/01
JO - JMIR formative research
DO - 10.2196/106157
UR - https://europepmc.org/article/MED/42962066
AB - Background There is evidence of rapid adoption of large language models (LLMs) in undergraduate medical education, with medical students using them to support their learning. This raises concerns about the accuracy of LLM-generated information that students are learning from. Retrieval-augmented generation (RAG), which grounds an LLM in curated materials, may be a strategy to improve the accuracy of this information.Objective This study explored medical students' perceptions after using an online tool, NotebookLM, that uses RAG.Methods A cohort of 284 second-year medical students at Bristol Medical School were encouraged to use NotebookLM over a period of 4 months (March 2025 to June 2025) to help them learn course content. Following this, volunteers were recruited to share their perspectives through interviews and a survey that was open for 4 months (June 2025 to September 2025). A total of 9 individuals participated: 2 through interviews and 7 through the survey. Qualitative data were analyzed using reflexive thematic analysis and supplemented by Likert scale data.Results NotebookLM was perceived as more reliable than standard LLM responses by interview participants, mainly due to the transparency of and control over information sources. Reliability ratings from survey participants were more mixed, but none rated unreliable. Students described it as an accessible, on-demand learning resource, which they perceived as improving efficiency and supporting self-directed study. They reported continued use of strategies to cross-check generated information. Participants demonstrated some awareness of ethical concerns, but it was unclear whether this altered their practice.Conclusions In this small, exploratory study, students described NotebookLM as improving trust in LLM outputs while maintaining their flexibility. The findings suggest that RAG may be a strategy to support students who use LLMs. However, further research is needed to formally evaluate RAG vs standard LLMs, including the impact of RAG on accuracy, curricular alignment, and the ability of students to meet learning outcomes.
SN - 2561-326X
ER -