Comparative evaluation of ChatGPT and Gemini feedback in scenario-based clinical reasoning among medical interns.
- 1Aydın Adnan Menderes University, Aydın Vocational School - Aydın, Turkey.
- 2Aydın Adnan Menderes University, Faculty of Education, Department of Computer and Instructional Technologies - Aydın, Turkey.
- 3Aydın Adnan Menderes University, Faculty of Medicine, Department of Medical Education - Aydın, Turkey.
Abstract
Objective
Artificial intelligence tools are increasingly used to provide individualized feedback in medical education. The aim of this study was to compare the quality of feedback generated by ChatGPT and Gemini during scenario-based clinical reasoning activities.Methods
A convergent mixed-methods study was conducted with 32 medical interns randomly assigned to ChatGPT (n=16) or Gemini (n=16). Following a virtual patient interview on iron deficiency anemia, artificial intelligence-generated feedback was evaluated by interns and medical education specialists using an 11-item rubric assessing clarity, accuracy, structure, identification of strengths and weaknesses, contribution to clinical reasoning, and overall educational value. Quantitative data were analyzed using descriptive statistics and Mann-Whitney U tests, while qualitative data were examined through content analysis.Results
Interns rated ChatGPT slightly higher than Gemini (1.80±0.20 vs. 1.75±0.26), whereas specialists reported similar scores (1.91±0.06 vs. 1.87±0.10). No statistically significant differences were identified (p>0.05). Both systems were perceived as clear, supportive, and constructive; however, insufficient patient-centeredness emerged as a common limitation of both artificial intelligence systems. Qualitative findings indicated that ChatGPT provided more structured and reflective feedback, whereas Gemini offered more concrete examples. Common limitations included insufficient patient-centeredness, lack of systematicity, and occasional unrealistic statements.Conclusion
ChatGPT and Gemini generated educationally useful feedback for scenario-based clinical reasoning activities. Although artificial intelligence tools cannot replace expert supervision, they may support personalized learning and reduce faculty workload when used under human oversight.
Related
- Comparative Evaluation of Large Language Model Interfaces in Third Molar Surgery Complication Scenarios: Response Quality, Clinical Content, Potential Clinical Risk, Readability, and Externally Observable Response Latency—A Cross-Sectional Comparative Benchmark Study
- CPT/HCPCS Code Recommendation from Clinical Notes: A Comparative Evaluation of AI Methods
- Proactive Prediction of Alarms in Hemodialysis Machines: A Comparative Evaluation of AI Models for Clinical Prognostic Support
- Evaluation of the Efficacy and Safety of Combination Therapy of Vamha and Myrha in the Management of PMOS: An Open-Label, Randomized, Multicentre, Comparative, Prospective Clinical Study
- Cross-cultural adaptation and comparative psychometric evaluation of four patient-reported outcome measures for breast cancer-related lymphedema: measurement properties and clinical discrimination
BibTeX
@article{ulku2026comparative,
title = {{Comparative evaluation of ChatGPT and Gemini feedback in scenario-based clinical reasoning among medical interns.}},
author = {Hilal Hatice Ülkü and Fulya Torun and Selcen Öncü},
journal = {Revista da Associacao Medica Brasileira (1992)},
year = {2026},
month = oct,
doi = {10.1590/1806-9282.20260869},
url = {https://europepmc.org/article/MED/42879590},
note = {CC BY},
}
RIS
TY - JOUR
TI - Comparative evaluation of ChatGPT and Gemini feedback in scenario-based clinical reasoning among medical interns.
AU - Hilal Hatice Ülkü
AU - Fulya Torun
AU - Selcen Öncü
PY - 2026
DA - 2026/10/12
JO - Revista da Associacao Medica Brasileira (1992)
DO - 10.1590/1806-9282.20260869
UR - https://europepmc.org/article/MED/42879590
AB - Objective Artificial intelligence tools are increasingly used to provide individualized feedback in medical education. The aim of this study was to compare the quality of feedback generated by ChatGPT and Gemini during scenario-based clinical reasoning activities.Methods A convergent mixed-methods study was conducted with 32 medical interns randomly assigned to ChatGPT (n=16) or Gemini (n=16). Following a virtual patient interview on iron deficiency anemia, artificial intelligence-generated feedback was evaluated by interns and medical education specialists using an 11-item rubric assessing clarity, accuracy, structure, identification of strengths and weaknesses, contribution to clinical reasoning, and overall educational value. Quantitative data were analyzed using descriptive statistics and Mann-Whitney U tests, while qualitative data were examined through content analysis.Results Interns rated ChatGPT slightly higher than Gemini (1.80±0.20 vs. 1.75±0.26), whereas specialists reported similar scores (1.91±0.06 vs. 1.87±0.10). No statistically significant differences were identified (p>0.05). Both systems were perceived as clear, supportive, and constructive; however, insufficient patient-centeredness emerged as a common limitation of both artificial intelligence systems. Qualitative findings indicated that ChatGPT provided more structured and reflective feedback, whereas Gemini offered more concrete examples. Common limitations included insufficient patient-centeredness, lack of systematicity, and occasional unrealistic statements.Conclusion ChatGPT and Gemini generated educationally useful feedback for scenario-based clinical reasoning activities. Although artificial intelligence tools cannot replace expert supervision, they may support personalized learning and reduce faculty workload when used under human oversight.
SN - 0104-4230
SN - 1806-9282
ER -