Comparison of ChatGPT-Generated Patient Information on Testicular Cancer Across Two Assessment Periods: Clinical Accuracy, Response Quality, and Citation Support
- 1Department of Urology, Acıbadem Bayındır Söğütözü Hospital, Ankara, Türkiye
- 2Department of Urology, Etimesgut Şehit Sait Ertürk State Hospital, Ankara, Türkiye
Abstract
Objective:: The aim of this study was to compare clinical accuracy, response quality, and citation characteristics of ChatGPT-generated testicular cancer information across 2 assessments 20 months apart.
Methods:: Sixty-one patient-oriented questions derived from European Association of Urology (EAU) Patient Information were submitted to ChatGPT-4o in December 2024 and repeated unchanged using ChatGPT-5.6 Luna in August 2026. Two urologists independently assessed accuracy against contemporaneous EAU guidelines and rated quality using the Global Quality Score (GQS) and modified DISCERN (mDISCERN). Citation quantity, source quality, and source-to-claim support were evaluated. Only the first response was retained; repeated-generation variability was not assessed.
Results:: Fully accurate responses were 86.9% (53/61) in 2024 and 93.4% (57/61) in 2026 (P = .125). Of 8 clinically relevant inaccuracies in 2024, 4 were corrected, 3 improved but remained misleading, and 1 persisted. GQS increased from 3.75 ± 0.77 to 4.82 ± 0.41 and mDISCERN from 30.37 ± 4.65 to 36.80 ± 1.93 (both P < .001). Citations per response decreased from 5.97 ± 3.53 to 4.79 ± 1.19 (P = .042), while high-quality unique sources increased from 77.4% to 96.9% (P < .001). Fully supported sources were 80.1% and 86.4% (P = .050); 6 sources (2.6%) in 2026 did not support the claims.
Conclusion:: Response quality and the use of high-quality sources were greater in 2026, whereas changes in clinical accuracy and source-to-claim support were less clear. Clinically meaningful errors persisted in guideline-sensitive topics. Because different model versions and interfaces were assessed, the findings represent model- and interface-specific snapshots, not a controlled head-to-head comparison.
Related
- Social comparison information is not always prioritized: Temporal performance feedback dominates self-assessment in dysphoria
- Nurse–patient communication when patients introduce AI-generated health information: a qualitative study of clinical nurses’ experiences and responses
- When Can Ultra-Short Paired-Comparison Assessment Recover Trait Information? A Simulation and Empirical Illustration
- Bayesian Borrowing of External Information in Clinical Trials: A Comparison of MAP, RMAP, and SAM Priors
- Contrast-enhanced ultrasound in assessment of testicular injury based on different degrees of testicular torsion in rabbits
BibTeX
@article{ekenci2026comparison,
title = {{Comparison of ChatGPT-Generated Patient Information on Testicular Cancer Across Two Assessment Periods: Clinical Accuracy, Response Quality, and Citation Support}},
author = {Berk Yasin Ekenci and Mehmet Altan},
journal = {Urology Research and Practice},
year = {2026},
month = nov,
doi = {10.5152/tud.2026.26067},
url = {https://doi.org/10.5152/tud.2026.26067},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Comparison of ChatGPT-Generated Patient Information on Testicular Cancer Across Two Assessment Periods: Clinical Accuracy, Response Quality, and Citation Support
AU - Berk Yasin Ekenci
AU - Mehmet Altan
PY - 2026
DA - 2026/11/02
JO - Urology Research and Practice
DO - 10.5152/tud.2026.26067
UR - https://doi.org/10.5152/tud.2026.26067
AB - Objective:: The aim of this study was to compare clinical accuracy, response quality, and citation characteristics of ChatGPT-generated testicular cancer information across 2 assessments 20 months apart. Methods:: Sixty-one patient-oriented questions derived from European Association of Urology (EAU) Patient Information were submitted to ChatGPT-4o in December 2024 and repeated unchanged using ChatGPT-5.6 Luna in August 2026. Two urologists independently assessed accuracy against contemporaneous EAU guidelines and rated quality using the Global Quality Score (GQS) and modified DISCERN (mDISCERN). Citation quantity, source quality, and source-to-claim support were evaluated. Only the first response was retained; repeated-generation variability was not assessed. Results:: Fully accurate responses were 86.9% (53/61) in 2024 and 93.4% (57/61) in 2026 (P = .125). Of 8 clinically relevant inaccuracies in 2024, 4 were corrected, 3 improved but remained misleading, and 1 persisted. GQS increased from 3.75 ± 0.77 to 4.82 ± 0.41 and mDISCERN from 30.37 ± 4.65 to 36.80 ± 1.93 (both P < .001). Citations per response decreased from 5.97 ± 3.53 to 4.79 ± 1.19 (P = .042), while high-quality unique sources increased from 77.4% to 96.9% (P < .001). Fully supported sources were 80.1% and 86.4% (P = .050); 6 sources (2.6%) in 2026 did not support the claims. Conclusion:: Response quality and the use of high-quality sources were greater in 2026, whereas changes in clinical accuracy and source-to-claim support were less clear. Clinically meaningful errors persisted in guideline-sensitive topics. Because different model versions and interfaces were assessed, the findings represent model- and interface-specific snapshots, not a controlled head-to-head comparison.
SN - 2980-1478
ER -