Can artificial intelligence counsel expectant mothers? Expert evaluation of ChatGPT’s educational performance in gestational diabetes mellitus
- 1Department of Obstetrics and Gynecology, Torbalı State Hospital, Torbalı, Izmir, Turkey
- 2Department of Gynecologic Oncology, İzmir City Hospital, Izmir, Turkey
- 3Department of Obstetrics and Gynecology, Tepecik Training and Research Hospital, Izmir, Turkey
Abstract
Background: Gestational diabetes mellitus (GDM) is an increasingly common pregnancy complication associated with adverse maternal–fetal outcomes and long-term risk of type 2 diabetes. Artificial intelligence (AI) tools such as ChatGPT have emerged as potential instruments for patient education, yet their reliability and safety in obstetric counseling remain underexplored. This study aimed to evaluate the educational quality of ChatGPT-5 responses to frequently asked questions about GDM.
Methods: This cross-sectional study was conducted between January and June 2025 at Torbalı State Hospital, İzmir, Türkiye, following STROBE guidelines. Twenty frequently asked questions about gestational diabetes mellitus, identified from routine antenatal counseling through clinician observation and consensus, were entered into ChatGPT-5 using a standardized prompt. Thirty physicians specializing in obstetrics and gynecology from Türkiye, the United Kingdom, Canada, and Germany independently rated the AI-generated answers on a 5-point Likert scale across five domains: accuracy, comprehensiveness, clarity, safety, and appropriateness. Reliability was analyzed using Cronbach’s α and intraclass correlation coefficient (ICC). Comparative analyses were performed using Friedman and Dunn–Bonferroni tests.
Results: Overall, ChatGPT-5 achieved high ratings for accuracy (4.23 ± 0.32), clarity (4.18 ± 0.27), and appropriateness (4.09 ± 0.31), while comprehensiveness (3.92 ± 0.29) and safety (3.88 ± 0.34) scored lower. Internal consistency was strong (Cronbach’s α = 0.87) and interrater reliability was good (ICC = 0.79). Among thematic domains, “Definition, causes, and risk factors” received the highest scores (4.20 ± 0.30), whereas “Follow-up and treatment” scored the lowest (3.96 ± 0.33). Safety ratings were significantly lower than accuracy and clarity (p = 0.018).
Conclusion: ChatGPT-5 demonstrated high accuracy and clarity in GDM counseling, suggesting its potential use as a physician-supervised educational support tool. However, limitations in comprehensiveness and safety highlight the need for guideline-based refinement and human oversight to ensure ethical and reliable integration of AI in obstetric patient education.
Trial registration: Not applicable. This study is an observational cross-sectional design and did not involve patient enrollment or intervention.
Supplementary Information: The online version contains supplementary material available at https://doi.org/10.1186/s12902-026-02490-w.
Related
- A pilot study to implement artificial intelligence-enabled, point-of-care obstetric ultrasound for gestational age estimation in Zambia: An evaluation protocol
- The ethical and regulatory framework of medical artificial intelligence (AI) system’s behavioral performance for image-based diagnostics: An ethical evaluation of the International and European Union Scope
- Artificial Intelligence and Anatomical Variation: A Narrative Review of Detection, Classification, and Educational Applications.
- Generative artificial intelligence use and interpersonal educational interaction among nursing students: self-regulated learning as a statistical mediator
- Educational Artificial Intelligence Software to Support Assessment of Atopic Dermatitis Severity
BibTeX
@article{balcı2026artificial,
title = {{Can artificial intelligence counsel expectant mothers? Expert evaluation of ChatGPT’s educational performance in gestational diabetes mellitus}},
author = {Mücahit Furkan Balcı and Celal Akdemi̇r and Fatih Yildirim},
journal = {BMC Endocrine Disorders},
year = {2026},
month = aug,
doi = {10.1186/s12902-026-02490-w},
url = {https://doi.org/10.1186/s12902-026-02490-w},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - JOUR
TI - Can artificial intelligence counsel expectant mothers? Expert evaluation of ChatGPT’s educational performance in gestational diabetes mellitus
AU - Mücahit Furkan Balcı
AU - Celal Akdemi̇r
AU - Fatih Yildirim
PY - 2026
DA - 2026/08/14
JO - BMC Endocrine Disorders
DO - 10.1186/s12902-026-02490-w
UR - https://doi.org/10.1186/s12902-026-02490-w
AB - Background: Gestational diabetes mellitus (GDM) is an increasingly common pregnancy complication associated with adverse maternal–fetal outcomes and long-term risk of type 2 diabetes. Artificial intelligence (AI) tools such as ChatGPT have emerged as potential instruments for patient education, yet their reliability and safety in obstetric counseling remain underexplored. This study aimed to evaluate the educational quality of ChatGPT-5 responses to frequently asked questions about GDM. Methods: This cross-sectional study was conducted between January and June 2025 at Torbalı State Hospital, İzmir, Türkiye, following STROBE guidelines. Twenty frequently asked questions about gestational diabetes mellitus, identified from routine antenatal counseling through clinician observation and consensus, were entered into ChatGPT-5 using a standardized prompt. Thirty physicians specializing in obstetrics and gynecology from Türkiye, the United Kingdom, Canada, and Germany independently rated the AI-generated answers on a 5-point Likert scale across five domains: accuracy, comprehensiveness, clarity, safety, and appropriateness. Reliability was analyzed using Cronbach’s α and intraclass correlation coefficient (ICC). Comparative analyses were performed using Friedman and Dunn–Bonferroni tests. Results: Overall, ChatGPT-5 achieved high ratings for accuracy (4.23 ± 0.32), clarity (4.18 ± 0.27), and appropriateness (4.09 ± 0.31), while comprehensiveness (3.92 ± 0.29) and safety (3.88 ± 0.34) scored lower. Internal consistency was strong (Cronbach’s α = 0.87) and interrater reliability was good (ICC = 0.79). Among thematic domains, “Definition, causes, and risk factors” received the highest scores (4.20 ± 0.30), whereas “Follow-up and treatment” scored the lowest (3.96 ± 0.33). Safety ratings were significantly lower than accuracy and clarity (p = 0.018). Conclusion: ChatGPT-5 demonstrated high accuracy and clarity in GDM counseling, suggesting its potential use as a physician-supervised educational support tool. However, limitations in comprehensiveness and safety highlight the need for guideline-based refinement and human oversight to ensure ethical and reliable integration of AI in obstetric patient education. Trial registration: Not applicable. This study is an observational cross-sectional design and did not involve patient enrollment or intervention. Supplementary Information: The online version contains supplementary material available at https://doi.org/10.1186/s12902-026-02490-w.
SN - 1472-6823
ER -