Staging and Grading Advanced Periodontitis with Large Language Models from Standardized Case Summaries: A Multicenter Concordance and Reproducibility Study
- 1Department of Periodontology, Faculty of Dentistry, Batman University, 72060 Batman, Türkiye
- 2Department of Periodontology, Faculty of Dentistry, Burdur Mehmet Akif Ersoy University, 15030 Burdur, Türkiye
- 3Department of Periodontology, Faculty of Dentistry, Alanya Alaaddin Keykubat University, 07410 Alanya, Türkiye
Abstract
Background/Objectives: To evaluate the concordance of three general-purpose large language models (LLMs), ChatGPT, Claude, and Gemini, with an adjudicated expert consensus classification of Stage, Grade, and Extent in Stage III–IV periodontitis when applying the 2018 American Academy of Periodontology/European Federation of Periodontology (AAP/EFP) rules to clinician-prepared, standardized case summaries. Methods: In this multicenter retrospective concordance study (two centers, 105 patients), two periodontologists independently classified each case from complete charts and radiographs; discordant cases were adjudicated. De-identified summaries containing pre-extracted clinical, radiographic, and modifier variables were presented to each LLM in three independent sessions per case; the modal decision was the primary index result, and single-session performance was also reported. Stage and Grade concordance (Cohen’s/weighted kappa) were co-primary endpoints; Extent and full concordance were secondary. Results: Modal Stage concordance was almost perfect for ChatGPT (κ = 0.98) and Gemini (κ = 0.92) and substantial for Claude (κ = 0.73); single-session Stage κ ranged from 0.73 to 0.98. Grade (weighted κ = 0.97–1.00) and Extent (κ = 0.93–1.00) concordance were almost perfect. Full concordance ranged from 85.7% (Claude) to 99.0% (ChatGPT). Every classifiable modal Stage discordance understated severity: Claude and Gemini assigned Stage II or I to 14 Stage III cases in which attachment loss met the Stage III threshold but radiographic bone loss was below 33%. Conclusions: Given pre-extracted variables and explicit instructions, the LLMs applied the classification rules with substantial to almost perfect concordance; this reflects rule application, not independent diagnosis. Performance was model-specific, Gemini showed greater session-to-session variability, and two models systematically understaged borderline Stage III cases. The findings support only clinician-supervised, educational use with local validation.
Related
- Beyond Breslow Thickness: Reproducibility and Histopathologic Concordance of Ultra-High-Frequency Ultrasound in Primary Cutaneous Melanoma
- PET Tumour Segmentation in Head and Neck Cancer: Reproducibility, Reader Dependence and Concordance with CT-Defined Tumour Volume in a 154-Patient Cohort
- Association between periodontitis and different insulin resistance indices in hypertensive patients: a multicenter cross-sectional study
- A standardized ECPR activation protocol and resuscitation efficiency, patient outcomes, and nursing cognitive load: a retrospective multicenter study
- Improved COPD Epidemiological Awareness Following Standardized Education Independently Associated with Better Inhaled Medication Adherence: A Multicenter Prospective Pre-Post Study
BibTeX
@article{kızıltoprak2026staging,
title = {{Staging and Grading Advanced Periodontitis with Large Language Models from Standardized Case Summaries: A Multicenter Concordance and Reproducibility Study}},
author = {Mehmet Kızıltoprak and Mustafa Karaca and Mustafa Özay Uslu},
journal = {Journal of clinical medicine},
year = {2026},
month = sep,
doi = {10.3390/jcm15197585},
url = {https://europepmc.org/article/PMC/PMC13665847},
note = {CC BY},
}
RIS
TY - JOUR
TI - Staging and Grading Advanced Periodontitis with Large Language Models from Standardized Case Summaries: A Multicenter Concordance and Reproducibility Study
AU - Mehmet Kızıltoprak
AU - Mustafa Karaca
AU - Mustafa Özay Uslu
PY - 2026
DA - 2026/09/30
JO - Journal of clinical medicine
DO - 10.3390/jcm15197585
UR - https://europepmc.org/article/PMC/PMC13665847
AB - Background/Objectives: To evaluate the concordance of three general-purpose large language models (LLMs), ChatGPT, Claude, and Gemini, with an adjudicated expert consensus classification of Stage, Grade, and Extent in Stage III–IV periodontitis when applying the 2018 American Academy of Periodontology/European Federation of Periodontology (AAP/EFP) rules to clinician-prepared, standardized case summaries. Methods: In this multicenter retrospective concordance study (two centers, 105 patients), two periodontologists independently classified each case from complete charts and radiographs; discordant cases were adjudicated. De-identified summaries containing pre-extracted clinical, radiographic, and modifier variables were presented to each LLM in three independent sessions per case; the modal decision was the primary index result, and single-session performance was also reported. Stage and Grade concordance (Cohen’s/weighted kappa) were co-primary endpoints; Extent and full concordance were secondary. Results: Modal Stage concordance was almost perfect for ChatGPT (κ = 0.98) and Gemini (κ = 0.92) and substantial for Claude (κ = 0.73); single-session Stage κ ranged from 0.73 to 0.98. Grade (weighted κ = 0.97–1.00) and Extent (κ = 0.93–1.00) concordance were almost perfect. Full concordance ranged from 85.7% (Claude) to 99.0% (ChatGPT). Every classifiable modal Stage discordance understated severity: Claude and Gemini assigned Stage II or I to 14 Stage III cases in which attachment loss met the Stage III threshold but radiographic bone loss was below 33%. Conclusions: Given pre-extracted variables and explicit instructions, the LLMs applied the classification rules with substantial to almost perfect concordance; this reflects rule application, not independent diagnosis. Performance was model-specific, Gemini showed greater session-to-session variability, and two models systematically understaged borderline Stage III cases. The findings support only clinician-supervised, educational use with local validation.
SN - 2077-0383
ER -