Comparative Analysis of Aesthetic Judgment between Orthodontists, General Dentists, and Different Versions of Large Language Models Using the IOTN-AC Index
- 1İstanbul Aydın University Faculty of Dentistry Department of Orthodontics, İstanbul, Türkiye
Abstract
Objective: This study aimed to compare the performance and diagnostic accuracy of general dentists and different artificial intelligence (AI)-based large language models (LLMs), including their older and newer versions, in photograph-based IOTN-AC index assessments, using orthodontists’ evaluations as the reference standard. Methods: The mean IOTN-AC scores derived from orthodontists were established as the reference (gold) standard. Cases were evaluated by general dentists and both earlier and newer versions of the ChatGPT, Claude, and Gemini families. Performance was analyzed using mean absolute error (MAE), Spearman’s correlation, Lin’s concordance correlation coefficient (CCC), systematic bias (Bland-Altman analysis), and diagnostic accuracy metrics (sensitivity/specificity) based on the clinical treatment need threshold of IOTN-AC ≥4.0. Results: General dentists demonstrated the closest agreement with the reference standard, exhibiting the lowest MAE (0.761) and the highest CCC (0.901, 95% confidence interval: 0.796-0.948). Conversely, all AI models exhibited a positive bias by systematically overestimating aesthetic impairment (+0.761 to +3.361). In the longitudinal analysis, although newer iterations of LLMs showed improved ranking performance (correlation), they regressed in absolute accuracy compared to earlier versions, exhibiting increased MAE values (e.g., Gemini 3 Pro MAE increased from 2.529 to 3.406; p=0.023). At the IOTN-AC ≥4.0 threshold, while LLMs demonstrated high sensitivity (84.6-100.0%), they critically failed to distinguish healthy cases, resulting in extremely low specificity (e.g., 5.9% for Gemini 3 Pro). Conclusion: Within the limitations of this study’s sample and pooled evaluation design, general dentists demonstrated higher reliability and closer agreement with the reference standard than AI models in orthodontic aesthetic assessment. The architectural advancements in newer model versions did not translate positively into absolute clinical accuracy, creating a performance paradox. Although current LLMs show potential as preliminary screening tools due to their high sensitivity, they carry a significant risk of over-diagnosis owing to their low specificity, remaining far from being independent clinical decision support systems.
Related
- Clinical Factors Influencing Cosmetic Lip Reduction Recommendations: A Comparative Study of Orthodontists and Oral and Maxillofacial Surgeons
- Multimedia versus audio-based education for improving dentists’ nutrition-related knowledge and behavior: A randomized comparative study
- Oncoplastic Surgery Versus Total Breast Reconstruction in Locally Advanced and Multicentric Carcinomas. A Matched Comparative Cohort Study of Oncologic and Aesthetic Outcomes
- Comparative Two-Year Outcomes of Three Different Three-Piece Intraocular Lenses in Flanged Intrascleral Fixation
- Comparative effectiveness of exercise and self-management interventions delivered through different modes of digital support on pain and patient-reported physical function in people with knee osteoarthritis: a systematic review and network meta-analysis of randomised controlled trials
BibTeX
@article{oncu2026comparative,
title = {{Comparative Analysis of Aesthetic Judgment between Orthodontists, General Dentists, and Different Versions of Large Language Models Using the IOTN-AC Index}},
author = {Rümeysa Cemre ÖNCÜ and Aylin PAŞAOĞLU BOZKURT},
journal = {Bezmiâlem Science},
year = {2026},
month = oct,
doi = {10.14235/bas.galenos.2026.82085},
url = {https://doi.org/10.14235/bas.galenos.2026.82085},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - Comparative Analysis of Aesthetic Judgment between Orthodontists, General Dentists, and Different Versions of Large Language Models Using the IOTN-AC Index
AU - Rümeysa Cemre ÖNCÜ
AU - Aylin PAŞAOĞLU BOZKURT
PY - 2026
DA - 2026/10/26
JO - Bezmiâlem Science
DO - 10.14235/bas.galenos.2026.82085
UR - https://doi.org/10.14235/bas.galenos.2026.82085
AB - Objective: This study aimed to compare the performance and diagnostic accuracy of general dentists and different artificial intelligence (AI)-based large language models (LLMs), including their older and newer versions, in photograph-based IOTN-AC index assessments, using orthodontists’ evaluations as the reference standard. Methods: The mean IOTN-AC scores derived from orthodontists were established as the reference (gold) standard. Cases were evaluated by general dentists and both earlier and newer versions of the ChatGPT, Claude, and Gemini families. Performance was analyzed using mean absolute error (MAE), Spearman’s correlation, Lin’s concordance correlation coefficient (CCC), systematic bias (Bland-Altman analysis), and diagnostic accuracy metrics (sensitivity/specificity) based on the clinical treatment need threshold of IOTN-AC ≥4.0. Results: General dentists demonstrated the closest agreement with the reference standard, exhibiting the lowest MAE (0.761) and the highest CCC (0.901, 95% confidence interval: 0.796-0.948). Conversely, all AI models exhibited a positive bias by systematically overestimating aesthetic impairment (+0.761 to +3.361). In the longitudinal analysis, although newer iterations of LLMs showed improved ranking performance (correlation), they regressed in absolute accuracy compared to earlier versions, exhibiting increased MAE values (e.g., Gemini 3 Pro MAE increased from 2.529 to 3.406; p=0.023). At the IOTN-AC ≥4.0 threshold, while LLMs demonstrated high sensitivity (84.6-100.0%), they critically failed to distinguish healthy cases, resulting in extremely low specificity (e.g., 5.9% for Gemini 3 Pro). Conclusion: Within the limitations of this study’s sample and pooled evaluation design, general dentists demonstrated higher reliability and closer agreement with the reference standard than AI models in orthodontic aesthetic assessment. The architectural advancements in newer model versions did not translate positively into absolute clinical accuracy, creating a performance paradox. Although current LLMs show potential as preliminary screening tools due to their high sensitivity, they carry a significant risk of over-diagnosis owing to their low specificity, remaining far from being independent clinical decision support systems.
SN - 2148-2373
ER -