Performance of Large Language Models Across Diagnostic Tasks in Oral and Maxillofacial Pathology: A Systematic Review
4 affiliations
- 1King Abdullah International Medical Research Center, Department of Maxillofacial Surgery and Diagnostic Sciences, College of Dentistry, King Saud Bin Abdulaziz University for Health Sciences, Ministry of National Guard Health Affairs, Riyadh, Kingdom of Saudi Arabia
- 2Department of Physiology, Azra Naheed Dental College, Superior University, Lahore, Pakistan
- 3King Abdullah International Medical Research Center, College of Dentistry, King Saud Bin Abdulaziz University for Health Sciences, Ministry of National Guard Health Affairs, Riyadh, Kingdom of Saudi Arabia
- 4Department of Oral Implantology, The Second Affiliated Hospital of Harbin Medical University, Harbin, China
Abstract
Introduction: Large language models (LLMs) are increasingly evaluated for diagnostic applications; however, studies in oral and maxillofacial pathology (OMFP) assess materially different diagnostic tasks and outcomes. This systematic review synthesized evidence according to the specific task evaluated.
Methods: PubMed, Web of Science, and Cochrane Central were searched without language or date restrictions, supplemented by gray-literature screening and citation tracking. Studies evaluating defined OMFP diagnostic tasks using task-appropriate outcomes were included. Screening, data extraction, and quality assessment were independently performed by 2 reviewers. Owing to clinical and methodological heterogeneity, evidence was synthesized narratively by diagnostic task.
Results: Sixteen studies published between 2024 and 2025 were included. Seven evaluated single-diagnosis identification, 6 differential-diagnosis generation or ranking, 5 binary or multiclass classification/detection, and 1 structured diagnostic report generation; counts were non-mutually exclusive. Exact-diagnosis performance varied considerably and did not demonstrate consistent specialist equivalence. Differential-diagnosis studies generally reported higher Top-k inclusion than Top-1 accuracy, but these outcomes did not establish exact diagnostic correctness. Classification studies frequently reported high specificity but variable sensitivity. The single report-generation study reported item-level rather than whole-case diagnostic outcomes.
Conclusion: Large language models may support selected OMFP tasks, particularly differential-diagnosis generation. However, exact-diagnosis performance remains inconsistent, classification sensitivity is task-dependent, and evidence for structured report generation is limited. Current evidence does not support a single overall estimate of diagnostic performance or autonomous diagnostic use.
Related
- Sycophancy in Large Language Models in Oral and Maxillofacial Radiology: The Impact of Misleading Context on Image- and Text-Based Diagnostic Scenarios
- Expert-Guided Visual Correction for Characterizing Diagnostic Performance and Error Patterns of Multimodal Large Language Models Using Periodontal In-Service Examination Images
- Diagnostic and Triage Performance of Large Language Models in Hand and Wrist Complaints: A Standardised Vignette Evaluation
- From underdiagnosis to scalable screening: Applications of artificial intelligence for vertebral compression fracture detection – a systematic review and meta-analysis of diagnostic performance
- Biomarkers in Acute Mesenteric Ischemia: A Systematic Review of Diagnostic Performance and Injury-Related Signals
BibTeX
@article{shujaat2026performance,
title = {{Performance of Large Language Models Across Diagnostic Tasks in Oral and Maxillofacial Pathology: A Systematic Review}},
author = {Sohaib Shujaat and Marryam Riaz and Hisham Albrikeet and Hongyang Ma},
journal = {Pathobiology},
year = {2026},
month = oct,
doi = {10.1159/pat/aehag008},
url = {https://doi.org/10.1159/pat/aehag008},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Performance of Large Language Models Across Diagnostic Tasks in Oral and Maxillofacial Pathology: A Systematic Review
AU - Sohaib Shujaat
AU - Marryam Riaz
AU - Hisham Albrikeet
AU - Hongyang Ma
PY - 2026
DA - 2026/10/16
JO - Pathobiology
DO - 10.1159/pat/aehag008
UR - https://doi.org/10.1159/pat/aehag008
AB - Introduction: Large language models (LLMs) are increasingly evaluated for diagnostic applications; however, studies in oral and maxillofacial pathology (OMFP) assess materially different diagnostic tasks and outcomes. This systematic review synthesized evidence according to the specific task evaluated. Methods: PubMed, Web of Science, and Cochrane Central were searched without language or date restrictions, supplemented by gray-literature screening and citation tracking. Studies evaluating defined OMFP diagnostic tasks using task-appropriate outcomes were included. Screening, data extraction, and quality assessment were independently performed by 2 reviewers. Owing to clinical and methodological heterogeneity, evidence was synthesized narratively by diagnostic task. Results: Sixteen studies published between 2024 and 2025 were included. Seven evaluated single-diagnosis identification, 6 differential-diagnosis generation or ranking, 5 binary or multiclass classification/detection, and 1 structured diagnostic report generation; counts were non-mutually exclusive. Exact-diagnosis performance varied considerably and did not demonstrate consistent specialist equivalence. Differential-diagnosis studies generally reported higher Top-k inclusion than Top-1 accuracy, but these outcomes did not establish exact diagnostic correctness. Classification studies frequently reported high specificity but variable sensitivity. The single report-generation study reported item-level rather than whole-case diagnostic outcomes. Conclusion: Large language models may support selected OMFP tasks, particularly differential-diagnosis generation. However, exact-diagnosis performance remains inconsistent, classification sensitivity is task-dependent, and evidence for structured report generation is limited. Current evidence does not support a single overall estimate of diagnostic performance or autonomous diagnostic use.
SN - 1015-2008
SN - 1423-0291
ER -