Benchmarking Gemini 1.5 Pro in Pediatric Emergency Radiology: Sensitivity Gap and Physiological Bias in the Immature Skeleton.
- 1Orthopedic Unit, Sant'Andrea Hospital, University of Rome La Sapienza, Via di Grottarossa 1035, 00189 Rome, Italy.
Abstract
Background/Objectives: To assess the diagnostic performance of Gemini 1.5 Pro, a general-purpose Large Multimodal Model (LMM), as a zero-shot, smartphone-based decision support tool to distinguish pediatric fractures from healthy controls. Methods: A retrospective diagnostic accuracy study was conducted on 182 pediatric radiographs (112 confirmed fractures, 70 healthy controls) using the Gemini 1.5 Pro mobile interface with a 4-question prompt protocol (Detection, Characterization, Skeletal Maturity, Management). Diagnostic metrics were calculated with 95% Confidence Intervals (95% CI). Ground truth was established by a senior musculoskeletal radiologist. Results: The model showed an overall sensitivity of 90.2% (95% CI: 83.1-95.0%) and specificity of 87.1% (95% CI: 77.0-93.9%). A significant sensitivity gap occurred between displaced (100.0%; 95% CI: 91.4-100.0%) and non-displaced fractures (84.5%; 95% CI: 74.0-92.0%; p = 0.007). Qualitative taxonomy revealed a primary physiological bias leading to a 12.8% false-positive rate due to growth plate misinterpretation, alongside minor characterization and localization errors (3.6% of correctly identified fractures). When fractures were correctly identified, proposed management strategies aligned with international standards in 95.0% of cases. Inter-observer reliability was excellent (weighted k = 0.96). Conclusions: Gemini 1.5 Pro displays robust clinical reasoning for obvious trauma. However, its zero-shot performance is limited by subtle morphological disruptions and developmental anatomical nuances such as open growth plates. It shows promise as a mobile clinical decision support system for non-specialists, but expert human supervision remains necessary to ensure safety in the complex radiographic landscape of the immature skeleton.
Related
- Artificial Intelligence in Emergency Radiology: Current Applications, Challenges, and Future Directions.
- Web-Based Digital Radiology Cuts Emergency Head CT Time in Limited-Resource Hospital
- Automated Calculation of the Pediatric Emergency Care Applied Research Network (PECARN) Intra-Abdominal Injury Rule from Unstructured Emergency Department Notes Using a Large Language Model
- Emergency Department Pediatric Early Warning System for Identifying Resuscitation-Defined Critical Illness in Children: A Prospective Diagnostic Classification Study in Vietnam
- A Community-Based Implementation of The Family Check-up Is Associated with Lower Pediatric Emergency Room Utilization and Acute Care Services: A Health Service Utilization and Cost Analysis Study
BibTeX
@article{benelli2026benchmarking,
title = {{Benchmarking Gemini 1.5 Pro in Pediatric Emergency Radiology: Sensitivity Gap and Physiological Bias in the Immature Skeleton.}},
author = {Cristiano Benelli and Luca Labianca and Vito Caiolo and Gabriele Altavilla and Nicola Maffulli},
journal = {Diagnostics (Basel, Switzerland)},
year = {2026},
month = sep,
doi = {10.3390/diagnostics16193112},
url = {https://europepmc.org/article/MED/42882268},
note = {CC BY},
}
RIS
TY - JOUR
TI - Benchmarking Gemini 1.5 Pro in Pediatric Emergency Radiology: Sensitivity Gap and Physiological Bias in the Immature Skeleton.
AU - Cristiano Benelli
AU - Luca Labianca
AU - Vito Caiolo
AU - Gabriele Altavilla
AU - Nicola Maffulli
PY - 2026
DA - 2026/09/24
JO - Diagnostics (Basel, Switzerland)
DO - 10.3390/diagnostics16193112
UR - https://europepmc.org/article/MED/42882268
AB - Background/Objectives: To assess the diagnostic performance of Gemini 1.5 Pro, a general-purpose Large Multimodal Model (LMM), as a zero-shot, smartphone-based decision support tool to distinguish pediatric fractures from healthy controls. Methods: A retrospective diagnostic accuracy study was conducted on 182 pediatric radiographs (112 confirmed fractures, 70 healthy controls) using the Gemini 1.5 Pro mobile interface with a 4-question prompt protocol (Detection, Characterization, Skeletal Maturity, Management). Diagnostic metrics were calculated with 95% Confidence Intervals (95% CI). Ground truth was established by a senior musculoskeletal radiologist. Results: The model showed an overall sensitivity of 90.2% (95% CI: 83.1-95.0%) and specificity of 87.1% (95% CI: 77.0-93.9%). A significant sensitivity gap occurred between displaced (100.0%; 95% CI: 91.4-100.0%) and non-displaced fractures (84.5%; 95% CI: 74.0-92.0%; p = 0.007). Qualitative taxonomy revealed a primary physiological bias leading to a 12.8% false-positive rate due to growth plate misinterpretation, alongside minor characterization and localization errors (3.6% of correctly identified fractures). When fractures were correctly identified, proposed management strategies aligned with international standards in 95.0% of cases. Inter-observer reliability was excellent (weighted k = 0.96). Conclusions: Gemini 1.5 Pro displays robust clinical reasoning for obvious trauma. However, its zero-shot performance is limited by subtle morphological disruptions and developmental anatomical nuances such as open growth plates. It shows promise as a mobile clinical decision support system for non-specialists, but expert human supervision remains necessary to ensure safety in the complex radiographic landscape of the immature skeleton.
SN - 2075-4418
ER -