Does predictive accuracy reflect anatomical attention in CVM staging? evidence from explainable held-out vendor evaluation.
7 affiliations
- 1Department of Orthodontics, Ningbo Stomatology Hospital, Ningbo, Zhejiang, China.
- 2Engineering Laboratory for Biomaterials and Tissue Regeneration, Ningbo Stomatology Hospital, Ningbo, Zhejiang, China.
- 3GreenOne Technology Consulting Co., Ltd, Jinan, Shandong, China.
- 4Department of Orthodontics, Jinan Stomatological Hospital, Jinan, Shandong Province, China.
- 5Shandong Provincial Health Commission Key Laboratory of Oral Diseases and Tissue Regeneration, Jinan Stomatological Hospital, Jinan, Shandong, China.
- 6Shandong Medical and Pharmaceutical University, Jinan, Shandong, China.
- 7Department of Orthodontics, HuiZhou Stomatological Hospital, Huizhou, Guangdong, China.
Abstract
Deep learning has shown promising performance for cervical vertebral maturation (CVM) staging, but reported accuracies vary widely across studies. Differences in image preprocessing and imaging systems may affect model performance, while the black-box nature of deep learning makes it unclear whether predictions are based on clinically relevant cervical vertebral morphology. A total of 908 lateral cephalometric radiographs from four imaging vendors were retrospectively collected. Of these, 708 images from three vendors were used for model development and internal validation, while 200 images from a held-out vendor were reserved for external testing. ResNet-50 and Swin Transformer were evaluated across six preprocessing pipelines. Classification accuracy, Macro-F1, adjacent-stage accuracy, and severe misclassification were assessed. Grad-CAM and attention rollout were used for qualitative interpretability analysis. External exact-stage accuracy was consistently lower than internal performance. The highest external accuracies were 0.47 for ResNet-50 with min-max plus Z-score normalization and 0.42 for Swin Transformer with min-max plus CLAHE. Adjacent-stage accuracy was substantially higher, but severe misclassifications persisted. Preprocessing effects were architecture-dependent, with no single strategy consistently improving both models. Explainability maps showed variable attention alignment with C2-C4 and occasional emphasis on non-classical anatomical or imaging-dependent regions. Deep learning performance for CVM staging did not generalize consistently across imaging conditions, and preprocessing influenced different architectures differently. These findings suggest that predictions from machine learning-based CVM models should be interpreted with caution, particularly when applied to data acquired under different imaging conditions.
Related
- Hybrid Explainable Deep Ensemble Framework with self-attention mechanism for early prediction of depression severity using PHQ-9 clinical and behavioral data
- Explainable deep graph attention learning with enhanced residual SwinV2 transformer for interpretable facial paralysis severity assessment
- An explainable machine-learning framework for seasonal early warning of West Nile virus using predictive climate data
- Comments on "Predictive Accuracy of Early Warning Score Systems for Detecting Critically Ill Patients in an Outpatient Setting."
- Computer Vision for Real-Time Pixel-Level Anatomical Segmentation in Neurosurgery: First-in-Human Clinical Evaluation and Iterative Development (IDEAL Stage 1)
BibTeX
@article{jiang2026does,
title = {{Does predictive accuracy reflect anatomical attention in CVM staging? evidence from explainable held-out vendor evaluation.}},
author = {Yanjun Jiang and Yang Lyu and Fengcong Wang and Ping Liu and Yilan Yang},
journal = {Frontiers in medicine},
year = {2026},
month = oct,
doi = {10.3389/fmed.2026.1971090},
url = {https://europepmc.org/article/MED/42953228},
note = {CC BY},
}
RIS
TY - JOUR
TI - Does predictive accuracy reflect anatomical attention in CVM staging? evidence from explainable held-out vendor evaluation.
AU - Yanjun Jiang
AU - Yang Lyu
AU - Fengcong Wang
AU - Ping Liu
AU - Yilan Yang
PY - 2026
DA - 2026/10/14
JO - Frontiers in medicine
DO - 10.3389/fmed.2026.1971090
UR - https://europepmc.org/article/MED/42953228
AB - Deep learning has shown promising performance for cervical vertebral maturation (CVM) staging, but reported accuracies vary widely across studies. Differences in image preprocessing and imaging systems may affect model performance, while the black-box nature of deep learning makes it unclear whether predictions are based on clinically relevant cervical vertebral morphology. A total of 908 lateral cephalometric radiographs from four imaging vendors were retrospectively collected. Of these, 708 images from three vendors were used for model development and internal validation, while 200 images from a held-out vendor were reserved for external testing. ResNet-50 and Swin Transformer were evaluated across six preprocessing pipelines. Classification accuracy, Macro-F1, adjacent-stage accuracy, and severe misclassification were assessed. Grad-CAM and attention rollout were used for qualitative interpretability analysis. External exact-stage accuracy was consistently lower than internal performance. The highest external accuracies were 0.47 for ResNet-50 with min-max plus Z-score normalization and 0.42 for Swin Transformer with min-max plus CLAHE. Adjacent-stage accuracy was substantially higher, but severe misclassifications persisted. Preprocessing effects were architecture-dependent, with no single strategy consistently improving both models. Explainability maps showed variable attention alignment with C2-C4 and occasional emphasis on non-classical anatomical or imaging-dependent regions. Deep learning performance for CVM staging did not generalize consistently across imaging conditions, and preprocessing influenced different architectures differently. These findings suggest that predictions from machine learning-based CVM models should be interpreted with caution, particularly when applied to data acquired under different imaging conditions.
SN - 2296-858X
ER -