Similar discrimination, different behaviour under synthetic occlusion: confidence and attribution reproducibility of deep learning models for intraoral ultrasonography
- 1Yonsei University College of Dentistry
Abstract
Objectives: To determine whether similar baseline discrimination conceals different responses to synthetic occlusion, whether loss of discrimination is accompanied by reduced confidence, and whether spatial attributions are reproducible in intraoral ultrasonography models. Methods: We analysed 477 images from 40 healthy volunteers, using acquisition position to label superior and inferior subregions. Five architectures were evaluated across five training seeds on a subject-independent test set of 97 images from eight volunteers. Images underwent constant-intensity occlusion at boundaries spanning 30-70% of image depth and an area-matched band-occlusion control. Attribution analyses assessed variation across seeds and class-activation methods and compared attribution-guided insertion with random insertion. Results: Seed-mean areas under the receiver operating characteristic curve ranged from 0.958 to 0.972, with rankings reversing across seeds. EfficientNet-B0 fell below 0.60 in 49 of 50 boundary-sweep conditions, compared with zero to two for the other architectures; the difference persisted with area-matched masking. Under 50% occlusion, its mean confidence on incorrect predictions rose to 0.989 and expected calibration error from 0.094 to 0.478. Between two inference runs, EfficientNet-B0's per-seed occluded AUCs differed by up to 0.161; seed means were 0.51-0.55 in both. Attribution patterns varied across seeds and methods.
Related
- Reproducibility of Patch Test Reactions and Interpretation of Doubtful Results: A Retrospective Study of Identical Haptens Tested Simultaneously in Different Series
- Role of Ultrasonography in the Diagnosis of Splenic Diseases
- Safe Ultrasonography‐Assisted Elbow Arthroscopy Technique
- The added value of contrast-enhanced ultrasonography to BI-RADS in the evaluation of breast lesions presenting with pathological nipple discharge
- A Gastric Glomus Tumor Diagnosed Preoperatively Using Contrast-enhanced Harmonic Endoscopic Ultrasonography and Endoscopic Ultrasonography-guided Fine Needle Aspiration
BibTeX
@misc{jung2026similar,
title = {{Similar discrimination, different behaviour under synthetic occlusion: confidence and attribution reproducibility of deep learning models for intraoral ultrasonography}},
author = {G. Jung and Y.-R. Heo and K.-S. Hu and H.-J. Kim},
howpublished = {medRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.03.26367069},
url = {https://www.medrxiv.org/content/10.64898/2026.11.03.26367069v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Similar discrimination, different behaviour under synthetic occlusion: confidence and attribution reproducibility of deep learning models for intraoral ultrasonography
AU - G. Jung
AU - Y.-R. Heo
AU - K.-S. Hu
AU - H.-J. Kim
PY - 2026
DA - 2026/11/03
PB - medRxiv
DO - 10.64898/2026.11.03.26367069
UR - https://www.medrxiv.org/content/10.64898/2026.11.03.26367069v1
AB - Objectives: To determine whether similar baseline discrimination conceals different responses to synthetic occlusion, whether loss of discrimination is accompanied by reduced confidence, and whether spatial attributions are reproducible in intraoral ultrasonography models. Methods: We analysed 477 images from 40 healthy volunteers, using acquisition position to label superior and inferior subregions. Five architectures were evaluated across five training seeds on a subject-independent test set of 97 images from eight volunteers. Images underwent constant-intensity occlusion at boundaries spanning 30-70% of image depth and an area-matched band-occlusion control. Attribution analyses assessed variation across seeds and class-activation methods and compared attribution-guided insertion with random insertion. Results: Seed-mean areas under the receiver operating characteristic curve ranged from 0.958 to 0.972, with rankings reversing across seeds. EfficientNet-B0 fell below 0.60 in 49 of 50 boundary-sweep conditions, compared with zero to two for the other architectures; the difference persisted with area-matched masking. Under 50% occlusion, its mean confidence on incorrect predictions rose to 0.989 and expected calibration error from 0.094 to 0.478. Between two inference runs, EfficientNet-B0's per-seed occluded AUCs differed by up to 0.161; seed means were 0.51-0.55 in both. Attribution patterns varied across seeds and methods.
ER -