Classification Performance of a Deep Learning Algorithm for Common Skin Lesions in the Emergency Department: A Multicenter Preliminary Validation Study
- 1Department of Emergency Medicine, School of Medicine, Kyungpook National University, Daegu 41944, Republic of Korea
- 2Department of Emergency Medicine, College of Medicine, Yeungnam University, Daegu 42415, Republic of Korea
- 3Department of Emergency Medicine, Daegu Catholic University Hospital, Daegu 42472, Republic of Korea
Abstract
Background/Objectives: Initial classification of skin lesions in the emergency department (ED) is difficult when dermatology consultations are limited. This study externally validated a deep learning (DL) algorithm for classifying three common ED skin lesions and explored whether algorithmic pre-classification was associated with changes in the classification performance of medical staff. Methods: A ResNet-50-based algorithm was developed using 6000 clinical images from 4815 patients. For external validation, 100 images from an independent institution were classified by the algorithm and nine medical staff members (three emergency medicine specialists, three residents, and three physician assistant [PA] nurses). Fifteen days later, the staff reclassified the same randomly reordered images with the algorithmic pre-classification provided. Results: In external validation, the algorithm showed an overall accuracy of 0.96 (95% confidence interval [CI], 0.90–0.99), lesion-specific balanced accuracies of 0.95–0.96, and a macro-average area under the receiver operating characteristic curve of 0.98 (95% CI, 0.95–1.00). Unassisted, the three PA nurses differed significantly from the algorithm for urticaria and herpes zoster (all p < 0.01, McNemar’s exact test). After receiving algorithmic assistance, all medical staff showed a balanced accuracy of 0.97 or higher for all three lesions. In the mixed-effects model, assistance was associated with higher odds of correct classification (odds ratio 25.9, 95% CI 11.7–57.2), and inter-rater agreement (Fleiss’ κ) increased from 0.70 to 0.97. Conclusions: The algorithm demonstrated high classification performance on independent external validation, and algorithmic assistance was associated with improved classification performance, with the largest absolute gains among medical staff with lower unassisted accuracy. As these findings are exploratory, they should be confirmed in randomized prospective studies.
Related
- Diagnostic Performance of a Deep Learning Model Using Phonocardiogram and Single‐Lead ECG Signals for Detecting Moderate‐or‐Greater Aortic Stenosis: Multicenter External Validation
- DeepFusion-HSK: a multimodal deep learning for subtype classification of herpes simplex keratitis with multicenter validation
- Infectious disease risk awareness and protective behaviors among adult emergency department patients: A multicenter cross-sectional study
- Ethnoracial disparities in emergency department triage are more pronounced in lower-risk visits: a multicenter study
- Emergency Department Overcrowding Despite Lower Post-Pandemic Volumes: A Multicenter Study
BibTeX
@article{lee2026classification,
title = {{Classification Performance of a Deep Learning Algorithm for Common Skin Lesions in the Emergency Department: A Multicenter Preliminary Validation Study}},
author = {Dong Eun Lee and Jong Kun Kim and Min-Seong Kwak and Kyung-Woo Lee and Sin-Youl Park},
journal = {Journal of Clinical Medicine},
year = {2026},
month = nov,
doi = {10.3390/jcm15218602},
url = {https://www.mdpi.com/2077-0383/15/21/8602},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Classification Performance of a Deep Learning Algorithm for Common Skin Lesions in the Emergency Department: A Multicenter Preliminary Validation Study
AU - Dong Eun Lee
AU - Jong Kun Kim
AU - Min-Seong Kwak
AU - Kyung-Woo Lee
AU - Sin-Youl Park
PY - 2026
DA - 2026/11/04
JO - Journal of Clinical Medicine
DO - 10.3390/jcm15218602
UR - https://www.mdpi.com/2077-0383/15/21/8602
AB - Background/Objectives: Initial classification of skin lesions in the emergency department (ED) is difficult when dermatology consultations are limited. This study externally validated a deep learning (DL) algorithm for classifying three common ED skin lesions and explored whether algorithmic pre-classification was associated with changes in the classification performance of medical staff. Methods: A ResNet-50-based algorithm was developed using 6000 clinical images from 4815 patients. For external validation, 100 images from an independent institution were classified by the algorithm and nine medical staff members (three emergency medicine specialists, three residents, and three physician assistant [PA] nurses). Fifteen days later, the staff reclassified the same randomly reordered images with the algorithmic pre-classification provided. Results: In external validation, the algorithm showed an overall accuracy of 0.96 (95% confidence interval [CI], 0.90–0.99), lesion-specific balanced accuracies of 0.95–0.96, and a macro-average area under the receiver operating characteristic curve of 0.98 (95% CI, 0.95–1.00). Unassisted, the three PA nurses differed significantly from the algorithm for urticaria and herpes zoster (all p < 0.01, McNemar’s exact test). After receiving algorithmic assistance, all medical staff showed a balanced accuracy of 0.97 or higher for all three lesions. In the mixed-effects model, assistance was associated with higher odds of correct classification (odds ratio 25.9, 95% CI 11.7–57.2), and inter-rater agreement (Fleiss’ κ) increased from 0.70 to 0.97. Conclusions: The algorithm demonstrated high classification performance on independent external validation, and algorithmic assistance was associated with improved classification performance, with the largest absolute gains among medical staff with lower unassisted accuracy. As these findings are exploratory, they should be confirmed in randomized prospective studies.
SN - 2077-0383
ER -