Quality Assessment and Model Training Impact of Synthetic Images in Medical Diagnostic Imaging: Systematic Review.
6 affiliations
- 1Information Systems Department, Catalan Health Service (CatSalut), Gran Via de les Corts Catalanes, 587, Barcelona, 08007, Spain, +34 93 403 85 85.
- 2Digitalization for the Sustainability of the Heathcare System (DS3) Research Group, Barcelona, Spain.
- 3Fundació Tic Salut i Social (FTSS), Barcelona, Spain.
- 4Process Unit, Directorate of Quality, Processes and Innovation, Vall d'Hebron University Hospital, Barcelona, Spain.
- 5Faculty of Engineering, Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark.
- 6Biostatistics Unit, Medical School, Universitat Autònoma de Barcelona, Bellaterra, Spain.
Abstract
Background
AI has increasingly been used to generate synthetic images to overcome the scarcity of large, regulatory-compliant medical image datasets. However, their quality and actual utility for algorithm development across deep learning architectures, medical settings, and image types have been poorly assessed.Objective
This study aimed to review the use of synthetic images for algorithm development and to assess their quality and effects on downstream algorithms trained on these images, according to AI architecture, medical setting, and image type.Methods
The authors conducted a systematic review of studies published between January 2020 and September 2025. PubMed, Scopus, IEEE Xplore, and Embase were searched for peer-reviewed studies evaluating synthetic images generated from human medical imaging data. Data were extracted for imaging modalities, generative architectures, image-quality assessment methods, downstream model performance, explainability approaches, reproducibility practices, and risk of bias. Given substantial heterogeneity and incomplete statistical reporting, findings were synthesized descriptively.Results
A total of 160 studies were included. Generative adversarial network-based architectures dominated image generation (119 studies, 74.4%), although hybrid and diffusion-based approaches became increasingly represented in recent years. Image-quality evaluation was highly heterogeneous: 50 studies included human-reader evidence, and 101 reported an automated image-quality assessment. Synthetic-only training generally underperformed real-data training, including a median Dice similarity coefficient change of -10.2% for segmentation. Adding synthetic data to real training sets was associated with median improvements in classification precision (+7.8%), recall (+5.3%), F₁-score (+5.5%), accuracy (+4.3%), and area under the curve (+4.0%), while the median Dice coefficient improvement was +1.2%. Explainability evidence was reported in 68 (42.5%) studies and was rarely validated against expert annotations. Reproducibility reporting was inconsistent, and only 38% (60/160) of studies were judged to have low overall risk of bias.Conclusions
Unlike previous reviews restricted to particular diseases, imaging modalities, or generative architectures, this review provides a cross-domain quantitative synthesis of both synthetic-image quality assessment and downstream model performance. Synthetic images currently appear more valuable as a complement to real training data than as a replacement, with modest improvements in classification performance, smaller gains in segmentation, and generally poorer performance with synthetic-only training. For researchers, developers, and regulators, these findings highlight the need for standardized multimethod image-quality evaluation, complete reporting, and rigorous external validation before synthetic-image pipelines are translated into clinical AI applications.
Related
- Effects of dual-task training on health-related quality of life and cognitive function in people with Parkinson’s disease: a systematic review and meta-analysis
- Artificial Intelligence-Based Burn Assessment Demonstrates High Diagnostic Accuracy: Systematic Review and Meta-Analysis
- Training and Assessment of Non-Technical Human Factor Skills Among Anesthesiology Residents: An Exploratory Educational Quality Improvement Initiative.
- Evidence quality assessment of traditional Chinese medicine and integrated traditional Chinese and Western medicine for acute upper respiratory infection in children: an umbrella review of systematic reviews and meta-analyses
- Quality assessment of synthetic data in healthcare: a critical appraisal with a focus on subgroup bias.
BibTeX
@article{folgueraprofitos2026quality,
title = {{Quality Assessment and Model Training Impact of Synthetic Images in Medical Diagnostic Imaging: Systematic Review.}},
author = {Júlia Folguera-Profitós and Marina Ramiro-Pareta and Gerard Carot-Sans and Alba Jiménez-Rueda and Júlia Aznar-Roig and Didier Domínguez-Herrera and Maria Mercedes Nogueras and Esmaeil Nadimi and Ferran Torres and Jordi Piera-Jiménez},
journal = {JMIR AI},
year = {2026},
month = oct,
doi = {10.2196/81472},
url = {https://europepmc.org/article/MED/42879762},
note = {CC BY},
}
RIS
TY - JOUR
TI - Quality Assessment and Model Training Impact of Synthetic Images in Medical Diagnostic Imaging: Systematic Review.
AU - Júlia Folguera-Profitós
AU - Marina Ramiro-Pareta
AU - Gerard Carot-Sans
AU - Alba Jiménez-Rueda
AU - Júlia Aznar-Roig
AU - Didier Domínguez-Herrera
AU - Maria Mercedes Nogueras
AU - Esmaeil Nadimi
AU - Ferran Torres
AU - Jordi Piera-Jiménez
PY - 2026
DA - 2026/10/15
JO - JMIR AI
DO - 10.2196/81472
UR - https://europepmc.org/article/MED/42879762
AB - Background AI has increasingly been used to generate synthetic images to overcome the scarcity of large, regulatory-compliant medical image datasets. However, their quality and actual utility for algorithm development across deep learning architectures, medical settings, and image types have been poorly assessed.Objective This study aimed to review the use of synthetic images for algorithm development and to assess their quality and effects on downstream algorithms trained on these images, according to AI architecture, medical setting, and image type.Methods The authors conducted a systematic review of studies published between January 2020 and September 2025. PubMed, Scopus, IEEE Xplore, and Embase were searched for peer-reviewed studies evaluating synthetic images generated from human medical imaging data. Data were extracted for imaging modalities, generative architectures, image-quality assessment methods, downstream model performance, explainability approaches, reproducibility practices, and risk of bias. Given substantial heterogeneity and incomplete statistical reporting, findings were synthesized descriptively.Results A total of 160 studies were included. Generative adversarial network-based architectures dominated image generation (119 studies, 74.4%), although hybrid and diffusion-based approaches became increasingly represented in recent years. Image-quality evaluation was highly heterogeneous: 50 studies included human-reader evidence, and 101 reported an automated image-quality assessment. Synthetic-only training generally underperformed real-data training, including a median Dice similarity coefficient change of -10.2% for segmentation. Adding synthetic data to real training sets was associated with median improvements in classification precision (+7.8%), recall (+5.3%), F₁-score (+5.5%), accuracy (+4.3%), and area under the curve (+4.0%), while the median Dice coefficient improvement was +1.2%. Explainability evidence was reported in 68 (42.5%) studies and was rarely validated against expert annotations. Reproducibility reporting was inconsistent, and only 38% (60/160) of studies were judged to have low overall risk of bias.Conclusions Unlike previous reviews restricted to particular diseases, imaging modalities, or generative architectures, this review provides a cross-domain quantitative synthesis of both synthetic-image quality assessment and downstream model performance. Synthetic images currently appear more valuable as a complement to real training data than as a replacement, with modest improvements in classification performance, smaller gains in segmentation, and generally poorer performance with synthetic-only training. For researchers, developers, and regulators, these findings highlight the need for standardized multimethod image-quality evaluation, complete reporting, and rigorous external validation before synthetic-image pipelines are translated into clinical AI applications.
SN - 2817-1705
ER -