Evaluating synthetic-data fidelity in two-group biomedical studies: a multidimensional validation framework
- 1University of Guelph
Abstract
Synthetic data increasingly support model development and privacy-conscious sharing in biomedicine. Two-group studies require synthetic data to reproduce within-group structure and between-group differences, yet marginal agreement or predictive performance may obscure multivariate and conditional-dependence changes. We present a multidimensional validation framework for class-conditional synthetic data and apply it to three datasets spanning sample-size and dimensionality regimes. Two controls and four generators spanning mixture, interpolation, hybrid, and latent-variable architectures (GMM, SMOTE, GMM-SMOTE, and CVAE, respectively) were assessed using predictive utility, real-synthetic distinguishability, marginal agreement, PCA and t-SNE geometry, pairwise dependence, and Graphical LASSO networks. Noise perturbation, within-class permutation, and reverse ablation probed the sources of real-synthetic distinguishability. Across 18 dataset-method comparisons, discriminator AUC ranged from 0.55 to 1.00, while mean feature-level KS statistics ranged from 0.027 to 0.282. Thus, strong performance under individual criteria coexisted with detectable differences and lost or synthetic-only dependencies. Rather than assigning a single fidelity score, the framework supports multidimensional fidelity reporting as a minimum standard for shared synthetic biomedical data.
Related
- Evaluating a Large Language Model for Emergency Triage Using Synthetic Clinical Dialogues: Toward a Multimodal Clinical Safety Framework
- A Multidimensional Disease Weighted Index (DWI) for Risk Assessment in COPD: A Proposed Conceptual Framework and Pre-Specified Validation Roadmap
- Development and psychometric validation of a multidimensional menstrual hygiene scale
- Multi-Source Data Fusion Integrating HPLC, GC-IMS, and Multidimensional Biomimetic Technologies for Evaluating the Effects of Different Processing Degrees on the Quality of Salt-Processed Semen Cuscutae.
- Effect Estimates, Effect Sizes, Confidence Intervals, and P-values: A Framework for Reporting and Interpreting Biomedical Research
BibTeX
@misc{tran2026evaluating,
title = {{Evaluating synthetic-data fidelity in two-group biomedical studies: a multidimensional validation framework}},
author = {T. Tran and M. S. Ghaemi and C. S. Korosec},
howpublished = {medRxiv},
year = {2026},
month = aug,
doi = {10.64898/2026.08.31.26361878},
url = {https://www.medrxiv.org/content/10.64898/2026.08.31.26361878v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Evaluating synthetic-data fidelity in two-group biomedical studies: a multidimensional validation framework
AU - T. Tran
AU - M. S. Ghaemi
AU - C. S. Korosec
PY - 2026
DA - 2026/08/31
PB - medRxiv
DO - 10.64898/2026.08.31.26361878
UR - https://www.medrxiv.org/content/10.64898/2026.08.31.26361878v1
AB - Synthetic data increasingly support model development and privacy-conscious sharing in biomedicine. Two-group studies require synthetic data to reproduce within-group structure and between-group differences, yet marginal agreement or predictive performance may obscure multivariate and conditional-dependence changes. We present a multidimensional validation framework for class-conditional synthetic data and apply it to three datasets spanning sample-size and dimensionality regimes. Two controls and four generators spanning mixture, interpolation, hybrid, and latent-variable architectures (GMM, SMOTE, GMM-SMOTE, and CVAE, respectively) were assessed using predictive utility, real-synthetic distinguishability, marginal agreement, PCA and t-SNE geometry, pairwise dependence, and Graphical LASSO networks. Noise perturbation, within-class permutation, and reverse ablation probed the sources of real-synthetic distinguishability. Across 18 dataset-method comparisons, discriminator AUC ranged from 0.55 to 1.00, while mean feature-level KS statistics ranged from 0.027 to 0.282. Thus, strong performance under individual criteria coexisted with detectable differences and lost or synthetic-only dependencies. Rather than assigning a single fidelity score, the framework supports multidimensional fidelity reporting as a minimum standard for shared synthetic biomedical data.
ER -