Detection of Pathological Voice Samples Using a Text-to-Speech-Inspired Approach With Multiple Model Architectures: Method Development and Comparative Evaluation Study.
4 affiliations
- 1Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Essen, Hufefelandstraße 55, Essen, 45147, Germany, 49 20172377817.
- 2Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen, Essen, Germany.
- 3Department of Otorhinolaryngology, Head and Neck Surgery, University Hospital Essen, University Duisburg-Essen, Essen, Germany.
- 4University of Lübeck, Institute of Medical Informatics, Lübeck, Germany.
Abstract
Background
Voice disorders are common, yet access to timely specialist assessment is often limited. Automated analysis of short voice recordings may support early screening, but achieving robust performance on heterogeneous clinical datasets remains challenging.Objective
This study aims to investigate the impact of synthetic speech augmentation on voice disorder recognition and classification using the Saarbrücken Voice Database (SVD) and to develop an adaptive ensemble with learnable weighting and optional sex embedding.Methods
Sentence recordings from the SVD were used for binary voice disorder recognition (VDR, n=1988) and 4-class voice disorder classification (VDC; n=1199). Two text-to-speech (TTS) models, XTTS and Kartoffelbox, generated 1 synthetic sample for each training instance. Synthetic speech quality was assessed in a blinded listening evaluation with 40 samples per TTS model. Baseline models included support vector machine (SVM), ResNet-18, long short-term memory, and transformer-based architectures, including Hidden-Unit BERT (HuBERT), Wav2Vec 2.0, and WavLM. Ensemble learning and learnable fusion (LF), with and without a sex embedding (LFS), were evaluated using bootstrapped performance estimates and standard classification metrics. Area under the receiver operating characteristic curve (AUC) was reported for VDR.Results
In the blinded evaluation, Kartoffelbox-generated synthetic samples were less often identified as synthetic than XTTS-generated synthetic samples (10/19, 52.6% vs 17/19, 89.5%). In pooled original and synthetic ratings, Kartoffelbox showed higher pathological-status identification than XTTS for pathological samples (22/29, 75.9% vs 19/29, 65.5%) and nonpathological samples (8/10, 80.0% vs 6/10, 60.0%), as well as higher diagnostic-class assignment (20/39, 51.3% vs 14/39, 35.9%). Among the baseline models, HuBERT showed the most balanced VDR performance (accuracy=0.85, F₁-score=0.84). Kartoffelbox-based augmentation further improved performance, with HuBERT achieving 0.88 sensitivity, 0.87 accuracy, and 0.85 F₁-score. LF showed minor improvements in threshold-dependent metrics (accuracy=0.87, F₁-score=0.86), while LFS further balanced sensitivity and specificity (accuracy=0.88, AUC=0.92). The equal-weight ensemble achieved the highest AUC among ensemble approaches (AUC=0.93). For VDC, HuBERT was the strongest baseline (accuracy=0.72, median F₁-score=0.63), and LF achieved the highest overall fusion performance (accuracy=0.73, F₁-score=0.66), while laryngitis remained one of the more challenging classes.Conclusions
High-quality synthetic speech augmentation can improve automated voice disorder analysis across heterogeneous clinical data, with effectiveness varying by the TTS model. Multimodel fusion of complementary architectures provided robust performance for screening and multiclass classification, with the strongest approach varying by metric and task. These findings support further evaluation of AI-assisted preassessment in clinical and telemedicine settings.
Related
- Multi-Version Evaluation of Deep Learning Architectures for Tibial Plateau Fracture Detection and Deployment in a Web-Based Clinical Support System.
- Comparative evaluation of Allplex RV master assay and biofire respiratory panel 2.1 plus for detection of respiratory viruses
- Comparative evaluation of convolutional and transformer-based object detection models for brain tumor localization in MRI
- Comparative evaluation of fractal analysis and canny edge-detection algorithms for the quantification and classification of bone trabecular patterns in cone-beam computed tomography
- Development and comparative pharmacokinetic evaluation of iodine-125- and fluorine-18-labelled analogues of 4-(benzo[d]thiazol-2-yl)phenylboronic acid for boron neutron capture therapy
BibTeX
@article{walter2026detection,
title = {{Detection of Pathological Voice Samples Using a Text-to-Speech-Inspired Approach With Multiple Model Architectures: Method Development and Comparative Evaluation Study.}},
author = {Marie Sophie Walter and Muhittin Demir and Mattias Heinrich and Felix Nensa and Stephan Lang and Nadia Sadok and René Hosch},
journal = {Journal of medical Internet research},
year = {2026},
month = oct,
doi = {10.2196/94285},
url = {https://europepmc.org/article/MED/42896146},
note = {CC BY},
}
RIS
TY - JOUR
TI - Detection of Pathological Voice Samples Using a Text-to-Speech-Inspired Approach With Multiple Model Architectures: Method Development and Comparative Evaluation Study.
AU - Marie Sophie Walter
AU - Muhittin Demir
AU - Mattias Heinrich
AU - Felix Nensa
AU - Stephan Lang
AU - Nadia Sadok
AU - René Hosch
PY - 2026
DA - 2026/10/19
JO - Journal of medical Internet research
DO - 10.2196/94285
UR - https://europepmc.org/article/MED/42896146
AB - Background Voice disorders are common, yet access to timely specialist assessment is often limited. Automated analysis of short voice recordings may support early screening, but achieving robust performance on heterogeneous clinical datasets remains challenging.Objective This study aims to investigate the impact of synthetic speech augmentation on voice disorder recognition and classification using the Saarbrücken Voice Database (SVD) and to develop an adaptive ensemble with learnable weighting and optional sex embedding.Methods Sentence recordings from the SVD were used for binary voice disorder recognition (VDR, n=1988) and 4-class voice disorder classification (VDC; n=1199). Two text-to-speech (TTS) models, XTTS and Kartoffelbox, generated 1 synthetic sample for each training instance. Synthetic speech quality was assessed in a blinded listening evaluation with 40 samples per TTS model. Baseline models included support vector machine (SVM), ResNet-18, long short-term memory, and transformer-based architectures, including Hidden-Unit BERT (HuBERT), Wav2Vec 2.0, and WavLM. Ensemble learning and learnable fusion (LF), with and without a sex embedding (LFS), were evaluated using bootstrapped performance estimates and standard classification metrics. Area under the receiver operating characteristic curve (AUC) was reported for VDR.Results In the blinded evaluation, Kartoffelbox-generated synthetic samples were less often identified as synthetic than XTTS-generated synthetic samples (10/19, 52.6% vs 17/19, 89.5%). In pooled original and synthetic ratings, Kartoffelbox showed higher pathological-status identification than XTTS for pathological samples (22/29, 75.9% vs 19/29, 65.5%) and nonpathological samples (8/10, 80.0% vs 6/10, 60.0%), as well as higher diagnostic-class assignment (20/39, 51.3% vs 14/39, 35.9%). Among the baseline models, HuBERT showed the most balanced VDR performance (accuracy=0.85, F₁-score=0.84). Kartoffelbox-based augmentation further improved performance, with HuBERT achieving 0.88 sensitivity, 0.87 accuracy, and 0.85 F₁-score. LF showed minor improvements in threshold-dependent metrics (accuracy=0.87, F₁-score=0.86), while LFS further balanced sensitivity and specificity (accuracy=0.88, AUC=0.92). The equal-weight ensemble achieved the highest AUC among ensemble approaches (AUC=0.93). For VDC, HuBERT was the strongest baseline (accuracy=0.72, median F₁-score=0.63), and LF achieved the highest overall fusion performance (accuracy=0.73, F₁-score=0.66), while laryngitis remained one of the more challenging classes.Conclusions High-quality synthetic speech augmentation can improve automated voice disorder analysis across heterogeneous clinical data, with effectiveness varying by the TTS model. Multimodel fusion of complementary architectures provided robust performance for screening and multiclass classification, with the strongest approach varying by metric and task. These findings support further evaluation of AI-assisted preassessment in clinical and telemedicine settings.
SN - 1439-4456
SN - 1438-8871
ER -