An artificial intelligence-based audio recognition system for dysarthria home-based rehabilitation: A proof-of-concept study
6 affiliations
- 1Department of Future Technology & Innovation - Data, AI & Innovation, Synapxe, Singapore, Singapore.
- 2Rehabilitation, Ng Teng Fong General Hospital and Jurong Community Hospital, National University Health System, Singapore, Singapore.
- 3Department of Post-Acute & Continuing Care, Jurong Community Hospital, National University Health System, Singapore, Singapore.
- 4Division of Rehabilitation Medicine, Department of Medicine, National University Hospital, National University Health System, Singapore, Singapore.
- 5Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.
- 6Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore.
Abstract
Dysarthria is a common motor speech disorder affecting individuals with neurological conditions, with high prevalence among stroke survivors. Effective treatment of dysarthria requires accurate assessment of severity and the provision of speech exercises tailored to the degree of impairment. However, traditional speech intervention faces several limitations, including limited accessibility to therapists and low treatment adherence rates. AI-driven solutions have the potential to address these limitations by automating the detection of dysarthria severity and recommending personalized exercises. Although substantial progress has been made in the automated detection and classification of dysarthria severity, the performance of these approaches on less controlled, real-world speech remains insufficiently characterized. This proof-of-concept study describes the development and evaluation of Whisper-ST1, a novel framework for classifying dysarthric speech severity from audio files by fine-tuning a pretrained automatic speech recognition model. Our framework builds on previous studies by incorporating real-world data alongside traditional research datasets, adapting a model architecture pretrained on a large corpus of audio data, applying a progressive unfreezing strategy to prevent catastrophic forgetting of relevant speech features while promoting flexible adaptation to the new task, and implementing an ordinal regression loss to minimize the impact of misclassification errors. Ablation experiments demonstrate the performance contributions of individual pipeline components in isolation. Whisper-ST1 achieves 79% [67% – 100%] per-speaker F1 in the classification of dysarthria severity when tested on real-world samples, and shows strong agreement with two speech therapists’ ratings of dysarthria (Weighted Cohen’s kappa of 0.814 [0.400 – 1.000] and 0.878 [0.647 – 1.000]), comparable to inter-therapist agreement (0.851 [0.550 – 1.000]). Model errors are predominantly confined to adjacent severity levels, suggesting preservation of ordinal structure. These findings demonstrate the feasibility of leveraging pretrained speech models for dysarthria severity classification in heterogeneous audio conditions. and provide a foundational step toward AI-assisted tools for remote monitoring and support of dysarthria rehabilitation. Author summary: Dysarthria is a motor speech disorder characterized by muscle problems which result in unclear or slurred speech. This disorder affects millions of people worldwide. Effective treatment requires intensive and frequent rehabilitation sessions with speech therapists, who provide personalized rehabilitation programs tailored to each individual dysarthria severity. However, speech therapy currently faces several major challenges, including high costs, low adherence rates, and limited therapist availability. These challenges are particularly pressing in Singapore, where an ageing population and a shrinking workforce necessitate innovative healthcare solutions. In this study, a multidisciplinary team of speech therapists, rehabilitation physicians, and data specialists developed Whisper-ST1, an Artificial Intelligence system that accurately assesses dysarthria severity from audio samples, closely matching expert therapists’ evaluations. Unlike previous work that relied solely on curated research data, we distinctively built Whisper-ST1 using both controlled clinical recordings and real-world speech samples, thus increasing the range of speech and recording conditions represented during model development. Whisper-ST1 classifies dysarthria severity at overall speaker level with 83% accuracy, 95% CI [58% – 100%], showing promising performance within this proof-of-concept study. With further clinical validation, the system may be suitable for integration into a mobile application to support remote, personalized speech therapy for individuals with dysarthria. Collectively, this work demonstrates the feasibility of AI-based dysarthria severity classification using speech recorded under diverse conditions.
Related
- Simulating Psychotherapeutic Sessions with Generative Artificial Intelligence: A Proof-of-Concept Study of In Silico Psychotherapy Research
- Hidden Signals in Serum Protein Electrophoresis: Artificial Intelligence from Laboratory Pattern Recognition to Clinical Validation
- Artificial Intelligence for developing a personalised functional profile and supporting upper-limb rehabilitation decision-making in children with unilateral Cerebral Palsy (AInCP): protocol for an observational study
- From Pattern Recognition to Decision Guidance: Artificial Intelligence in Interstitial Lung Disease Diagnosis and Surveillance
- Artificial Intelligence-Driven Nanobody Engineering for Pathogen Recognition: From Functional Epitope Prioritization to Multimodal Foundation Models
BibTeX
@article{sancenon2026artificial,
title = {{An artificial intelligence-based audio recognition system for dysarthria home-based rehabilitation: A proof-of-concept study}},
author = {Vicente Sancenon and Pamela Y S Oh and Elvina K W Tay and Jarrod Z X Lee and Megan L E Soon and Han Leong Goh and Nicodemus Oey and Andy W A Ta},
journal = {PLOS Digital Health},
year = {2026},
month = oct,
doi = {10.1371/journal.pdig.0001794},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001794},
note = {CC BY},
}
RIS
TY - JOUR
TI - An artificial intelligence-based audio recognition system for dysarthria home-based rehabilitation: A proof-of-concept study
AU - Vicente Sancenon
AU - Pamela Y S Oh
AU - Elvina K W Tay
AU - Jarrod Z X Lee
AU - Megan L E Soon
AU - Han Leong Goh
AU - Nicodemus Oey
AU - Andy W A Ta
PY - 2026
DA - 2026/10/21
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001794
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001794
AB - Dysarthria is a common motor speech disorder affecting individuals with neurological conditions, with high prevalence among stroke survivors. Effective treatment of dysarthria requires accurate assessment of severity and the provision of speech exercises tailored to the degree of impairment. However, traditional speech intervention faces several limitations, including limited accessibility to therapists and low treatment adherence rates. AI-driven solutions have the potential to address these limitations by automating the detection of dysarthria severity and recommending personalized exercises. Although substantial progress has been made in the automated detection and classification of dysarthria severity, the performance of these approaches on less controlled, real-world speech remains insufficiently characterized. This proof-of-concept study describes the development and evaluation of Whisper-ST1, a novel framework for classifying dysarthric speech severity from audio files by fine-tuning a pretrained automatic speech recognition model. Our framework builds on previous studies by incorporating real-world data alongside traditional research datasets, adapting a model architecture pretrained on a large corpus of audio data, applying a progressive unfreezing strategy to prevent catastrophic forgetting of relevant speech features while promoting flexible adaptation to the new task, and implementing an ordinal regression loss to minimize the impact of misclassification errors. Ablation experiments demonstrate the performance contributions of individual pipeline components in isolation. Whisper-ST1 achieves 79% [67% – 100%] per-speaker F1 in the classification of dysarthria severity when tested on real-world samples, and shows strong agreement with two speech therapists’ ratings of dysarthria (Weighted Cohen’s kappa of 0.814 [0.400 – 1.000] and 0.878 [0.647 – 1.000]), comparable to inter-therapist agreement (0.851 [0.550 – 1.000]). Model errors are predominantly confined to adjacent severity levels, suggesting preservation of ordinal structure. These findings demonstrate the feasibility of leveraging pretrained speech models for dysarthria severity classification in heterogeneous audio conditions. and provide a foundational step toward AI-assisted tools for remote monitoring and support of dysarthria rehabilitation. Author summary: Dysarthria is a motor speech disorder characterized by muscle problems which result in unclear or slurred speech. This disorder affects millions of people worldwide. Effective treatment requires intensive and frequent rehabilitation sessions with speech therapists, who provide personalized rehabilitation programs tailored to each individual dysarthria severity. However, speech therapy currently faces several major challenges, including high costs, low adherence rates, and limited therapist availability. These challenges are particularly pressing in Singapore, where an ageing population and a shrinking workforce necessitate innovative healthcare solutions. In this study, a multidisciplinary team of speech therapists, rehabilitation physicians, and data specialists developed Whisper-ST1, an Artificial Intelligence system that accurately assesses dysarthria severity from audio samples, closely matching expert therapists’ evaluations. Unlike previous work that relied solely on curated research data, we distinctively built Whisper-ST1 using both controlled clinical recordings and real-world speech samples, thus increasing the range of speech and recording conditions represented during model development. Whisper-ST1 classifies dysarthria severity at overall speaker level with 83% accuracy, 95% CI [58% – 100%], showing promising performance within this proof-of-concept study. With further clinical validation, the system may be suitable for integration into a mobile application to support remote, personalized speech therapy for individuals with dysarthria. Collectively, this work demonstrates the feasibility of AI-based dysarthria severity classification using speech recorded under diverse conditions.
SN - 2767-3170
ER -