Assessing spoken discourse in aphasia using multimodal artificial intelligence

Abstract

Discourse analysis can reliably predict real-world communicative success, yet is rarely implemented clinically, due to challenges in manually generating stimulus-specific main concept inventories (MCIs) and scoring patient narratives. We evaluated whether automatically derived macrolinguistic (main concept, sentiment) and microlinguistic (fluency, lexical, syntactic) features extracted from movie clip narrations can augment the clinical assessment of discourse in aphasia. The MCIs generated directly from each clips video, audio and subtitles using a vision-language model approximated the main concepts healthy controls produced. Group classification (control: n=51; aphasia: n=54) based on the discourse features was excellent (AUC=0.95, 95% CI [0.90, 0.99]), with macrolinguistic features showing the largest group differences (main concept completeness: d=-1.84, 95% CI [-2.30, -1.38]) and strongest associations with aphasia severity (semantic distance to inventory: r=-0.79, 95% CI [-0.87, -0.66]). Findings support the feasibility and scalability of automated criterion-based discourse assessment using ecological stimuli, and highlight its potential for clinical implementation.

Bridging Languages, Rebuilding Communication: Best Practices for Assessing and Treating Discourse for Bilingual People with Aphasia
Danielle Fahey, Niharika M K, Reem S. W. Alyahya +1
PsyArXiv31 Oct 2026PreprintAll rights reservedFull textNeuroscienceOSF
Nonverbal Cognition is Associated with Discourse Production in Persons with Aphasia: Evidence from Expository and Picture Description Tasks
Hana Kim, Manaswita Dutta
PsyArXiv1 Oct 2026PreprintAll rights reservedFull textNeuroscience Psychiatry & mental healthOSF
Mapping Artificial Intelligence Research in Aphasia: An AI-Assisted Scoping Review and Semantic Network Analysis
Mengyang Qiu, Siena Sun, Sara NassrElgrgawi +1
PsyArXiv16 Sep 2026PreprintCC BY 4.0Full textNeuroscienceOSF
Cortical activity during narrative discourse production in individuals with post-stroke aphasia and controls measured via functional near-infrared spectroscopy
E. J. Braun, E. A. Carpenter, Y. Gao +3
medRxiv5 Jun 2026PreprintCC BY-NC-NDFull textNeurosciencemedRxiv
Artificial Intelligence Versus Conventional Scoring for Assessing Small Bowel Capsule Endoscopy Cleanliness in Crohn's Disease
Raphaëlle Rouveyre, Catherine Le Berre, Romain Leenhardt +10
United European Gastroenterology Journal9 Jun 2026Peer-reviewedCC BY-NC-ND 4.0Full textGeneral biology General medicineHAL

Cite

BibTeX RIS

BibTeX

@misc{marte2026assessing,
  title = {{Assessing spoken discourse in aphasia using multimodal artificial intelligence}},
  author = {M. J. Marte and S. H. Lee and S. Wang and K. Goldin and M. Varkanitsa and J. M. Girard and R. Brito and X. Wei and J. T. Baker and Y. Tie and S. Kiran and E. Liebenthal},
  howpublished = {medRxiv},
  year = {2026},
  month = sep,
  doi = {10.64898/2026.09.27.26364108},
  url = {https://www.medrxiv.org/content/10.64898/2026.09.27.26364108v1},
  note = {CC BY-NC-ND},
}

RIS

TY  - UNPB
TI  - Assessing spoken discourse in aphasia using multimodal artificial intelligence
AU  - M. J. Marte
AU  - S. H. Lee
AU  - S. Wang
AU  - K. Goldin
AU  - M. Varkanitsa
AU  - J. M. Girard
AU  - R. Brito
AU  - X. Wei
AU  - J. T. Baker
AU  - Y. Tie
AU  - S. Kiran
AU  - E. Liebenthal
PY  - 2026
DA  - 2026/09/27
PB  - medRxiv
DO  - 10.64898/2026.09.27.26364108
UR  - https://www.medrxiv.org/content/10.64898/2026.09.27.26364108v1
AB  - Discourse analysis can reliably predict real-world communicative success, yet is rarely implemented clinically, due to challenges in manually generating stimulus-specific main concept inventories (MCIs) and scoring patient narratives. We evaluated whether automatically derived macrolinguistic (main concept, sentiment) and microlinguistic (fluency, lexical, syntactic) features extracted from movie clip narrations can augment the clinical assessment of discourse in aphasia. The MCIs generated directly from each clips video, audio and subtitles using a vision-language model approximated the main concepts healthy controls produced. Group classification (control: n=51; aphasia: n=54) based on the discourse features was excellent (AUC=0.95, 95% CI [0.90, 0.99]), with macrolinguistic features showing the largest group differences (main concept completeness: d=-1.84, 95% CI [-2.30, -1.38]) and strongest associations with aphasia severity (semantic distance to inventory: r=-0.79, 95% CI [-0.87, -0.66]). Findings support the feasibility and scalability of automated criterion-based discourse assessment using ecological stimuli, and highlight its potential for clinical implementation.
ER  -

Abstract

Related

Cite