Analysis and improvement of surface electromyography-based American Sign Language hand gesture recognition
Abstract
Surface electromyography (sEMG) allows sign language gestures to be recognized from forearm muscle activity without visual access to the hand. Gestures that produce similar forearm activation patterns, however, remain difficult to distinguish. This study examined how these similarities relate to classification errors and whether short-term temporal context improves recognition of static American Sign Language (ASL) gestures. We recorded 32-channel forearm sEMG from 10 participants performing 24 static gestures from the ASL alphabet and evaluated a dual-branch convolutional-Transformer classifier using both participant-specific and shared multi-user models. We also applied a Multi-Window Model that integrated the current and preceding window representations through causal attention after offline zero-phase filtering. On identical target windows, single-window classification achieved letter-only accuracies of 86.71% and 85.23% in the two settings, respectively. Nine low-recall letters formed three signal-similar groups, {A, M, N, S, T}, {E, O}, and {U, V}, with classification errors strongly concentrated within these groups. Spatial activation and independent-component analyses further showed that greater similarity in forearm sEMG patterns was associated with greater classification confusion. With the Multi-Window Model, letter-only accuracy increased to 90.77% in the participant-specific setting and 89.01% in the multi-user setting, with improvements observed across all participants in the participant-specific setting and larger gains for the nine low-recall letters. These findings demonstrate that integrating recent signal history can improve the accuracy of sEMG-based sign language gesture recognition, while persistent errors among highly similar gestures indicate an important limitation of the information available from forearm sEMG alone.
Related
- Precise modeling of task-related sensorimotor activation based on simultaneous surface electromyography
- It's a small world of signs: Semantic network structure in American Sign Language based on free associations
- Grounding-Specific Effects of Valence in Word Recognition: Evidence from Pain-Related Language.
- Decoding antibody-antigen recognition: the BL8 language and its application in calibrating deep learning predictions
- Assessing accelerometry as an alternative to electromyography for mouse sleep scoring
BibTeX
@article{wang2026analysis,
title = {{Analysis and improvement of surface electromyography-based American Sign Language hand gesture recognition}},
author = {Yuanxi Wang and Lun Shu and Yasuharu Koike},
journal = {Frontiers in Neuroscience},
year = {2026},
month = nov,
doi = {10.3389/fnins.2026.1978950},
url = {https://www.frontiersin.org/articles/10.3389/fnins.2026.1978950},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Analysis and improvement of surface electromyography-based American Sign Language hand gesture recognition
AU - Yuanxi Wang
AU - Lun Shu
AU - Yasuharu Koike
PY - 2026
DA - 2026/11/03
JO - Frontiers in Neuroscience
DO - 10.3389/fnins.2026.1978950
UR - https://www.frontiersin.org/articles/10.3389/fnins.2026.1978950
AB - Surface electromyography (sEMG) allows sign language gestures to be recognized from forearm muscle activity without visual access to the hand. Gestures that produce similar forearm activation patterns, however, remain difficult to distinguish. This study examined how these similarities relate to classification errors and whether short-term temporal context improves recognition of static American Sign Language (ASL) gestures. We recorded 32-channel forearm sEMG from 10 participants performing 24 static gestures from the ASL alphabet and evaluated a dual-branch convolutional-Transformer classifier using both participant-specific and shared multi-user models. We also applied a Multi-Window Model that integrated the current and preceding window representations through causal attention after offline zero-phase filtering. On identical target windows, single-window classification achieved letter-only accuracies of 86.71% and 85.23% in the two settings, respectively. Nine low-recall letters formed three signal-similar groups, {A, M, N, S, T}, {E, O}, and {U, V}, with classification errors strongly concentrated within these groups. Spatial activation and independent-component analyses further showed that greater similarity in forearm sEMG patterns was associated with greater classification confusion. With the Multi-Window Model, letter-only accuracy increased to 90.77% in the participant-specific setting and 89.01% in the multi-user setting, with improvements observed across all participants in the participant-specific setting and larger gains for the nine low-recall letters. These findings demonstrate that integrating recent signal history can improve the accuracy of sEMG-based sign language gesture recognition, while persistent errors among highly similar gestures indicate an important limitation of the information available from forearm sEMG alone.
SN - 1662-453X
ER -