C-TransDance: A Style-Conditioned Gated CNN-Transformer for Real-Time Multi-Style Dance Quality Assessment Using Vision-Based Pose Sensing.
- 1Department of Dance, Sookmyung Women's University, Seoul 04310, Republic of Korea.
- 2Department of Computer Information Engineering, Kunsan National University, Gunsan 54150, Republic of Korea.
Abstract
Vision-based human motion assessment requires a model to capture both temporal execution patterns and context-dependent quality criteria. This problem is particularly challenging in multi-style dance, where similar kinematic patterns may receive different quality evaluations under different stylistic conventions. We propose C-TransDance, a style-conditioned gated CNN-Transformer for real-time multi-style dance quality assessment using pose-based visual sensing. The system extracts 33 body landmarks from a single front-facing camera with MediaPipe BlazePose GHUM and represents each motion sequence using normalized joint coordinates, confidence, velocity, and acceleration. A convolutional branch for local spatio-temporal execution patterns and a Transformer branch for long-range temporal dependencies are combined through learnable gated fusion and modulated by a learned dance style embedding with an explicit feature-style interaction. Multi-task heads jointly predict continuous quality scores, quality classes, and structured feedback categories, and temporal attention and joint-level sensitivity are coupled with constrained feedback templates to expose inspectable movement evidence. The framework was evaluated on 1152 clips from 96 participants across four dance styles, annotated independently by three qualified instructors, using participant-independent splits and five random seeds. On the held-out test set (173 clips, 15 unseen participants), C-TransDance achieved an MAE of 4.86 ± 0.16, an RMSE of 6.38 ± 0.22, an accuracy of 82.1% ± 1.1%, and a macro-F1 of 0.822 ± 0.011 compared with an MAE of 5.63 for the strongest style-agnostic pose baseline. Within this backbone and dataset, removing style conditioning increased MAE by 15.6%, compared with 12.8% after removing the convolutional branch and 9.3% after removing the Transformer branch. Because the baselines do not receive the style label and style is partly confounded with routine in the dataset, these results are interpreted as controlled evidence that style information is a major factor, not as a general architectural ranking. The complete system operated at 28.8 FPS with 34.7 ms end-to-end latency, supporting the technical feasibility of style-aware pose sensing for instructor-supervised feedback.
Related
- Fuzzy Fault Detection and Band-Conditioned Data Quality Assessment for Sensor Streams: A Continuous Glucose Monitoring Case Study
- Region-Aware Degradation-Conditioned Vision-Language Learning for Blind Image-Quality Assessment of Power Field-Operation Imagery
- Artificial intelligence-enabled monitoring of pesticide contamination in agricultural water systems: a comprehensive review of intelligent sensing technologies for water quality surveillance and environmental risk assessment
- Inverse design of multi-band terahertz metamaterial sensors with nested topologies via a transformer-conditioned variational autoencoder
- Semblance Distance Analysis for Channel Quality Assessment in Distributed Acoustic Sensing
BibTeX
@article{zhang2026ctransdance,
title = {{C-TransDance: A Style-Conditioned Gated CNN-Transformer for Real-Time Multi-Style Dance Quality Assessment Using Vision-Based Pose Sensing.}},
author = {Shan Zhang and Yantong Liu and Seong-Yoon Shin and Hyun-Ae Lee},
journal = {Sensors (Basel, Switzerland)},
year = {2026},
month = oct,
doi = {10.3390/s26206628},
url = {https://europepmc.org/article/MED/42944491},
note = {CC BY},
}
RIS
TY - JOUR
TI - C-TransDance: A Style-Conditioned Gated CNN-Transformer for Real-Time Multi-Style Dance Quality Assessment Using Vision-Based Pose Sensing.
AU - Shan Zhang
AU - Yantong Liu
AU - Seong-Yoon Shin
AU - Hyun-Ae Lee
PY - 2026
DA - 2026/10/20
JO - Sensors (Basel, Switzerland)
DO - 10.3390/s26206628
UR - https://europepmc.org/article/MED/42944491
AB - Vision-based human motion assessment requires a model to capture both temporal execution patterns and context-dependent quality criteria. This problem is particularly challenging in multi-style dance, where similar kinematic patterns may receive different quality evaluations under different stylistic conventions. We propose C-TransDance, a style-conditioned gated CNN-Transformer for real-time multi-style dance quality assessment using pose-based visual sensing. The system extracts 33 body landmarks from a single front-facing camera with MediaPipe BlazePose GHUM and represents each motion sequence using normalized joint coordinates, confidence, velocity, and acceleration. A convolutional branch for local spatio-temporal execution patterns and a Transformer branch for long-range temporal dependencies are combined through learnable gated fusion and modulated by a learned dance style embedding with an explicit feature-style interaction. Multi-task heads jointly predict continuous quality scores, quality classes, and structured feedback categories, and temporal attention and joint-level sensitivity are coupled with constrained feedback templates to expose inspectable movement evidence. The framework was evaluated on 1152 clips from 96 participants across four dance styles, annotated independently by three qualified instructors, using participant-independent splits and five random seeds. On the held-out test set (173 clips, 15 unseen participants), C-TransDance achieved an MAE of 4.86 ± 0.16, an RMSE of 6.38 ± 0.22, an accuracy of 82.1% ± 1.1%, and a macro-F1 of 0.822 ± 0.011 compared with an MAE of 5.63 for the strongest style-agnostic pose baseline. Within this backbone and dataset, removing style conditioning increased MAE by 15.6%, compared with 12.8% after removing the convolutional branch and 9.3% after removing the Transformer branch. Because the baselines do not receive the style label and style is partly confounded with routine in the dataset, these results are interpreted as controlled evidence that style information is a major factor, not as a general architectural ranking. The complete system operated at 28.8 FPS with 34.7 ms end-to-end latency, supporting the technical feasibility of style-aware pose sensing for instructor-supervised feedback.
SN - 1424-8220
ER -