ASD-FER: An Annotated Multi-Class Benchmark and Comparative Study for Facial Emotion Recognition in Children with Autism Spectrum Disorder
- 1Department of Computer Science, LSU New Orleans, New Orleans, Louisiana, USA
Abstract
Autism Spectrum Disorder (ASD) affects social interaction and communication, and automatically recognizing emotions from the facial expressions of children with ASD remains challenging. A key barrier to facial emotion recognition (FER) in children with ASD is the absence of ASD-specific datasets with proper emotion-level annotations. To address this, we present ASD-FER, a benchmark of 2,779 facial images of children with ASD annotated across six emotion categories by three independent annotators using majority voting. We evaluate nine models -- CNN, SVM, AlexNet, ResNet34, ResNet50, MobileNetV2, and three backbone configurations of our proposed task-specific architecture, EQFormer: ViT, Swin-T, and a compact Hybrid CNN-Transformer (HybridT) -- under stratified 10-fold cross-validation and a held-out 90/10 evaluation. ResNet50 led conventional models at 84.89% mean cross-validation accuracy; among transformer-based models, ViT led with 88.27%, followed by Swin-T at 88.02% and HybridT at 86.58%. On the held-out set, ViT achieved the highest overall accuracy (91.01%), while Swin-T obtained the best balanced accuracy (74.47%) and macro-F1 (0.7078); HybridT remained competitive at 89.57% accuracy with a fraction of the parameters. To support explainability, Grad-CAM visualizations highlighted attention on emotion-relevant facial regions -- eyes, eyebrows, and mouth -- with Swin-T the most spatially focused and the HybridT the most parameter-efficient (2.66M). Class imbalance and cross-dataset generalization remain open challenges.
Related
- Association of Halliwick-Based Aquatic Physical Activity with Health-Related Quality of Life in Children and Adolescents with Autism Spectrum Disorder: A Cross-Sectional Comparative Study.
- Identification of Peripheral Blood mIR-128-3p Molecular Signature in Children with Autism Spectrum Disorder
- A Comparative Benchmark of Biomedical Language Models for Concept Normalization from Real-World Text
- Influence of a full-spectrum aqueous extract of black cardamom (Amomum subulatum Roxb.) fruits on focus/alertness and executive function: a randomized, double-blind, placebo-and active- controlled, 14-day comparative study in healthy working professionals
- Guideline for intervention and rehabilitation in children with autism spectrum disorder (2026 edition)
BibTeX
@misc{chowdhury2026asdfer,
title = {{ASD-FER: An Annotated Multi-Class Benchmark and Comparative Study for Facial Emotion Recognition in Children with Autism Spectrum Disorder}},
author = {S. Y. Chowdhury and B. Banik and T. A. Knaus and C. D. Moore and H. F. Ansah and H. Hammad and P. Ghimire and M. T. Hoque and S. Banerjee},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.14.754143},
url = {https://www.biorxiv.org/content/10.64898/2026.10.14.754143v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - ASD-FER: An Annotated Multi-Class Benchmark and Comparative Study for Facial Emotion Recognition in Children with Autism Spectrum Disorder
AU - S. Y. Chowdhury
AU - B. Banik
AU - T. A. Knaus
AU - C. D. Moore
AU - H. F. Ansah
AU - H. Hammad
AU - P. Ghimire
AU - M. T. Hoque
AU - S. Banerjee
PY - 2026
DA - 2026/10/14
PB - bioRxiv
DO - 10.64898/2026.10.14.754143
UR - https://www.biorxiv.org/content/10.64898/2026.10.14.754143v1
AB - Autism Spectrum Disorder (ASD) affects social interaction and communication, and automatically recognizing emotions from the facial expressions of children with ASD remains challenging. A key barrier to facial emotion recognition (FER) in children with ASD is the absence of ASD-specific datasets with proper emotion-level annotations. To address this, we present ASD-FER, a benchmark of 2,779 facial images of children with ASD annotated across six emotion categories by three independent annotators using majority voting. We evaluate nine models -- CNN, SVM, AlexNet, ResNet34, ResNet50, MobileNetV2, and three backbone configurations of our proposed task-specific architecture, EQFormer: ViT, Swin-T, and a compact Hybrid CNN-Transformer (HybridT) -- under stratified 10-fold cross-validation and a held-out 90/10 evaluation. ResNet50 led conventional models at 84.89% mean cross-validation accuracy; among transformer-based models, ViT led with 88.27%, followed by Swin-T at 88.02% and HybridT at 86.58%. On the held-out set, ViT achieved the highest overall accuracy (91.01%), while Swin-T obtained the best balanced accuracy (74.47%) and macro-F1 (0.7078); HybridT remained competitive at 89.57% accuracy with a fraction of the parameters. To support explainability, Grad-CAM visualizations highlighted attention on emotion-relevant facial regions -- eyes, eyebrows, and mouth -- with Swin-T the most spatially focused and the HybridT the most parameter-efficient (2.66M). Class imbalance and cross-dataset generalization remain open challenges.
ER -