Dual-stream attention and feature fusion for human activity recognition via wearable sensors
Abstract
Multi-sensor-based human activity recognition (HAR) has attracted considerable attention in recent years. Despite significant advancements achieved through deep learning, extracting refined and subtle features from multiple sensors placed on different parts of the body remains a challenge. Accurate sensor modality attention and temporal information processing are crucial for enhancing model performance. In this paper, we propose a new Temporal and Sensor-Modality Attention with Feature Fusion Network (TSFF). TSFF consists of two modules: the Temporal and Sensor Modality Attention block (TS) and the Global and Local Feature Fusion block (GLFF). The TS module extracts sensor modality attention and temporal attention in parallel using two branches, while the GLFF module dynamically fuses features from the sensor modality attention branch and the temporal stream through global and local attention mechanisms. We evaluated TSFF on three widely used public datasets–PAMAP2, Opportunity, and WISDM. When integrated with the ResNet backbone, TSFF achieved accuracies of 94.64%, 92.18%, and 97.96% on these datasets, respectively. Ablation studies further indicate that jointly modelling temporal and sensor-modality information with adaptive global–local fusion improves performance over the corresponding backbone under the evaluated settings. Visual analyses additionally provide qualitative evidence that the learned representations exhibit clearer class separation and differentiated weighting patterns across sensors and time steps.
Related
- Compact target-aware stability-based feature selection for cross-dataset wearable activity monitoring
- The future of wearable sweat sensors in intensive care units: Toward a new era of noninvasive, continuous real-time monitoring
- Identifying unique gait phenotypes across neuromotor conditions using wearable inertial sensors and topological data analysis
- Adaptive personalization for wearable non-invasive continuous glucose sensors.
- A wearable device dataset for mental health assessment using laser Doppler flowmetry and fluorescence spectroscopy sensors
BibTeX
@article{fan2026dualstream,
title = {{Dual-stream attention and feature fusion for human activity recognition via wearable sensors}},
author = {Jiangtao Fan and Anish Jindal and Amir Atapour-Abarghouei},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-75273-x},
url = {https://www.nature.com/articles/s41598-026-75273-x},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Dual-stream attention and feature fusion for human activity recognition via wearable sensors
AU - Jiangtao Fan
AU - Anish Jindal
AU - Amir Atapour-Abarghouei
PY - 2026
DA - 2026/10/12
JO - Scientific Reports
DO - 10.1038/s41598-026-75273-x
UR - https://www.nature.com/articles/s41598-026-75273-x
AB - Multi-sensor-based human activity recognition (HAR) has attracted considerable attention in recent years. Despite significant advancements achieved through deep learning, extracting refined and subtle features from multiple sensors placed on different parts of the body remains a challenge. Accurate sensor modality attention and temporal information processing are crucial for enhancing model performance. In this paper, we propose a new Temporal and Sensor-Modality Attention with Feature Fusion Network (TSFF). TSFF consists of two modules: the Temporal and Sensor Modality Attention block (TS) and the Global and Local Feature Fusion block (GLFF). The TS module extracts sensor modality attention and temporal attention in parallel using two branches, while the GLFF module dynamically fuses features from the sensor modality attention branch and the temporal stream through global and local attention mechanisms. We evaluated TSFF on three widely used public datasets–PAMAP2, Opportunity, and WISDM. When integrated with the ResNet backbone, TSFF achieved accuracies of 94.64%, 92.18%, and 97.96% on these datasets, respectively. Ablation studies further indicate that jointly modelling temporal and sensor-modality information with adaptive global–local fusion improves performance over the corresponding backbone under the evaluated settings. Visual analyses additionally provide qualitative evidence that the learned representations exhibit clearer class separation and differentiated weighting patterns across sensors and time steps.
SN - 2045-2322
ER -