Multimodal sEMG and Accelerometer Data Fusion for Fine-Grained Hand Gesture Recognition - 12/08/26
, Sofiane Boudaoud, Imad RidaAbstract |
Surface electromyography (sEMG) has become a key sensing modality for hand gesture recognition in rehabilitation, prosthetic control, and human–computer interaction. However, its sparse activation patterns and sensitivity to subject- and recording-related variability pose significant challenges, particularly for fine-grained gestures involving subtle finger and wrist motions. To address these limitations, we propose an attention-enhanced multimodal fusion framework that integrates raw sEMG and accelerometer (ACC) signals. The proposed network employs narrow-kernel temporal convolutions in dedicated modality-specific streams to preserve fine-grained neuromuscular patterns, followed by a dual-attention fusion module combining multi-head self-attention (MHSA) and gated attention to model interactions between sEMG and ACC embeddings and improve gesture separability. Experiments on the NinaPro DB2 and DB7 databases show that the proposed fusion model outperforms both the sEMG-only and ACC-only models. The feature-level fusion model achieves accuracies of 96.42% on DB2 and 97.03% on DB7 without data augmentation. After applying the selected combined augmentation strategy based on time warping and scaling, the accuracy further increases to 97.56% and 98.19%, respectively. A decision-level fusion variant also achieves comparable performance while keeping the sEMG and ACC classifiers independent. Detailed gesture-level analysis further shows that the largest gains are obtained for low-amplitude and highly similar gestures, highlighting the value of multimodal fusion for improving the separability of subtle hand movements.
Il testo completo di questo articolo è disponibile in PDF.Graphical abstract |
Highlights |
• | A multimodal sEMG-ACC fusion framework is proposed for hand gestures. |
• | ACC data help distinguish gestures with similar sEMG patterns. |
• | MHSA and gated attention model cross-modal feature interaction. |
• | Time warping and scaling improve recognition performance. |
• | The method achieves 97.56% on DB2 and 98.19% on DB7. |
Keywords : Accelerometer (ACC), Attention Mechanism, Decision Fusion, Feature Fusion, Hand Gesture Recognition, Surface Electromyography (sEMG)
Mappa
Benvenuto su EM|consulte, il riferimento dei professionisti della salute.
L'accesso al testo integrale di questo articolo richiede un abbonamento.
Già abbonato a @@106933@@ rivista ?
