一种双路并行的大规模手势识别模型

A large-scale gesture recognition model with dual-path parallel

文章以大规模手势为研究对象,提出一种基于肌电信号(electromyography,EMG)分支和惯性测量单元(inertial measurement unit,IMU)分支的双路并行手势识别模型。首先,设计双路并行模型来充分提取数据特征,EMG分支利用二维卷积神经网络设计双流结构,分别关注EMG信号的空间和通道变化,IMU分支在卷积长短时记忆(convolutional long short-term memory,ConvLSTM)网络基础上引入时间机制,将空间信息与时间信息融合;其次,对模型预训练并根据预训练模型进行参数微调,提高模型泛化性;最后,在500个常用的中国手语手势上进行测试,结果表明,该模型平均识别率为82.1%,与SignSpeaker和CG-Recognizer相比分别提高了21.0%和6.8%。

In this paper,a dual-path parallel gesture recognition model based on the electromyography(EMG)branch and the inertial measurement unit(IMU)branch is proposed for large-scale gestures.Firstly,the dual-path parallel model is designed to fully extract the data features.The EMG branch uses a two-dimensional convolutional neural network to design a dual-stream structure to focus on the spatial and channel variations of EMG signals,respectively.The IMU branch introduces a temporal mechanism based on the convolutional long short-term memory(ConvLSTM)network to fuse spatial and temporal information.Secondly,the model is pre-trained and the parameters are fine-tuned according to the pre-trained model to improve the generalization of the model.Finally,the model is tested on 500 commonly used Chinese sign language gestures,and the average recognition rate of the model is 82.1%,which is 21.0%and 6.8%higher than that of SignSpeaker and CG-Recognizer,respectively.