可分离长短期注意力网络的手势识别方法

Gesture recognition method with separable long short-term attention networks

在人机交互领域中,大多数手势识别算法无法有效地消除采集背景对待提取手势区域的影响。与此同时,对手势运动信息的准确建模也存在困难。针对目前人机交互中的上述问题,提出利用深度可分离残差卷积长短期记忆(LSTM)网络的方法对动态手势的特征信息进行建模和识别。首先,利用常规3D卷积操作对输入的视频帧进行特征的初步提取,通过较大的卷积核尺寸以扩大其感受野;然后,通过可分离卷积残差操作对输入的浅层特征进行特征的再提取,实现对高维特征的提取建模;最后,将经过前两个阶段提取出的特征信息经过3D池化操作后输入到LSTM网络中,对输入的视频数据的时序信息进行建模,并在输入中引入注意力机制。在大规模孤立手势数据集上进行的相关实验结果表明,所提方法的准确率与原始的围绕稀疏关键点的混合特征(MFSK)+视觉词袋(BoVW)+支持向量机(SVM)网络相比提高了21.02个百分点。

Most gesture recognition algorithms in the human-computer interaction field cannot effectively eliminate the influence of the acquisition background on the extraction gesture area.At the same time,it is difficult to accurately model the motion information of the gesture.In view of the above problems in human-computer interaction,separable Long Short-Term Memory(LSTM)network for gesture recognition was proposed to model and recognize the feature information.First,the preliminary extraction of the input video frame by conventional 3D convolution operation was carried out.A large convolutional size was chosen to expand the receptive field.Then,the shallow features were re-extracted with separable convolutional residual operation and constructed the model of high-dimensional features.Finally,the feature information extracted through the first two steps was entered into a LSTM network after 3D pooling.The timing information of the video data was modeled,and attention mechanism was introduced into the input.Experimental results on a large-scale isolated gesture dataset show that the accuracy of the proposed method is 21.02 percentage points higher than that of the original MFSK(Mixed features around Sparse Keypoints)+BoVW(Bag of Visual Words)+SVM(Support Vector Machine)network.