基于注意力机制的动态手势识别方法

Dynamic gesture recognition method based on attention mechanism

实时识别动态手势是一项艰巨的任务,因为系统永远无法知道手势在视频流中何时或从何处开始和结束。由于其各种应用,许多研究人员一直致力于基于视觉的手势识别。提出了一种基于3D卷积神经网络(3D-CNN)和长短期记忆(LSTM)网络相结合的深度学习框架,整个架构同时融合了注意力机制(CBAM)。所提出的架构从视频序列输入中提取时空信息,同时避免大量计算。3D-CNN用于提取光谱和空间特征,然后将特征图像提供给注意力机制模块,在增强图像特定区域的表征能力的同时加强特征的表达,最后通过LSTM网络进行分类。实验结果表明,所提方法能很好地识别动态手势,识别率达到了95.58%,验证了所提方法的有效性和可能性。

Recognizing dynamic gestures in real-time is a difficult task because the system can never know when or where the gestures begin and end in the video stream.Due to its various applications,many researchers have been working on vision-based gesture recognition.This paper proposes a deep learning framework based on the combination of 3D Convolutional Neural Network(3D-CNN)and Long Short-Term Memory(LSTM)network,and the whole architecture also incorporates the Attention Mechanism(CBAM).The proposed architecture extracts spatiotemporal information from video sequence input while avoiding computationally intensive.3D-CNN is used to extract spectral and spatial features,and then provide the feature image to the attention mechanism module to enhance the representation ability of specific regions of the image while telling the model what to pay attention to,and finally classify it through the LSTM network.the experimental results show that the proposed method can recognize dynamic gestures well,and the recognition rate reaches 95.82%,which verifies the effectiveness of the proposed method.and possibility.