时域注意力Dense-TCNs在多模手势识别中的应用

Application of Time Domain Attention Dense-TCNsin Multimodal Gesture Recognition

为增强时间卷积网络(TCNs)在时间特征提取方面的能力,提出一种基于三维密集卷积网络与改进TCNs的多模态手势识别方法。通过时空特征表示方法将手势视频分析任务分为空间分析和时间分析两部分。在空间分析中采用三维DenseNets学习短期的时空特征,在时间分析中使用TCNs提取时间特征。在此基础上引入注意力机制,使用时域维度的压缩-激励网络调整每个TCNs层特征在时间维度上的权值比重。分别在VIVA和NVGesture两个动态手势数据集上对该方法进行评价,实验结果表明,该方法在VIVA数据集上的正确率为91.54%,在NVGesture数据集上的正确率为86.37%,且与最新的MTUT方法水平相近。

In order to enhance the temporal feature extraction ability of Temporal Convolutional Networks(TCNs),this paper proposes a multimodal gesture recognition method based on 3D Dense convolutional Networks(3D-DenseNets)and improved TCNs.3D-DenseNets are used in spatial analysis to effectively learn short-term temporal and spatial features,and TCNs are used to extract temporal features in temporal analysis.On this basis,the attention mechanism is introduced,and the time-domain compression-stimulation network is used to adjust the weight ratio of each TCN layer feature in the time dimension.The method is evaluated on two dynamic gesture data sets,VIVA and NVGesture.Experimental results show that the proposed method achieves an accuracy rate of 91.54%on VIVA and 86.37%on the benchmark of NVGesture,reaching a level similar to that of the latest MTUT method.