基于时空注意力深度增强差分图卷积的骨架行为识别

A Skeleton-Based Action Recognition with Spatiotemporal Attention Depth Enhance Differential Graph Convolution

时空卷积神经网络是行为识别的主流方法之一,但传统时空图卷积神经网络在空间特征聚合存在数据冗余与时间特征提取不充分的问题,针对该问题该文提出了一种时空注意力深度增强差分图卷积网络(ST-DEdGCN)模型。首先,在空间上通过深度增强差分图卷积(DEdGC)动态地学习不同通道中节点拓扑与节点梯度信息,有效地聚合不同通道中的关节特征。其次,通过时空卷积模块在时间维度上对全局时间信息进行建模,得到高效的序列特征信息。最后在NTU RGB + D 60和NTU RGB + D 120两个数据集进行了实验,实验结果表明时空注意力深度差分图卷积网络模型在空间特征的有效聚合和时空信息的有效提取方面优于当前主流方法,为行为识别及其相关研究提供了新的技术途径。

Spatiotemporal convolution neural network is one of the mainstream methods of action recognition, but the traditional spatiotemporal graph convolution neural network while having the problems of data redundancy and insufficient temporal feature extraction. To tackle the problem, a novel Spatio Temporal attention Depth Enhance difference Graph Convolution Network (ST- DEdGCN) model is proposed in this paper. Firstly, the Depth Enhance difference Graph Convolution (DEdGC) in space is proposed to dynamically learn joint topology and joint gradient information in different channels, and the joint features in different channels are effectively aggregated. Secondly, the Spatiotemporal Attention Temporal Convolution Network is proposed to model the global temporal joint information in time, and obtain efficient temporal feature information. Finally, the proposed algorithm is verified on the public skeleton action data sets NTU RGB + D 60 and NTU RGB + D 120. The results further verify the superiority to aggregate spatial features and to extract spatial-temporal information of this model, and provide a new technical approach for action recognition.