基于全局分割图卷积网络的骨骼双人交互行为识别

Skeleton-based Double Human Interaction Recognition Based on Global Segmentation Graph Convolutional Network

图卷积网络(GCN)在双人交互识别方面应用非常广泛,但是传统图卷积网络对于节点特征的学习不够充分,尤其在双人交互行为中,每个节点的特征往往包含多个人的信息,导致节点特征不够准确。为了有效提取双人交互行为关节点之间的相关性特征,聚合双人之间的特征信息,提出全局分割图卷积网络(GS-GCN),进行基于骨骼的双人交互行为识别。GS-GCN包含全局分割图卷积(GSGC)和层次聚合注意力(HAA)模块,GSGC将图卷积(GCN)和全局分割图(GS-Graph)相结合,将双人交互识别看成全局性识别,并具有多个邻接矩阵,可提取单人特征和双人全局特征。此外,单人的层次和双人层次的重要性各不相同,为了突出交互行为之间的交互信息,引入层次聚合的注意力模块(HAA),突出交互行为中更加明显的语义信息。在NTU-RGB+D、NTU-RGB+D 120和SBU双人交互数据集上进行实验,验证了模型有效性,与其他双人交互识别方法相比,该模型性能更优。

Graph convolutional network(GCN)is widely used in interaction recognition,but the traditional graph convolutional network is not enough to learn node features.Especially in interaction,the features of each node often contain information of multiple people,resulting in inaccurate node features.In order to effectively extract the correlation features between the nodes between interaction behaviors and aggregate the feature information,a global segmentation graph convolution network(GS-GCN)is proposed,which consists of global segmentation graph convolution(GSGC)and hierarchical aggregation attention(HAA)modules.GSGC combines graph convolution(GCN)and global segmenta-tion graph(GS-Graph),treats two-person interaction recognition as global recognition,has multiple adjacency matrices,and extracts single-person feature and two-person global feature.In addition,the importance of single level and double level is different.In order to highlight the interactive information between interaction behaviors,the attention module of hierarchical aggregation(HAA)is introduced to highlight the more obvious semantic information in interaction behaviors.Through experiments on NTU-RGB+D,NTU-RGB+D 120 and SBU two-person interaction data sets,the validity of the model is verified,and it is obviously superior to other two-person interaction recognition methods.