基于双流多关系GCNs的骨架动作识别方法

Skeleton-based Action Recognition Method with Two-Stream Multi-relational GCNs

基于图卷积网络(graph convolutional networks,GCNs)的骨架动作识别方法只能对关节点间的单一关系进行建模,缺少描述多种关系的能力.借鉴知识图谱描述实体之间不同关系的思想,提出一种基于关节点流和肢体流的双流多关系GCNs人体骨架动作识别方法,对图结点间的自然连接关系、对称关系和全局关系进行建模,各种特征在网络中同步传输并有效融合.运动的全身协作过程中,每个部位的交互范围有限且依赖于具体动作,提出基于Non-local机制的top K全局邻接关系自适应计算方法,为每个结点动态选择交互强度较大的前K个结点作为全局关系邻接点.实验结果表明,所提出的双流多关系网络在Kinetics和NTU-RGB+D数据集上取得了较好的动作识别效果.

The interaction of human body parts in motion is diverse,but the existing skeleton-based action recognition methods with GCNs(graph convolutional networks)can only model a single relationship between joints.The idea of knowledge graphs was used to describe the different relationships between entities and a two-stream multi-relational GCNs action recognition method was proposed based on joints and body parts,by which the models of the natural connection relationship,symmetric relationship,and global relationship among nodes were established.The features of each relationship were synchronously transmitted and effectively fused in the network.In the process of global cooperation of human body parts,the interaction range of each part was limited and depends on specific actions.An adaptive top K global adjacency relationship calculation method was proposed based on Non-local algorithm,in which the nodes with the top K interaction strength were selected dynamically as the adjacent nodes for each node.The experimental results show that the proposed two-stream multi-relational network achieves good action accuracy on the Kinetics dataset and the NTU-RGB+D dataset.