摘要
基于图卷积网络(graph convolutional networks,GCNs)的骨架动作识别方法只能对关节点间的单一关系进行建模,缺少描述多种关系的能力.借鉴知识图谱描述实体之间不同关系的思想,提出一种基于关节点流和肢体流的双流多关系GCNs人体骨架动作识别方法,对图结点间的自然连接关系、对称关系和全局关系进行建模,各种特征在网络中同步传输并有效融合.运动的全身协作过程中,每个部位的交互范围有限且依赖于具体动作,提出基于Non-local机制的top K全局邻接关系自适应计算方法,为每个结点动态选择交互强度较大的前K个结点作为全局关系邻接点.实验结果表明,所提出的双流多关系网络在Kinetics和NTU-RGB+D数据集上取得了较好的动作识别效果.
The interaction of human body parts in motion is diverse,but the existing skeleton-based action recognition methods with GCNs(graph convolutional networks)can only model a single relationship between joints.The idea of knowledge graphs was used to describe the different relationships between entities and a two-stream multi-relational GCNs action recognition method was proposed based on joints and body parts,by which the models of the natural connection relationship,symmetric relationship,and global relationship among nodes were established.The features of each relationship were synchronously transmitted and effectively fused in the network.In the process of global cooperation of human body parts,the interaction range of each part was limited and depends on specific actions.An adaptive top K global adjacency relationship calculation method was proposed based on Non-local algorithm,in which the nodes with the top K interaction strength were selected dynamically as the adjacent nodes for each node.The experimental results show that the proposed two-stream multi-relational network achieves good action accuracy on the Kinetics dataset and the NTU-RGB+D dataset.
作者
刘芳
乔建忠
代钦
石祥滨
LIU Fang;QIAO Jian-zhong;DAI Qin;SHI Xiang-bin(School of Computer Science&Engineering,Northeastern University,Shenyang 110169,China;School of Computer Science,Shenyang Aerospace University,Shenyang 110136,China;College of Information,Shenyang Institute of Engineering,Shenyang 110136,China)
出处
《东北大学学报(自然科学版)》
EI
CAS
CSCD
北大核心
2021年第6期768-774,共7页
Journal of Northeastern University(Natural Science)
基金
辽宁省自然科学基金资助项目(2019-MS-254,20180550337)
辽宁省教育厅科研项目(JYT2020050,JL-2019)
中央引导地方科技发展专项资金资助项目(2021JH6/10500127).
关键词
动作识别
骨架
图卷积网络
多关系
top
K
action recognition
skeleton
GCNs(graph convolutional networks)
multi-relation
top K