顺序主导和方向驱动下基于点边特征的人体动作识别方法

Sequence-driven and direction-driven human action recognition method based on joint-bone features

人体运动是肢体运动方向、关节活动顺序以及动作幅度相互协调的过程.然而,现有方法往往直接对原始3D骨骼关节点信息进行建模,容易忽略肢体关节活动的顺序关系、运动方向性以及动作幅度变化影响.因此,提出一种顺序主导和方向驱动下基于点边特征的骨骼卷积神经网络,通过刻画人体关节点运动顺序、帧间距离和骨骼边方向向量等特征对人体动作分类识别.该网络包含顺序主导单元和方向驱动单元.顺序主导单元对骨骼边末端关节点进行建模,利用关节点的排列方式、帧间距离信息对关节活动顺序和肢体变化幅度进行表征.方向驱动单元利用骨骼边方向向量信息表征肢体运动的方向性.最后,将顺序主导单元与方向驱动单元进行特征融合,对人体日常行为动作进行分类识别.实验结果表明,在两个大型数据集NTU-RGB+D60和NTU-RGB+D120上的实验结果分别较基准方法提升了2.6%、3.5%和5.9%、6.1%.因此,所提出方法能有效利用多特征之间的协同互补性对人类日常行为运动进行深层次刻画,提高人体动作识别的精度.

Human action is the process of coordinating the direction of limb movement,the sequence of joint activity and the amplitude of motion.However,existing methods tend to directly model the original 3D skeletal joint information,which easily ignores the sequential relationship between limb joint activities,motion directionality and movement amplitude variation.Therefore,this paper proposes a skeletal convolutional neural network based on point-bone features in a sequence-driven and direction-driven manner to recognize human actions by characterizing the sequence of human joint point movements,inter-frame distances and skeletal bone direction vectors.The network consists of a sequence-driven unit and a direction-driven unit.The sequence-driven unit models the joint points at the end of the skeletal bone,and characterizes the sequence of joint movements and the magnitude of limb changes by using the joint arrangement and inter-frame distance information.The direction-driven unit uses the direction vector information of the skeletal bone to characterize the directionality of the limb movement.Finally,the sequence-driven unit is fused with the direction-driven unit features maps to classify and recognize human daily behavioral actions.The experimental results show that the results on two large datasets,NTU-RGB+D60 and NTU-RGB+D120,improve 2.6%,3.5%and 5.9%,6.1%,respectively,compared with the benchmark method.The proposed method can effectively utilize the synergistic complementarity between multiple features to deeply characterize human daily behavioral movements and effectively improve the accuracy of human action recognition.