关节点连接历史图与卷积神经网络结合的双人交互动作识别

Human interaction recognition based on joint connection history maps and convolutional neural network

人体关节点数据可以精确表征人体运动的三维信息,卷积神经网络能够提取二维图像中的深层次特征,因此将关节点数据与卷积神经网络结合用于双人交互动作识别具有很好的研究意义。目前将两者结合的方法中,多数不能充分利用关节点的时空关系,导致识别率不高。提出一种新的关节点数据的时空特征表示方法,即关节点连接历史图。首先将关节点数据中的关节点按照人体骨架顺序连接得到关节点连接图,然后将每帧关节点连接图中的关节点和关节点连线按照时间顺序谱编码得到关节点连接历史图,最后将其馈送到卷积神经网络得到最终的识别结果。实验结果表明,关节点连接历史图与CNN结合可以准确识别双人交互动作,在国际公开的SBU Kinect interaction数据库测试中达到94.12%的识别率,充分证明了所提出算法的有效性。

The joint data can accurately represent three-dimensional information of human motion,and CNN can extract the deep features from images.Hence it is significant to combine them for human interaction recognition.Most of existing methods could not make full use of the spatial position relationship and time sequence in the joint data,which results in low recognition results.This paper proposes a new spatio-temporal feature representation method of joint data,that is,joint connection history map.First,connect the joint points according to the order of human skeleton to obtain the joint point connection map.Then,code the joint points and connections of each frame according to the time sequence spectrum to obtain the joint connection history map.Finally,feed it to the convolutional neural network to get the final recognition result.Experimental results show that the combination of the joint connection history map and CNN can identify human interaction exactly.The recognition rate reaches 94.12%in the SBU Kinect interaction database,which fully proves the effectiveness of the proposed algorithm.