基于双流独立循环神经网络的人体行为识别

Human action recognition with Two-stream independently recurrent neural network

提出了一种基于双流独立循环神经网络的人体行为识别方法。该方法结合人体行为的骨架时间结构和空间结构构建双通道神经网络。对于时间通道,根据人体运动学原理设计结合分层网络和堆叠网络的神经网络结构。对于空间通道,将原始骨架经过视角不变三维坐标变换,并把骨架坐标转换为坐标向量,有效克服人体体型和视角等差异带来的影响,以增强骨架空间表达能力。最后,由时间通道和空间增强通道分别经独立循环神经网络并综合评分识别人体行为。实验表明,所提出的模型在NTU RGB+D数据集上具有较好的识别率,结果证实该方法的有效性。

A two-stream independently recurrent neural network was proposed for human skeleton action recognition.It combined the temporal and spatial structure of the human skeleton to construct a dual-channel neural network.A network combining hierarchical network with the stacked network was designed according to human kinematics principles for the temporal channels.For the spatial channel,the original skeletons were transformed into coordinate vector through the three-dimensional coordinate transformation with an unchanged view to effectively overcome the influence of the size,shape of the human skeleton,angle and other differences and enhance the spatial expression ability of skeleton.Finally,the human behaviours were recognized by the temporal channel and the enhanced spatial channel respectively through an independent recurrent neural network and total score.Experiments show that the proposed method has a reasonable recognition rate on NTU RGB+D data sets,and the results confirm the effectiveness of the proposed method.