基于长效递归深度卷积网络的实时人物动作识别

Real-time Human Action Recognition Based on Long-term Recurrent Deep Convolutional Networks

为高准确度识别实时人物动作模式,对长效递归卷积网络(LRCN)进行改进,得到长效递归深度卷积网络(LRDCN)。LRDCN使用多帧叠加的RGB图像与光流图像作为网络输入,将基于RGB图像的人体动作特征与基于光流图像的人体动作特征进行加权融合,结合卷积神经网络(CNN)对Two-stream算法进行扩展作为最终的人体动作特征,传入长效递归深度卷积网络进行序列学习,得出实时动作序列。在UCF-101和Weizman数据集上训练LRDCN,分别测试单一动作识别准确度与动作序列检测精确度。实验结果表明,神经网络层数增加不会引起梯度爆炸问题,单一动作识别准确率达到91.2%,动作序列识别准确率达到93.8%。LRDCN不仅能精确识别单个动作及动作序列,并且对长视频序列有较高的适应性。

In order to achieve high-accuracy real-time human action pattern recognition,a long-term recursive deep convolutional network(LRDCN)is obtained by improving the long-term recursive neural network(LRCN).LRDCN uses multi-frame superimposed RGB images and optical flow images as network inputs,combines the human action feature based on RGB images with the human motion feature based on optical flow images by weighted fusion.Combined with convolutional neural network(CNN),two-stream algorithm is extended as the final human action feature,which is transferred into LRDCN for sequence learning,and the real-time human action feature is obtained.LRDCN is trained on UCF-101 and Weizman datasets to test the accuracy of single action recognition and that of action sequence detection.The experimental results show that the increase of neural network layers will not cause gradient explosion,and the accuracy of single action recognition is 91.2%,and the accuracy of action sequence recognition is 93.8%.LRDCN can not only recognize the single action and action sequence accurately,but also has high adaptability to the long video sequences.