基于光流和深度运动图的行为识别算法

Action Recognition Algorithm Based on Optical Flow and Depth Motion Map

为了融合不易受光照等环境因素影响的深度信息和RGB视频序列中丰富的纹理信息,提出一种基于光流和深度运动图(Depth Motion Map,DMM)的人体行为识别算法.首先从RGB视频序列获取彩色信息(RGB视频帧)和光流信息,并且从同步的深度视频序列获取深度信息,以增强特征互补性,其次把3种特征信息分别作为基于ResNet101的空间流网络、时间流网络和深度流网络的输入,通过LSTMs进行特征融合,最后将特征送入Softmax层得到每个行为类别的概率值.实验结果表明,在具有挑战性的UTD-MHAD数据集和MSR Daily Activity 3D数据集上的行为识别准确率分别为94.86%和97.69%,在与该领域中的同类算法比较中表现优异.

In the report,in order to fuse the depth information with the rich texture information in RGB video sequences,a human action recognition algorithm based on optical flow and depth motion map(DMM)was proposed.Firstly,the color information(RGB video frame)and optical flow information were obtained from RGB video sequence,and depth information was obtained from synchronous depth video sequence to enhance feature complementarity;Secondly,the three kinds of characteristic information are used as the input of ResNet101 based spatial-stream network,temporal-stream network,and depth-stream network,and LSTMs were used for the feature fusing.Finally,the feature is sent to the Softmax layer to get the probability value of each behavior category.The results showed that the behavioral recognition accuracy on the challenging UTD-MHAD dataset and MSR Daily Activity 3D dataset are 94.86%and 97.69%,respectively,which are excellent in comparison with similar algorithms in the field.