多尺度输入3D卷积融合双流模型的行为识别方法

Multi-scale 3D Convolution Fusion Two-Stream Networks for Action Recognition

基于视频的行为识别技术在计算机视觉领域有广泛的应用.针对当前存在的网络模型不能有效结合视频数据中的时空信息,并且缺乏对不同尺度数据之间的融合信息进行考虑等问题,提出一种结合双流网络以及3D卷积神经网络的多尺度输入3D卷积融合双流模型.首先利用2D残差网以及多尺度输入3D卷积融合网络获取视频中的时空维度信息;然后将2层网络得到的实验结果进行决策相加,有效地提升网络对视频中时空特征提取的能力;最后通过在多尺度输入3D卷积融合网络对不同尺度的数据进行不同策略的融合,提高了网络对不同尺度数据的泛化能力.实验结果表明,文中模型在数据集UCF-101以及HMDB-51的识别准确率分别为90.5%与66.3%;相比于其他方法,该模型能取得更高的识别精度,体现出文中方法的优越性与鲁棒性.

Action recognition technology based on videos has been widely used in the field of computer vision.The existing networks cannot effectively combine the spatio-temporal information of video data and lacks consideration of fusion information between different scale data.This paper proposes a multi-scale 3D convolution fusion two-stream network that combines the two-stream network and the 3D convolution neural network.Firstly,the spatial and temporal dimension information of videos are obtained by using 2D residual networks and multi-scale 3D convolution fusion networks.Then,experimental results of the two networks are combined with fusion,to effectively improve the ability of the network to extract the spatio-temporal features of videos.Finally,the generalization ability of the network to different scale data is improved by the fusion of different strategies in multi-scale 3D convolution fusion network.The model was experimented and test in the data set of UCF-101 and HMDB-51,the experimental results were 90.5%and 66.3%,compared with other algorithms,the proposed model can achieve higher recognition accuracies and embody the superiority and the robustness of the algorithm.