摘要
针对现有的双流卷积神经网络,无法充分的融合视频的时序信息,从而对视频的行为理解不充分的问题,提出了一种改进的双流网络模型.首先在原双流网络中,分别将VGG-16神经网络替换为改进的ResNet神经网络,对单帧RGB图像特征进行预处理,将提取到的数据特征输入到改进的残差网络中.其次,在时间流部分,将连续光流图作为改进的ResNet网络结构的输入.最后,将得到的空间静态信息和运动信息在Fusion层进行融合,利用Softmax最大似然函数完成行为识别的任务,得到最终结果.实验结果表明:在UCF-101和HMDB-51数据集上,识别算法的平均精度分别为94.2%和68.4%,与传统方法相比,准确率有所提升,验证了该方法的有效性.
Aiming at the problem that the existing two stream convolutional neural network can not fully fusing the timing information of video,and thus can not fully understand the behavior of video,an improved dual-flow network model is proposed.Firstly,in the original two stream network,the VGG-16 neural network is replaced by the improved ResNet neural network.The features of single frame RGB image are preprocessed,and the extracted data features are input into the improved residual network.Secondly,in the time flow part,the continuous optical flow diagram is used as the input of the improved ResNet network structure.Finally,the spatial static information and motion information are fused in the Fusion layer,and the maximum likelihood function of Softmax is used to complete the task of behavior recognition,and the final result is obtained.Experimental results show that on UCF-101 and HMDB-51 datasets,the average accuracy of the proposed algorithm is 94.2%and 68.4%,respectively.Compared with traditional methods,the accuracy is improved,which verifies the effectiveness of the proposed method.
作者
贾永乐
周李涌
刘月峰
弓彦章
JIA Yongle;ZHOU Liyong;LIU Yuefeng;GONG Yanzhang(Information Engineering School,Inner Mongolia University of Science and Technology,Baotou 014000,China;Baotou Data Research and Application Center,Discipline Inspection and Supervision Big Data Laboratory,Inner Mongolia Autonomous Region,Baotou 014000,China)
出处
《内蒙古科技大学学报》
CAS
2023年第2期145-148,共4页
Journal of Inner Mongolia University of Science and Technology
基金
内蒙古自治区自然科学基金资助项目(2019M506021)
内蒙古自治区研究生教育教学改革研究与实践资助项目(YJG20191012710)
内蒙古科技大学专项资助项目(2019ZD025)。
关键词
ResNet
光流图
时空特征
人体行为识别
ResNet
optical flow graph
the space-time characteristics
human action recognition