With the rapid development of deep learning technology,behavior recognition based on video streams has made great progress in recent years.However,there are also some problems that must be solved:(1)In order to improv...With the rapid development of deep learning technology,behavior recognition based on video streams has made great progress in recent years.However,there are also some problems that must be solved:(1)In order to improve behavior recognition performance,the models have tended to become deeper,wider,and more complex.However,some new problems have been introduced also,such as that their real-time performance decreases;(2)Some actions in existing datasets are so similar that they are difficult to distinguish.To solve these problems,the ResNet34-3DRes18 model,which is a lightweight and efficient two-dimensional(2D)and three-dimensional(3D)fused model,is constructed in this study.The model used 2D convolutional neural network(2DCNN)to obtain the feature maps of input images and 3D convolutional neural network(3DCNN)to process the temporal relationships between frames,which made the model not only make use of 3DCNN’s advantages on video temporal modeling but reduced model complexity.Compared with state-of-the-art models,this method has shown excellent performance at a faster speed.Furthermore,to distinguish between similar motions in the datasets,an attention gate mechanism is added,and a Res34-SE-IM-Net attention recognition model is constructed.The Res34-SE-IM-Net achieved 71.85%,92.196%,and 36.5%top-1 accuracy(The predicting label obtained from model is the largest one in the output probability vector.If the label is the same as the target label of the motion,the classification is correct.)respectively on the test sets of the HMDB51,UCF101,and Something-Something v1 datasets.展开更多
基金the National Science Fund for Distinguished Young Scholars,No.61425002the National Natural Science Foundation of China,Nos.91748104,61632006,61877008+3 种基金Program for ChangJiang Scholars and Innovative Research Team in University,No.IRT_15R07Program for the Liaoning Distinguished Professor,Program for Dalian High-level Talent Innovation Support,No.2017RD11the Scientific Research fund of Liaoning Provincial Education Department,No.L2019606the Science and Technology Innovation Fund of Dalian,No.2018J12GX036.
文摘With the rapid development of deep learning technology,behavior recognition based on video streams has made great progress in recent years.However,there are also some problems that must be solved:(1)In order to improve behavior recognition performance,the models have tended to become deeper,wider,and more complex.However,some new problems have been introduced also,such as that their real-time performance decreases;(2)Some actions in existing datasets are so similar that they are difficult to distinguish.To solve these problems,the ResNet34-3DRes18 model,which is a lightweight and efficient two-dimensional(2D)and three-dimensional(3D)fused model,is constructed in this study.The model used 2D convolutional neural network(2DCNN)to obtain the feature maps of input images and 3D convolutional neural network(3DCNN)to process the temporal relationships between frames,which made the model not only make use of 3DCNN’s advantages on video temporal modeling but reduced model complexity.Compared with state-of-the-art models,this method has shown excellent performance at a faster speed.Furthermore,to distinguish between similar motions in the datasets,an attention gate mechanism is added,and a Res34-SE-IM-Net attention recognition model is constructed.The Res34-SE-IM-Net achieved 71.85%,92.196%,and 36.5%top-1 accuracy(The predicting label obtained from model is the largest one in the output probability vector.If the label is the same as the target label of the motion,the classification is correct.)respectively on the test sets of the HMDB51,UCF101,and Something-Something v1 datasets.
