基于CNN-LSTM双流融合网络的危险行为识别

Dangerous Behavior Recognition Based on CNN-LSTM Dual-Stream Fusion Network

针对目前人体危险行为识别过程中由于时空特征挖掘不充分导致精度不够的问题,对传统双流卷积模型进行改进,提出了一种基于CNN-LSTM的双流卷积危险行为识别模型。该模型将CNN网络与LSTM网络并联,其中CNN网络作为空间流,将人体骨架空间运动姿态分为静态与动态特征进行分别提取,两者融合作为空间流的输出;在时间流中采用改进的可滑动长短时记忆网络,以增加人体骨架时序特征的提取能力;最后将两个分支进行时空融合,利用Softmax对危险动作做出分类识别。在公开的NTU-RGB+D数据集和Kinetics数据集上的实验结果表明,改进后模型的平均跨角度(Cross view,CV)精度达到92.5%,平均跨视角(Cross subject,CS)精度为87.9%。所提方法优于改进前及其他方法,可以有效地对人体危险动作做出识别,同时对于模糊动作也有较好的区分效果。

To solve the problem of insufficient spatial and temporal feature in the process of dangerous behavior recognition,this paper improves the traditional dual-stream convolution model and proposes a new dual-steam convolution dangerous behavior recognition model based on CNN-LSTM.In this model,CNN network and LSTM network are connected in parallel.CNN network is used as the spatial flow.The spatial motion attitude information of human skeleton is divided into static and dynamic.These features are fused as the output of the spatial flow.In order to increase the ability of extracting temporal features of human skeleton,an improved temporal sliding LSTM network is used in the time stream.Finally,the two branches are fused in time and space,and the dangerous actions are classified and identified by Softmax.Experimental results on NTU RGB D and Kinetics datasets show that the average cross view(CV)accuracy of the improved model is 92.5%and the average cross subject(CS)accuracy is 87.9%.The proposed method is superior to that before improvement and other methods.It can effectively recognize dangerous human actions and has good discrimination effect for fuzzy actions.