DCFF-Net:基于人体骨骼点的双流跨级特征融合动作识别网络

DCFF⁃Net: Dual⁃stream cross⁃level feature fusion networkfor skeleton⁃based action recognition

在基于骨骼的动作识别任务中,骨骼点特征对于动作识别来说至关重要。针对现有方法存在输入特征不足、特征融合策略粗糙、参数量大等问题,提出一种基于人体骨骼点的双流跨级特征融合网络。首先,针对特征输入,用欧氏距离骨架特征(EDSF)和余弦角度骨架特征(CASF)两种局部关节特征来表征人体骨骼序列,帮助网络识别不同体态和体态相似的人体动作;其次,考虑到部分动作类别的运动轨迹与全局运动的相关性,引入全局运动特征(GMF)弥补局部关节特征在此类动作上识别精度不足的问题;此外,为了加强不同特征之间的信息交互,提出一种跨级特征融合模块(CLFF),对不同特征层、不同属性的动作特征进行特征互补,丰富了网络的特征形式;最后,网络采用一维卷积(Conv1D)进行搭建,减轻了模型的计算负担。实验结果表明,所提模型在JHMDB身体动作数据集上获得了84.1%的识别准确率,在SHREC手势动作数据集上分别获得了97.4%(粗糙数据集)和95%(精确数据集)的识别准确率,取得了与先进方法相当的性能。

In the skeleton⁃based action recognition task,skeleton features are crucial for action recognition.In view of the insufficient input features,rough feature fusion strategies,and a large number of parameters in the existing methods,a dual⁃stream cross⁃level feature fusion network(DCFF⁃Net)based on skeleton is proposed.For feature input,two local joint features,Euclidean distance skeleton features(EDSF)and cosine angle skeleton features(CASF),are used to characterize the human skeleton sequence to help the network identify human body movements in different postures and similar postures.Considering the correlation between the motion trajectories of some action categories and global motion,global motion features(GMF)are introduced to make up for the lack of recognition accuracy of local joint features in such actions.In addition,in order to strengthen the information interaction among different features,a cross⁃level feature fusion(CLFF)module is proposed to complement the action features of different feature layers and different attributes,which enriches the characteristics of the network form.The network is built with Conv1D,which reduces the computational burden of the model.Experimental results show that the proposed model achieves a recognition accuracy of 84.1%on the body action dataset JHMDB and 97.4%(coarse dataset)and 95%(fine dataset)on the gesture action dataset SHREC.To sum up,the proposed network achieves the performance comparable to the advanced methods.