基于关节点数据关注RGB视频的双人交互行为识别

Two-person interactive behavior recognition based on joint point data focusing on RGB video

基于RGB视频与关节点数据结合的双人交互行为识别具有广阔的应用前景。针对RGB视频与关节点数据两种异构特征不能较好融合的问题,提出一种利用关节点数据关注RGB视频的双人交互行为识别方法。利用关节点数据分析得到主要运动部位并关注RGB运动特征,以此来融合RGB视频提供的全局外观信息和由关节点数据分析得出的局部运动信息,并进行有效的表征,解决以往双人交互行为特征中统一对待所有信息而忽略主要运动信息的问题,提高异构特征融合的合理性,并利用卷积神经网络提取深层特征进行双人交互行为识别。在国际标准的NTU RGB+D数据库下进行测试,结果表明:该算法提高了相似动作的识别准确率,证明了本算法的可行性。

The two-person interactive behavior recognition based on the combination of RGB video and joint point data has broad application prospects.Aiming at the problem that two heterogeneous features of RGB video and joint point data cannot be well integrated,a two-person interactive behavior recognition method that used joint point data to focus on RGB video was proposed.Joint point data analysis was used to obtain the main movement parts,and attention was paid to the RGB motion features.In this way,the global appearance information provided by the RGB video and the local motion information obtained from joint point data analysis were fused,and the features of them were effectively represented.The problem that all information was treated in a unified manner without considering the main motion information in the previous two-person interaction behavior characteristics was solved,and the rationality of heterogeneous feature fusion was improved.Convolutional neural network was used to extract deep features and perform two-person interactive behavior recognition.The method was tested using the international standard NTU RGB+D database.The results showed that the algorithm improves the recognition accuracy of similar actions,which proves the feasibility of the algorithm.