融合场景及交互性特征的多人行为识别

Multi person behavior recognition based on scene and interactive features

人类的行为复杂多样,场景、外观、位置等信息均与行为息息相关.本文针对如何高效地综合利用这些信息的问题,提出了融合场景及交互性特征的多人行为识别方法,使用双通道的方式分别提取个体外观特征与场景特征.对于个体通道,采用注意力机制模块来关注与行为相关度更大的区域,并将提取的个体外观特征结合位置特征输入图卷积网络进行关系推理.其中,图卷积网络采用了余弦相似度的方法度量个体特征之间的相关性,并结合个体之间的位置特征进行关系推理;对于场景通道,使用在place365数据集上预训练的ResNet-50提取场景特征.最后,本文将个体以及场景通道所得的特征进行加权融合,得到群组以及所有个体的行为识别结果.在Collective Activity Dataset(CAD)数据集上的实验表明,该方法能提高行为识别的准确率,群组行为以及个体行为的准确率分别达到了92.29%与78.19%.

Human behavior is complex and diverse, and the information such as scene, appearance and location are closely related to human behavior. Aiming at the problem of how to make efficient comprehensive use of these information, a multi-person behavior recognition method integrating scene and interactive features was proposed, and the individual appearance features and scene features were extracted by two channels. For the individual channel, the attention mechanism module was used to focus on the areas with greater correlation with behavior, and the extracted individual appearance features combined with location features were input into the graph convolution network for relational reasoning. Among them, the graph convolution network used the cosine similarity method to measure the correlation between individual features, and combined the position features between individuals for relationship reasoning;For the scene channel, scene features were extracted by using ResNet-50 pretrained on place365 dataset. Finally, the final features obtained from individual channels and scene channels were weighted and fused to obtain the behavior recognition results of groups and all individuals. The experimental results on the Collective Activity Dataset(CAD) show that this method can improve the accuracy of behavior recognition, and the accuracy of group behavior and individual behavior reaches 92.29% and 78.19%.