基于时空自注意力转换网络的群组行为识别

Spatio-temporal transformer network for group activity recognition

个体间关系信息的获取是群组行为识别中关键问题。为了获取更加丰富的关系信息,本文提出了一种时空自注意力转换网络(Spatio-Temporal Transformer Network)。空间自注意力转换模块可以同时处理群组中的所有个体,包括其外观特征和位置特征,以便提取个体间空间关系信息。使用时序自注意力转换模块进行时序建模。为了获得更加丰富有效的关系信息,提出了全局空间注意图,用以增强模型空间关系推理能力,使用时序掩膜优化时序自注意力转换模块。通过在Volleyball和Collective Activity数据集上实验验证,结果表明本文方法性能优于其它方法。

The relationship information between individuals is a key problem in group activity recognition. In order to obtain richer relational information,a spatio-temporal transformer network is proposed in this paper. The spatial transformer module processes all individuals in the cluster simultaneously,including their appearance features and location features,to extract information on the spatial relationships between individuals. Then the temporal transformer module is used for temporal sequence modeling. In order to obtain richer and effective relationship information,a global spatial attention map is proposed to enhance the model’s spatial relational reasoning ability,and a temporal mask is used to optimize the temporal transformer module. We verify our model on Volleyball and Collective Activity datasets,and the experimental results show that the proposed method achieves advanced performance.