基于时空交互注意力模型的人体行为识别算法

Human Action Recognition Algorithm Based on Spatio-Temporal Interactive Attention Model

针对传统的双流网络不能有效提取视频序列中的有效帧和帧中的有效区域,而导致识别准确率低的问题,提出了一种基于时空交互注意力模型(STIAM)的人体行为识别算法。首先,利用两个不同的深度学习网络分别提取空间和时间特征;其次,设计一种掩模引导的空间注意力模型,用于计算每一帧上的显著性位置;然后,设计一种光流引导的时间注意力模型,用于定位每个视频中的显著性帧;最后,分别将时间、空间注意力获得的权重与空间特征、时间特征进行加权融合,使模型实现时空交互性。在UCF101和Penn Action数据集上与现有的方法进行比较,实验结果表明,STIAM具有较好的特征提取能力,可以明显提升行为识别的精度。

A human action recognition algorithm is proposed based on spatio-temporal interactive attention model(STIAM)to solve the problem of low recognition accuracy.This problem is caused by the incapability of the two-stream network to effectively extract the valid frames in each video and the valid regions in each frame.Initially,the proposed algorithm applies two different deep learning networks to extract spatial and temporal features respectively.Subsequently,a mask-guided spatial attention model is designed to calculate the salient regions in each frame.Then,an optical flow-guided temporal attention model is designed to locate the saliency frames in each video.Finally,the weights obtained from temporal and spatial attention are weighted respectively with spatial features and temporal features to make this model realize the spatio-temporal interaction.Compared with the existing methods on UCF101 and Penn Action datasets,the experimental results show that STIAM has high feature extraction performance and the accuracy of action recognition is obviously improved.