基于特征交互和聚类的行为识别方法

Action Recognition Based on Feature Interaction and Clustering

针对现有行为识别方法缺乏对时空特征关系建模的问题,提出一种基于特征交互和聚类的行为识别方法.首先设计一种混合多尺度特征提取网络提取连续帧的时间和空间特征;然后基于Non-local操作设计一种特征交互模块实现时空特征的交互;最后基于三元组损失函数设计一种难样本选择策略来训练识别网络,实现时空特征的聚类,提高特征的鲁棒性和判别性.实验结果表明,与基线方法TSN相比,所提方法的准确度在UCF101数据集上提高了23.25个百分点,达到94.82%;在HMDB51数据集上提高了20.27个百分点,达到44.03%.

To mitigate the problem that the action recognition methods lack the modeling of spatiotemporal feature relationship,an action recognition method based on feature interaction and clustering is proposed.Firstly,a mixed multi-scale feature extraction network is designed to extract spatial and temporal features of continuous frames.Secondly,a feature interaction module is designed based on non-local operation to realize spatiotemporal feature interaction.Finally,based on the triplet loss function,a hard sample selection strategy is designed to train the recognition network,thus realizing spatiotemporal feature clustering and improving the robustness and discrimination of the features.Experimental results show that compared with TSN,the accuracy of on the UCF101 dataset is increased by 23.25 percentage points to 94.82%.On the HMDB51 dataset,the accuracy is increased by 20.27 percentage points to 44.03%.