基于全局时空感受野的高效视频分类方法

Efficient Video Classification Method Based on Global Spatiotemporal Receptive Field

在现有混合卷积神经网络架构(2D+3D)的视频分类方法中,卷积滤波器都是对局部区域进行操作,无法捕获大范围的时空依赖关系,特征通道之间缺乏相互依赖关系,传统的三维卷积核无法很好地建模时空特征.针对这些问题,提出了一种基于全局时空感受野的高效视频分类方法(CS-NL-SECO).首先将传统的三维卷积核分解成空域卷积核和时域卷积核,来更好地学习时空特征.然后在已有混合架构中的底层二维网络引入通道和空间注意力,通过学习自动获取每个特征通道的权重,依照权重关注重要的特征而抑制不相关的背景.最后在高层三维网络中引入全局时空感受野,学习全局时空特征表示自动捕获大范围的时空依赖关系.并在UCF101、HMDB51、Kinetics以及Something-something这四个视频分类常用的公有数据集上进行了实验,结果表明该方法无论在速度和精度上都远好于原方法,并且整体性能达到了最新方法的基准.

In the video classification methods of mixed convolutional neural network architecture(2 D+3 D),Convolution filters all operate on local regions and cannot capture a wide range of spatiotemporal dependencies,there is a lack of interdependence between feature channels,and traditional 3 D convolution kernels cannot model spatiotemporal features.To address these issues,this paper proposes an efficient video classification method based on global spatiotemporal field(CS-NL-SECO).First,channels and spatial attention mechanism is introduced into the low-level 2 D networks of the mixed architecture to automatically select and focus on specific areas in important features to suppress irrelevant backgrounds.Second,global spatiotemporal receptive fields are introduced into highlevel 3 D networks,and global spatiotemporal features are learned to automatically capture a wide range of spatiotemporal dependencies.the method was tested on four public datasets commonly used for video classification:UCF101,HMDB51,Kinetics,and Something-something.This method is far better than baseline in terms of speed and accuracy,and the overall performance achieves the benchmark of state-of-the-art.