基于3D全时序卷积神经网络的视频显著性检测

Video Saliency Detection Based on 3D Full ConvLSTM Neural Network

视觉是人类感知世界的重要途径之一。视频显著性检测旨在通过计算机模拟人类的视觉注意机制,智能地检测出视频中的显著性物体。目前,基于传统方法的视频显著性检测已经达到一定的水平,但是在时空信息一致性利用方面仍不能令人满意。因此,文中提出了一种基于全时序卷积神经网络的视频显著性检测方法。首先,利用全时序卷积对输入视频进行空间信息和时间信息的时空特征提取;然后,利用3D池化层进行降维;其次,在解码层中用3D反卷积和3D上采样对前端特征进行解码;最后,通过把时空信息有机地提取与融合,来有效地提升显著图的质量。实验结果表明,所提算法在3个广泛使用的视频显著性检测数据集(DAVIS,FBMS,SegTrack)上的性能优于当前主流的视频显著性检测方法。

Video saliency detection aims to mimic human’s visual attention mechanism of perceiving the world via extracting the most attractive regions or objects in the input video.At present,it is still a challenge for video saliency detection.Traditional video saliency-detection models have reached a certain level,but exploiting the consistency of spatio-temporal information is unsatisfactory.In order to solve this issue,this paper proposes a video saliency-detection model based on 3D full ConvLSTM neural network.Firstly,the full-time convolution is utilized to extract spatio-temporal features from the input video,and then the 3D pooling layer is explored for dimensionality reduction.Secondly,the extracted features are decoded by 3D deconvolution in the decoding layer,and the interpolation algorithm is applied to restore the saliency map to the original size of the original image.The proposed method extracts the time and space information jointly so as to effectively enhance the completeness of the saliency map.Experimental results show that the performance of the proposed algorithm is superior to state-of-the-art video saliency detection methods based on three widely used data sets(DAVIS,FBMS,SegTrack)for video saliency detection.