基于深度空时特征编码的视频显著性检测

Video Saliency Detection based on Encoding of Deep Spatiotemporal Features

针对目前视频显著性检测算法中存在难以提取鲁棒的空时特征和缺乏有效的空时显著性融合模型问题,提出基于深度空时特征编码的视频显著性检测算法。首先运用FlowNet提取视频帧的深度光流场,并基于全局对比度模型构造时间特征线索;其次,基于卷积网络提取每一帧图像的语义显著性区域,构造空间特征线索;最后,通过联合前后帧的空时显著性编码网络实现空时显著线索的融合,并得到最终显著图。在公开数据集上的实验结果表明,所提算法在检测精度上优于目前的主流算法,具有较强的鲁棒性。

Aiming at the problem that it is difficult to extract robust spatiotemporal features and lack of effective spatiotemporal features fusion model in the current video saliency detection algorithm, a video saliency detection algorithm based on deep spatiotemporal features coding is proposed. Firstly, FlowNet is used to extract the depth optical flow field of the video frame, and based on the global contrast model; the temporal feature cues are constructed. Then based on the convolutional network, the semantic saliency region of each frame image is extracted, and the spatial feature cues are constructed. Finally, by combining the spatiotemporal saliency coding network of the front and back frames, the fusion of spatiotemporal significant cues is realized, and the final saliency map acquired. Experiment on the public dataset indicates that the proposed algorithm is superior to the current mainstream algorithms in detection accuracy and has fairly strong robustness.