基于深度残差网络的双阶段视频显著性检测

TWO-STAGE VIDEO SALIENCY DETECTION BASED ON DEEP RESIDUAL NETWORK

为了进一步推进视频显著性检测的研究,提出一种以深度残差网络和U-net为基本结构的双阶段视频显著性检测方法。用静态图像和视频序列训练第一阶段模型来分别提高模型对于空间特征和时序特征的学习能力;通过调整基本结构的输入端,融合连续三帧视频序列以及第一阶段得到的显著图作为每次的输入来训练第二阶段的模型,增强模型学习的时序特征;融合周期性学习率,使得学习率周期性变化,保证模型在训练的每个阶段可以利用到最佳学习率,以此更好更快地达到收敛。实验表明,该方法在两个公开视频数据集上的检测效果均超过了当前主流的方法,检测精度更高,鲁棒性更好。

In order to further promote the research of video saliency detection,we proposed two-stage video saliency detection based on deep residual network and U-net.The static image and video sequence were used to train the model in the first stage to improve the learning ability of the model for spatial and temporal features.By adjusting the input end of the basic structure,fusing three consecutive video sequences and saliency images obtained in the first stage as input,the second stage model was trained to enhance the temporal characteristics of model learning.In addition,the cyclical learning rate was fused to make the learning rate change periodically,which ensured that the model could use the optimal learning rate in each stage of the training process to achieve better and faster convergence.Experiments show that the proposed method is better than the current mainstream methods on both open video datasets,and has higher detection accuracy and better robustness.