基于深度学习特征匹配的视频超分辨率方法

Video Super-resolution Method Based on Deep Learning Feature Warping

视频复原的目标是从给定的退化视频序列中把潜在的高质量视频复原出来。现有的视频复原方法主要集中在如何有效地找到相邻帧之间的运动信息,然后利用运动信息建立相邻帧之间的匹配。与这些方法不同,文中提出了基于深度学习特征匹配的方法来解决视频超分辨率问题。首先,通过深度卷积神经网络计算出相邻帧之间的运动信息;然后,采用一个浅层深度卷积神经网络从输入的视频帧中提取特征,基于估计到的运动信息,将浅层深度卷积神经网络提取到的特征匹配到中间视频帧对应的特征中,并将得到的特征进行有效融合;最后,采用一个深度卷积神经网络重建视频帧。大量的实验结果验证了基于深度学习特征匹配的方法能有效地解决视频超分辨率问题。与现有的基于视频帧匹配的方法相比,所提方法在现有的公开视频超分辨率数据集上取得了较好的效果。

Video restoration aims to restore potential clear videos from a given degraded video sequence.Existing video restoration methods usually focus on modeling the motion information among adjacent frames and establishing the alignment among them.Different from these methods,this paper proposes a feature warping method based on deep learning for video super-resolution.Firstly,the proposed algorithm estimates the motion information based on deep convolutional neural networks.Then,it develops a shallow deep convolutional neural network to estimate the features from input frames.Based on the estimated motion information,this paper warps the deep features to those of the central frames.Next,the proposed method fuses the deep features effectively.Finally,this paper proposes a restoration network which is able to reconstruct clear frames.Experimental results de-monstrate the effectiveness of the proposed algorithm.The proposed algorithm performs well on the benchmark datasets compared to existing methods.