基于可变形非局部三维卷积网络的视频超分辨率重建算法

Video Super Resolution with Deformable Non-local 3D Convolutional Network

视频超分辨率(VSR)技术的目标是找出从相应的低分辨率(LR)视频序列重建高分辨率(HR)视频的最佳重建方案。提出了一种新颖的可变形非局部三维卷积网络(DNL-3DCNN)能有效地利用时空信息和参考帧与相邻帧之间的全局相关性。具体来说,非局部结构(Non-Local)同时增强了输入帧的时空信息中所需要的精细细节。此外,残差可变形三维卷积(R3D)获得了卓越的时空建模能力和运动感知建模的灵活性。此外,残差密集连接网络(RRDB)再进行重建处理,以充分利用输入到重建模块的层级特征。在基准数据集上进行的定量和定性实验表明,与现有的较为先进的VSR方法相比,所提方法在PSNR指标上提高了1.19db,在SSIM指标上提高了约5.95%。消融性实验确认提出的三个模块均带来了一定的性能增益,实验结果验证了所提算法在视频超分辨率时空信息重建领域的有效性。

This paper proposes a novel deformable non-local 3D convolutional network(DNL-3DCNN) capable of efficiently exploiting spatio-temporal information and global correlation between reference frames and adjacent frames.Specifically,the non-local structure(Non-Local) enhances both the fine details required in the spatio-temporal information of the input frames.In addition,the residual deformable 3D convolution(R3D) obtains superior spatio-temporal modeling capability and flexibility in motion-aware modeling.The residual densely connected network(RRDB) is then reconstructed to take full advantage of the layer-level features input to the reconstruction module.Quantitative and qualitative experiments conducted on benchmark datasets show that the proposed method in this paper improves 1.19 db in PSNR metrics and about 5.95% in SSIM metrics compared with existing more advanced VSR methods.The ablation experiments confirm that all three modules proposed in this paper bring certain performance gains,and the experimental results verify the effectiveness of the proposed algorithm in the field of video super-resolution spatio-temporal information reconstruction.