基于自适应可分离卷积核的视频压缩伪影去除算法

Video compression artifact removal algorithm based on adaptive separable convolution network

针对目前视频质量增强和超分辨率重建等任务中常采用的光流估计相关算法只能估计像素点间线性运动的问题,提出了一种新型多帧去压缩伪影网络结构。该网络由运动补偿模块和去压缩伪影模块组成。运动补偿模块采用自适应可分离卷积代替传统的光流估计算法,能够很好地处理光流法不能解决的像素点间的曲线运动问题。对于不同视频帧,运动补偿模块预测出符合该图像结构和像素局部位移的卷积核,通过局部卷积的方式实现对后一帧像素的运动偏移估计和像素补偿。将得到的运动补偿帧和原始后一帧联结起来作为去压缩伪影模块的输入,通过融合包含不同像素信息的两视频帧,得到对该帧去除压缩伪影后的结果。与目前最先进的多帧质量增强(MFQE)算法在相同的训练集和测试集上训练并测试,实验结果表明,峰值信噪比提升(ΔPSNR)较MFQE最大增加0.44 dB,平均增加0.32 dB,验证了所提出网络具有良好的去除视频压缩伪影的效果。

The existing optical flow estimation methods, which are frequently used in video quality enhancement and super-resolution reconstruction tasks, can only estimate the linear motion between pixels. In order to solve this problem, a new multi-frame compression artifact removal network architecture was proposed. The network consisted of motion compensation module and compression artifact removal module. With the traditional optical flow estimation algorithms replaced with the adaptive separable convolution, the motion compensation module was able to handle with the curvilinear motion between pixels, which was not able to be well solved by optical flow methods. For each video frame, a corresponding convolutional kernel was generated by the motion compensation module based on the image structure and the local displacement of pixels. After that, motion offsets were estimated and pixels were compensated in the next frame by means of local convolution. The obtained compensated frame and the original next frame were combined together as input for the compression artifact removal module. By fusing different pixel information of the two frames, the compression artifacts of the original frame were removed. Compared with the state-of-the-art Multi-Frame Quality Enhancement(MFQE) algorithm on the same training and testing datasets, the proposed network has the improvement of Peak Signal-to-Noise Ratio(ΔPSNR) increased by 0.44 dB at most and 0.32 dB on average. The experimental results demonstrate that the proposed network performs well in removing video compression artifacts.