基于时空融合的多帧压缩视频增强方法

Multi-frame compressed video enhancement based on spatio-temporal fusion

为了减少视频的存储和传输开销,通常对视频进行有损压缩处理以减小体积,往往会在视频中引入各类不自然效应,造成主观质量的严重下降。基于单帧的压缩图像复原方法仅利用当前帧有限的空间信息,效果有限。而现有的多帧方法则大多采用帧间对齐或时序结构来利用相邻帧信息以加强重建,但在对齐性能上仍有较大的提升空间。针对上述问题,提出一种基于多帧时空融合的压缩视频复原方法,通过设计的深度特征提取块和自适应对齐网络实现更优的对齐融合,充分地利用多帧时空信息以重建高质量视频。该方法在公开测试集上(HEVC HM16.5低延时P配置)优于所有对比方法,并在客观指标上(峰值信噪比PSNR)相比于目前最先进的方法STDF取得了平均0.13 dB的提升。同时,在主观比较上,该方法也取得了领先的效果,重建出更干净的画面,实现了良好的压缩不自然效应去除效果。

In order to reduce the storage and transmission cost of video,lossy compression is in frequent use,which however would incur various types of artifacts in the video and affect users’subjective visual experience.The single frame method cannot be directly applied to video processing,because they independently process each video frame,limiting the use of spatial information and causing limited effectiveness.Inter-frame alignment or temporal structure was often adopted in multi-frame methods to enhance the reconstruction results by utilizing the temporal information,but there remains much room for improvement in alignment performance.To solve the above problems,a multi-frame spatio-temporal compression artifact removal method was proposed to achieve better alignment fusion through the alignment fusion design.This method efficiently utilized the multi-frame spatio-temporal information to reconstruct high quality videos.The experimental results show that the proposed method outperforms other comparative methods on a number of test compressed videos with different resolutions(HM16.5 under low delay P),and that it can achieve an average improvement of 0.13 dB on the objective index peak signal to noise ratio(PSNR)compared with the state-of-the-art multi-frame method STDF.Meanwhile,the proposed method can yield promising results in subjective comparisons,reconstructing a clearer picture with a good effect of compression artifact removal.