基于双参考帧的多源着色结果融合的视频着色方法

Video colorization method based on fusion of multi-source colorization results using dual reference frames

对黑白视频着色时,为了更好地利用参考帧信息,提出一种基于双参考帧的多源着色结果融合的着色方法.首先,采用硬注意力融合子模块融合双参考帧提供的颜色信息,以防止双帧语义匹配模块着色时不合理参考信息导致的颜色模糊问题.然后,使用多源着色结果融合模块融合通过双帧光流传播模块、双帧语义匹配模块以及遮挡信息得到的着色结果,进而产生更优质的着色结果.实验结果表明,该方法在Davis30测试集上的峰值信噪比(PSNR)、结构相似性(SSIM)和颜色分布一致性指数(CDC)分别为37.36 dB、0.980 5、0.003 748,说明该方法能够通过多种融合方式充分利用双参考帧的信息为灰度帧着色,并生成美观且具有较好时间一致性的着色结果.

To better leverage reference frame information,a colorization method for black-and-white videos based on fusion of multi-source colorization results using dual reference frames is proposed.First,a hard attention fusion submodule is employed to fuse color information from two reference frames,preventing color blurring caused by unreasonable reference information during the coloring of the dual-frame semantic matching module.Then,a multi-source colorization result fusion module fuses colorization results obtained through the dual-frame optical flow propagation module,the dual-frame semantic matching module and occlusion information,thereby producing a better colorization result.Experimental results show that the peak signal to noise(PSNR),the structural similarity(SSIM),and the color distribution consistency index(CDC)of the method on the Davis30 test dataset are 37.36 dB,0.9805,0.003748,respectively.This demonstrates that the method can fully utilize information from dual reference frames to colorize grayscale frames through multiple fusion methods and generate aesthetically pleasing and temporally consistent colorization results.