基于深度神经网络的电影彩色化研究

FILM COLORIZATION BASED ON DEEP NEURAL NETWORK

针对电影彩色化面临着上色质量和时序稳定性的双重挑战,提出一种带有循环结构的生成对抗网络,可用于电影的自动彩色化,不需要任何参考帧和人工干预。该网络基于经典的条件生成对抗网络:生成器用于生成彩色图像,完成彩色化任务;鉴别器用于鉴别真伪,提升生成器性能。引入循环结构和时序一致性损失,用于整合时序信息,解决上色的稳定性问题。实验结果表明,该方法在保证单帧图像上色的同时,可以有效减少生成的电影序列中的闪烁现象。

Aimed at the dual challenges of coloring quality and temporal consistency, a generative adversarial network with recurrent structure is proposed, which can be used in automatic film colorization without any reference frames and manual interventions. The network was based on classic conditional generation adversarial networks(cGAN). The generator was used to generate color frames and complete the task of colorization. The discriminator was used to identify real or fake frames and improve the performance of the generator. Recurrent structure and sequence consistency loss were introduced to utilize sequence information and solve the problem of coloring stability. The experimental results show that the network structure can effectively reduce flicker in generated movie sequences while ensuring the color of single frame image.