场景线稿动漫效果的自动上色算法

Automatic Colorization Algorithm with Anime Effect for Scene Sketches

场景线稿具有线条语义多样化的特点,直接应用现有的人像线稿图自动上色算法对其着色容易出现上色错误或棋盘效应等结果失真的现象.针对上述问题,文中提出动漫效果自动上色算法.基于条件生成对抗网络,改进和增强人像线稿图自动上色算法中常用的U型网络(U-Net)生成器的结构,设计双层信息抽取的生成器网络(DIEU-Net),自动完成场景线稿到动漫效果的上色.DIEU-Net设计用于抽取场景线稿浅层显著信息的双卷积子模块(IESS).构建双层IESS与残差结构的集成模块,插入生成器的不同阶段,增强网络在与线稿关联的颜色、位置等重要特征上的全域学习能力,缓和网络加深带来的梯度消失等网络退化问题.同时采用“卷积+上采样”操作替换U-Net生成器中原有的反卷积操作,抑制生成结果中棋盘效应的发生.实验表明,文中算法能较好地克服结果失真的问题,上色效果合理、自然,具有较好的应用推广性,可应用于多种类型景物线稿图的动漫上色.

When the existing automatic portrait coloring algorithms are directly applied to scene sketches,distortion phenomena are caused,such as wrong colorization and checkerboard artifacts,due to the diversified line semantics of scene sketches.To address this issue,an automatic colorization algorithm with anime effect for scene sketches is put forward.The structure of U-Net generator in the existing automatic portrait coloring algorithms is improved and enhanced based on the conditional generative adversarial network.A double-layer information extraction U-Net(DIEU-Net)is designed for automatic anime effect colorization of scene sketches.Firstly,the double-convolution sub-module prominence-information extraction of a scene sketch(IESS)is designed.Then,a module integrating double-layer IESS and residual structure is inserted into different stages of the proposed generator.Thus,the global learning ability of the generator on important features,like colors and positions related to the sketch,are enhanced,and the network degradation problems caused by vanishing gradients as the network deepens,are alleviated.Moreover,the deconvolution in U-Net is replaced by the operations of convolution and upsample to suppress the occurrence of the checkerboard artifacts.Experimental results show that the proposed algorithm performs well in avoiding the distortion phenomenon and achieves more reasonable and natural coloring effect than other algorithms.Furthermore,the proposed algorithm can be applied to automatic anime coloring of various types of scene sketches.