基于中国写意风格迁移的动漫视频生成模型

Animation video generation model based on Chinese impressionistic style transfer

目前生成式对抗网络(GAN)已经被用于图像的动漫风格转换。然而,现有基于GAN的动漫生成模型主要以日本动漫和美国动漫为对象,集中在写实风格的提取与生成,很少关注到中国风动漫中写意风格的迁移,因此限制了GAN在国内广大动漫制作市场中的应用。针对这一问题,通过将中国写意风格融入到GAN模型,提出了一种新的中国风动漫生成式对抗网络模型CCGAN,用以自动生成具有中国写意风格的动漫视频。首先,通过在生成器中增加反向残差块,构造了一个轻量级的深度神经网络模型,以降低视频生成的计算代价。其次,为了提取并迁移中国写意风格中图像边缘锐利、内容构造抽象、描边线条具有水墨质感等性质,在生成器中构造了灰度样式损失和颜色重建损失,以约束真实图像和中国风样例图像在风格上的高层语义一致性,并且在判别器中构造了灰度对抗损失和边缘促进对抗损失,以约束重构图像与样例图像保持相同的边缘特性。最终,采用Adam算法最小化上述损失函数,从而实现风格迁移,并将重构图像组合为视频。实验结果表明,与目前最具代表性的风格迁移模型CycleGAN与CartoonGAN相比,所提CCGAN可从以《中国唱诗班》为例的中国风动漫中有效地学习到中国写意风格,同时显著降低了计算代价,适合于大批量动漫视频的快速生成。

At present,Generative Adversarial Network(GAN)has been used for image animation style transformation.However,most of the existing GAN-based animation generation models mainly focus on the extraction and generation of realistic style with the targets of Japanese animations and American animations.Very little attention of the model is paid to the transfer of impressionistic style in Chinese-style animations,which limits the application of GAN in the domestic animation production market.To solve the problem,a new Chinese-style animation GAN model,namely Chinese Cartoon GAN(CCGAN),was proposed for the automatic generation of animation videos with Chinese impressionistic style by integrating Chinese impressionistic style into GAN model.Firstly,by adding the inverted residual blocks into the generator,a lightweight deep neural network model was constructed to reduce the computational cost of video generation.Secondly,in order to extract and transfer the characteristics of Chinese impressionistic style,such as sharp image edges,abstract content structure and stroke lines with ink texture,the gray-scale style loss and color reconstruction loss were constructed in the generator to constrain the high-level semantic consistency in style between the real images and the Chinese-style sample images.Moreover,in the discriminator,the gray-scale adversarial loss and edge-promoting adversarial loss were constructed to constrain the reconstructed image for maintaining the same edge characteristics of the sample images.Finally,the Adam algorithm was used to minimize the above loss functions to realize style transfer,and the reconstructed images were combined into video.Experimental results show that,compared with the current representative style transfer models such as CycleGAN and CartoonGAN,the proposed CCGAN can effectively learn the Chinese impressionistic style from Chinese-style animations such as Chinese Choir and significantly reduce the computational cost,indicating that the proposed CCGAN is suitable for the rapid generation of animation videos with large quantities.