联合图像域间和域内信息建模的图像风格转换

Joint Intra-Domain and Inter-Domain Information Modeling for Image-to-Image Translation

针对当前图像风格转换算法缺乏建模图像域间语义信息和域内长范围信息的能力,提出了一种联合图像域间和域内信息建模的图像风格转换算法SSC-GAN.通过提出语义残差连接,提取图像域内的语义特征,增强模型建模图像域间语义信息差异的能力;同时,将注意力机制引入图像风格转换任务中,解决卷积缺乏图像域内长范围信息建模能力的问题.SSC-GAN可以在不增加计算量的情况下,显著提升图像风格转换的表现.在图像风格转换数据集vangh2photo和selfie2anime上对SSC-GAN进行训练、评估和验证,结果表明,SSC-GAN不仅能取得极佳的视觉效果,而且在FID和KID指标上分别平均下降了1.3和1.1,证明了SSC-GAN的有效性.

To solve the disability of modeling inter-domain semantic and intra-domain long-range information in current image style transferring algorithms this article proposes a novel image style transferring algorithm SSC-GAN.This method extracts the semantic features by constructing the semantic shortcut connections,thereby enhancing its capability of modeling semantic difference between image domains.Meanwhile,the self-attention mechanism is introduced for the modeling of long-range dependency in the image domain.The SSC-GAN can significantly improve the performance of image style transferring without extra computation.Through the extensive experiments on vangh2photo and selfie2anime datasets,the proposed method achieves excellent visual effects and reduces FID and KID by 1.3 and 1.1 on average respectively,which verifies the effectiveness of SSC-GAN.