基于双专用注意力机制引导的循环生成对抗网络

Cycle generative adversarial network guided by dual special attention mechanism

现有基于循环生成对抗网络的图像生成方法通过引入独立通用的注意力模块,在无匹配图像转换任务中取得了较好的效果,但同时也增加了模型复杂度与训练时间,而且难以关注到图中关键区域的所有细节,图像生成效果仍有提升的空间。针对上述问题,提出一种基于双专用注意力机制引导的循环生成对抗网络(Dual-SAG-CycleGAN),分别对生成器和判别器采用不同的注意力机制进行引导。首先,提出一种名为SAG(Special Attention-mechanism Guided)的专用注意力模块来引导生成器工作,在提升生成图像质量的同时降低网络的复杂度;然后,对判别器采用基于CAM(Class Activation Mapping)的专用注意力机制引导模块,抑制生成器生成无关的噪声;最后,提出背景掩码的循环一致性损失函数,引导生成器生成更加精准的掩码图,更好地辅助图像转换。实验证明,本文方法与现有同类模型相比,网络模型参数量降低近32.8%,训练速度快34.5%,KID与FID最低分别可达1.13和57.54,拥有更高的成像质量。

The existing image generation methods based on cycle generative adversarial network have achieved excellent results in unpaired image to image translation tasks by introducing a separate generic attention module,but they also increase the model complexity and training time,and it is difficult to focus on all the details of key regions in the image,and there is still room for improvement in image generation results.To solve these problems,this paper proposes a dual special attention-mechanism guided cycle generative adversarial network architecture for unpaired image transformation translation(Dual-SAG-CycleGAN).Firstly,to improve the quality of the generated images and reduce the complexity of the network,a special attention module called SAG(Special Attention-mechanism Guided)is proposed to guide the generator.Then,to suppress the generation of extraneous noise by the generator,a discriminator based on special attention mechanism of CAM(Class Activation Mapping)is introduced.Finally,a cyclic consistency loss function for the background mask is introduced to guide the network to generate a more accurate mask map,which can better aid for image translation.Experiments demonstrate that compared with similar existing models,our proposed method can reduce the maximum number of parameters by 32.8%,speed of train 34.5%faster,and generate higher quality images with a minimum KID and FID of 1.13 and 57.54,respectively.