TCSNGAN:基于Transformer和谱归一化CNN的图像生成模型

TCSNGAN:image generation model based on Transformer and CNN with spectral normalization

生成对抗网络(generative adversarial network,GAN)已成为图像生成问题中常用的模型之一,但是GAN的判别器在训练过程中易出现梯度消失而导致训练不稳定,以致无法获得最优化的GAN而影响生成图像的质量。针对该问题,设计满足Lipschitz条件的谱归一化卷积神经网络(CNN with spectral normalization,CSN)作为判别器,并采用具有更强表达能力的Transformer作为生成器,由此提出图像生成模型TCSNGAN。CSN判别器网络结构简单,解决了GAN模型的训练不稳定问题,且能依据数据集的图像分辨率配置可调节的CSN模块数,以使模型达到最佳性能。在公共数据集CIFAR-10和STL-10上的实验结果表明,TCSNGAN模型复杂度低,生成的图像质量优;在火灾图像生成中的实验结果表明,TCSNGAN可有效解决小样本数据集的扩充问题。

GAN has become one of the commonly-used image generation models.However,the discriminator of GAN is prone to the vanishing gradient problem in the training process,which leads to the instability of training.So that it is difficult to obtain the optimal GAN,and the quality of generation image is poor.To solve this problem,it designed a CNN with spectral normalization which satisfied the Lipchitz condition as the discriminator.Together with the Transformer generator,this paper proposed an image generation model,namely TCSNGAN(Transformer CSN GAN).The network structure of discriminator was simple,which could solve the problem of training instability of GAN model,and could configure the number of adjustable CSN modules according to the image resolution of data sets to achieve the optimal performance of the model.Experiments on public datasets CIFAR-10 and STL-10 show that the proposed TCSNGAN model has low complexity,and the generated image quality is good.And the experiments of fire image generation task demonstrates the effectiveness of small-sample dataset augmentation.