基于单阶段生成对抗网络的文本生成图像方法

Text-to-image generation method based on single stage generative adversarial network

为了提高生成图像质量,提出新的文本生成图像方法,整体框架采用单阶段文本生成图像主干.在原有模型只使用句子信息生成图像的基础上,使用注意力机制把单词信息融入图像特征,采用合理地融入更多文本信息的方式提高生成图像的质量.引入对比损失,使相同语义图像之间更加接近,不同语义图像之间更加疏远,从而更好地保证文本与生成图像之间的语义一致性.在生成器中采用动态卷积来增强生成器的表达能力.实验结果表明,所提方法在数据集CUB(Fréchet inception distance(FID)从12.10提升到10.36)和数据集COCO(FID从15.41提升到12.74)上都获得了较好的性能提升.

A novel text-to-image generation method was proposed to enhance the quality of generated images,utilizing single-stage text-to-image generation backbone.On the basis of the original model that exclusively used sentence information for image generation,an attention mechanism was employed to integrate word information into image features.The quality of generated images was improved by judiciously incorporating additional textual information in a reasonable manner.The introduction of contrast loss makes the same semantic images closer and different semantic images more distant,so as to better ensure the semantic consistency between the text and the generated image.Dynamic convolution was used in the generator to enhance the expression ability of the generator.Experimental results illustrate that the proposed method obtains substantial performance improvements in both the CUB(Fréchet inception distance(FID)from 12.10 to 10.36)and COCO(FID from 15.41 to 12.74)datasets.