基于生成对抗网络的情感语义描述与生成

Emotional Semantic Description and Generation Based on Generative Adversarial Network

图像语义描述是指利用计算机对图像中的语义内容进行描述。虽然基于卷积神经网络和循环神经网络的编解码框架在生成图像描述方面已经取得不错的效果,但是在语义丰富性上还有待提升,并且基于循环神经网络的图像模型在生成单词时容易产生偏差累积,造成描述不够准确。为解决此问题,使用生成对抗网络来生成图像描述并且加入情感语料库,使生成的描述语句更加人性化,同时为了在特征提取时保留图像的特别关注点,使特征提取更加充分和完善,加入注意力机制。实验结果表明,所用方法在精确度和语义丰富性上均有很大提升。

Image caption refers to the use of the computer to describe the semantic content in the image. Although the encoding and decoding framework based on convolutional neural network and the cyclic neural network has achieved good results in generating image captions,it still needs to be improved in semantic richness,and the image model based on cyclic neural network is prone to deviation accumulation when generating words,resulting in the inaccurate description. In order to solve this problem,the generative adversarial network is used to generate image description,and the emotional corpus is added to make the generated description sentences more humanized. At the same time,in order to retain the special concerns of the image during feature extraction,feature extraction is more sufficient and perfect,and the attention mechanism is added. The experimental results show that the accuracy and semantic richness of the method is greatly improved.