基于关键词指导的图像中文描述生成

Image Description Generation in Chinese Based on Keywords Guidance

图像描述生成技术可以加速图文内容的生产,因而有着广泛的应用前景。为了满足实际需要,我们提出了一种基于编码–解码框架的新方法。我们的模型通过融合图像和文本特征作为输入来指导图像描述的生成,文本特征包含图像的关键词信息作为图像特征的补充。实验结果表明,关键词信息加强了图像到图像描述的映射,本文模型比未融合关键词信息的模型具有更好的性能,并且不同的关键词信息对图像描述的生成有一定的控制作用。

Technology for generating image description has a wide range of applications as it can speed up the production of graphic content. To meet the practical requirements, we propose a new method based on the encoder-decoder framework. Our model guides the generation of image description by fusing image and text features as input. The text features contain the semantics of keywords of images, as a supplement to image information. The experimental results show that keyword information enhances the mapping from image to image description. The model in this paper has better performance than the model without fusing keyword information, and different keyword information has certain control over the generation of image description.