基于深度学习的图像描述综述

Overview of Image Captions Based on Deep Learning

图像描述旨在通过提取图像的特征输入到语言生成模型中最后输出图像对应的描述,来解决人工智能中自然语言处理与计算机视觉的交叉领域问题——智能图像理解.现对2015—2020年间图像描述方向有代表性的论文进行汇总与分析,以不同核心技术作为分类标准将图像描述大致划分为基于Encoder-Decoder框架的图像描述、基于注意力机制的图像描述、基于强化学习的图像描述、基于生成对抗网络的图像描述和基于新融合数据集的图像描述五大类.使用NIC、Hard-Attention和Neural Talk三个模型在真实数据集MS-COCO数据集上进行实验,并从BLEU1、BLEU2、BLEU3、BLEU4四处平均评分对比分析,展示三个模型效果.本文点明了未来图像描述的发展趋势,并指出了图像描述将要面临的挑战和可深入挖掘的研究方向.

Image caption aims to extract the features of the image and input the description of the final output image into the language generation model,which solves the intersection of natural language processing and computer vision in artificial intelligence-image understanding.Summarize and analyze representative thesis of image description orientation from 2015 to 2020,different core technologies as classification criteria,it can be roughly divided into:image caption based on Encoder-Decoder framework,image caption based on attention mechanism,image caption based on reinforcement learning,image caption based on Generative Adversarial Networks,and based on new fusion data set these five categories.Use three models of NIC,Hard-Attention and Neural Talk to conduct experiments on the real data set MS-COCO data set,and compare the average scores of BLEU1,BLEU2,BLEU3,and BLEU4 to show the effects of the three models.This article points out the development trend of image caption in the future,and the challenges that image caption will face and the research directions that can be digged in.