融合图像场景与目标显著性特征的图像描述生成方法

Image caption based on fusion of image scene and target saliency feature

图像描述生成是图像人工智能领域的重要研究方向之一。现有方法大多仅使用单一图像特征,导致无法完全描述图像中多个目标或者无法准确表达目标间的关系。提出方法通过场景特征解码模块和目标显著性特征解码模块分别对图像场景特征和目标显著性特征进行解码,并将解码后的两种特征进行融合,实现图像目标属性与目标间关系的信息互补。在MSCOCO数据集上进行实验,结果相较于基准方法有一定的提升,表明该模型对图像内容的描述更加准确和完善,对图像信息表达更加丰富。

Image caption is one of the important research directions in the field of image artificial intelligence.Most of the existing methods only use single image feature,which makes it impossible to fully describe multiple objects in the image or accurately express the relationship between objects.The proposed method decoded the image scene feature and the target salient feature respectively through the scene feature decoding module and the target salient feature decoding module,and fused the decoded features to realize the information complementarities between the image target attribute and the target relationship.The experimental results on MSCOCO dataset show that the proposed method has a certain improvement compared with the benchmark method,which shows that the description of image content is more accurate and perfect,and the expression of image information is more abundant.