基于注意力和强化学习的遥感图像描述方法

Remote Sensing Image Caption Method Based on Attention and Reinforcement Learning

针对当前遥感目标检测方法只能识别出遥感目标的类别及位置,无法生成与遥感图像内容相关文本描述的问题,提出了一种基于注意力和强化学习的遥感图像描述方法。首先,采用卷积神经网络构建编码器,提取遥感图像的特征。其次,利用长短期记忆网络搭建解码器,学习图像特征与文本语义特征间的映射关系。然后,引入注意力机制,增强模型对显著性特征的关注,减少无关背景特征的干扰。最后,采用强化学习策略,根据离散且不可微的评价指标直接对模型进行优化,消除暴露偏差及优化方向不一致的缺陷。在公开遥感图像描述数据集中的实验结果表明,本方法的检测精度较高,对密集小目标、雾气积聚、背景特征与目标特征相似等复杂环境下的遥感图像具有良好的描述性能。

The current remote sensing object detection methods,only identifying the category and location of remote sensing objects,cannot generate text caption related to the contents of remote sensing images.A remote sensing image caption method based on attention and reinforcement learning is proposed in this paper to solve this problem.First,the convolution neural network is used to construct an encoder and thereby extract remote sensing image features.Secondly,a decoder is built through the long short-term memory network to learn the mapping relationships of the image features with text semantic features.Thirdly,the attention mechanism is introduced to enhance the attention of the model on salient features and reduce the interference of irrelevant background features.Finally,the reinforcement learning strategy is adopted to optimize the model directly according to the discrete and non-differentiable evaluation indexes and thus to eliminate the defects of exposure bias and inconsistent optimization directions.Experimental results of public data sets of remote sensing image caption show that the method achieves high detection accuracy and has good caption performance for remote sensing images in complex environments such as dense small targets,fog accumulation,and similar background and object features.