融合对象和多尺度视觉特征的遥感图像描述模型

Remote sensing image caption model with fusion of object and multiscale visual feature

基于遥感图像多尺度、无法准确提取微小物体、物体类别易混淆的问题,提出了一种融合对象和多尺度视觉特征的遥感图像描述模型(Fusion of Object and Multiscale Visual Feature,FO-MSV),通过构建的对象提取器分析文本信息,提取其中的对象信息;设计了一种多尺度交互模块,获取遥感图像的多尺度视觉特征,以适应多尺度的特点;为了充分利用对象信息并融合视觉信息,提出了一种新的对象-视觉特征融合机制,调整视觉上下文和对象上下文之间的平衡。基于该领域内三个数据集的实验结果表明,该模型能明显提升描述的性能,与其他先进模型相比具有竞争力。

Aiming at the problems that remote sensing image has multiscale features and the object categories are easy to be confused,cannot accurately extract the tiny objects from images,a new remote sensing image caption model(FO-MSV)is proposed,which analyzes the text information through the constructed object extractor,to extract the object information.A multiscale interaction module is designed to obtain the multiscale visual features of remote sensing images to adapt to the characteristics of multiscale.In order to make full use of object information and fuse visual information,a new object-visual feature fusion mechanism is proposed to adjust the balance between visual context and object context.Experimental results on three datasets show that the proposed model can significantly improve the performance of captions and is competitive compared with other advanced models.