基于多流融合生成对抗网络的遥感图像融合方法

Remote Sensing Image Fusion Based on Generative Adversarial Network with Multi-stream Fusion Architecture

由于强大的高质量图像生成能力,生成对抗网络在图像融合和图像超分辨率等计算机视觉的研究中得到了广泛关注。目前基于生成对抗网络的遥感图像融合方法只使用网络学习图像之间的映射,缺乏对遥感图像中特有的全锐化领域知识的应用。该文提出一种融入全色图空间结构信息的优化生成对抗网络遥感图像融合方法。通过梯度算子提取全色图空间结构信息,将提取的特征同时加入判别器和具有多流融合架构的生成器,设计相应的优化目标和融合规则,从而提高融合图像的质量。结合WorldView-3卫星获取的图像进行实验,结果表明,所提方法能够生成高质量的融合图像,在主观视觉和客观评价指标上都优于大多先进的遥感图像融合方法。

The generative adversarial network receives extensive attention in the study of computer vision such as image fusion and image super-resolution,due to its strong ability of generating high quality images.At present,the remote sensing image fusion method based on generative adversarial network only learns the mapping between the images,and lacks the unique Pan-sharpening domain knowledge.This paper proposes a remote sensing image fusion method based on optimized generative adversarial network with the integration of the spatial structure information of panchromatic image.The proposed algorithm extracts the spatial structure information of the panchromatic image by the gradient operator.The extracted feature would be added to both the discriminator and the generator which uses a multi-stream fusion architecture.The corresponding optimization objective and fusion rules are then designed to improve the quality of the fused image.Experiments on images acquired by WorldView-3 satellites demonstrate that the proposed method can generate high quality fused images,which is better than the most of advanced remote sensing image fusion methods in both subjective visual and objective evaluation indicators.