结合双胞胎结构与生成对抗网络的半监督遥感图像融合

Semi-Supervised Remote Sensing Image Fusion Method Combining Siamese Structure with Generative Adversarial Networks

针对当前遥感图像融合算法中存在的标签图像难获取和光谱畸变等问题,提出一种采用双胞胎结构的半监督遥感图像融合方法.采用了由生成器和鉴别器组成的生成对抗网络结构,其中生成器包含编码器和解码器.首先,对多光谱图像进行放大并转换到HSV空间;将多光谱图像的V通道和全色图像分别送入编码器中的双胞胎网络后,通过卷积层和多重跳层连接模型来提取图像特征;然后,将获得的特征送入解码器进行图像重构;再利用鉴别器对融合后的V通道图像进行鉴别,从而获得最优融合结果;最后,将融合后的V通道与多光谱图像的H,S通道拼接起来获得最终的融合图像.另外,设计了一种复合损失函数进行模型训练.在QuickBird卫星遥感图像数据集上的实验表明,该方法有效提高了融合图像中的空间细节信息和色彩信息,与对比算法相比,其融合图像在主观视觉质量和客观评价指标上都具有一定的优势.

To solve the problems of acquisition of label image and spectral distortion in the current remote sensing image fusion,a semi-supervised remote sensing image fusion method using Siamese structure is proposed.This method adopted a generative adversarial network structure composed of generator and discriminator,in which the generator contains two encoders and a decoder.First,the multispectral image is amplified and converted into HSV color space.Then,the V channel of the multispectral image and panchromatic images are respectively input into the Siamese network of the encoder,and the image features are extracted through the convolutional layer and the multi-skip connection layer model.Third,the obtained feature map is input to the decoder for image reconstruction.And the fused V channel image is identified by the discriminator,so as to obtain the optimal fusion result.Finally,the fused V channel is concatenated with the H and S channels of the multispectral image to obtain the final fused image.In addition,a compound loss function is designed.Experiments on QuickBird satellite remote sensing image dataset show that this method can effectively improve spatial details and color information in fused images.Compared with the contrast algorithms,the fusion images have certain advantages in subjective visual quality and objective evaluation index.