基于双分支多尺度残差融合嵌套的SAR和多光谱图像融合架构与实验

Architecture and Experiments of SAR and Multispectral Image Fusion Based on Double-Branch Multiscale Residual-Fusion Nesting

基于深度学习融合合成孔径雷达(SAR)和多光谱(MS)图像的方法主要通过增加卷积层数量描述网络模型尺度,但未能提高算法对不同尺度空间细节特征的提取能力。该文设计双分支的多尺度残差融合嵌套连接网络架构(Double-branch Multiscale Residual-fusion Nested-connections Net,DMRN-Net),将融合任务划分为细节提升和光谱保持两部分:在细节提升分路中,将SAR和MS图像中的高频信息分别经过多深度特征提取层、多尺度残差融合网络层及嵌套连接解码器得到重建图像;在光谱保持分路中,通过融合上采样后的MS图像和细节提升分路结果,将光谱信息注入融合图像中,从而得出融合结果。通过DMRN-Net和传统算法以及普通双分支网络的对比实验表明,DMRN-Net在主观判断和客观评价上均取得较好的融合结果,能在保持光谱信息的基础上,进一步增加图像的空间细节信息,验证了DMRN-Net在图像融合领域的重要价值。

The method of deep learning fusion of synthetic aperture radar(SAR)and multispectral(MS)images describes the scale of network model mainly by increasing the number of convolution layers.However,this method fails to improve the ability of extracting spatial details at different scales.In order to solve this problem,this paper combines the double-branch network,multiscale residual module and nested connection network to design a double-branch multiscale residual-fusion nested-connections net(DMRN-Net)for fusion of SAR and MS images.In DMRN-Net,the fusion task is divided into detail improvement of the image and spectrum preservation:in the detail improvement branch,the high-frequency information in the SAR and MS images is firstly processed,and then the high-frequency information is passed through the multi-depth feature extraction layer,multiscale residual fusion network layer and embedding layer respectively,and the reconstructed image is obtained by connecting the decoder;in the spectrum preservation branch,the up-sampled MS image and the acquisition results of the detail improvement branch are first fused,and the spectral information is injected into the fusion image.The comparison experiment between DMRN-Net,traditional algorithm and double-branch network shows that DMRN-Net achieves good fusion results in both subjective judgment and objective evaluation,which verifies the important value of the proposed algorithm in the field of SAR and MS image fusion in terms of spectrum and details.