基于差异双分支编码器的多阶段图像融合方法

Multi-stage Image Fusion Method Based on Differential Dual-Branch Encoder

在现有的红外和可见光图像融合方法中,融合图像中的细节信息丢失严重,视觉效果不佳.针对上述问题,文中提出基于差异双分支编码器的多阶段图像融合方法.通过两支不同结构的编码器提取多模态图像的特征,增强特征的多样性.设计多阶段的融合策略,实现精细化图像融合.首先,在差异双分支编码器中,对两个编码分支提取的差异性特征进行初级融合.然后,在融合阶段,对多模态图像的显著性特征进行中级融合.最后,使用远程横向连接将差异双分支编码器的浅层特征传送给解码器,同时指导融合过程和图像重建.对比实验表明,文中算法可增强融合图像的细节信息,并在视觉效果和客观评价上都较优.

In the existing infrared and visible image fusion methods,the details of the fused image are lost seriously and the visual effect is poor.Aiming at the problems,a multi-stage image fusion method based on differential dual-branch encoder is proposed.The features of multi-modal images are extracted by two encoders with different network structures to enhance the diversity of features.A multi-stage fusion strategy is designed to achieve refined image fusion.Firstly,primary fusion is performed on the differential features extracted by the two encoding branches in the differential dual-branch encoder.Then,mid-level fusion on the saliency features of the multi-modal images is conducted in the fusion stage.Finally,the long-range lateral connections are adopted to transmit shallow features of the differential dual-branch encoder implemented to the decoder and guide the fusion process and the image reconstruction simultaneously.Experimental results show the proposed method enhances the detailed information of the fused images and achieves better performance in both visual effect and objective evaluation.