基于GAN和自注意力机制的红外与可见光图像融合方法

Infrared and visible light image fusion method based on GAN and attention mechanism

为了进一步提升生成对抗网络融合方法对图像中信息提取的能力,提出一种基于GAN和自注意力机制相结合的红外与可见光图像融合方法。在生成器模型中使用双通道设计,分别对红外图像和可见光图像进行特征提取,并且在生成器中引入自注意力机制增强每个像素之间的依赖关系,使得融合图像可以较好地保存红外图像和可见光图像中的信息。在判别器模型中,采用马尔可夫判别器(PatchGAN)作为融合框架的判别器。实验数据结果表明,相比现有图像融合方法,得到的融合图像细节纹理更丰富,对比度更高,具有更好的视觉效果,并且在相似性指标、相关系数、峰值信噪比等客观评价指标上均有不同程度的提高。

In order to improve the ability of generative adversarial network fusion methods to extract information from infrared and visible light images,a new visible and infrared image fusion method based on generative adversarial network is proposed.In the generator model,dual channels are used to extract features from infrared and visible light images,and attention mechanism is intro⁃duced to enhance the dependency relationship between each pixel,Enable the fusion of images to better preserve information from in⁃frared and visible light images.And Markov discriminator(PatchGAN)is used as the discriminator for the fusion framework in this pa⁃per.The experimental results show that compared to existing image fusion methods,the fused images obtained by this method have rich⁃er texture details,higher contrast,better visual effects,and have varying degrees of improvement in objective evaluation indicators such as similarity,correlation coefficient,and peak signal-to-noise ratio.