基于注意力密集网络的伪彩色红外与可见光图像融合

Pseudo-color infrared and visible image fusion based on attention-dense network

针对现有红外与可见光图像融合算法中存在融合图像的纹理细节不清晰,红外信息和纹理细节的显示不平衡等问题,提出了一种基于注意力密集网络的伪彩色红外与可见光图像融合方法。首先对灰度的红外图像进行伪彩色处理再与彩色的可见光图像组成多通道数据输入融合网络。其次,设计了一种由卷积层和带有注意力模块的密集连接块组成的生成器网络结构,关注源图像的关键信息,增强网络提取源图像信息的能力。最后,利用红外像素、可见光像素、可见光梯度和红外梯度构建内容损失函数,以保持融合图像中红外目标和纹理细节的平衡。与5种具有代表性的融合方法进行定性和定量比较。结果表明,该方法所获得融合图像的峰值信噪比、信息熵、平均梯度和互信息取得最优值,分别为31.6841、6.5581、6.0096、3.0960。定量以及定性结果证明所提融合方法使融合图像具有更为丰富的纹理细节以及良好的视觉效果。

Current infrared and visible image fusion algorithms often suffer from issues such as unclear texture details in the fused image and an unbalanced display of infrared information and texture details.In this paper,we propose an image fusion method of pseudo-color infrared and visible images based on attention-dense network.The greyscale infrared image is first processed in pseudo-color and then combined with the colored visible image to form a multi-channel data input fusion network.Secondly,a generator network structure consisting of convolutional layers and densely connected blocks with attention modules is designed to focus on the key information of the source image and enhance the ability of the network to extract information from the source image.Finally,the content loss function is constructed by using infrared pixels,visible pixels,visible gradient and infrared gradient to keep the stability of infrared target and texture details in the fused image.Qualitative and quantitative comparisons are made with five representative fusion methods.The results show that the peak signal-to-noise ratio,information entropy,average gradient,and mutual information of the fused images obtained by this method achieve the optimal values of 31.6841,6.5581,6.0096,and 3.0960,respectively.The quantitative and qualitative results demonstrate that the proposed fusion method results in a fused image with richer texture details and good visual effects.