基于GAN改进的红外光与可见光图像融合算法研究

An Improved Infrared and Visible Image Fusion Algorithm Based on GAN

针对夜晚户外场景下,传统的单一鉴别器生成对抗网络(GAN)容易忽略红外光的亮度信息和边缘信息的问题,提出一种基于注意力机制与双鉴别器的红外光与可见光图像融合算法。首先,为了有针对性地获得红外光图像的目标信息和可见光图像的背景纹理信息,在生成器网络中引入通道注意力机制;其次,使用双鉴别器的生成对抗网络,并设计一种新的鉴别器输入,在提高训练稳定性的同时更好地保留源图像信息;最后,损失函数设置为对抗损失、结构相似性损失和梯度损失,以约束鉴别器使其生成细节信息丰富的融合图像。在TNO数据集上的实验结果表明,所提算法得到的融合图像梯度变化更明显、边缘更加清晰,更符合人眼视觉效果。

Aiming at the problem that a single traditional discriminator Generative Adversarial Network(GAN)tends to ignore the brightness information and edge information of infrared light in the outdoor scene at night,a fusion algorithm of infrared and visible images based on attention mechanism and dual discriminators is proposed.Firstly,in order to pertinently obtain target information of infrared images and background texture information of visible images,a channel attention mechanism is introduced into the generator network.Secondly,the GAN with two discriminators is used,and a new discriminator input is designed to improve the training stability while better preserving the source image information.Finally,the loss functions are set as adversarial loss,structural similarity loss and gradient loss to constrain the discriminator for generating fusion images with rich details.The experimental results on the TNO dataset show that the fusion image obtained by this algorithm has more significant gradient changes and clearer edges,which is more in line with human visual effects.