基于GAN轻量化改进的红外与可见光图像融合算法

An Infrared and Visible Light Image Fusion Algorithm Based on GAN Lightweight Improvement

普通神经网络难以生成符合人眼视觉的红外与可见光融合图像,且网络模型复杂、占用内存过大。本文改进现有的生成对抗网络(GAN)框架:首先,在生成器中融入深度卷积和逐点卷积,设计小卷积核的卷积网络以减少网络参数;其次,对源图像进行掩膜处理以减少提取特征过程中源图像信息的丢失;然后将处理后的图像和生成器得到的融合图像共同输入到鉴别器中,以增强网络对可见光图像保留源图像信息的能力;最后,在性能评价阶段将损失函数设置为梯度损失、对抗损失和内容损失函数,以约束融合图像,使其包含更多的可见光图像的背景信息以及红外图像的目标信息。在TNO image fusion dataset上进行了仿真实验,结果表明所提算法在降低网络复杂度、减少运算参数的同时可得到细节丰富、目标明确的融合图像。

Ordinary neural networks are difficult to generate infrared and visible light fusion images that conform to human vision,and the network model is complex and occupies too much memory.The existing Generative Adversarial Network(GAN)framework is improved.Firstly,deep convolution and point by point convolution are integrated into the generator,and a convolutional network with small convolution kernels is designed to reduce network parameters.Secondly,mask processing is applied to the source image to reduce the loss of source image information during feature extraction.Then,the processed image and the fused image obtained by the generator are jointly input into the discriminator to enhance the network's ability to retain source image information for visible light images.Finally,in the performance evaluation stage,the loss functions are set as gradient loss,adversarial loss,and content loss functions to constrain the fusion image to contain more background information of visible light images and target information of infrared images.The results of simulation experiments on the TNO image fusion dataset show that the proposed algorithm can obtain fused images with rich details and clear targets while reducing network complexity and operational parameters.