基于改进生成对抗网络模型的红外与可见光图像融合

Fusion of Infrared and Visible Image by Using Improved Generative Adversarial Network Model

为增强融合图像的视觉效果,减少计算的复杂度,解决传统红外与可见光图像融合算法存在的背景细节丢失问题,提出了一种生成对抗网络框架下基于深度可分离卷积的红外与可见光图像融合方法。首先,在生成器中对源图像进行深度卷积与逐点卷积运算,得到源图像的特征映射信息;其次,通过前向传播的方式更新网络参数,得到初步的单通道融合图像;再次,在红外及可见光判别器中,使用深度可分离卷积分别对源图像与初步融合图像进行像素判别;最后,在损失函数的约束下,双判别器不断将更多的细节信息添加到融合图像中。实验结果表明,相比于传统的融合算法,该方法在信息熵、平均梯度、空间频率、标准差、结构相似性损失和峰值信噪比等评价指标上分别平均提高了1.63%、1.02%、3.54%、5.49%、1.05%、0.23%,在一定程度上提升了融合图像的质量,丰富了背景的细节信息。

To enhance the visual effect of fused image,reduce the computational complexity,and solve the problem of background detail loss in the traditional infrared and visible image fusion algorithm,an infrared and visible image fusion method is proposed based on depthwise separable convolution under the framework of generative adversarial network(GAN).First,in the generator,the source image is subjected to depthwise convolution and pointwise convolution operations to obtain the feature mapping information of the source image.Second,the network parameters are updated through forward propagation to obtain preliminary single-channel fused image.Third,in the infrared and visible discriminator,depthwise separable convolution is used to distinguish the pixels of the source image and the preliminary fused image respectively.Finally,under the constraint of the loss function,the dual discriminator continually adds more detailed information to the fused image.The experimental results reveal that compared with the traditional fusion algorithm,the proposed method averagely improves the evaluation indicators including information entropy,average gradient,spatial frequency,standard deviation,structural similarity loss,and peak signal-to-noise ratio by 1.63%,1.02%,3.54%,5.49%,1.05%,0.23%,respectively,which enhances the fused image quality to a certain extent and enriches the background detail information.