GAN网络的红外与可见光图像融合方法研究

Research on Infrared and Visible Images Fusion Method of GAN Network

为使红外与可见光图像融合后的目标图像更突出、背景信息更丰富,研究一种生成对抗网络(GAN)的红外与可见光图像融合方法。首先,利用语义分割网络deeplabv3+将红外和可见光图像分别分割为目标图像和背景图像;然后,将不同区域的图像分别输入到非对称的双输入GAN融合网络中,得到对比度较高的融合目标图像和细节信息丰富的融合背景图像;最后,将两幅融合图像加权得到最终的融合图像。实验结果表明:GAN网络的红外与可见光图像融合算法能很好地提高融合图像目标的对比度,同时融合图像也能包含更多的背景信息;融合图像相较于残差网络的融合图像和融合生成对抗网络的融合图像,其信息熵、互信息、结构相似性、标准差4个参数的值更高,融合效果更好。

In order to make the target more prominent and the background information more abundant after infrared and visible image fusion, an infrared and visible image fusion method based on GAN segmentation is proposed.First, infrared and visible images are segmented into target and background images respectively by using the semantic segmentation of the deeplabv3+network, and then the images of different regions are input into the improved asymmetric dual-input GAN fusion network, high contrast fusion target image and detail-rich fusion background image can be obtained.Finally, the final fusion image is obtained by weighting the two fusion images.The experimental results show that the infrared and visible image fusion algorithm based on GAN segmentation can enhance the contrast of the fused image and contain more background details.The values of the four parameters including experimental data in the information entropy, mutual information, structural similarity, the standard deviation are higher and fusion effect is better.