基于无监督显著性掩码引导的红外与可见光图像融合网络

UMGN:An Infrared and Visible Image Fusion Network Based on Unsupervised Significance Mask Guidance

在具有挑战性的拍摄环境中,使用单张红外或可见光图像很难捕获清晰详细的纹理信息以及热辐射信息。然而,红外和可见光图像融合允许保存来自红外图像的热辐射信息和来自可见光图像的纹理细节。现有的许多方法在融合过程中直接生成融合图像,忽略了对源图像像素级权重贡献的估计,强调了不同源图像之间的学习。为此,提出了基于无监督显著性掩码引导的红外与可见光图像融合网络,利用密集结构在源图像中进行全面的特征提取。它产生一个权重估计概率来评估每个源图像对融合图像的贡献。此外,由于红外与可见光图像缺乏真实标签,难以使用有监督学习,UMGN还引入了显著性掩码,便于网络集中学习红外图像的热辐射信息和可见光纹理信息。在训练过程中还引入了加权保真度项和梯度损失,以防止梯度退化。与大量其他最先进的方法进行对比实验,结果证明了所提出的UMGN方法的优越性和有效性。

In challenging shooting environments,it is difficult to capture clear and detailed texture information and thermal radiation information using a single infrared or visible image.However,infrared and visible image fusion allows the preservation of thermal radiation information in infrared images and texture details in visible light images.Many existing methods directly generate fused images in the fusion process,ignore the estimation of pixel-level weight contribution of source images,and emphasize the learning between different source images.For this reason,an infrared and visible image fusion based on unsupervised significance mask guidance network is proposed,which uses DenseNet structure to extract comprehensive features from source images.It produces a weight estimation probability to evaluate the contribution of each source image to the fused image.Since infrared and visible images lack ground truth,it is difficult to use supervised learning.UMGN also introduces the significance mask to facilitate the network to focus on learning the thermal radiation information and visible light texture information of infrared images.A weighted fidelity term and gradient loss are also introduced in the training process to prevent gradient degradation.A large number of comparative experiments with other advanced methods prove the superiority and effectiveness of the proposed UMGN method.