基于梯度方向直方图的红外与可见光融合网络的损失函数

Soft Histogram of Gradients Loss:A loss Function for Optimization of the Image Fusion Networks

基于深度学习的图像融合方法实现了良好的图像融合性能,近年来经过快速发展,被广泛应用于生物特征识别、自动驾驶和目标追踪等方面。深度学习网络在提取图像的重要纹理细节和保存重要信息等方面依然存在许多挑战。因此,提出了一种适用于红外与可见光图像融合网络的损失函数,在损失函数中引入了梯度方向直方图(HOG)损失,HOG特征可以反映图像局部的梯度方向和梯度大小,用HOG特征作损失函数可以提升网络提取图像细节信息的能力。将HOG损失与多尺度结构相似性损失相结合,用设计的损失函数训练了NestFuse、Res2Fusion和UNFusion 3个红外与可见光图像融合网络。在TNO数据集上,所提模型将融合图像的标准差(SD)分别提高2.1476%、1.2273%和1.4444%,将融合图像的视觉信息保真度(VIF)分别提高1.6529%、1.4936%和1.2902%;在RoadScene数据集上,所提模型将融合图像的SD分别提高1.0083%、1.1669%和0.7214%,将融合图像的VIF分别提高1.8093%、1.8063%和1.0406%。实验结果表明,所提损失函数可以从源图像中提取更多有效信息。

Image fusion methods based on deep learning have achieved excellent image fusion performance and have been widely used in biometric recognition,automatic driving and target tracking.However,it is still challenging to extract important texture details and preserve information of images.Therefore,a loss function for infrared and visible image fusion networks is presented.We employ the histogram of oriented gradient(HOG)to calculate the loss function.HOG feature can reflect the direction and magnitude of local gradient in the image,and using HOG feature as the loss function can improve the ability of the network to extract image details.We combine HOG loss with multi-scale structural similarity loss,and train NestFuse,Res2Fusion and UNFusion infrared and visible image fusion networks with the designed loss function.On the TNO dataset,our model increases the standard deviation(SD)of fused images by 2.1476%,1.2273%and 1.4444%respectively,and increases the visual information fiedity(VIF)of fused images by 1.6529%,1.4936%and 1.2902%respectively.On the RoadScene dataset,our model increases the SD of the fused images by 1.0083%,1.1669%and 0.7214%respectively,and increases the VIF of the fused images by 1.8093%,1.8063%and 1.0406%respectively.The experimental results show that the proposed loss function can extract more effective information from the source image.