基于梯度转移和自编码器的红外与可见光图像融合

Infrared and visible image fusion based on gradient transfer and auto-encoder

针对红外与可见光图像融合过程中图像特征提取不充分、易丢失中间层信息和训练时间长的问题,本文提出一种基于梯度转移结合自编码器的端到端轻量级图像融合网络结构。该网络结构由编码器、融合层和解码器三部分组成。首先,将Bottleneck块引入编码器当中,利用卷积层和Bottleneck块对输入的红外图像和可见光图像进行特征提取得到深度特征图,将梯度提取算子引入融合层,对获得的深度特征图进行处理获得对应的梯度图。其次,将每张深度特征图和对应的梯度图在融合层通过融合层策略对应融合,重新设计损失函数,将融合后的特征图和梯度图进行拼接并输入解码器进行解码重建出融合图像。最后,选取目前的代表性融合方法进行对比验证,在SF、MI、VIF、Qabf、SCD、AG、EN和SD 8个融合质量评估指标上,前七个指标性能提升显著,分别提高57.4%、54.6%、28.3%、74.2%、23.8%、43.1%、1.3%,第八个指标性能接近。并进行网络模型参数分析和时间复杂度对比实验,本文算法模型参数为12352,算法用时为1.1246 s。实验结果表明:本文方法可以快速生成目标清晰、轮廓明显、纹理突出、符合人类视觉感受的融合图像。

In order to solve the problems of inadequate feature extraction,easy to lose middle layer information and long training time during infrared and visible image fusion,this paper proposes an end-to-end lightweight image fusion network structure based on gradient transfer and autoencoder.The network structure is composed of encoder,fusion layer and decoder.First,a Bottleneck block is introduced into the encoder,which is a Bottleneck block,and it uses the convolution layer and a bottleneck block to extract features from input infrared and visible images to get depth feature graphs.Then,the gradient extraction operator is introduced into the fusion layer,and the obtained depth feature graphs are processed to obtain the corresponding gradient graphs.Then,each depth feature map and the corresponding gradient map are fused in the fusion layer through the fusion layer strategy,the loss function is redesigned,the fused feature map and gradient map are spliced and input to the decoder for decoding to reconstruct the fusion image.Finally,the current representative fusion methods were selected for comparison and verification.In the eight fusion quality evaluation indexes of SF,MI,VIF,Qabf,SCD,AG,EN and SD,the performance of the first seven indexes improved significantly.They are improved by 57.4%,54.6%,28.3%,74.2%,23.8%,43.1%and 1.3%respectively,and the performance of the eighth index is similar.In addition,network model parameter analysis and time complexity comparison experiment were conducted.The algorithm parameter in this paper is 12352,and the algorithm time is 1.1246 seconds.Experimental results show that the proposed method can quickly generate the fusion image with clear target,clear outline,prominent texture and human visual perception.