基于自编码器的红外与可见光图像融合算法

Infrared and visible image fusion algorithm based on autoencoder

针对目前红外与可见光图像融合过程中,图像特征提取不充分、中间层信息丢失以及融合图像细节不够清晰的问题,提出了一种基于自编码器的端到端图像融合网络结构。该网络由编码器、融合网络和解码器3部分组成。将高效通道注意力机制和混合注意力机制引入到编码器和融合网络中,利用卷积残差网络(convolutional residual network,CRN)基本块来提取并融合红外图像和可见光图像的基本特征,然后将融合后的特征图输入到解码器进行解码,重建出融合图像。选取目前具有典型代表性的5种方法在主客观方面进行对比。在客观方面,较第2名平均梯度、空间频率和视觉保真度分别提升了21%、10.2%、7.2%。在主观方面,融合后的图像目标清晰、细节突出、轮廓明显,符合人类视觉感受。

To address the current problems of inadequate image feature extraction,loss of information in the middle layer and insufficient details of fused images in the process of infrared and visible image fusion,this paper proposes an end-to-end image fusion network structure based on a self-encoder,which consists of three parts:encoder,fusion network and decoder.Firstly,the efficient channel attention mechanism and hybrid attention mechanism are introduced into the encoder and fusion network.The CRN(convolutional residual network)base blocks are used to extract and fuse the basic features of infrared images and visible images.The fused feature images are input to the decoder to reconstruct the fused images.Five representative methods are selected to compare with subjective and objective aspects.In the objective aspect,compared with the second place,AG、SF and VIF have increased by 21%,10.2%,and 7.2%.In the subjective aspect,significantly with clear targets,prominent details and obvious outline,which is in line with human visual perception.