基于自编码器的红外与可见光图像融合算法

Infrared and Visible Image Fusion Algorithm Based on Auto-encoder

基于深度学习的红外与可见光图像融合是图像处理领域的研究热点之一,为丰富融合图像的细节信息、突出红外目标,基于自编码器网络提出一种可进行端到端训练的融合算法。首先,将编码器设计成双分支结构,并针对红外与可见光图像的特性设置不同的特征提取方式;其次,为有效地融合红外与可见光的互补信息,设计了一种可学习的融合策略,并使整个框架能够进行端到端的学习;最后,解码器对通过融合策略所获得的特征进行重构,进而得到融合图像。基于TNO数据集进行实验验证,定性和定量的结果表明,算法能够生成红外目标突出、细节信息丰富的融合图像,相较于其他算法,融合性能更加优越。

Fusion of infrared and visible images based on deep learning is one of the research focuses in the field of image processing.To enrich the detail information of fused images and highlight the infrared targets,this paper proposes a fusion algorithm that can be trained end-to-end based on a self-encoder network.Firstly,the encoder is designed as a two-branch structure,and different feature extraction methods are set according to the characteristics of infrared and visible images.Secondly,in order to effectively fuse the complementary information of infrared and visible images,a learnable fusion strategy is designed to make sure the whole framework can be learned end-to-end.Finally,based on the feature maps constructed by the fusion strategy,the decoder is reconstructed to obtain the fused images.The qualitative and quantitative results of the verification experiments based on the TNO dataset show that the proposed algorithm is able to generate fused images with prominent infrared targets and rich texture detail information,and the fusion performance is superior to that of other algorithms.