结合信息感知与多尺度特征的红外与可见光图像融合

Infrared and Visible Image Fusion Combining Information Perception and Multiscale Features

现有的基于深度学习图像融合算法无法同时满足融合效果与运算效率,且在建模过程中大部分采用基于单一尺度的融合策略,无法很好地提取源图像中上下文信息。为此本文提出了一种基于信息感知与多尺度特征结合的端到端图像融合网络。该网络由编码器、融合策略和解码器组成。具体来说,通过编码器提取红外与可见光图像的多尺度特征,并设计特征增强融合模块来融合多个尺度的不同模态特征,最后设计了一个轻量级的解码器将不同尺度的低级细节与高级语义信息结合起来。此外,利用源图像的信息熵构造一个信息感知损失函数来指导融合网络的训练,从而生成具有丰富信息的融合图像。在TNO、MSRS数据集上对提出的融合框架进行了评估实验。结果表明:与现有的融合方法相比,该网络具有较高计算效率;同时在主观视觉评估和客观指标评价上都优于其它方法。

Existing image fusion algorithms based on deep learning are unable to satisfy the demands of computational efficiency and fusion effect.Most have also adopted a fusion strategy based on a single-scale model,which cannot effectively extract the contextual information in images.This study proposes an end-to-end image fusion network based on information perception and multiscale features.The network consists of an encoder,a fusion strategy,and decoder.Specifically,the multiscale features of the infrared and visible images were extracted by the encoder,and a feature complementary enhancement module was designed to fuse different modal multiscale features.Finally,the lightweight decoder was designed to combine the low-level details and high-level semantic information.In addition,the information entropy of the source image was used to construct an information-sensing loss function to train the fusion network and generate the fused image with rich information.The proposed fusion framework was evaluated on the TNO and MSRS datasets.The results show that compared with existing fusion methods,the proposed network was superior to other methods in terms of both subjective visual description and objective index evaluation,with higher computational efficiency.