基于渐进式差异感知注意力的红外和可见光图像融合算法

Infrared and visible image fusion algorithm based on progressive difference-aware attention

红外图像与可见光图像融合是图像融合的重要研究方向.不同的图像源可以提供互补的知识.融合成的图像包含更多信息,能获得更好的识别和分析性能.目前的方法可以分为传统方法和基于深度学习的方法两种.本文在现有方法的基础上,提出了一种新的渐进式跨模态差异感知图像融合网络,建立了端到端的可见光-红外图像融合模型.模型采用基于卷积神经网络(convolutional neural network,CNN)的框架作为主干,由渐进式特征提取器和图像重构器组成.首先,本算法对可见光图像和红外图像分别建立两条特征提取支路,在两支路间引入了差异感知注意力模块(differential aware attention module,DAAM),该模块使本网络能够在特征提取阶段逐步整合互补信息.因此,特征提取器可以从红外和可见光图像中完全提取出共同和互补的特征.然后,将提取出的深度特征通过中间融合策略进行融合,将可见光图像和红外图像的特征结合起来,以获得尽可能好的融合效果.再通过图像重构器恢复出融合图像.最后,通过与其他相关方法的比较,对本方法的性能进行了测试,实验结果表明,所提出的方法能有效提高融合效果.

The fusion of infrared images and visible images is an important research direction in image fusion.Different image sources can provide complementary knowledge.The fused image contains more information,which leads to better recognition and analysis performance.Currently,there are two main approaches:traditional methods and deep learning-based methods.This paper proposes a new progressive cross-modal difference-aware image fusion network based on existing methods and establishes an end-to-end visibleinfrared image fusion model.The model adopts a CNN-based framework as the backbone,consisting of a progressive feature extractor and an image reconstructor.Firstly,the algorithm establishes two feature extraction branches for visible and infrared images,respectively,and introduces a differential aware attention module(DAAM)between the two branches.This module enables the network to gradually integrate complementary information in the feature extraction stage.Therefore,the feature extractor can fully extract common and complementary features from both infrared and visible images.Then,the extracted deep features are fused through an intermediate fusion strategy,combining the features of visible and infrared images to obtain the best possible fusion result.The fused image is then reconstructed using an image reconstructor.Finally,the performance of the proposed method is tested by comparing it with other relevant methods,and the experimental results show that the proposed method can effectively improve the fusion effect.