引入特征交互的红外与可见光图像自适应融合

Adaptive fusion of infrared and visible light images with introduction of feature interaction

在图像融合领域,现有的基于卷积神经网络(CNN)或Transformer架构的方法存在两个局限性:首先,浅层纹理特征与深层语义特征之间无法有效聚合;其次,红外与可见光特征的权重比例无法自适应变化。本文提出一种引入特征交互的红外与可见光图像自适应融合方法。首先,构建一种基于Transformer的特征交互模块,聚合跨尺度特征信息,增强特征表达能力。其次,设计一种融合模块,自适应地调整特征权重比例。所提出的融合方法通过两阶段训练策略完成。第一个阶段,应用创新的特征交互概念训练编码器,增强特征表达,重建特征图像。第二个阶段,基于设计的权重自适应调整模块训练红外与可见光特征融合任务。公开数据集的实验结果表明,与现有方法相比,本方法在主观和客观的评价方面均优于其他典型方法。

In the field of image fusion,existing methods based on Convolution Neural Network(CNN)or Transformer architectures have two limitations:first,effective aggregation between shallow texture features and deep semantic features is not possible;second,the weight ratio of infrared and visible light features cannot be adaptively changed.In this paper,an adaptive fusion method of infrared and visible images that introduce feature interaction is proposed.Firstly,a feature interaction module based on Transformer is constructed to aggregate cross-scale feature information and enhance feature representation.Secondly,a fusion module is designed to adaptively adjust the feature weight ratio.The proposed fusion method is completed by a two-stage training strategy.In the first stage,the encoder is trained using innovative feature interaction concepts to enhance feature representation and reconstruct feature images.In the second stage,the weight adaptive adjustment module based on the design trains the infrared and visible feature fusion task.The experimental results on publicly available datasets show that this method is superior to other typical methods in terms of subjective and objective evaluation compared with the existing methods.