基于坐标注意力的重参数化红外与可见光图像融合网络

Re-parameterized infrared and visible image fusion network based on coordinate attention

针对现有大多数基于深度网络的图像融合方法网络架构复杂、计算成本高,且没有充分考虑多模态图像的固有特性进而难以实现跨模态特征的丰富交互两个问题,提出一种基于坐标注意力机制的重参数化红外与可见光图像融合网络.该网络引入重参数化技巧,并结合残差学习进行特征提取,以在保证融合质量的同时提高计算效率.其次,为增强跨模态特征间的交互性、充分利用多模态图像信息,构建基于坐标注意力的融合模块以生成融合特征.最后,考虑到特征提取过程中可能伴随的信息丢失,设计融合特征增强模块,以利用浅、中层特征进行信息补偿.实验表明,本文方法不仅具有更为低廉的计算成本,且在保证良好视觉效果的同时,实现多个客观评价指标的提升.

Aiming at the two problems that most existing deep network-based fusion approaches have complex network architecture with high computation cost,and they fail to adequately consider the intrinsic characteristics of multi-modal images which results in insufficient information interaction of cross-modality features,a re-parameterized infrared and visible image fusion network based on coordinate attention is proposed.In this network,re-parameterization technique is introduced and combined with residual learning to perform feature extraction for computing efficiency and satisfying fusion quality.Moreover,to improve the interactivity of cross-modality features and fully utilize multi-modal image information,a coordinate attention-based fusion module is devised to yield fused feature.Considering the information loss during extracting process,a fused feature enhance module which leverages the preceding cross-modality features to implement feature compensation is further developed.Extensive experiments demonstrate that the proposed method not only has lower computational cost,but also achieves the improvement of multiple objective evaluation metrics while ensuring good visual effects.