基于空洞卷积与双注意力机制的红外与可见光图像融合

Infrared and Visible Image Fusion Based on Dilated Convolution and Dual Attention Mechanism

针对红外与可见光图像融合算法中多尺度特征提取方法损失细节信息,且现有的融合策略无法平衡视觉细节特征和红外目标特征,出了基于空洞卷积与双注意力机制(Dilated Convolution and Dual AttentionMechanism,DCDAM)的融合网络。该网络首先通过多尺度编码器从图像中提取原始特征,其中编码器利用空洞卷积来系统地聚合多尺度上下文信息而不通过下采样算子。其次,在融合策略中引入双注意力机制,将获得的原始特征输入到注意力模块进行特征增强,获得注意力特征;原始特征和注意力特征合成最终融合特征,得在不丢失细节信息的情况下捕获典型信息,同时抑制融合过程中的噪声干扰。最后,解码器采用全尺度跳跃连接和密集网络对融合特征进行解码生成融合图像。通过实验表明,DCDAM比其他同类有代表性的方法在定性和定量指标评价都有提高,体现良好的融合视觉效果。

The multiscale features extraction method in infrared and visible image fusion algorithms loses detail information.Existing fusion strategies also cannot balance the visual detail and infrared target features.Therefore,a fusion network via a dilated convolution and dual-attention mechanism(DCDAM)is proposed.First,the network extracts the original features from the image through a multiscale encoder.The encoder systematically aggregates the multiscale context information through dilated convolution instead of using downsampling operator.Second,a dual-attention mechanism is introduced into the fusion strategy,and the original features are input into the attention module for feature enhancement to obtain the attention features.The original and attention features were combined into the final fusion feature.The mechanism captured the typical information without losing details and suppressed the noise during the fusion process.Finally,the decoder used a full-scale jump connection and dense network to decode the fusion features and generate the fused image.The experimental results show that the DCDAM is better than other representative methods in qualitative and quantitative index evaluations and has a good visual effect.