基于互信息特征分离表达的红外与可见光图像融合

Infrared and Visible Image Fusion Based on Separate Expression of Mutual Information Features

针对红外与可见光图像融合方法存在的对源图像特征分离不充分、可解释性低且融合规则难以准确设计等问题,提出基于互信息特征分离表达的红外与可见光图像融合方法,有效分离特征的同时保留源图像的典型信息。首先,采用互信息约束的编码网络提取特征,最大化源图像与特征间互信息来保留源图像的特征表示,同时通过最小化私有和公有特征的互信息来达到分离表达的目的;其次,特征融合阶段设计了层级特征自适应融合模块来有效融合不同层级的特征信息,减小语义差距并调整感受野,增强网络对特征的学习能力;此外,损失函数采用软加权强度损失来平衡红外与可见光特征分布;最后,对比实验的主客观评价结果表明,所提方法能有效融合红外与可见光图像的重要信息,具有良好的视觉感知。

To solve the challenges associated with the inadequate separation of source image features,low interpretability,and difficulty of designing accurate fusion rules,this paper proposes an infrared(IR)and visible image fusion method based on mutual information feature separation and representation,which effectively separates features while preserving the typical information of the source image.First,a mutual information constrained coding network is used to extract the features,maximize the mutual information between the source image and features to retain the feature representation of the source image,and minimize the mutual information of private and public features to achieve separation and representation.In addition,the loss function adopts a soft weighted intensity loss to balance the distribution of IR and visible features.Objective and subjective evaluation results of comparison experiments indicate that the proposed method can effectively fuse important information regarding IR and visible images and has good visual perception.