卷积神经网络结合NSST的红外与可见光图像融合

Infrared and visible image fusion of convolutional neural network and NSST

传统的多尺度红外与可见光图像融合方法,所提取的图像特征固定,并不能很好的应用于各类复杂的图像环境,而深度学习可以自主选择合适图像特征,改良特征提取单一性问题,因此提出一种基于卷积神经网络与非下采样剪切波变换(NSST)相结合的红外与可见光图像融合方法。首先,用卷积神经网络提取红外目标与背景的二分类图,利用调频(FT)显著性检测算法对分类图进行精准分割,同时,利用NSST将源图像多尺度、多方向进行分解;其次,利用目标显著性结合自适应模糊逻辑算法进行低频子带融合,利用高频系数局部方差对比度方法进行高频子带融合;最后,通过NSST逆变换得到融合后图像。实验结果表明:相比于传统图像融合算法,该方法在信息熵、平均梯度、空间频率、互信息和交叉熵等多个客观评价指标上至少分别提高了0.01%、0.30%、1.43%、2.32%、1.14%。一定程度提高了融合图像对比度,丰富了背景细节信息,更有利于人眼识别,可以广泛的应用于光电侦察、光电告警、多传感器信息融合等光电信息领域。

Traditional multi-scale infrared and visible image fusion methods couldnot be well applied to all kinds of complex image environments, because the extracted image features were fixed. However, deep learning could independently select appropriate image features to solve the unicity in feature extraction of multi-scale methods.Therefore, an infrared and visible image fusion method based on the combination of convolutional neural network and non-subsampled shear wave transform(NSST) was proposed. Firstly, the binary classification map of the infrared target and background was extracted by convolutional neural network, and the classification map was accurately segmented by frequency-tuned(FT) saliency detection algorithm. At the same time, the NSST was used to decompose the source image in multiple scales and directions;Secondly, the target saliency combined with adaptive fuzzy logic algorithm was used for the fusion of low frequency sub-bands, and the high frequency coefficient local variance contrast method was used for the fusion of high frequency sub-bands;Finally, the fused image was obtained through the inverse transformation of NSST. The experiment results show that compared with the traditional image fusion algorithm, this method improves objective evaluation indicators such as information entropy, average gradient, spatial frequency, mutual information and cross entropy at least increased by 0.01%,0.30%, 1.43%, 2.32%, 1.14%, respectively. The contrast of fusion image is greatly improved, and the background details are enriched, which is more conducive to human eye recognition. It can be widely used in electro-optical reconnaissance, electro-optical warning, multi-sensor information fusion and other electro-optical information fields.