采用对比学习的多阶段Transformer图像去雾方法

A Multi-Stage Transformer Network for Image Dehazing Based on Contrastive Learning

为了解决现有图像去雾方法在图像局部去雾以及纹理细节恢复等方面始终不理想以及处理非均匀雾质始终不彻底的问题,提出了一种采用对比学习的多阶段自注意力模块(Transformer)的图像去雾MSTCNet方法。首先,利用信道级Transformer模块作为基本的特征提取模块,充分地捕获特征信道之间的长距离依赖关系;其次,通过提出的多监督对比学习方法最大限度地挖掘正负样本信息,使去雾图像在投影后的隐空间中更靠近清晰图像,同时远离有雾图像;最后,利用多阶段渐进式网络结构和可变形自注意力机制有效地整合图像局部细粒度特征和全局粗粒度信息。本文在2个合成数据集和3个真实数据集上对所提出的方法进行了大量的实验,结果表明:所提出的MSTCNet方法在5个数据集上的峰值信噪比(PSNR)分别提高了1.49、1.45、0.11、1.45和0.22 dB,在通用数据集与非数据集的测试中均超越已有的方法,在浓雾质、非均匀雾质以及均匀雾质的测试中均表现出最佳的去雾视觉效果,并达到最高的客观评价指标值。

A multi-stage Transformer network for image dehazing based on contrastive learning is proposed to solve the problem that existing image dehazing methods fail to achieve the desired results in local image dehazing and detail restoration, and the non-homogeneous haze cannot be removed completely all the way. First, the channel-wise Transformer block is utilized as the primary feature extraction block to adequately capture the mutual long-range dependencies among channels. Second, the multi-modality supervised contrastive learning is introduced to maximize the capturing efficiency of information from the contrastive samples, so that the restored image is closer to the clear image in the embedding space while staying as far away from the hazy image as possible. Finally, a hierarchical multi-patch structure and deformable Transformer blocks are employed to effectively integrate the local and global structural information of the hazy image. Moreover, a large number of tests have been conducted on the proposed method by using two synthetic data sets and the three real data sets. The results show that the proposed MSTCNet achieves a higher peak signal-to-noise ratio(PSNR) gain of 1.49, 1.45, 0.11, 1.45 and 0.22 dB on five datasets, respectively. It outperforms existing methods in both general and non-data sets, shows the best visual effect of dehazing in removing the dense, non-homogeneous and uniform haze, and achieves the highest objective evaluation index value.