非对称端到端的无监督图像去雨网络

Asymmetric unsupervised end-to-end image deraining network

现有的基于学习的单幅图像去雨网络大都关注雨天图像中雨痕对于视觉成像的影响,而忽略了雨天环境下由于空气中湿度的增加所产生的雾气对视觉成像的影响,因此造成去雨后图像的生成质量低、纹理细节信息模糊等问题。针对该问题,提出一种非对称端到端的无监督图像去雨网络模型,该模型主要包含雨雾去除网络、雨雾特征提取网络和雨雾生成网络,并由它们组成两个不同数据域映射转换模块:Rain-Clean-Rain和Clean-Rain-Clean。上述三个子网络构成并行的两条转换路径:去雨路径和雨雾特征提取路径。在雨雾特征提取路径上,提出一种基于全局和局部注意力机制的雨雾感知提取网络,利用雨雾特征存在的全局自相似性和局部差异性学习雨-雾相关特征;在去雨路径上,引入雨天图像退化模型和上述提取的雨雾相关特征作为先验知识以增强雨雾图像生成的能力,从而约束雨雾去除网络,提高它从雨天数据域到无雨数据域的映射转换能力。在不同雨天图像数据集上的实验结果表明,与较先进的去雨方法CycleDerain相比,在合成雨雾数据集HeavyRain上所提方法的峰值信噪比(PSNR)提升了31.55%,能适应不同的雨天场景,具有更好的泛化性,并且能更好地复原图像的细节和纹理信息。

Existing learning-based single-image deraining networks mostly focus on the effect of rain streaks in rainy images on visual imaging,while ignoring the effect of fog on visual imaging due to the increase of humidity in the air in rainy environments,thus causing problems such as low generation quality and blurred texture detail information in the derained images.To address these problems,an asymmetric unsupervised end-to-end image deraining network model was proposed.It mainly consists of rain and fog removal network,rain and fog feature extraction network and rain and fog generation network,which form two different data domain mapping conversion modules:Rain-Clean-Rain and Clean-Rain-Clean.The above three sub-networks constituted two parallel transformation paths:the rain removal path and the rain-fog feature extraction path.In the rain-fog feature extraction path,a rain-fog-aware extraction network based on global and local attention mechanisms was proposed to learn rain-fog related features by using the global self-similarity and local discrepancy existing in rain-fog features.In the rain removal path,a rainy image degradation model and the above extracted rain-fog related features were introduced as priori knowledge to enhance the ability of rain-fog image generation,so as to constrain the rain-fog removal network and improve its mapping conversion capability from rain data domain to rain-free data domain.Extensive experiments on different rain image datasets show that compared to state-of-the-art deraining method CycleDerain,the Peak Signal-to-Noise Ratio(PSNR)is improved by 31.55%on the synthetic rain-fog dataset HeavyRain.The proposed model can adapt to different rainy scenarios,has better generalization,and can better recover the details and texture information of images.