一种结合自注意和多尺度生成对抗网络的图像去雨方法

An image raindrop removal method based on self-attention and multi-scale generative adversarial network

为去除雨天拍摄照片上的雨滴,针对被雨滴所覆盖区域未知,雨滴区域中大多数背景信息已经丢失,以及需要提升图像清晰度和对全局信息关注度的问题,在生成对抗网络中生成网络的自动编码器结构中添加自注意层,并在判别网络中引入多尺度判别器。通过注意力分布图的引导,自注意层的优化和多尺度判别器的评估,生成网络在关注雨滴区域的前提下进一步关注全局信息,多尺度判别器可由粗到细更好地判别雨滴图像与清晰图像之间的差距。实验完成了所提方法与其他方法的对比,以及自对比,并用峰值信噪比和结构相似性进行评估,结果表明了所提方法的有效性,其质量和指标数值均高于其他方法。

In order to remove raindrops from images taken on rainy days,aiming at the issues that the area covered by raindrops is unknown,most of the background information in the raindrop area has been lost,and the image clarity and global information attention are needed to improve,a self-attention layer is added to the self-encoding structure,and a multi-scale discriminator is introduced into the discriminant network.Guided by the attention distribution map,the optimization of the self-attention layer and the evaluation of the multi-scale discriminator,the generating network considers the global information more under the premise of paying attention to the raindrop area.The multi-scale discriminator can better distinguish the gap between the raindrop image and the clear image from coarsely to finely.The experiment completed the comparison between the proposed method and other methods,the self-comparison,and the evaluation with the peak signal-to-noise ratio and structural similarity,which proves that the proposed method is effective and its quality and index values are higher than other methods.