基于注意力机制及多尺度特征融合的图像去雨

Image Deraining Based on Attention Mechanism and Multi-scale Feature Fusion

针对去除不同雨纹的同时恢复图像背景细节的问题,提出一种基于注意力机制及多尺度特征融合的图像去雨方法。该网络采用双分支结构,分别用于雨纹去除和背景恢复。雨纹提取模块采用跨空间学习的多尺度注意力机制,通过多尺度上下文信息捕捉、均值计算、权重计算和整体信息综合,帮助改善雨纹去除任务中的图像质量,提高去雨效果。背景恢复模块包括多尺度特征提取部分和特征融合部分,采用多个扩张卷积层,每个卷积层具有不同的扩张因子,以扩大感受野,提取多尺度的图像背景特征;使用大核卷积对提取的多尺度特征信息进行融合调整,从而更准确地进行背景恢复。在多个公开数据集上的实验结果表明:所提方法能够有效去除真实雨图像场景中的雨纹,同时可以更好地恢复图像背景的细节信息。

To restore background details while removing different rain patterns,an image rain re-moval method based on attention mechanism and multi-scale feature fusion is proposed.The net-work adopts a two-branch structure,which is used for rain stripe removal and background recovery respectively.The rain-stripe extraction module uses a multi-scale attention mechanism of cross-space learning to help improve the image quality in the rain-stripe removal task and improve the rain removal effect through multi-scale context information capture,mean calculation,weight calcu-lation and overall information synthesis.The background recovery module includes multi-scale fea-ture extraction part and feature fusion part,and adopts multiple extended convolution layers,each with different expansion factors,to enlarge the receptive field and extract multi-scale image back-ground features.Large nuclear convolution is used to fuse and adjust the extracted multi-scale fea-ture information,so as to recover the background more accurately.The experimental results on sev-eral public data sets show that the proposed method can effectively remove the rain lines in the real rain image scene,and can better recover the details of the image background.