基于双注意力残差循环单幅图像去雨集成网络

Single Image De-raining Using a Recurrent Dual-attention-residual Ensemble Network

降雨会严重降低拍摄图像质量和影响户外视觉任务.由于不同图像中,雨的形状、方向和密度不同,导致单幅图像去雨是一项困难的任务.提出一种新的基于双注意力的残差循环单幅图像去雨集成网络(简称RDARENet).在网络中,因为上下文的信息对于去除雨痕十分重要,所以首先采用多尺度的扩张卷积网络去获得更大的感受野.雨痕信息可以认为是多个雨层特征的叠加,为了更好地提取雨痕的特征和恢复背景图层信息,运用了通道和空间注意力机制的残差网络.通道注意力能够反映不同雨层的权重,而空间注意力则通过相邻空间特征之间的关系增强区域的表征.随着网络的加深,防止低层信息的丢失,采用级联的残差网络和长短时间记忆网络,将低层特征信息传递到高层中去,逐阶段地去除雨痕.在网络的输出部分,采用集成学习的方式,将每个阶段的输出结果通过门控网络加权相加,得到最终的无雨图像.实验结果表明,去雨和恢复纹理细节的效果都得到较大提升.

Rain streaks can severely degrade the quality of captured images and affect outdoor vision.However,due to non-uniform in shape,direction,and density of rain in different images,it is a difficult task to remove rain from a single image.This study proposes a single image de-raining using an ensemble recurrent dual-attention-residual network,called RDARENet.In the network,as contextual information is very important for the process of rain removal,a multi-scale dilated convolution network is firstly adopted to acquire large receptive field.Rain streaks can be regarded as the accumulation of multiple rain streaks layers,the residual of the channel attention and spatial attention mechanisms are used to extract the features of the rain streaks and restore the background layer information.The channel attention can assign different weights to rain streaks layers,and the spatial attention enhances the representation of the area through the relationship between adjacent spatial features.With the deepening of the network,to prevent the loss of low-level information,a cascaded residual network and a long-term memory network are used to transfer low-level feature information to the high-level and remove rain streaks stage by stage.In the output of the network,ensemble learning method is adopted to weight the output of each stage through the gated network,and add to get the clean image.Extensive experiments demonstrate that the effect of removing rain and restoring texture details is greatly improved.