渐进式多尺度特征融合的图像去镜头雨滴方法

Multi-scale progressive fusion networkfor single image raindrop removal

针对附着在镜头上的雨滴会造成图像退化和背景信息丢失问题,提出一种渐进式多尺度特征融合的单幅图像去镜头雨滴的方法,以采样至不同尺度的镜头雨滴附着图像为输入,通过重复堆叠的残差模块和循环神经网络进行渐进式雨滴特征提取,捕获从局部到全局的雨滴特征信息,在不同空间维度上探索互补的信息来更好地表征尺寸形状多变、位置随机分布的雨滴,对多尺度的雨滴特征进行融合,最后按照线性叠加物理模型恢复出清晰背景图像。使用公开数据集进行训练得到去镜头雨滴模型,在其测试集以及真实采集的镜头雨滴附着图像上进行了测试,在量化指标和可视化结果方面都取得了较好的效果,并表现出良好的泛化性能。

Aiming at the problem that raindrops attached to the lens will cause image degradation and loss of background information, this paper presents a multi-scale progressive network for single image raindrop removal. It takes images with adherent raindrops sampled to several scales as input, achieves raindrop feature extraction by stacking identical residual blocks and recurrent neural units, captures raindrop feature information from local to global, explores complementary information in different spatial dimensions to better represent randomly distributed raindrops with varying size and shape. The multi-scale raindrop features are fused and the clear background image is restored according to the linear superposition physical model. We utilize public dataset with adherent raindrops for training and evaluate the model on both test set and real data qualitatively and quantitatively. Experiments show that the proposed method achieves better performance and generalization comparing to state-of-art methods with respect to numerical metrics and visual results.