深度卷积先验引导的鲁棒图像层分离方法及其应用

Deep Convolutional Prior Guided Robust Image Separation Method and Its Applications

图像层分离是根据任务需要将观测图像分解成两个独立且具有实际意义的组成成分。图像恢复领域中的很多问题在本质上都可以被理解为两个不同层的分离,如自然图像去雨、本质图像分解、反射层去除等。因此,做好图像层分解工作对解决这些问题有极大的推动作用。由于这个问题的求解具有病态性,已有的方法大多都是通过设计一个复杂的模型先验来约束所需要的两层。然而,复杂的先验会导致目标函数难以被优化求解,算法的有效性也不能很好地发挥出来。更重要的是,这些方法只能针对其中某一个特定的任务,不能迁移到其他应用上,泛化能力不强。为了弥补上述不足,文中提出了一个自适应的灵活优化框架,将深度网络整合到图像层分离迭代过程中。近年来,深度学习在特征提取上的优势逐步体现,在低级图像处理领域也越来越多地被采用。因此,该算法使用深度结构替代传统模型先验来刻画不同层的特征,同时,为了减少网络对训练数据的依赖并提升算法的有效性,将深度信息与传统优化框架相结合。具体来讲,首先基于MAP(最大后验概率)建立能量函数。然后使用ADMM(交替方向乘子法)将该模型分解成3个子问题。前两个子问题用来估计分离的两层,其中鉴于卷积操作在特征刻画上的优势,使用其作为隐式先验刻画问题属性;最后一个子问题通过优化的手段得到最终的精确结果。通过深度卷积先验引导优化迭代,既保持了深度结构对特征的刻画优势,又保留了传统模型优化的稳定性。最后,将所提方法应用到多种图像恢复问题上,包括自然图像去雨和反射层去除。与同类方法的定性与定量比较表明,所提方法在可视效果和数值结果上均表现出了极大的优势,证明了该方法具有强大的泛化能力和有效性。

Single image layer separation aims to divide the observed image into two independent and practical components based on the requirement of tasks.Many tasks in computer vision can be understood as the separation of two different layers essentially,such as single image rain streak removal,intrinsic image decomposition and reflection removal.Therefore,an excellent image layer decomposition method would promote the solution of these problems greatly.Since there is only one known variable,two variables need to be recovered.This problem is fundamentally ill-posed.Most exis- ting approaches tend to design complex priors according to the different characteristics between the two separated layers .However,loss function with complex prior regularization is hard to be optimized.Performance is also compromised by the fixed iteration schemes and less data fitting ability.More importantly, these conventional prior based methods can only be applied to one specific task as they are weak in generalization.To partially mitigate the limitations mentioned above ,this paper developed a flexible optimization technique to incorporate deep architectures into optimization iterations for adaptive image layer separation.As we all know,the convolutional neural network is a network structure composed of convolutions and other non-linear operations.The convolution operation uses different convolution kernels to extract different features for a given image,so the convolution kernel has very strong capabilities for feature extraction.Recently,the advantages of deep learning in feature extraction have been gradually reflected and are increasingly used in the low-level image processing.Therefore,the proposed method uses deep convolutional prior instead of traditional model prior to characterize different layers.At the same time,in order to reduce the network’s dependence on training data and improve the effectiveness of the algorithm on different tasks,deep information is combined with traditional optimization framework.Specifically,energy function using MAP (Maximum A Posteriori) is built and then the model is transfered to three subproblems based on ADMM (Alternating Direction Method of Multipliers).The first two subproblems are to estimate two approximate separated layers,and the other subproblem is to solve the final result.In other words,deep convolutional networks are used to guide the process of model optimization.In this way,the proposed method not only retains the advantage of feature extraction in deep structure,but also maintains the stability of traditional model optimization and improves the effectiveness of networks.Finally,this method is applied to a variety of ima- ge restoration tasks,including single image rain streak removal and reflection removal.By comparing this method with several tasks-specific methods including conventional model methods and deep learning methods respectively,this me- thod shows great advantages in both visual effects and numerical results.It reveals that this method has a strong genera- lization in multi-tasks and outperforms other methods in each task.