零参考样本下的逆光图像深度学习增强方法

Deep learning based backlight image enhancement method derived of zero-reference samples

目的随着数码设备的普及,拍照成为记录生活的一种主流方式。但是周围环境的不可控因素会导致用户获取到逆光图像。传统的图像增强方法大多是全局增强,通常存在增强过度或增强程度不够的问题。而基于深度学习的图像增强方法大多是针对低照度图像增强任务,此类方法无法同时兼顾逆光图像中欠曝光区域和过曝光区域的增强问题,且在网络训练时需要成对的数据集。方法提出一个基于注意力机制的逆光图像增强网络ABIEN(attention-based backlight image enhancement network),该网络学习逆光图像与增强图像之间像素级的映射参数,解决无参考图像的问题,同时使用注意力机制使网络关注欠曝光区域和过曝光区域的增强。为了解决无法获取成对图像数据集的问题,所设计的网络学习逆光图像与恢复图像之间的映射参数,并借助该参数进行迭代映射以实现图像增强;为了在增强欠曝光区域的同时还能抑制过曝光区域增强过度的问题,通过引入注意力机制帮助网络关注这两个不同区域的增强过程;为了解决大多数图像恢复中都会出现的光晕、伪影等问题,采用原始分辨率保留策略,在不改变图像大小的情况下将主网络各个深度的特征信息充分利用,以削弱该类问题对增强图像的影响。结果通过将本文方法与MSRCR(multi-scale retinex with color restoration)、Fusion-based(fusion-based method)、Learnning-based(learning-based restoration)、NPEA(naturalness preserved enhancement algorithm)和Ex CNet(exposure correction network)等方法进行对比,本文方法得到的增强图像从主观上看曝光度更好、颜色保留更真实、伪影更少;从客观指标来看,本文方法在LOE(lightness order error)上取得了最好的效果,在VLD(visibility level descriptor)和CDIQA(contrast-distorted image quality assessment)上表现也很好;从处理时间上来看,本文方法的处理时间相对较短,可以满足在现实场景中的实时应用。结论提出的逆光图像增强方法通过结合注意力机制和原始分辨率保留策略,可以帮助网络学习各个层次的特征信息,充分挖掘图像内容信息,在矫正图像亮度的同时,还能更好地恢复图像细节。

ObjectiveDigital photos have been evolved in human life.However,backlight images are captured due to its unidentified factors in the context of its scenarios.Without careful control of lighting,important objects can disappear in the backlight areas,causing backlight images to become a fatal problem of image quality degradation.The theoretical cause of backlight image is that the object being photographed is located right between the light source and the shooting lens,the overall dynamic range of the light in the same picture is extremely large.Due to the limitation of the photosensitive element,the general camera cannot incorporate all the levels of detail into the latitude range,resulting in poor shooting results,which further causes problems such as barren visual quality of the entire image,color degradation of meaningful areas and loss of detail information in the image.Current image enhancement methods are focused on the aspect of global enhancement,and there is an issue of excessive enhancement or insufficient enhancement for backlight images.Moreover,deep learning based image enhancement method is mainly related to the low-illumination image enhancement task,which cannot take the backlight images enhancement of underexposed and overexposed regions into account simultaneously.We illustrate an attention-based backlight image enhancement network(ABIEN),which can resolve non-pairing image sets via learning the pixel-wise mapping relationship between the backlight image and the enhanced image,and facilitate network training to enhance underexposed and overexposed regions.MethodFirst,our demonstrated network is designated to learn the mapping parameters between the backlight image and the restored image to obtain paired datasets in an iterative way,and the enhanced image is obtained based on learned mapping parameters to transform the backlight image.Pixel-level parameters avoid the disadvantages of the previous methods without distinction enhancement and achieve targeted enhancement.Next,in order to enhance the underexposed region and suppress the overexposed region,the attention mechanism is carried out to focus on the two aspects of trained network.Experiments show that the attention mechanism in the network can distinguish the underexposed region and overexposed region in the backlight image more accurately,and promote the optimal mapping parameters generated by the network.Finally,in order to solve the problems of artifact and halo in most image restoration works,we harness a strategy of retaining the original resolution to extract the features of each depth of the backboned network.The feature information extraction based on this strategy solves the problem of poor feature information caused by single scale resolution.Besides,the artifact and halo issues can be further deducted.ResultIn comparison of multi-scale retinex with color restoration(MSRCR),fusion-based method(Fusion-based),learning-based restoration(Learningbased),naturalness preserved enhancement algorithm(NPEA)and exposure correction network(ExCNet)methods,our demonstration is focused on enhanced image exposure more,which are more real color retention and fewer artifacts.Lightness order error(LOE),visibility level descriptor(VLD)and contrast-distorted image quality assessment(CDIQA)are utilized to evaluate the image quality restored by different methods.LOE index is used to evaluate the changes of image brightness order statistics caused by halo,artifact,contour and ringing,which affect the visual perception quality of the image.The smaller LOE value is,the better the restoration method is.VLD is computing the ratio between the gradient of the visible edges between the image before and after the restoration method,and the higher value of VLD means the better visual quality of the image.CDIQA can be regarded as an indicator to evaluate the content richness of an image.A high CDIQA value indicates better image quality.By comparing the proposed method with others,it is proved that our LOE value,VLD and CDIQA indicators have their priority both.The processing running speed of our method is relatively faster which can meet the real-time application in real scenario.ConclusionOur backlight image enhancement method can melt the attention mechanism and the original resolution retention strategy into the feature learning at all layers of the network and mine the detailed information overall.Subjective and objective evaluations were conducted to corroborate the superiority of the proposed approach over the others.The experimental results show that the proposed enhancement method can satisfactorily rectify the underexposure and overexposure problems that coexist in backlight images,demonstrating its superiority in restoring image details over the other competitors.Additionally,the artifacts in the image are effectively suppressed,both backlight and non-backlight regions without introducing annoying side effects.A high processing efficiency makes the proposed method have a good application prospect.