YCbCr空间分治的双分支低照度图像增强网络

Dual-branch low-light image enhancement network via YCbCr space divide-and-conquer

目的现有的低照度图像增强算法通常在RGB颜色空间采用先增强后去噪的方式提升对比度并抑制噪声,由于亮度失真和噪声在RGB颜色空间存在复杂的耦合关系,往往导致增强结果不理想。先增强后去噪的方式也放大了原本隐藏在黑暗中的噪声,使去噪变得困难。为有效处理亮度失真并抑制噪声,提出了一个基于YCbCr颜色空间的双分支低照度图像增强网络,以获得正常亮度和具有低噪声水平的增强图像。方法由于YCbCr颜色空间可以分离亮度信息与色度信息,实现亮度失真和噪声的解耦,首先将低照度图像由RGB颜色空间变换至YCbCr颜色空间,然后设计一个双分支增强网络,该网络包含亮度增强模块和噪声去除模块,分别对亮度信息和色度信息进行对比度增强和噪声去除,最后使用亮度监督模块和色度监督模块强化亮度增强模块和噪声去除模块的功能,确保有效地提升对比度和去除噪声。结果在多个公开可用的低照度图像增强数据集上测试本文方法的有效性,对比经典的低照度图像增强算法,本文方法生成的增强图像细节更加丰富、颜色更加真实,并且含有更少噪声,在LOL(low-light dataset)数据集上,相比经典的KinD++(kindling the darkness),峰值信噪比(peak signal-to-noise ratio,PSNR)提高了3.09 dB,相比URetinex(Retinex-based deep unfolding network),PSNR提高了2.74 dB。结论本文提出的空间解耦方法能够有效地分离亮度失真与噪声,设计的双分支网络分别用于增强亮度和去除噪声,能够有效地解决低照度图像中亮度与噪声的复杂耦合问题,获取低噪声水平的亮度增强图像。

Objective The images acquired at night or backlight conditions always have poor visibility and have details that are hidden in the dark.Moreover,due to insufficient lighting and limited exposure time,the number of incident photons on these images decreases,thereby resulting in a large amount of non-negligible noise.Therefore,improving the contrast and removing noise from low-light images present a challenge.The existing low-light image enhancement algorithms usually enhance the contrast and suppress the noise in the RGB color space by way of enhancing and then denoising.However,due to the complex coupling relationship between brightness distortion and noise in the RGB space,enhancing-and-then denoising methods usually amplify the noise that is originally hidden in the dark,thus increasing the difficulty of the denois⁃ing task,affecting the aesthetic quality of images,and constraining subsequent image processing tasks on the aspects of image classification,object detection,and recognition.To effectively deal with brightness distortion and noise,a dualbranch low-light image enhancement network based on YCbCr space is proposed in this paper to yield enhanced images with minimal color distortion and less noise.Method The YCbCr space can separate luminance information from chromi⁃nance information.Experiments show that brightness distortion is mostly observed in luminance information and that chro⁃minance information is heavily polluted with noise.This paper designs a corresponding network structure based on the divide-and-conquer method to deal with different degradation modes,where different modules can effectively deal with a specific distortion and reduce the difficulty of network learning.A novel dual-branch network in YCbCr space is then devel⁃oped to realize the decoupling of luminance distortion and noise,which can enhance the luminance information and denoise the chrominance information.First,the low-light images are transformed from the RGB space to the YCbCr space.Sec⁃ond,the images in the YCbCr space are inputted into the proposed dual-branch enhancement network for low-light image enhancement.This network comprises a luminance enhancement module,a noise removal module,and a fusion module.The luminance enhancement module performs contrast enhancement on the luminance information,the noise removal mod⁃ule denoises the chrominance information,and the fusion module combines the features of luminance and chrominance to obtain the final enhanced results.The luminance channel Y and chrominance channels Cb and Cr serve as inputs of the luminance enhancement and noise removal modules,respectively.Given that the luminance distortion is often non-local,the luminance enhancement module adopts a U-shaped network that can obtain the rich brightness context information of the image via an encoder and decoder.Specifically,in the U-Net network,the encoder expands the receptive fields of con⁃volutions through a pooling operation,and at the bottleneck layer,the large receptive field can extract non-local luminance information for contrast enhancement.This non-local information can then be extended to the global scale by up-sampling at the decoder layer by layer.Given that noise damages the texture of low-light images,the noise removal module uses a multi-scale denoising network to enrich the details of the image and effectively learn its multi-scale features.The noise removal module consists of a multi-residual channel attention block,pixel detection,and pixel shuffle.The residual chan⁃nel attention block,which is commonly found in several networks,is suitable for feature extraction.Instead of up-sampling and down-sampling,the pixel unshuffled and shuffle operations are used to obtain multi-scale input images,remove noise at multi-scale features,fuse multiple features,and eventually obtain valuable features for denoising while avoiding informa⁃tion loss in the images.The brightness supervision and chrominance supervision modules are also used to strengthen the functions of the brightness enhancement and noise removal modules,which can help boost contrast and remove noise effec⁃tively.The supervision modules ensure the features of brightness enhancement and noise removal.Several simple convolu⁃tion operations are then applied to concatenate the outputs of the brightness enhancement and noise removal modules and obtain the final results in the YCbCr space.An additional transform operation is applied to convert the images from the YCbCr space to the RGB space.Result The effectiveness of the proposed algorithm is tested on several paired and unpaired public low-light image enhancement datasets,such as the low-light dataset(LOL),vision enhancement in the low-light condition dataset,multi-exposure image fusion dataset,and naturalness preserved enhancement dataset.The proposed method is also compared with several classical low-light enhancement algorithms that have achieved excellent results in lowlight image enhancement.These algorithms are implemented using public codes,and their parameters are set by referring to their original papers.These methods are then objectively evaluated based on their visual results and by using several quality metrics,including peak signal-to-noise ratio(PSNR),structure similarity index,natural image quality evaluator,and learned perceptual image patch similarity.The images generated by the proposed method have richer details,more realistic color,and less noise compared with those generated by the other algorithms.Moreover,on the LOL dataset,com⁃pared with kindling the darkness(KinD++),the PSNR of the proposed method increases by 3.09 dB,while compared with the Retinex-based deep unfolding network that uses Transformer as its basic module,the PSNR of the proposed method increases by 2.74 dB.Conclusion The proposed spatial decoupling method effectively decouples the brightness distortion and noise of low-light images.The designed dual branch network can enhance the brightness and remove noise,thus effec⁃tively solving the complex coupling problem of brightness and noise in low-light images and obtaining noise-free brightness enhanced images.Extensive experiments demonstrate the effectiveness and generalization of this method,which outper⁃forms the state-of-the-art both qualitatively and quantitatively.