基于盲点网络的低照度图像自监督增强

Self-Supervised Enhancement of Low-Light Images Based on Blind Spot Networks

针对现有的低照度图像增强算法在增强的同时其图像中仍含有残留噪声、网络训练中将产生恒等映射以及成对数据集获取困难的问题,本文提出了一种基于盲点网络的自监督微光增强网络。首先,采用双边多尺度融合直方图均衡化的方法对图像亮度进行调整,以此来克服传统直方图增强方法中的信息颜色损失;其次,所设计的去噪网络可以自适应地从原始图像中进行学习,同时采用像素混洗下采样解耦相邻像素空间中的相关性;最后,为保持图像空间和颜色的一致性设计了相关的损失函数。实验证明,本文算法克服了现有算法存在残留噪声、网络训练中产生恒等映射的问题,有效地提高了低照度图像质量。

Objective Images can be conceptualized as a vivid linguistic schema that communicates information and elicits emotions via distinct elements comprising lines, colors, shapes, textures, etc. The human visual apparatus demonstrates an elevated sensitivity and recognition proficiency towards these visual components, thereby amassing a lot of information and enriching experience from simple image observations. Additionally, images exert a perceptible influence on human vision.For instance, variations in color, contrast, brightness, and other factors can trigger diversified reactions within the human visual system. However, due to suboptimal environmental lighting, equipment limitations, and the photographer proficiency, the resultant images frequently fail to meet the anticipated outcomes. Among the multitude of factors impinging on image quality, the pervasive influence of environmental lighting conditions, particularly in low-light environments, is the most remarkable. Low-light images can be characterized as images captured in lighting conditions that are insufficient to fully stimulate the brightness capture function of the camera. Consequently, the output image is not even on the fringe of possessing an exemplary histogram distribution. In such predicaments, the implementation of a specialized algorithm becomes imperative to facilitate image enhancement, thereby delivering an optimized image and bolstering overall performance.Methods To solve the problems of residual noise, identity mapping in network training, and pairwise data acquisition,we propose a self-supervised low-light enhancement network based on a blind spot network. Firstly, the technique of bilateral multi-scale fusion histogram equalization is utilized to adjust the image brightness and thus overcome the information color loss prevalent in traditional histogram enhancement methods. Secondly, the designed denoising network can adaptively learn from the original image, while pixel shuffle downsampling is employed to decouple the correlation in adjacent pixel spaces. Lastly, related loss functions are designed to maintain the consistency of image space and color.Results and Discussions Initially, we delve into the performance of network models with varying stride factors and convolution kernel sizes(Table 1 and Fig. 7). As the stride factor ascends, a parallel increment in model performance ensues, culminating in a peak at stride factor of 5. On the contrary, a continued escalation in stride factor degrades the network performance. As the stride factor widens, the spatial correlation of noise signals decreases, and pixel correlation also diminishes due to an extended distance between the pixels. Only when the noise eradicates a greater proportion of image details, manifesting as aliasing artifacts, does the performance indicator plummet. In contrast, smaller convolution kernels have proven their supremacy in effective image detail capture. To measure the effectiveness of our proposed method, we conduct a comparative experimental analysis using 11 diverse methodologies. Meanwhile, we employ four tangible evaluation metrics, including peak signal-to-noise ratio(PSNR), structural similarity(SSIM), color deviation(Delta E), and natural image quality evaluation(NIQE). During utilizing the LOL test set, outcomes indicate certain limitations inherent to traditional methodologies(Table 2 and Fig. 8). These limitations range from handling local optimal solutions and resulting ramifications of lackluster color, deficient brightness, and conspicuous noise, to consequential discrepancies in color and brightness, blurring, and obscured details. However, our proposed method exhibits remarkable superiority and succeeds in visually retaining the input image color and aligning the overall image brightness much closer to the real image. Furthermore, against the backdrop of most existing enhancement techniques, our proposal stands out with preeminent outcomes across defining evaluation metrics. Additionally, we perform multiple generalization evaluation experiments inclusive of enhancing low-light images captured in actual world settings(Fig. 9). The performance of our proposed method is sound, and features optimum brightness, homogeneous color dissemination, and vividly delineated details. Eventually, we quantify the influence of the loss function on image enhancement by ablation experiments(Table 3and Fig. 10). Meanwhile, we discover that the custom-designed loss function has a profound bearing on the images,thereby authenticating its efficacy. In summary, when juxtaposed with other available strategies, the proposed algorithm demonstrates superior efficacy.Conclusions We propose an enhancement method for low-light images based on blind spot network. The low-light image enhancement task is divided into two sub-tasks of enhancement and denoising, and a set of loss functions without reference is designed to guide network training. By adopting a self-supervised enhancement technique, the limitation of paired data required by many traditional enhancement algorithms is overcome. Meanwhile, blind spot convolution technology is employed to ensure that the identity mapping phenomenon is avoided during the training, which can enhance the network robustness, remove the noise generated during the enhancement, and improve the generalization ability. The experimental results show that our method is superior to the existing methods in image quality and visual effect. At the same time, it is also compared with some other classical image enhancement algorithms, which proves that this method has certain advantages and can provide references for the enhancement of low-light images.