结合空间域和频域信息的双分支低光照图像增强网络

Dual-branch low-light image enhancement network combining spatial and frequency domain information

针对低光照图像增强中纹理细节模糊和颜色失真的问题,从空间域和频域信息结合的角度出发,提出一个端到端的轻量级双分支网络(SAFNet)。SAFNet使用基于Transformer的空间域处理模块和频域处理模块在空间域分支和频域分支分别对图像的空间域信息和傅里叶变换后的频域信息进行处理,并通过注意力机制引导两个分支的特征进行自适应融合,得到最终增强的图像。此外,针对频域信息提出一个频域损失函数作为联合损失函数的一部分,通过联合损失函数在空间域和频域都对SAFNet进行约束。在公开数据集LOL和LSRW上进行实验,在LOL上,SAFNet在客观指标结构相似性(SSIM)和学习感知图像块相似度(LPIPS)两项指标上分别达到0.823和0.114;在LSRW上,峰值信噪比(PSNR)和SSIM分别达到17.234 dB和0.550,均优于LLFormer(Low-Light Transformer)、IAT(Illumination Adaptive Transformer)、KinD(Kindling the Darkness)++等主流方法,且网络参数量仅为0.07×10^(6);在DarkFace数据集上,使用SAFNet作为预处理步骤对待检测图像进行增强,可以使人脸检测平均精确率从52.6%提升至72.5%。实验结果表明,SAFNet能有效提高低光照图像的质量,并能显著改善下游任务低光照人脸检测的性能。

To address the problems of blurred texture details and color distortion in low-light image enhancement,an end-to-end lightweight dual-branch network by combining spatial and frequency information,named SAFNet,was proposed.Transformer-based spatial block and frequency block were adopted by SAFNet to process spatial information and Fourier transformed frequency information of input image in spatial and frequency branchs,respectively.Attention mechanism was also applied in SAFNet to fuse features captured from spatial and frequency branchs adaptively to obtain final enhanced image.Furthermore,a frequency-domain loss function for frequency information was added into joint loss function,in order to constrain SAFNet on both spatial and frequency domains.Experiments on public datasets LOL and LSRW were conducted to evaluate the performance of SAFNet.Experimed results show that SAFNet achieved 0.823,0.114 in metrics of Structural SIMilarity(SSIM)and Learned Perceptual Image Patch Similarity(LPIPS)on LOL,respectively,and 17.234 dB,0.550 in Peak Signal-to-Noise Ratio(PSNR)and SSIM on LSRW.SAFNet achieve supreme performance than evaluated mainstream methods,such as LLFormer(Low-Light Transformer),IAT(Illumination Adaptive Transformer),and KinD(Kindling the Darkness)++with only 0.07×10^(6) parameters.On DarkFace dataset,the average precision of human face detection is increased from 52.6%to 72.5%by applying SAFNet as preprocessing step.Above experimental results illustrate that SAFNet can effectively enhance low-light images quality and improve performance of low-light face detection for downstream tasks significantly.