多尺度特征融合的低照度光场图像增强算法

Low-Light Enhancement Method for Light Field Images by Fusing Multi-scale Features

光场图像(LFI)记录了目标场景丰富的三维结构和纹理等信息,在多种计算机视觉任务中拥有巨大优势。但是,低光照条件下采集的光场图像存在亮度低、噪声大等问题,降低了图像质量。提出了一种多尺度特征融合的低照度光场图像增强算法,引入数码单反相机(DSLR)图像来监督网络的训练以提升低照度光场图像的质量。为了充分挖掘和利用光场信息,通过角度和空间Transformer在不同尺度上对光场图像进行特征提取,捕获每个子孔径图像的互补信息以及局部和远程依赖关系。提出一个循环融合模块,利用长短时记忆网络保留不同尺度特征的长时记忆,同时通过局部和全局融合层自适应地聚合整个特征空间中的有用信息。设计了一个4D残差模块从聚合的特征重建目标光场子视图。此外,还构建了一个低照度LFI和正常光照DSLR图像配对的数据集来训练所提出的网络。实验结果表明,所提网络能够有效地提升低照度光场图像的质量,相比其他算法拥有明显的优势。

Light field images(LFI)record rich 3D structural and textural details of target scenes,which have great advantages in a wide range of computer vision tasks.However,LFI captured under low-light conditions always suffer from low brightness and strong noise,which may seriously degrade the quality of LFI.In this paper,a low light LFI enhancement method by fusing multi-scale light field(LF)structural features is proposed,which adopts digital single-lens reflex camera(DSLR)images to supervise the generated enlightened LFI.To explore and exploit light field structural features,angular and spatial Transformers are introduced to extract LF structural features from LFI at different scales,i.e.,the complementary information between sub-views and local and long-range dependencies within each sub-view.A recurrent fusion module is proposed to preserve the long-term memory of features at diffe rent scales by using a long-short-term memory network,while adaptively aggregating LF structural features in the entire feature space through local and global fusion layers.A 4D residual reconstruction module is designed to reconstruct target LFI sub-views from the aggregated features.In additional,a dataset of low-light LFI and normal light DSLR image pairs is constructed to train the proposed network.Extensive experiment demonstrates that the proposed network can effectively improve the quality of low-light LFI,and it obviously outperforms other state-of the-art methods.