融合全局和局部特征的光场图像空间超分辨率算法

A spatial super-resolution method for light filed images by fusing global and local features

光场相机传感器有限的空间分辨率阻碍了光场图像处理相关研究的进展.提出一种融合全局和局部特征的光场图像空间超分辨率算法,提高了对光场子视点全局关系建模的能力.由于光场相机捕捉的图像亮度较低,严重影响了超分辨率图像的质量,提出一个改进的4D零参考深度曲线估计网络(4D Zero-DCE-Net),充分利用光场全部子视点信息来提高光场图像的亮度.为了解决光场图像空间分辨率低的问题,提出一个基于生成对抗网络的光场图像空间超分辨率网络模型.生成器包含三个部分:第一部分是Transformer和4D卷积以并行方式结合的网络结构,能以较浅的网络层捕捉图像的全局和局部细节信息;第二部分是一个交互融合注意力模块IFAM(Interactive Fusion Attention Module),能有效地融合上述两个分支得到的全局自注意力和局部细节信息;第三部分是一个重建模块PS-PA(Pixel Shuffle-Pixel Attention),能提高整个光场的空间分辨率.最后,利用相对判别器来指导生成器的训练.实验结果表明,提出的算法和其他算法相比,峰值信号比(PSNR)至少提升了1 dB.

The limited spatial resolution of sensor of light field camera hinders the progress of light field image processing related research.This paper proposes a spatial super-resolution algorithm for light field images by integrating global and local features,which improves the ability of modeling the global relationship between light field sub-views.Since brightness of captured light field images is low and seriously affects the quality of the super-resolution image,this paper proposes an improved 4D Zero-DCE-Net to make full use of all sub-views of a light field to enlighten light field images.In order to solve the problem of low spatial resolution of light field images,we propose a spatial super-resolution network model of light field images based on generative adversarial network.The generator consists of three parts.The first part is a network structure that combines Transformer and 4D convolution in a parallel manner.It captures global and local details of the images with a shallower network layer.The second part proposes an interactive fusion attention module(IFAM)to effectively fuse the global self-attention and local detail information from the above two branches.The third part is a reconstruction module(PSPA)to improve the spatial resolution of the entire light field.Finally,the relative discriminator is used to guide the training of the generator.Extensive experimental results show that our proposed method improves the PSNR(Peak Signal to Noise Ratio)performance index by at least 1 dB than other methods.