融合全局与局部视角的光场超分辨率重建

Light field super-resolution using global and local multi-views

针对光场相机结构和像素传感器分辨率的限制导致光场图像空间分辨率和角度分辨率都较低的问题,提出一种融合全局与局部视角的光场超分辨率重建算法,同时提高光场图像的空间分辨率和角度分辨率。首先根据待重建新视角的位置,自适应选择局部视角,利用空间超分辨率卷积神经网络提高全局视角和局部视角的空间分辨率,然后提取并融合全局视角和局部视角在新视角处映射图像的深度特征和颜色特征,通过角度分辨率卷积神经网络重建获得新视角图像。实验结果表明,与现有方法相比,峰值信噪比(PSNR)提高约3 dB,结构相似性指数(SSIM)提高约0.02,有效地解决了遮挡情况下重建新视角局部目标丢失现象,同时更好地保持新视角的边缘信息,获得了更优的重建效果。

The spatial and angular resolution are comparatively low owing to the limitation of the resolution of its pixel sensor which is due to design reasons. To address the above problem,this paper presented a new super-resolution method based on global and local views to simultaneously enhance both the spatial and angular resolutions of a light field image.First of all,it adaptively chose local views according to the position of the novel view,and then used spatial super-resolution convolution neural network to get high resolution sub-images such as global-view and local-view images,subsequently it extracted and fused the depth and color features from the image obtained by backward warping the input views.Then it put the results into the angular super-resolution network which produced a novel view image as the final result. The experimental results demonstrate that the reconstruction effect of this method is superior to the state-of-the-art’s. And in peak signal to noise ratio get 3 dB promotion and structural similarity image measure upgrade about 0.02. In addition,it is not only able to improve spatial and angular resolution of multi-view images,but also to maintain edge information of the novel view,thus effectively avoiding the loss of objects and yielding a better reconstruction effect.