联合傅里叶卷积与通道注意力的光场角度重建

Light field angular reconstruction with joint Fourier convolution and channel attention

光场相机能够同时捕获光线的强度和方向信息,但由于成像传感器尺寸的限制,无法同时获得高空间和角度分辨率的光场图像。提出了一种联合傅里叶卷积和通道注意力的光场角度重建方法,通过使用稀疏光场图像4个边角位置的参考视图,可以间接地重建出密集光场图像。考虑到光场数据的内在4D结构,采用通道级密集快速傅里叶残差卷积块,在空域和频域对光场图像的空间和角度相关性进行建模,然后采用基于全局响应归一化的通道注意块,以实现通道间的自适应融合。此外,还提出了一种改进的视点加权间接合成方法,通过为每个参考视图分配一个置信图,为参考视图之间建立联系以合成更真实的新视图。实验结果表明,相比于现有先进的光场角度重建算法IRVAE,所提方法的重建光场图像质量在自然光场数据集30Scenes,Occlusion和Reflective上的平均PSNR分别提高了0.08,0.13和0.13 dB。所提方法在保证光场角度一致性的前提下取得了清晰的重建结果。

Light field cameras,capable of capturing both the intensity and direction of light,are utilized in applications like foreground occlusion removal and depth estimation.However,the limited size of imaging sensors restricts the simultaneous achievement of high spatial and angular resolution in light field images.This paper introduces a method combining Fourier convolution and channel attention for light field angular reconstruction,which indirectly creates dense light field images from sparse ones using reference views from the image's four corners.This method leverages the light field image's inherent 4D structure,employ⁃ing channel⁃level dense fast Fourier residual convolution blocks to model the spatial and angular correla⁃tions in both spatial and frequency domains.Channel attention blocks,utilizing global response normalization,then adaptively fuse these channels.Furthermore,an enhanced viewpoint⁃weighted indirect synthesis approach is proposed,assigning a confidence map to each reference view to improve the synthesis of new,realistic views by establishing relationships between reference views.Experimental results demonstrate that our method outperforms the advanced light field angular reconstruction technique IRVAE,showing an average PSNR improvement of 0.08,0.13,and 0.13 dB on the natural light field dataset 30Scenes,Occlu⁃sion,and Reflective,respectively,ensuring clear reconstruction results with angular consistency in the light field.