基于学习和模型相结合的光场超分辨率方法

Light Field Super-Resolution Method Based on Combined Learning and Model Methods

随着深度学习技术的发展,基于深度学习的图像超分辨率方法在多个领域取得了目前最优的性能和效果,具有广阔的发展前景.光场(light field)图像可以同时捕捉角度信息和空间信息,近年来在3D重建和虚拟现实应用中得到广泛应用.由于传感器分辨率的限制,光场相机(或光场显微镜)通过牺牲空间分辨率来换取角度分辨率.因此,有限的空间分辨率给相关应用的发展带来了很多困难,成为光场成像的主要瓶颈,光场图像的超分辨率及重建成为重要的研究领域.与传统自然图像超分往往基于双三次退化差值模型相比,基于光场图像的超分辨率模型优化算法具有清晰的物理解释.针对光场图像的超分辨率问题,受LADMM迭代优化算法和卷积神经网络结构的启发,提出一种基于学习和基于模型相结合的新型光场超分辨率网络.该网络的主体部分包含多个神经网络层,每层都与传统LADMM优化算法中每次迭代更新步骤有明确的对应关系.LADMM算法参数的可学习化与残差网络结合,使得本文提出的网络在具有较高的性能同时保持了可解释性.斑马鱼光场数据集中心角度图像的实验结果表明:本文的方法与现有的主流方法相比,具有较好的图像超分辨率能力,同时具有一定的去噪效果.果蝇光场数据集的实验结果表明,本文的方法同传统LADMM优化算法相比,具有更快的运行速度和更好的超分辨率性能.

With the development of deep learning technology,the image super-resolution method based on deep learning has achieved optimal performance in many research fields and has a broad development prospect.Light field images,which can capture angular and spatial information,have been widely used in three-dimensional reconstruction and virtual reality applications in recent years.Because of the limitation of sensor resolution,light field cameras(or light field microscopes)trade spatial resolution for angular resolution.Therefore,the limited spatial resolution has brought many difficulties to the development of related applications,which has become the main bottleneck of light field imaging.The super-resolution and reconstruction of light field images have become an important research field.Compared with the traditional natural image super-resolution model,which is usually based on bicubic degradation,the optimization algorithm of the super-resolution model based on a light field image has a clear physical ex-planation.Aiming at the problem of light field image super-resolution,we propose a new type of super-resolution light field network based on combined learning and model methods,inspired by the LADMM optimization algorithm and convolutional neural network structure.The main part of the network contains multiple neural network layers,where each layer has a clear corresponding relationship with each iteration update step in the traditional LADMM optimization algorithm.The learnability of parameters of the LADMM algorithm combined with a residual network gives the proposed network a high performance and maintains interpretability.Experimental results of the centrality angle image of the zebrafish light field dataset show that the proposed method has better super-resolution capability and a certain denoising effect compared with the existing mainstream methods.Experimental results of the drosophila light field dataset show that the proposed method has faster running speed and better super-resolution performance com-pared with the LADMM optimization algorithm.