结合图像特征迁移的光场深度估计方法

Light Field Depth Estimation Method Combining Image Feature Transfer

光场相机可以通过单次曝光同时采集空间中光线的位置信息和角度信息,在深度估计领域具有独特优势。目前光场真实场景数据集的深度标签难以获取且准确度不高,因此现有的多数光场深度估计方法依赖光场合成场景数据集进行训练,但合成数据集与真实数据集在图像特征分布上的差异,导致网络在将子孔径图像与深度图之间的映射关系应用于真实数据集时容易出现偏差。提出一种新的光场深度估计方法,利用基于对抗学习的图像翻译网络,使合成场景子孔径图像逼近真实场景子孔径图像的特征分布。在图像翻译网络中实施多视图角度一致性约束,保证图像翻译前后不同视角子孔径图像之间的视差关系保持不变。设计一种多通道密集连接深度估计网络,利用多通道输入模块充分提取不同方向子孔径图像堆栈特征,并通过密集连接模块进行特征融合,提升网络特征提取和特征传递的效率。在光场合成数据集4D Light Field Benchmark和光场真实数据集Stanford Lytro Light Field上的实验结果表明:与Baseline网络相比,该网络的均方误差和坏像素率平均降低23.3%和8.6%;与EPINET、EPI_ORM、EPN+OS+GC等方法相比,基于该网络的估计方法有效提升了深度估计的准确度,具有良好的鲁棒性和泛化能力。

Light-field cameras can simultaneously collect the position and angle details of light in space through a single exposure,which possesses unique advantages in the field of depth estimation.As the depth labels of light-field real-scene datasets are difficult to obtain and the accuracy is not high,most existing light-field depth estimation methods rely on a large number of light-field synthetic scene datasets for training.However,the difference in the image feature distributions between the synthetic and real datasets leads to deviations in the mapping relationship between the sub-aperture image and depth map learned by the network in the synthetic dataset when applied to the real dataset.A new light-field depth estimation method is proposed in this study.First,the image translation network based on adversarial learning is used to approximate the feature distribution of the real-scene image using the synthetic-scene-centered sub-aperture image,thereby implementing the multi-view angle consistency constraint in the image translation network to ensure the sub-apertures of different views before and after image translation.The disparity relationship between the images does not change.Second,a multi-channel Dense Connection(DC)depth estimation network is designed,in which the multi-channel input module extracts the features of sub-aperture image stacks along different directions.The feature fusion is performed using the DC module,which improves the efficiencies of network feature extraction and feature transfer.Finally,the experimental results of the light-field synthetic dataset,i.e.,4D Light Field Benchmark,and light-field real dataset,i.e.,Stanford Lytro Light Field,indicate that the values of the Mean Square Error(MSE)and Bad Pixel(BP)indicators of the proposed network are reduced by 23.3%and 8.6%compared with the Baseline network results,which are comparable to the existing ones.Compared with the EPINET,EPI_ORM,and EPN+OS+GC methods,the proposed estimation method based on the network above effectively improves depth estimation accuracy and demonstrates better robustness and generalization ability.