基于深度估计与自我运动联合优化的三维重建

3D reconstruction based on joint optimization of depth estimation and ego-motion

机器人安全导航或执行高级任务时需要稠密的三维重建。图像准确的深度信息以及位姿是三维重建的基础。RGB-D相机获得的深度图的分辨率较低、精度有限,而且RGB-D相机易受玻璃或者纯黑色物体的影响,传统的位姿计算方法不够准确。为了解决这些问题,提出了一种三维重建系统,使用神经网络预测RGB图像的深度以及位姿,然后进行重建。针对物体的轮廓会对深度图的最终预测起决定性作用的问题,提出了轮廓损失函数。针对RGB图像易受光线以及噪声影响的问题,首次增加了RGB图像的特征图作为网络的输入。设计了特征损失来联合优化深度估计和自我运动。在TUM RGB-D,ICL-NUIM数据集上,明显提高了深度图的质量、定位结果以及三维重建的效果。

Dense 3 D reconstruction is required for robots to safely navigate or perform advanced tasks.The accurate depth information and pose of the image are the basis of 3 D reconstruction.The depth map obtained by the RGB-D camera has low resolution and limited precision, and RGB-D cameras are susceptible to glass or pure black objects.The traditional pose calculation method is not accurate enough.In order to solve these problems, a 3 D reconstruction system is proposed, which uses neural network to predict the depth and pose of the RGB images and then reconstruct them.For the problem that the contour of the object plays a decisive role in the final prediction of the depth map, the contour loss function is innovatively proposed.To solve the problem that RGB images are susceptible to light and noise, the feature map of RGB images is added as the input of the network for the first time.Feature loss is designed to jointly optimize depth estimation and ego-motion.On the TUM RGB-D and ICL-NUIM datasets, the quality of the depth map, the localization results and the effect of 3 D reconstruction are significantly improved.