基于语义边缘驱动的实时双目深度估计算法

Real-time Binocular Depth Estimation Algorithm Based on Semantic Edge Drive

针对立体匹配中不适定区域视差边缘模糊、视差不平滑、单个物体视差不连续、存在空洞的问题,提出了一种轻量化的实时双目深度估计算法,将场景图、通过语义分割得到的语义标签图和通过边缘检测得到的边缘细节图作为辅助损失,以地面真值图为主要损失,构造了联合损失函数,以更好地监督视差图的生成。此外,构造了一个轻量化的特征提取模块,以降低特征提取模块的冗余性,从而更好地简化特征提取步骤,提高了网络的实时性和轻量性。最后利用由粗到精的思想实现视差图的渐进细化过程,利用低分辨率视差图变形与高分辨率特征图融合的方式,分阶段生成不同尺度的视差图,细节特征逐渐丰富,从而获得了最终的精准视差图。在KITTI 2012数据集上得到1.72%的3px错误率,在Middlebury 2014数据集中,Vintge错误率为1.23%,Playroom错误率为2.23%,Recycle错误率为1.65%,并且在Scene Flow数据集上计算时间低至0.76 s,内存占用量为2.4 G,显著提高了立体匹配算法在不适定区域的准确性和计算效率,能够满足工程实践中的实时性要求,对于实时三维重建任务有着很重要的指导意义。

Aiming at the problem of ill-posed regions with blurred disparity edges,unsmooth disparity,discontinuous disparity of a single object,and holes in stereo matching,a lightweight real-time binocular depth estimation algorithm is proposed,which uses the semantic tags obtained by semantic segmentation of the scene graph and the edge detail images obtained by edge detection as auxi-liary loss,and the ground truth image as the main loss,to construct the joint loss function which can better supervise the generation of the disparity map.In addition,a lightweight feature extraction module is constructed to reduce the redundancy of the feature extraction stage,which can better simplify the feature extraction steps,and improve the real-time and lightness of the network.Finally,the idea of from coarse to fine is used to realize the gradual refinement process of the disparity map with fusion of low-resolution disparity map deformation and high-resolution feature map to generate disparity maps of different scales in stages,meanwhile,the detailed features are gradually enriched,thus obtaining the final accurate disparity map.The 3px error rate of 1.72%is obtained on the KITTI 2012 dataset,the Vintge error rate on the Middlebury 2014 dataset is 1.23%,the Playroom error rate is 2.23%,and the Recycle error rate is 1.65%.Meanwhile,the calculation time on the Scene Flow dataset reaches 0.76 s with 2.4G memory occupation,which significantly improves the accuracy and computational efficiency of stereo matching algorithms in the ill-posed regions,meets the real-time requirements in engineering practice,and has important guiding significance for real-time 3D reconstruction tasks.