基于图像分割的深度视频校正算法

A depth video rectification algorithm based on image segmentation

在基于深度图的虚拟视点绘制过程中,由于通过深度估计软件获取的深度视频存在大量的失真,从而导致绘制的虚拟视点中存在纹理失真和缺失现象。本文围绕深度视频失真类型,提出一种基于分割的深度视频校正算法。利用彩色深度一致性信息分区域校正深度失真,以解决由于深度块失真造成的虚拟视点纹理缺失问题。首先,提取彩色视频运动和边缘区域,得到彩色视频边缘和运动区域掩模图;其次,在边缘和运动信息的辅助下,对彩色图像进行Mean Shift聚类,并将不同类别区域赋以不同的标签;最后,分别统计不同类别连通区域对应的深度直方图,利用其峰值校正深度视频中深度彩色非一致区域。实验结果表明,本文提出的基于分割块的深度视频校正算法优于部分基于像素的滤波算法,可以有效地校正深度视频块失真,解决虚拟视点边缘失真和纹理缺失问题,同时虚拟视点质量平均提高了0.20dB。

In based on depth image virtual viewpoint rendering,depth video produced by depth estimation software exists lots of distortions which cause distortion and texture loss in virtual viewpoint.To enhance the quality of depth video produced by depth estimation software,a depth image rectification algorithm based on image segmentation is proposed to tackle depth video distortion in this paper.The consistency of color and depth map is utilized to correct the depth distortion in different regions,which consequently improves the virtual view quality.Firstly,the edge and the motion regions of texture video are detected,from which motion and edge masks can be derived.Secondly,aided by edge motion information of color image,the color image is classified by mean shift method and different labels are assigned to different classes.Finally,the corresponding convergence regions in depth map are statistically analyzed,and depth histogram is obtained.The depth relative to the peak in depth histogram is used to correct the inconsistent region.Experimental results show that this depth video rectification algorithm is better than based pixel on filtering,rectifies the block distortion of depth video,solves the problem of texture loss and edge distortion,and improves virtual view quality by 0.20 dB.