基于投票决策的实时遮挡处理技术

Real-time virtual and real occlusion processing technology based on voting decision

针对当前基于深度信息的虚实遮挡处理技术面临的实时性差和精度低的问题,提出一种基于局部区域深度估计和基于patch相似性噪声点投票融合的实时虚实遮挡处理算法。该算法将真实场景视频序列作为输入,首先利用局部区域深度估计算法通过稀疏重建估算出稀疏关键点的深度信息,对稀疏深度施加目标区域的约束限制深度向周围像素的传播,从而快速恢复出目标区域的相对深度图;然后,噪声点投票融合算法利用深度比较确定虚实物体的前后位置关系,基于patch相似性和投票决策的方法对区域内的像素进行投票和融合绘制;最后输出具有真实遮挡关系的融合效果。实验结果表明,该算法不仅可以提高虚实遮挡的实时性,还能够获得真实和虚拟场景不同空间关系下的良好融合效果。

Aiming at the problems of poor real-time performance and low accuracy faced by the current virtual and real occlusion processing technology based on depth information,a real-time virtual and real occlusion processing algorithm was proposed based on local area depth estimation and patch similarity-based fusion of noise points.The algorithm took the real scene video sequence as the input.Firstly,the depth information of sparse key points was estimated using the local area depth estimation algorithm through sparse reconstruction,and the sparse depth was imposed on the target area to limit the depth propagation to the surrounding pixels,so as to quickly restore the relative depth map of the target area.Then,the noise point voting fusion algorithm employed depth comparison to determine the front and back position relationship of the virtual and real objects,and voted and merged the pixels in the area based on the patch similarity and voting decision method.Finally,the fusion effect with real occlusion relationship was output.The experimental results show that the proposed algorithm can not only improve the real-time performance of virtual and real occlusion,but also obtain a good fusion effect under different spatial relationships between real and virtual scenes.