基于低分辨率彩色指导图像的深度图像超分辨率重建

Low-Resolution RGB Image Guided Depth Image Super-Resolution

传统的以彩色图像为指导的深度图像超分辨率(SR)重建方法,参考图像必须为高分辨率彩色图像,彩色图像的分辨率决定了深度图像的放大上限。同时,实际应用中可能只存在低分辨率彩色图像,此时上述方法也不再适用。为此,探讨使用任意分辨率彩色图像为指导的深度图像SR重建方法。首先,使用大量不同类别的图像SR算法对输入彩色图像进行上采样,得到高分辨率彩色图像并以此作为指导图像,然后采用基于二阶总广义变分方法,将由低分辨率彩色图像重建得到的图像作为正则约束项,添加图像边缘信息,构建目标函数,将深度图像SR重建问题转化为最优化问题,再通过原-对偶方法求解,最终得到高分辨率深度图像。探讨了之前被相关方法所忽略的情形,该方法可以适用于任意分辨率的彩色指导图像。并且通过相关实验发现了令人惊异的现象,即通过使用低分辨率彩色图像放大后作为指导,可以得到与使用高分辨率彩色指导图像相近甚至更好的结果,对相关问题的研究和应用具有一定参考意义。

In traditional methods of RGB image guided depth image super-resolution,the reference images are required to be high-resolution intensity images and its resolution determines the upper limit of the depth image upsampling.Moreover,in some situations only low-resolution RGB images are available,thus the traditional methods are unpractical.In this paper an arbitrary resolution RGB image guided depth image super-resolution is proposed.First,we use different image super-resolution algorithm for the input RGB image upsampling,so that a high-resolution reference RGB image can be obtained.Then we increase the resolution of the input depth image by using the second-order total generalized variation based method and adding edge cues from the reference image obtained in above step.Then the final energy objective function is defined and depth image super-resolution can be transformed into optimization problem,which can be solved by primal-dual energy minimization scheme.Finally the high-resolution depth image is generated.This paper explores the cases previously ignored by the relevant method and the proposed method can be applied to arbitrary resolution RGB images.Through the relevant experiments,we found an amazing phenomenon that,by using low-resolution color image up-sampling as a guide,we can get similar to or even better results compared with using high-resolution intensity guided image.This conclusion has some reference significance for the research and application of related issues.