基于CNN的彩色图像引导的深度图像超分辨率重建

Super-resolution reconstruction of depth image guided by color image based on CNN

深度图像表达了三维场景内物体之间的相对距离信息,根据深度图像表达的信息,人们能够准确的获得物体在空间中的位置以及不同物体之间的相对距离,使得深度图像在立体视觉等领域有着广泛的应用。然而受RGB-D传感器硬件条件的限制,获取的深度图像分辨率低,无法满足一些具有高精度要求的实际应用的需求。近年来深度学习特别是卷积神经网络(CNN)在图像处理方面获得了非常大的成功。为此提出了一种基于CNN的彩色图像引导的深度图像超分辨率重建。首先,利用CNN学习彩色图像的边缘特征信息与深度图像的深度特征信息,获得边缘纹理清晰的高分辨率深度图像;再通过不同大小尺寸滤波核的卷积层,进一步优化深度图像的边缘纹理细节,获得更高质量的高分辨率深度图像。实验结果表明,相较于其他方法,该方法 RMSE值更低,重建的图像也能更好的恢复图像边缘纹理细节。

The depth image indicates the relative distance between the objects in the three-dimensional scene. According to the information expressed by the depth image, the position of the object in space and the relative distance between different objects can be accurately obtained, so that the depth image has a wide range of applications in areas such as stereo vision. However, due to the limitations of RGB-D sensor hardware, the acquired resolution of depth image is low, which cannot meet the requirements of some practical applications with high precision. Deep learning, especially the convolutional neural networks(CNN), has achieved great success in image processing in recent years. In this light, this paper proposes a super-resolution reconstruction method of depth image guided by color image based on CNN. First, the CNN are used to obtain the edge feature information of color images and the depth feature information of depth images, so as to obtain high-resolution depth images with clear edge texture. Then, the edge texture details of the depth image are further optimized by the convolution layer of the filter kernels of different sizes, so as to obtain the depth image with higher resolution. The experimental results show that the RMSE value of the method proposed is lower than that of other methods, and the reconstructed image shows clearer edge texture details.