基于RGB-D图像的移动端点云分割方法研究

Research on point cloud segmentation method of mobile devices based on RGB-D image

近年来,深度传感器和三维激光扫描仪的普及推动了三维点云处理方法的快速发展。针对传统的前后端分离的点云分割模式,提出了一种使用移动端设备进行三维数据采集与处理的一体化技术方案。基于谷歌的AR Core开发平台,进行了安卓设备上的深度图获取实验,深度图可进一步转换为点云数据;通过对模型轻量化方法的研究,改进了PointNet网络,使模型参数量减少为原来的1/5,同时具有约73%的分割精度;最后利用TensorFlow Lite移动端深度学习框架,将改进的PointNet网络成功部署到了安卓智能手机上,量化后的tflite模型仅268 kB大小,在启用GPU加速后,对单幅场景点云数据的推断速度约为0.7 s。实验结果表明了提出方法的可行性。

In recent years,the popularity of depth sensors and 3D laser scanners has promoted the rapid development of 3D point cloud processing methods.Aiming at the traditional point cloud segmentation mode with front-end and back-end separation,an integrated technical solution for 3D data collection and processing using mobile devices is proposed.Based on Google’s AR Core development platform,the depth map acquisition experiment on Android devices is carried out,the depth map can be further converted into point cloud data;through the research on the light-weight method of the model,the PointNet network is improved,the model parameters are reduced to 1/5 of the original,while it had a segmentation accuracy of about 73%.Finally,using the TensorFlow Lite mobile terminal deep learning framework,the improved PointNet network is successfully deployed on Android smartphone,and the quantized tflite model is only 268 kB in size.After enabled GPU acceleration,the inference speed of single scene point cloud data is about 0.7 s.The experimental results show the feasibility of the proposed method.