基于RGB-D相机的果树三维重构与果实识别定位

Research on 3D Reconstruction of Fruit Tree and Fruit Recognition and Location Method Based on RGB-D Camera

为实现对果园果实机器人采摘提供科学可靠的技术指导,提出了一种基于RGB-D相机的苹果果树三维重构以及进行果实立体识别与定位的方法。使用RGB-D相机快速获取自然光照条件下果树的彩色图像和深度图像,通过融合果树图像彩色信息和深度信息实现了果树的三维重构;对果树的三维点云进行R-G的色差阈值分割和滤波去噪处理,获得果实区域的点云信息;基于随机采样一致性的点云分割方法对果实点云进行三维球体形状提取,得到每个果实质心的三维空间位置信息和果实半径。实验结果表明,提出的果树三维重构和果实立体识别与定位方法具有较强的实时性和鲁棒性,在0.8~2.0 m测量范围内,顺光和逆光环境中果实正确识别率分别达95.5%和88.5%;在果实拍摄面的点云区域被遮挡面积超过50%的情况下正确识别率达87.4%;果实平均深度定位偏差为8.1 mm;果实平均半径偏差为4.5 mm。

In order to provide a scientific and reliable technical guidance for fruit harvesting robot in orchard,a method was proposed in this paper to reconstruct 3D image for apple tree and carry out recognition and location for each apple fruit. Firstly,the color image and depth image of the fruit trees were taken by an RGB-D camera,and the 3D reconstruction of each fruit tree was carried out by fusing its color and depth information. Then,3D point cloud of the fruit region were segmented from tree 's point cloud by applying the color threshold of R-G. Finally,the 3D shape of each fruit point cloud was extracted and its 3D spatial position information and radius were also obtained by using iteratively the RANSAC( Random sample consensus) algorithm to fit each fruit to a pre-defined apple model. The experimental results showed that the proposed method of 3D reconstruction of apple tree and recognition and location of its fruits had good real-time performance and strong robustness. In the measurement range of 0. 8 ~ 2. 0 m,the correct recognition rates of fruits under frontlighting and backlighting conditions were95. 5% and 88. 5% respectively,and the correct recognition rate was 87. 4% in the case that the sheltered area of fruit point clouds was over 50%,besides,the average position calculation error of the fruit was 8. 1 mm,and the average radius calculation error was 4. 5 mm.