基于图像与深度信息融合的西梅果实识别定位方法

Recognition and positioning method of prune fruit based on fusion of image and depth information

为实现西梅采摘机器人在复杂自然生长环境下对西梅果实的快速、准确识别定位,提出一种基于图像与深度信息融合的西梅果实识别定位方法。首先采集西梅图像,并通过数据增强建立阴天、沙尘等环境和不同角度下的西梅图像数据集,然后利用彩色图像基于YOLOv7网络模型快速识别西梅果实,获取果实表面中心点,再通过匹配融合彩色图和深度信息,采用深度距离分割剔除背景干扰噪声,实现西梅果实表面中心点的三维空间定位。试验结果表明,西梅果实识别模型可在多种果实生长分布场景下实现西梅果实的识别,其识别F1值最高为95.8%,最低为83.2%;融合图像与深度信息的定位方法具有良好的定位效果,当深度距离小于1 m时,算法在各轴向上的定位误差均在0.005 m内;当深度距离为1.5 m时,误差最高为0.013 m,可满足西梅果实的识别定位要求。

In order to realize the rapid and accurate identification and positioning of prune fruit by the prune picking robot in the complex natural growth environment,a method for prune fruit identification and positioning based on the fusion of image and depth information was proposed.Firstly,the image of prune was collected,and data enhancement was used to establish prune image data sets under cloudy days,dusty environment and different angles,and then the color image was used to quickly identify prune fruits based on the YOLOv7 network model,the center point of the fruit surface was obtained.By matching and blending the color image and depth information,the depth distance segmentation was adopted to eliminate the background interference noise,to realize the three‑dimensional spatial positioning of the center point on the prune fruit surface.The experimental results show that the prune fruit recognition model can realize the recognition of prune fruits in a variety of fruit growth and distribution scenarios,and its recognition F1 value is 95.8%at the highest and 83.2%at the lowest,the positioning method of fusion image and depth information has a good effect positioning effect,when the depth distance is less than 1 m,the positioning error of the algorithm in each axis is within 0.005 m,when the distance is 1.5 m,the maximum error is up to 0.013 m,which can meet the requirements of recognition and positioning of prune fruit.