基于改进YOLO v5的复杂环境下柑橘目标精准检测与定位方法

Accurate Detection and Localization Method of Citrus Targets in Complex Environments Based on Improved YOLO v5

针对自然环境下柑橘果实机械化采收作业环境复杂和果实状态多样等情况,提出了一种多通道信息融合网络——YOLO v5-citrus,以解决柑橘果实识别精准度低、果实分类模糊和定位精准度低等难题。将不同的柑橘目标通过不同遮挡条件分为“可采摘”和“难采摘”两类,这种分类策略可指导机器人在真实果园中顺序摘取,提高采摘效率并减少机器人本体和末端执行器损坏率。YOLO v5-citrus中,在颈部网络插入多通道信息融合模块,对柑橘的深浅特征信息进行处理,提高柑橘采摘状态识别精度,同时修改颈部网络拼接方法,针对目标柑橘大小进行识别,训练后在识别部分嵌入聚类算法模块,将训练部分识别模糊的柑橘目标进行最后区分。识别后进行深度图像和彩色图像的像素对齐,并通过坐标系转换获取柑橘目标三维坐标。在使用多种增强技术处理的数据集中,YOLO v5-citrus比原始YOLO v5在平均精度均值和精确率上分别提高2.8个百分点与3.7个百分点,表现出更优异的泛化能力。与YOLO v7和YOLO v8等其他主流网络架构相比较,保持了更高的检测精度和更快的检测速度。通过真实果园的检测与定位试验,得到柑橘目标的三维坐标识别定位系统的定位误差为(1.97 mm,0.36 mm,9.63 mm),满足末端执行器的抓取条件。试验结果表明,该模型具有较强的鲁棒性,满足复杂环境下柑橘状态识别要求,可为柑橘园机械采收设备提供技术支持。

Aiming at the challenges of mechanized citrus fruit harvesting in natural environments,such as complex environments and diverse fruit states,a multi-channel information fusion network(YOLO v5-citrus)was developed,to solve the problems of low accuracy of citrus fruit recognition,fuzzy fruit classification and low accuracy of localization.Different citrus targets were categorized into“pickable”and“hard-to-pick”by different occlusion conditions,and this classification strategy guided the robot to pick them sequentially in a real orchard,which improved the picking rate and reduced the damage rate of the robot body and end-effector.In YOLO v5-citrus,a multi-channel information fusion module was inserted into the neck network to process the depth feature information of citrus to improve the recognition accuracy of the citrus picking state.At the same time,the splicing method of the neck network was modified to recognize the size of the target citrus.The clustering algorithm module was embedded in the recognition part after training to make the final distinction between the citrus targets blurred by the recognition in the training part.Pixel alignment of a depth image and a color image was performed after recognition and 3D coordinates of citrus targets were obtained by coordinate system transformation.In the dataset processed using multiple enhancement techniques,YOLO v5-citrus improved mAP and precsion by 2.8 percentage points and 3.7 percentage points,respectively,compared with the original YOLO v5,respectively.It maintained higher detection accuracy and faster detection speed than other mainstream network architectures such as YOLO v7 and YOLO v8.Through the detection and localization test in the real orchard,the localization error of the 3D coordinate recognition localization system for the citrus target was obtained as(1.97 mm,0.36 mm,9.63 mm),which satisfied the grasping condition of the end-effector.The experimental results showed that the model had strong robustness,meeting the requirements of citrus state recognition in complex environments,and can provide technical support for mechanical harvesting equipment in large-field citrus orchards.