基于改进Mask R-CNN的番茄侧枝修剪点识别方法

Recognition method for the pruning points of tomato lateral branches using improved Mask R-CNN

为解决番茄枝叶修剪机器人无法准确识别番茄侧枝修剪点的问题,提出基于改进Mask R-CNN模型的番茄侧枝修剪点的识别方法。将Mask R-CNN的主干网络ResNet50替换为MobileNetv3-Large来降低模型复杂度和提升运行速度,并对部分特征图添加ECA(Efficient Channel Attention)注意力机制,以提升模型精度;通过改进的Mask R-CNN模型预测番茄侧枝与主枝的分割掩膜和边框位置;针对部分单根枝条被分割成多段掩膜的问题,通过掩膜边界框宽高比区分侧枝和主枝,分析同一枝条相邻掩膜约束条件,然后将符合约束条件的掩膜进行合并连接;根据修剪点在主枝附近的特点确定修剪点所在端,确定靠近修剪端端点的中心点作为侧枝的修剪点。试验结果表明,改进的Mask R-CNN模型平均分割图片时间为0.319 s,召回率和精确率分别为91.2%和88.6%,掩膜平均合并成功率为86.2%,修剪点识别平均准确率为82.9%。该研究为番茄枝叶修剪机器人的研发提供参考。

Branch and leaf pruning has been one of the most important links in the process of tomato planting for reducing the disease rate and increasing economic benefits.However,the manual pruning of tomato branches and leaves cannot fully meet large-scale production in recent years,due to the time-consuming and labor-intensive task.An accurate and rapid identification of the pruning position can be greatly contributed to the automatic operation of tomato branch and leaf pruning.In this study,a Recognition method was proposed for the pruning point of the tomato lateral branch using an improved Mask R-CNN.Firstly,the backbone network of ResNet50 in the original Mask R-CNN was replaced with the MobileNetv3-Large to reduce the model complexity.Efficient Channel Attention was added to the feature map C3 and C4,in order to focus more on the features of the lateral and main branch rather than other features.Then,the tomato lateral and main branches were predicted using the improved Mask R-CNN.Three steps were selected to avoid some single branches taken as multiple masks.The lateral and main branch masks were first distinguished by the aspect ratio of the bounding boxes.The overlap and pole constraints were then analyzed for the adjacent masks that belonged to the same branch.The masks with similar constraints were finally merged and joined in the images.The pruning point of the lateral branch was only positioned at one of the two ends of the lateral branch.The lateral pruning point identification was proposed with the help of the main branch,in order to determine the coordinate of the lateral pruning point.The range near the main branch was first determined.And then the branch pruning end was determined by estimating which one of the lateral branch left and right endpoints was in the range.The center point close to the endpoint of the pruning end was finally determined as the pruning point of the lateral branch.The original and improved Mask R-CNN were also compared to verify the detection performance of the lateral and main branches.The recall rate and precision of the original Mask R-CNN were 87.9%and 93.3%,respectively,whereas,the recall rate and precision of the improved Mask R-CNN were 91.2%and 88.6%,respectively.The number of backbone network parameters in the improved Mask R-CNN was only 21.1%of that in the original one.The average segmentation time of the improved Mask R-CNN decreased by 0.038 s than before.The results showed that the backbone network of MobileNetv3-Large reduced the model parameters with the high speed in the improved Mask R-CNN.More branches were recognized,particularly when adding the Efficient Channel Attention mechanism into the feature map C3 and C4.Lateral and main branches that were divided into multiple masks were selected randomly to verify the performance of merging masks.The merging success rate of lateral branch masks was lower than that of the main branch masks,due to the more outstanding curved shape of the lateral branch.The average success rate of merging masks was 86.2%,indicating the excellent performance of merging masks.The presence of multiple pruning points was effectively reduced,where the single branch was normally taken as the multiple masks.Some images were selected randomly in the test set to verify the recognition accuracy for the pruning point of the lateral branch.The result showed that the recognition success rate on sunny days was higher than that on cloudy.The average recognition success rate was 82.9%,which fully met the requirements of lateral branch pruning.This finding can provide the technical support for the tomato branch and leaf pruning automatically.