基于语义分割的作物垄间导航路径识别

Navigation path recognition between crop ridges based on semantic segmentation

针对目前农作物垄间导航路径识别目前存在准确性、实时性差、通用性弱及深度学习模型解释困难等问题,该研究在Unet模型的基础上进行剪枝与优化,提出了保留Unet模型特征跳跃连接优势的Fast-Unet模型,并以模型所识别的导航路径为基础,通过最小二乘法回归生成垄间导航线与偏航角。该研究首先在棉花垄间导航路径数据集上进行模型训练,随后将训练的模型迁移至玉米、甘蔗等小样本数据集进行导航路径识别,通过使用梯度加权类激活映射法对模型识别过程与迁移学习过程进行解释,对各模型识别结果进行可视化对比。Fast-Unet模型对棉花、玉米、甘蔗导航路径提取精度指标平均交并比分别为0.791、0.881和0.940。模型推理速度为Unet的6.48倍,在单核CPU上处理RGB图像的推理速度为64.67帧/s,满足农作物导航路径识别的实时性需求。研究结果可为田间智能农业装备的导航设备研制提供技术与理论基础。

A navigation path has been widely considered as one of the most important sub-tasks of intelligent agricultural equipment in field operations.However,there are still some challenges remaining on the recognition of current navigation paths between crop ridges,including the accuracy,real-time performance,generalization,and difficulty in the interpretation of deep learning models.In this research,a new Fast-Unet model was proposed to accurately and rapidly recognize the navigation path between crop ridges using semantic segmentation.The jump connection of the Unet model was also retained to generate the navigation line and yaw angle using the least square regression.Specifically,a cotton dataset of inter-ridge navigation path consisted of 800 images,640 of which was set as the training set,160 of that as the validation set.Subsequently,two datasets of 100 images each were constructed for the navigation paths of sugarcane and cotton ridges,which were divided into 50 images in the training set,and 50 images in the verification set.The training strategy was selected as the data augmentation and learning rate adjustment.The training order was ranked as the corn first,and then the sugarcane dataset.The Mean Intersection over Union(MIoU)was utilized as the accuracy indicator of the Fast-Unet model,which was 0.791 for cotton,0.881 for maize,and 0.940 for sugarcane.Furthermore,the least-squares regression was selected to calculate the navigation path of maize and sugarcane with good linearity between the ridges.Additionally,the navigation line was selected to further calculate the yaw angle.The mean difference between the predicted yaw angle of maize and sugarcane navigation path and the labeled were 0.999°and 0.376°under the Fast-Unet model,respectively.In terms of real-time performance,the inference speed of the Fast-Unet model was 6.48 times higher than that of Unet.The inference speed was 64.67 frames per second to process the RGB image data on a single-core CPU,while the number of parameters of the Fast-Unet model was 6.24%of that of Unet model.Correspondingly,the computing devices were deployed with weak computing power,thereby performing real-time calculations.A gradient weighted class activation mapping(Grad-CAM)was also used to visually represent the final feature extraction of model recognition and transfer learning.More importantly,the special features were highlighted on the navigation path between crop ridges in the optimized Fast-Unet structure,concurrently to remove a large number of redundant feature maps,while retaining only the most crucial feature extractors.The transfer learning also presented a larger activation area than the direct training,where the activated area matched the main road to be identified.In summary,the improved model can be fully realized the real-time recognition of maize navigation path.The finding can also provide technical and theoretical support to the development of navigation equipment for intelligent agricultural machinery in the field.