融合毫米波雷达与机器视觉的田间道路视觉引导线提取

Fusion of millimeter wave radar and machine vision for visual guide line extraction on field roads

针对传统机器视觉算法难以获取准确视觉引导线的问题,该研究提出了一种利用深度学习算法获取视觉引导线的方法,并将毫米波雷达探测的目标运动状态信息考虑到视觉引导线的提取中,以安全避开障碍物。首先通过数据预处理方法过滤部分雷达目标数据,再利用多目标跟踪算法过滤干扰数据并对动态目标持续跟踪,为后续数据融合提供准确的雷达目标数据。然后制作田间道路环境数据集,搭建基于Deeplabv3 plus的改进语义分割网络。继而利用时间戳对齐和基于最小二乘法的坐标变换实现毫米波雷达数据与视觉数据在时间与空间维度的同步。最后基于无动态目标状态信息和有动态目标状态信息两种情况提出了相应的视觉引导线提取策略,进行相关试验测试了视觉引导线提取方法的效果。相比于人工测量的真实道路中点,在无动态目标场景下提取的视觉引导线的平均误差为1.60~9.20像素;而在有动态目标的场景下,能够成功获取避障的视觉引导线。研究结果有助于提升丘陵山区农业机械的智能化水平。

Field roads have been the most important transportation ways for commercial grain and agricultural products in hilly and mountainous regions in China.Especially,the youth labor can rapidly migrate from the rural to the urban areas against everincreasing urbanization.Current manual production cannot fully meet the large-scale and precision agriculture in recent years,owing to the protracted yield cycle and low rate of return.Therefore,it is urgent to implement the mechanization and intelligent agriculture for the national food security in hilly and mountainous regions.Among them,the autonomous and safe operation of intelligent machinery can be critical to navigate or circumvent obstacles on field roads.However,the conventional machine vision cannot accurately and rapidly construct the visual guidance lines in such terrains.In this study,the extraction was proposed to enhance the visual guidance and obstacle avoidance using millimeter-wave radar and vision fusion.The state information was also detected from the target objects on the field road.The specific steps were as follow.Preprocessing techniques were employed to filter the portion of the radar object data.A multi-target tracking was utilized to eliminate the interference data for the continuous tracking of dynamic objects.Accurate radar object data was then obtained for subsequent data fusion.A semantic segmentation network was created using Deeplabv3+,and then leveraged a dataset of the adjacent field roads.The millimeter-wave radar and vision data were synchronized in both time and space via the timestamp alignment and least squares-based coordinate transformation.Extraction approach was then established for the visual guidance lines,particularly for the scenarios where dynamic target state was both available and unavailable.A series of experiments were carried out to validate the extraction of visual guide line.The average errors of detection were ranged from 1.60 to 9.20 pixels at the real road midpoints in the scenes without dynamic targets.Moreover,the visual guidance lines were successfully extracted for the obstacle avoidance proactively when encountering dynamic targets.Evidently,the integrated approach was effectively overcome the constraints of traditional machine vision,thereby enhancing the safety and reliability of machinery operation in hilly and mountainous terrains.Meanwhile,the efficacy was depended mainly on the precise projection of millimeter-wave radar data onto the visual plane.Simultaneously,the variables were markedly introduced the positional discrepancies,such as the posture of platform(including pitch and roll).Furthermore,the driveability of the operational area was effectively evaluated in the forthcoming regions.At the same time,the real-world conditions were more complex in the frequent presence of common elements like livestock,vehicles,and tricycles.The resilience of approach can be expected to improve the radar data processing for the detection of static objects.Moreover,positional deviations can also be mitigated from mechanical movements.As such,the precise visual guidance lines were established in the hilly and mountainous terrains.Therefore,the radar data processing can be focused on the dataset expanding and position deviation that induced by various factors.The intelligent level of agricultural machinery can be improved in hilly and mountainous area.The finding is of great significance to consolidate national food security.