多类复杂水果目标的通用三维感知框架

General 3D Perception Framework for Multiple Types Complex Fruit Objects

相比于理想的实验室环境,在户外果园采样的水果图像受更多复杂因素的干扰,如背景混杂、枝叶遮挡、光照不均和机械振动等,是制约视觉水果采摘机器人进一步发展和应用的关键因素。为此,提出一种适应于复杂采摘环境的通用三维感知框架。首先,构建并标定双目视觉系统;然后,借助深度目标检测网络去除复杂背景并获得水果所在的图像区域;最后,对水果区域进行立体匹配和三角测量,得到水果的三维点云与空间位置。该框架不依赖任何人工设计,可提高机器人在户外感知水果的准确度和稳定性,也可在不同的采摘任务中集成,为提高视觉水果采摘机器人感知的实用性提供有效的理论依据与技术支持。

Compared with an ideal laboratory environment,fruit images sampled in an outdoor orchard are often disturbed by more noises,such as complex backgrounds,occlusion of branches and leaves,uneven illumination and mechanical vibration,etc.,which have always been the main constraints on the further development of visual fruit picking robots.In response to this problem,a general three-dimensional perception framework adapted to the complex picking environment was developed.First,the binocular vision system was constructed and calibrated;then,the complex background was removed with the help of a robust deep object detection network to obtain the image area where the fruit is located;finally,the fruit area was conducted stereo matching operation and triangulated to obtain the spatial position of the fruits.The framework has fully combined the robustness of the deep neural network and the excellent perception ability of the stereo vision system.It does not rely on any artificially designed features and can improve the accuracy and stability of the robot's perception of outdoor fruits.This research provides an theoretical basis and technical support for improving the stability and practicability of the fruit picking robot.