基于多模态信息的机器人视觉识别与定位研究

Visual identification and location algorithm for robot based on the multimodal information

针对目前物体识别定位算法中,图像信息来源单一、处理过程复杂与定位误差大等问题,提出一种基于多模态信息的视觉识别与定位方法,通过提取二维图像和点云图像的多模态信息,实现对物体的识别与定位。先利用彩色相机获取目标的二维图像信息,通过轮廓检测与匹配处理进行轮廓识别,再提取图像SIFT特征进行定位跟踪,得到物体的位置信息;同时采用RGB-D相机获取目标的三维点云图像信息,经过预处理、欧式聚类分割、VFH特征计算、KD树搜索得到最佳模板,进行点云图像的识别,并经点云聚类配准获得物体方向信息。最后,利用上述二维图像和点云图像处理所得物体信息,完成对目标的识别与定位。通过机器臂抓取实验对本文方法的效果进行了验证,结果表明,采用二维图像和点云图像的多模态信息进行处理,能够有效对不同形状的目标物体进行识别与定位,与仅采用二维或点云单模态图像信息的处理方法相比,定位误差可减小54.8%,方向误差减少50.8%,具有较好的鲁棒性和准确性。

To overcome the problem of a single image source, complex processing and inaccurate positioning, a visual identification and location algorithm based on multi-modal information is proposed, and the fusion processing is performed by extracting the multimodal information of the two-dimensional image and the point cloud image to realize object recognition and positioning. Firstly the target 2 D image information is obtained by RGB camera. The contour is recognized through the contour detection and matching process. Then the image SIFT feature is extracted for location tracking and the position of the object is obtained. Meanwhile obtaining a point cloud image by RGB-D camera and the best model can be sorted through pre-processing, Euclidean cluster segmentation, computing VFH feature and KD-tree searching, identifying the point cloud image. Then the orientation is obtained by registering the point clouds. Finally, the two-dimensional images and point cloud image are used to process object information, complete the identification and positioning of the target. The effect of the method is verified by the robotic gripping experiment. The result shows that the multi-modal information of two-dimensional image and point cloud image can be used to identify and locate different target objects. Compared with the processing method using only two-dimensional or point cloud single-mode image information, the positioning error can be reduced to 50%, the robustness and accuracy are better.