基于视觉推理的机器人多物体堆叠场景抓取方法

Robotic grasping in multi-object stacking scenes based on visual reasoning

基于视觉的机器人抓取方法是研究智能机器人抓取问题的重要思路.本文提出了一种基于机器视觉推理的适用于多物体堆叠场景的机器人抓取方法,算法包含了场景理解和抓取规划两个步骤.在第一步的场景理解中,本文的算法包含两个主要部分:视觉操作关系推理和抓取部位检测.在视觉操作关系推理中,本文提出了一种基于深度卷积网络的视觉操作关系网络(Visual Manipulation Relationship Network, VMRN),以对物体和操作关系进行实时推理.在视觉操作关系网络中,通过设计物体对池化层,实现了物体检测和操作关系推理的端对端训练,提升了算法的速度和性能.在第二步的抓取部位检测中,本文提出了基于有向锚点框的全卷积视觉抓取部位检测网络,实现了对物体抓取部位的实时检测,并在康奈尔抓取数据集上取得了目前最高的精度.在抓取规划中,通过结合场景深度信息和抓取部位检测结果,获取当前被抓取物体的抓取点和对应的抓取向量,并通过坐标系变换将Kinect坐标系的抓取向量映射为机器人坐标系的抓取向量,完成当前抓取.实验结果表明,本文提出的机器人抓取方法能够在多物体堆叠环境下按照正确顺序完成抓取任务,并成功抓取目标物体.

Vision is one of the most important ways to solve robotic grasping problem. We propose a framework that can help the robot grasp target object in multi-object scenes based on visual reasoning, which includes two stages: Perception and execution. During percetion stage, our proposed framework includes two parts: Visual manipulation relationship reasoning and robotic grasp detection. In visual manipulation relationship reasoning, Visual Manipulation Relationship Network(VMRN) is proposed to simultainously detect objects and gets manipulation relationships between each pair of objects. We design Object Pairing Pooling Layer to implement endto-end training of object detection and visual manilupation relationship reasoning in VMRN, which makes the algorithm faster and more robust. In robotic grasp detection, fully convolutional grasp detection network based on oriented anchor box is proposed to implement real-time grasp detection for any object, which is the state-of-the-art robotic grasp detection algorithm on the standard Cornell Grasp Dataset. During execution stage, by combining depth information and perception results, grasp point and grasp vector in camera coordinate are computed first. Then, they are transformed into robot coordinate for robot to execute grasping motion.Experimental results show that our framework can help robot grasp target in multi-object scene in the right order.