基于合成数据集的多目标识别与6-DoF位姿估计

Multi-Object Recognition and 6-DoF Pose Estimation Based on Synthetic Datasets

多目标识别及六自由度(6-DoF)位姿估计是实现物料无序堆放状态下机器人自动分拣的关键。近年来,基于深度神经网络的方法在目标识别及位姿估计领域受到广泛关注,但此类方法依赖大量训练样本,而样本的采集及标注费时费力,限制了其实用性。其次,当成像条件差、目标相互遮挡时,现有位姿估计方法无法保证结果的可靠性,进而导致抓取失败。为此,文中提出了一种基于合成数据样本的目标识别、分割及位姿估计方法。首先,以目标对象的3维(3D)几何模型为基础,利用3D图形编程工具生成虚拟场景的多视角RGB-D合成图像,并对生成的RGB图像及深度图像分别进行风格迁移和噪声增强,从而提高合成数据的真实感,以适应真实场景的检测需要;接着,利用合成数据集训练YOLOv7-mask实例分割模型,运用真实数据进行测试,结果验证了该方法的有效性;然后,以分割结果为基础,基于ES6D目标位姿估计模型,提出了一种在线姿态评估方法,以自动滤除严重失真的估计结果;最后,采用基于主动视觉的位姿估计校正策略,引导机械臂运动到新的视角重新检测,以解决因遮挡而导致位姿估计偏差的问题。在自行搭建的6自由度工业机器人视觉分拣系统上进行了实验,结果表明,文中提出的方法能较好地适应复杂环境下工件的识别与6-DoF姿态估计要求。

Multi-object recognition and 6-DoF(degree of freedom)pose estimation are the key to achieve automatic sorting of robots in the state of unordered stacking of materials.In recent years,methods based on deep neural networks have received much attention in the multi-object recognition and 6-DoF pose estimation fields.Such methods rely on a large number of training samples,however,the collection and labeling of samples is time-consuming and laborious,which limits its application.In addition,when the imaging conditions are poor and the targets are occluded by each other,the existing pose estimation methods cannot guarantee the reliability of the results,resulting in grasping failures.To this end,this paper presented a method for target recognition,segmentation and pose estimation based on synthetic data samples.Firstly,multi-view RGB-D synthetic images of virtual scenes were generated using 3D graphics programming tools based on the 3D geometric models of the target objects,and then style transfer and noise enhancement was performed,respectively,on the generated RGB images and the depth images to improve their realism,so that they are suited for the detection in real scenes.Next,the YOLOv7-mask instance segmentation model was trained with synthetic datasets and tested by real data.The results demonstrate the effectiveness of the proposed method.Secondly,the ES6D model was utilized to estimate target poses based on the segmentation results,and an online posture evaluation method was proposed to automatically filter out severely distorted estimation results.Finally,a pose estimation correction strategy based on active vision technique was proposed to guide the robot arm to move to a new viewpoint for re-detection,which can effectively solve the problem of pose estimation deviation caused by occlusion.The above methods have been verified on a self-built 6-DoF industrial robot vision sorting system.The experimental results show that the proposed algorithm can well meet the requirements of recognition and 6-DoF posture estimation of common workpieces in complex environments.