基于多深度相机融合的机械臂抓取系统

Manipulator grasping system based on multi depth camera fusion

针对单深度相机视角下获取的物体深度图像存在噪声干扰、遮挡等问题,使物体点云形状缺失或变形,3D特征存在不稳定性,提出一种多深度相机融合方法。利用多相机标定结果拼接各个相机视角下的点云数据,通过k邻域去噪算法除去点云中的噪声点,再对去噪点云数据进行体素化降采样,使拼接后的点云在空间均匀分布,从而获取完整的物体形状。同时建立了一个端到端的神经网络,通过输入多相机拍摄的RGB-D图像来预测二指平行夹爪的3D位置和3D方向。为优化预测抓取结果,改进了神经网络中对预测结果的鲁棒性评估,提出力螺旋空间场评价方法,提高了4%的成功率。最后将该抓取系统应用于AR5机械臂,实现了91%的抓取成功率。

To solve the problems of noise interference and occlusion in the object depth image obtained from the perspective of single depth camera,the shape of object point cloud is missing or deformed,and the three-dimensional features are unstable.Therefore,a multi depth camera fusion method was proposed.Using the results of multi-camera calibration,the point cloud data from various camera perspectives were stitched together.Noise points in the point cloud were removed through k neighborhood denoising algorithm.Subsequently,the denoised point cloud data underwent voxelization and subsampling methods to ensure a uniform spatial distribution of the stitched point cloud,thereby obtaining a complete representation of the object's shape.Simultaneously,an end-to-end neural network was established to predict the 3D position and 3D orientation of a two-finger parallel gripper based on RGB-D images captured by multiple cameras.To optimize the predicted grasping results,the robustness evaluation of the prediction results in neural network was improved,and the evaluation method of force spiral space field was proposed,which improved the success rate by 4%.Using AR5 manipulator in the actual grasping system,the grasping success rate was 91%.