面向人机交互的手势指向估计方法

Estimation of gesture pointing for human-robot interaction

针对人机共融环境中机器人与人之间的交互问题,提出了一种面向人机交互场景的手势指向估计方法,通过人体指向手势,以实现机器人对工作平面上指向目标点的信息交互。首先,基于RGB-D相机与VICON人体动作捕捉系统,构建时间同步的视觉指向手势位姿数据集,其中的每个样本包含人体指向手势的RGB-D图像和指向手势的位姿真值;其次,提出融合语义与几何信息的指向手势位姿估计多层次神经网络模型;然后,设计融合位置点误差ΔP和方向角度误差Δθ的射线近似损失函数,并基于构建的数据集,对指向手势位姿估计模型进行训练;最后,在实验室环境中进行了人机交互实验与模型验证。实验结果表明,在距离相机5 m的范围内,指向手势检测的平均精度为98.4%,指向手势位姿的平均位置误差为34 mm,平均角度误差为9.94°,进而实现工作平面上的手势指向目标点的平均误差为0.211 m。

To solve the problem of interaction between robot and human in the human-robot integration environment,this article proposes an estimation method of gesture pointing for human-robot interaction scenes by pointing gesture to achieve the information interaction between the robot and the target point on the workplane.First,based on the RGB-D camera and the human motion capture system VICON,a time-synchronized visual pointing gesture position dataset is established.Each sample contains the RGB-D image of the pointing gesture and the true value of the pointing gesture pose.Secondly,a multi-level neural network model is formulated for estimating pointing gesture pose by combining semantic and geometric information.Thirdly,a ray approximation loss function is designed,which combines the position erroraP and direction angle error Ao.The pointing gesture pose estimation model is trained based on the constructed dataset.Finally,human-robot interaction experiments and model validation are implemented in the laboratory environment.In the range of 5 m from the camera,results show that the average precision of pointing gesture detection is 98.4%,the average position error of pointing gesture pose is 34 mm,and the average angle error is 9.94.The average error of gesture pointing to the target point on the workplane is 0.21l m.