基于手势识别的工业机器人操作控制方法

Operation control method for industrial robots based on hand gesture recognition

针对目前操作工人与工业机器人之间的交互还是采用比较机械化的交互方式,设计使用Kinect传感器作为手势采集设备,并使用人的手势来对工业机器人进行控制的方法.首先,使用深度阈值法与手部骨骼点相结合的方法,从Kinect传感器获取的数据中准确地提取出手部图像.在提取过程中,操作员无需佩戴任何设备,对操作员所站位置没有要求,对背景环境也没要求.然后,用稀疏自编码网络与Softmax分类器结合的方法对手势图像进行识别,手势识别过程包含预训练和微调,预训练是用逐层贪婪训练法依次训练网络的每一层,微调是将整个神经网络看成一个整体微调整个网络的参数,手势识别的准确率达到99.846%.最后,在自主研发的工业机器人仿真平台上进行实验,在单手和双手手势下都取得了不错的效果,实验结果验证了手势控制工业机器人的可行性和可用性.

The human-computer interaction modes between operators and industrial robots are rather mechanized currently. In order to solve the problem, a hand gesture control method by using Kinect sensor as a hand gesture acquisition equipment to control industrial robots was proposed. Firstly, the method of combining depth threshold algorithm and hand bones points was used to extract the hand gesture images accurately from the data obtained by a Kinect infrared camera. In the process of extraction, the operator did not need to wear any equipment, while it had no requirements to operator location and background environment. Then the method of combining deep autoencoder network and Softmax classifier was used for hand gesture image recognition. The hand gesture recognition included pretraining and fine tuning. The greedy layerwise approach was leveraged to train each layer of network in turn in pretraining, while all layers of the neural network were treated as a whole to fine tune the parameters of the entire network in fine tuning. The hand gesture recognition accuracy was up to 99. 846%. Finally, the experiments were conducted on self-developed industrial robot simulation platform, the good results had been achieved in one hand and both hands gestures. The experimental results show that the proposed method by using hand gesture to control the industrial robot is feasible and available.