基于手语视觉单词特征的手语字母识别研究

Study of Sign Language Alphabet Recognition Based on Sign Language Visual Word Features

为有效识别手语字母,提出一种手语视觉单词(SLVW)的识别方法。采用Kinect获取手语字母视频及其深度信息,在深度图像中,通过计算获得手语手势的主轴方向角和质心位置以调整搜索窗口,利用基于深度图像信息的DI_CamShift方法对手势进行跟踪,进而使用基于深度积分图像的Ostu方法分割手势,并提取其尺度不变特征变换数据。将局部特征描述子表示的图像小区域量化生成SLVW,统计一幅手语图像中的视觉单词频率,用词包模型表示手语字母,并用支持向量机进行识别。实验结果表明,该方法不受颜色、光照和阴影的干扰,具有较高的识别准确性和鲁棒性,对复杂背景手语视频中的30个手语字母的平均识别率达到96.21%。

In order to effectively recognize the sign language alphabet, this paper presents an algorithm based on Sign Language Visual Word（SLVW）. It uses Kinect to obtain the video and depth image information of sign language gestures, calculates spindle direction angle and mass center position of the depth image to adjust the search window and for gesture tracking which depends on depth image information DI_CamShift. An Ostu method based on depth integral image is used to gesture segmentation, and the Scale Invariant Feature Transform（SIFT） data are extracted. It generates SLVW from small regions represented by local feature descriptors. After counting the frequency of visual words in a sign language alphabet image, it builds Bag of Words（BoW） to describe manual alphabets and uses Support Vector Machine（SVM） for recognition. Experimental results show that this method has high recognition accuracy and good robustness. Meanwhile, all of color, light and shadow have no effect on it. The average recognition rate of 30 sign language alphabets in the sign language video under complex background is 96.21%.