基于加速计与表面肌电传感器信息融合的手语识别方法

A Method for Sign Language Recognition Based on Information Fusion of Accelerometer and Surface Electromyography Sensors

目的探索基于加速计和多通道表面肌电传感器信息融合方法用于手语识别的效果。方法根据多通道表面肌电信号的瞬时能量自动检测手语动作,采用多级决策树融合多传感器信息,实现对中国手语词的连续识别。结果在中国手语30类典型单手词汇和模拟真实情景对话的16个例句识别实验中,该方法获得了令人满意的识别效果,分别取得了96.7%和93.7%的平均词识别率。结论基于两类传感器信息融合的手语识别方法在大词汇量连续手语识别和智能人机交互领域具有很好的应用潜力。

Objective To investigate the feasibility of sign language recognition （SLR） based on the information fusion of accelerometer （ACC） and surface electromyography （EMG）. Methods A data segmentation algorithm was proposed firstly to automatically detect sign word segments within a sequence of performed signs according to the instantaneous energy stream of multi-channel surface electromyography （EMG）, and then a hierarchical decision tree was constructed for the information fusion of ACC and EMG signals to realize continuous recognition of Chinese Sign Language （CSL）. Results For the recognition of 30 typical single-hand CSL words and 16 kinds of dialog sentences, the average recognition accuracies of words by using our method reached up to 96.7% and 93.7% respectively. Conclusion The SLR method based on multi-sensor information fusion extends the potential applications in the fields of large-vocabulary continuous SLR and intelligent human-computer interaction.