基于迁移学习的跨对象手语手势识别方法

Transfer Learning Based Cross-object Sign Language Gesture Recognition Method

手语是听障人士重要的交流工具,准确识别手语可以减少健全人和听障人士之间的交流障碍。一般深度学习识别模型的性能高度依赖于所采集的数据,这导致模型跨对象泛化能力较差。因此,通过迁移学习的方法设计一种具有跨对象泛化能力的手语手势识别模型。首先,使用特征提取器融合表面肌电流(Surface Electromyography,sEMG)信号和惯性传感器(Inertial Measurement Unit,IMU)信号。然后,提出一种域对抗训练方法,其可以仅依靠源域数据完成特征提取器和域分类器的对抗训练,实现特征提取从源域到目标域的迁移。最后,在手势分类器中利用域不变特征实现手语手势跨对象识别,提高了模型的泛化能力。实验表明,在包含200种手语手势共60000条手语样本数据集上,所提模型可将手语跨对象识别准确率提高到85.1%。

Sign language is an important communication tool for hearing impaired people,and accurate recognition of sign language can reduce the communication barrier between able-bodied and hearing impaired people.The performance of general deep learning recognition models is highly dependent on the collected data,which leads to poor cross-object generalization ability of the models.Therefore,this paper designs a sign language gesture recognition model with cross-object generalization capability through a transfer learning approach.Firstly,a feature extractor is used to fuse the surface electromyography(sEMG)signal and the inertial measurement unit(IMU)signal.Then,a domain adversarial training method is proposed,which can complete the adversarial training of the feature extractor and domain classifier by relying on the source domain data only,and realize the migration of feature extraction from the source domain to the target domain.Finally,domain-invariant features are used in the gesture classifier to achieve sign language gesture cross-object recognition,which improves the generalization ability of the model in this paper.Experiments show that the proposed model can improve the accuracy of sign language cross-object recognition to 85.1%on a dataset containing 200 sign language gestures with a total of 60000 sign language samples.