面向智慧博物馆的基于毫米波雷达稳健的手语识别

mmWave radar based robust sign language recognition for the smart museum

智慧博物馆是利用物联网、人工智能等设备或技术,构建人、物、空间信息交互通道的博物馆新形态。手语识别技术既能让听障语障观众无障碍参观博物馆,也有助于解析观众自然状态下的手势互动。然而,基于摄像头或可穿戴设备的方法在博物馆中可能有隐私安全或使用不便等问题。提出一种基于毫米波雷达稳健的手语识别方法,首先提取不同手势相对于雷达距离和速度随时间变化的特征,其次采用基于物理意义的增强处理,最后设计残差网络进一步剔除两种特征预处理后的与环境相关信息,对其进行特征融合并实现分类。实验表明,该方法可以有效识别手语,在测试环境和用户位置改变时也能达到平均90%以上的精度,为智慧博物馆的手语手势识别提供了一种新方法。

A smart museum is a new form of a museum,which uses devices or technologies including the Internet of things(IoT)and artificial intelligence(AI)to build the information interaction channels between people,things,and space.Sign language recognition not only assists the visitors who have hearing or speech impairment to visit the mu-seum without barriers but also helps study the visitors’natural gesture interaction.However,the methods based on cameras and wearable devices mayhave issues like privacy or usability when applied to museum spaces.Therefore,a robust sign language recognition method based on millimeter-wave radar was proposed.Different features of distance and velocity changes relative to the radar device were firstly extracted in this method,then a physical data enhance-ment method was adopted to expand the training data.Finally,a ResNet based on the pre-processed distance time features and Doppler time features was designed to further remove the environment-related information and perform feature fusion for classification.Experimental results show that this method can effectively recognize sign language and achieve an averaged recognition accuracy of over 90%when the testing environment and the user's location change,providing a new method for smart museum wireless sign language and gesture recognition.