基于点云轨迹和压缩多普勒的跨场景手势识别

Cross-scene Gesture Recognition Based on Point Cloud Trajectories and Compressed Doppler

毫米波雷达能够用于各种感知任务,如活动识别、手势识别、心率感知等。手势识别作为其中的研究热点,可实现无接触人机交互。目前大多数手势识别研究使用点云或距离多普勒图通过神经网络进行识别感知,但是这些方法存在一些问题。首先,这些方法鲁棒性较差,被感知人员或其位置发生变化都会影响接收到的毫米波信号,降低感知精度。其次,这些方法将完整的距离多普勒图输入神经网络进行识别,由于图中存在较多与感知任务无关的区域,模型复杂且难以专注于感知任务。为解决这些问题,首先从连续多帧点云数据中建立手势轨迹,然后将连续多帧距离多普勒图进行局部切割并压缩获得二维局部多普勒图,最后将点云轨迹和二维局部多普勒图分别经过神经网络特征提取后,对特征进行拼接,通过全连接神经网络进行分类。实验结果表明,所提方法专注于手势,能够达到98%的识别准确率,在人员变化和位置变化情况下对新用户和在新位置的识别准确率分别能够达到93%和92%,高于现有方法。

Millimeter wave radar can be used for various sensing tasks,such as activity recognition,gesture recognition,heart rate perception.Among them,gesture recognition is a research hotspot,which can realize contactless human-computer interaction.Most existing studies on gesture recognition make use of point cloud or range-Doppler for pattern recognition through neural networks to achieve sensing.However,there are some problems.Firstly,the robustness of these methods is poor.The changes of the user and his/her location affect the received millimeter wave signals,causing the accuracy of the sensing model to reduce.Secondly,these methods input the complete range-Doppler map into the neural network,which makes the model complicated and makes it difficult for the model to focus on the sensing task,because there are many unrelated regions to the sensing task.To solve these problems,this paper first builds the gesture trajectory from multiple continuous frames of point cloud,and then cuts and compresses the multiple continuous range-Doppler maps to obtain the two-dimensional local Doppler map.Finally,the features are extracted from the point cloud trajectory and the two-dimensional local Doppler map respectively by the neural networks,concatenated and classified by a fully-connected neural network.Experiments show that the proposed method focuses on gestures and can achieve a recognition accuracy of 98%,and can achieve a recognition accuracy of 93%for new users and 92%for new locations in the cases of user changes and location changes,better than the state of the art.