基于FMCW雷达的双流融合神经网络手势识别方法

Two-Stream Fusion Neural Network Approach for Hand Gesture Recognition Based on FMCW Radar

针对传统光学摄像头和无线技术的手势识别方法受光照环境影响和空间纵向、横向特征不全的问题,该文提出一种基于调频连续波(Frequency Modulated Continuous Wave,FMCW)雷达信号的双流融合神经网络(Two-Stream Fusion Neural Network,TS-FNN)手势识别方法.首先,利用二维快速傅立叶变换(Fast Fourier Transform,FFT)求取中频信号的频谱,估计手势的距离和速度,并利用多重信号分类(Multiple Signal Classification,MUSIC)方法计算角度.其次,利用这三维参数在时间上的累积,将一个手势动作映射为32帧距离-速度矩阵图和角度时间图.最后,建立TS-FNN进行手势特征提取和特征融合.实验结果表明,TS-FNN方法与传统卷积神经网络相比,手势的平均识别准确率提升了约5%.

To deal with the problem of easily being affected by illumination environment of the traditional optical camera based hand gesture recognition method and the incomplete spatial and lateral characteristics of the wireless based hand gesture recognition method,this paper proposes a frequency modulated continuous wave (FMCW) radar signal based two-stream fusion neural network (TS-FNN) for hand gesture recognition.Firstly,the spectrum of the IF signal is obtained by two-dimensional Fast Fourier Transform (2D-FFT),the range and speed of the gesture are estimated,and the angle is calculated by the Multiple Signal Classification (MUSIC) method.Secondly,using the accumulation of three-dimensional parameters in time,a gesture action is mapped to a 32-frame range-speed matrix diagram and an angular-time map.Finally,TS-FNN is established for gesture feature extraction and classification.The experimental results show that compared with the existing methods,the TS-FNN method improves the average recognition accuracy by about 5%.