基于深度迁移学习的钢琴演奏手势识别技术研究

Research on Piano Performance Gesture Recognition Technology Based on Deep Transfer Learning

应用传感器实现钢琴教学时,为随时获取钢琴演奏者的演奏手势是否规范,需要合理的钢琴演奏手势识别技术,为此提出基于深度迁移学习的钢琴演奏手势识别技术研究。采用IU-EKF算法实现钢琴演奏手势的定姿,获取演奏者的演奏手势姿态,将该演奏手势信息作为数据样本,利用MEMS惯性传感器采集钢琴演奏手势姿态数据,并通过状态空间模型做出手势姿态估计。以该模型为基础,利用多特征提取方法,获取手势特征,并对不同特征作出归一化处理,将处理后的结果输入到极限学习机(VGG-16)网络模型中,通过该模型的深度迁移学习与训练,实现钢琴演奏手势的识别。经实验验证:该方法能有效提取演奏者手背、手指下关节、手指上关节的各角度特征,且相较于其他方法该方法具有较高的识别精度,能够在不同的时间有效识别手指上、下关节俯仰角的变化情况。

When applying sensors to realize piano teaching,a reasonable gesture recognition technology for piano performance is needed to obtain whether the player’s performance gestures are standardized at any time.Therefore,a research on gesture recognition technology for piano performance based on deep transfer learning is proposed.The IU-EKF algorithm is used to realize the posture of piano gestures,and the gesture posture of the player is obtained.The performance gesture information is taken as the data sample,and the MEMS inertial sensor is used to collect the gesture posture data of piano performance,and the gesture posture estimation is made through the state space model.Based on the model,the multi-feature extraction method is used to obtain gesture features,and the different features are normalized.The processed results are input into the extreme learning machine(VGG-16)network model,and the recognition of piano playing gestures is realized through the deep transfer learning and training of the model.The experimental results show that the proposed method can effectively extract the angle features of the dorsal hand,lower finger joint and upper finger joint of the performer.Compared with other methods,the proposed method has higher recognition accuracy and can effectively identify the changes of the pitch angle of the upper and lower finger joints at different times.