摘要
应用传感器实现钢琴教学时,为随时获取钢琴演奏者的演奏手势是否规范,需要合理的钢琴演奏手势识别技术,为此提出基于深度迁移学习的钢琴演奏手势识别技术研究。采用IU-EKF算法实现钢琴演奏手势的定姿,获取演奏者的演奏手势姿态,将该演奏手势信息作为数据样本,利用MEMS惯性传感器采集钢琴演奏手势姿态数据,并通过状态空间模型做出手势姿态估计。以该模型为基础,利用多特征提取方法,获取手势特征,并对不同特征作出归一化处理,将处理后的结果输入到极限学习机(VGG-16)网络模型中,通过该模型的深度迁移学习与训练,实现钢琴演奏手势的识别。经实验验证:该方法能有效提取演奏者手背、手指下关节、手指上关节的各角度特征,且相较于其他方法该方法具有较高的识别精度,能够在不同的时间有效识别手指上、下关节俯仰角的变化情况。
When applying sensors to realize piano teaching,a reasonable gesture recognition technology for piano performance is needed to obtain whether the player’s performance gestures are standardized at any time.Therefore,a research on gesture recognition technology for piano performance based on deep transfer learning is proposed.The IU-EKF algorithm is used to realize the posture of piano gestures,and the gesture posture of the player is obtained.The performance gesture information is taken as the data sample,and the MEMS inertial sensor is used to collect the gesture posture data of piano performance,and the gesture posture estimation is made through the state space model.Based on the model,the multi-feature extraction method is used to obtain gesture features,and the different features are normalized.The processed results are input into the extreme learning machine(VGG-16)network model,and the recognition of piano playing gestures is realized through the deep transfer learning and training of the model.The experimental results show that the proposed method can effectively extract the angle features of the dorsal hand,lower finger joint and upper finger joint of the performer.Compared with other methods,the proposed method has higher recognition accuracy and can effectively identify the changes of the pitch angle of the upper and lower finger joints at different times.
作者
黄丹
HUANG Dan(Nanchang Hangkong University,Nanchang Fenghe 330063;Fuyang Preschool Teachers College,Fuyang 236000,China)
出处
《河北北方学院学报(自然科学版)》
2022年第9期1-7,13,共8页
Journal of Hebei North University:Natural Science Edition
基金
2019阜阳幼儿师范高等专科学校校级质量工程项目:“大规模在线开放课程(MOOC):儿童古筝教学”(ZLGC2019MOOC001)。
关键词
深度迁移学习
钢琴演奏手势
手势识别
VGG-16
惯性传感器
融合定姿
deep transfer learning
gesture for piano playing
gesture recognition
VGG-16
inertial sensor
fusion pose determination