基于声学音素向量和孪生网络的二语者发音偏误确认

Non-native Mispronunciation Verification Using Acoustic Tonal Phone Embedding and Siamese Networks

随着自动大规模语音识别的不断发展,以自动语音识别为基础的计算机辅助发音教学也随之进步,作为传统教学方法的补充,它极大地弥补了传统教育资源不足以及传统教育方法无法及时给学习者反馈的缺陷。二语学习者的发音偏误确认和评价在计算机辅助发音训练中是较为重要的研究课题之一。针对二语者发音偏误的确认任务中缺少二语偏误发音标注问题,该文提出了一种基于声学音素向量和孪生网络的方法,将带有配对信息的成对的语音特征作为系统输入,通过神经网络将语音特征映射到高层表示,期望将不同的音素区分开。训练过程引入了孪生网络,依照输出的两个音素向量是否来自于同一类音素来调整和优化输出向量之间的距离,并通过相应的损失函数实现优化过程。结果表明使用基于余弦最大间隔距离损失函数的孪生网络获得了89.93%的准确率,优于实验中其它方法。此方法应用在发音偏误确认任务时,不使用标注的二语发音偏误数据训练的情况下,也获得了89.19%的诊断正确率。

With the continuous development of automatic speech recognition,the pronunciation errors verification and evaluation of second language(L2)learners has become one of the most important research topics in computer assisted pronunciation training.To deal with the lack of labeled mispronunciation speech data,a method based on acoustic phone embedding and Siamese network is proposed in this paper.A pair of acoustic phone segments with a pair-wise label is used as a system input,and speech features are mapped to high level representation through neural network to differentiate different types of phones.The Siamese network is optimized by tell whether two output embeddings are from same type of phones or not.Results show that accuracy of Siamese network based on cosine hinge loss function achieves the best accuracy of 89.93%,and accuracy of diagnosis is 89.19%in pronunciation error verification task.