COMPARISON OFEMA-SYNCHRONIZED AND STAND-ALONE SPEECH BASED ON SPEECH RECOGNITION

基于语音识别的EMA同步语音与独立语音的比较研究

Synchronized acoustic-articulatory data is the basis of various applications,such as exploring the fundamental mechanisms of speech production,acoustic to articulatory inversion(AAI),and articulatory to acoustic mapping(AAM).Numerous studies have been conducted based on the synchronized ElectroMagnetic Articulograhy(EMA)data and acoustic data.Hence,it is necessary to make clear whether the EMA-synchronized speech and stand-alone speech are different,and if so,how it affects the performance of the applications that are based on synchronized acoustic-articulatory data.In this study,we compare the differences between EMA-synchronized speech and stand-alone speech from the aspect of speech recognition based on the data of a male speaker.It is found that:i)the general error rate of EMA-synchronized speech is much higher than that of stand-alone speech;ii)apical vowels and apical/blade consonants are more significantly affected by the presence of EMA coils;iii)parts of vowel and consonant tokens are confused with the sounds who use the same articulator or the articulators nearby,such as confusion among apical vowels and confusion among apical and blade consonants;iv)the confusion of labial tokens demonstrates a diverse pattern.

同步声学—发音信号同步记录了语音声学信号和对应的发音器官的位置和形状。因此,同步声学—发音信号是语音的生理发音机制、声学机制、发音逆推、基于发音数据的语音合成等领域研究的基石。时至今日,有多种不同的设备被用于采集同步的发音和声学数据,如超声、x-ray微束、EMA、以及实时MRI等。在这些设备中,EMA能同时记录多个发音器官的运动,具有高时间分辨率,采集的数据便于后期处理和分析,并且对被试无害,因而得到了广泛的应用。通常在使用EMA采集同步声学—发音信号时,我们将传感器粘贴在发音器官上(如上下唇、下颚、舌尖、舌叶、舌背等位置),这些传感器通常占据一定的空间(2.2mm×2.4mm×1.8mm)并且通过线缆与外部采集模块相连。在发音过程中,传感器和线缆可能会影响发音时声道收紧部位的形状和对发音的精准控制,导致产生的EMA同步语音与独立的语音在声学上存在差异,对语音的生理发音机制、声学机制、发音逆推、基于发音数据的语音合成等领域的研究产生不利影响。这一问题还没有得到广泛关注,目前只有少量的研究从感知的角度比较了EMA同步语音的发音正确率和可懂度,还没有研究考察感知正确的EMA同步语音与独立语音是否有差异,以及这些差异是否会影响其他任务。本文从语音识别的角度,通过对比分析EMA同步语音与独立语音的识别率和混淆矩阵我们发现:1)感知正确的EMA同步语音的正确率显著低于独立语音的正确率;2)舌尖元音和舌尖/舌叶辅音更容易受到传感器的影响,其他元音和辅音也受到不同程度的影响,这与人们通常的认知不同;3)在EMA同步语音中,元音和辅音容易与使用相同或相近发音器官的元音和辅音混淆;4)唇辅音的混淆模式有些发散,如/p/、/p^(h)/除部分被混淆成唇音外还被混淆成舌根擦音/x/,不少/f/被识别生成/k/,/k^(h)/,/x/。这些发现说明由于在发音器官上粘贴了传感器,除舌尖音外,其他发音的准确性和发音方式都受到一定程度的影响。