基于特定韵母发音事件分析的语音唇动一致性判决方法

Lip Motion and Voice Consistency Recognition Method Based on Specific Vowel Pronunciation Events Analysis

针对现有一致性判决方法主要对整句(段)话进行分析,并无对分析内容加以筛选,存在运算繁琐及结果易受静音等弱关联片段影响等不足,以唇型变化显著的韵母发音单元为研究重心,通过分析聚类后各韵母类别的音唇关联度,选出更具代表性的特定韵母单元并结合位置时延分析,提出了基于特定韵母发音事件分析的音唇一致性判决方法。该方法先分割并识别出特定韵母单元;然后求出以上各韵母发音事件的音唇相关度,并对特定韵母出现位置的时延分布进行分析;最后融合特定韵母事件的音唇相关度得分与位置时延分析评分进行一致性判决。通过实验对该方法与其他方法进行了对比,结果表明,该算法在识别性能上优于多种整句分析的比较算法,同时也相应降低了运算量。

The traditional lip motion and voice consistency recognition method is to analyze the whole sentence without filtering the content,which is complicate in computation and its results are vulnerable to weak related segments such as mute.The vowels which with significant lip shape changes were researched in depth.By analyzing the audio and lip motion correlation of each vowel category clustered by lip sequence features,a more representative specific phonological pronunciation unit was selected as the analysis object.Combined with audio-visual delay analysis,a consistent recognition method based on specific vowel pronunciation events analysis was proposed.Firstly,the selected unit was segmented and identified.Then the correlation degree of each specific vowel event was obtained,and the delay distribution of each specific vowel occurrence position was statistically scored.Finally,a consistency judgment was made by combining the vowel pronunciation event audio-visual correlation score with the position delay analysis score.Compared with other methods through experiments,results show that the proposed method is superior in recognition performance and reduces the amount of computation.