基于汉语视频三音素的可视语音合成

Visual Speech Synthesis Algorithm Based on Chinese Visual Triphone

为了合成具有真实感的视频序列,该文提出一种基于汉语视频三音素的可视语音合成方法。根据汉语的发音规律和音素与视素的对应关系,该文提出"视频三音素"的概念。在此基础上,建立隐马尔可夫(HMM)训练与合成模型,在训练过程中使用了视频音频联合特征,并加入了动态特征。在合成过程中,连接视频三音素HMM模型形成句子HMM,并从中提取特征参数,合成可视语音。从主观和客观评估结果来看,合成视频的真实感强,满意度较高。

In order to synthesize real video sequence, a visual speech synthesis algorithm based on Chinese visual triphone is proposed. According to Chinese pronunciation principle and the relationship between phoneme and viseme, conception of ‘visual triphone’ is presented. Hidden Markov Model（HMM） is established based on visual triphones. In the training stage, combined features including visual features and audio features are used. In the synthesis stage, sentence HMM is constructed by concatenating triphone HMMs, from which the feature parameters are extracted. From the result of subjective and objective evaluation, the synthesized video is real and satisfied.