视觉驱动的语音合成系统中唇形轮廓的傅里叶描述

Lip contour description based on Fourier descriptors in speech synthesis system driven by visual-speech

为了能够自动、快速地表示唇读系统中所必须的唇形轮廓特征,将傅里叶描述子用于唇形轮廓的描述和识别过程中,采用边界傅里叶变换的方法,得到非对称唇形模型中唇形轮廓的傅里叶描述子,用来刻画唇动过程中唇形轮廓的形状信息,并将傅里叶描述子φ作为唇形轮廓的特征向量,应用于基于隐马尔可夫模型(HMM)的视觉驱动语音合成系统。基于独立汉字发音的实验表明,单纯采用前15或20个傅里叶描述子就能够有效地刻画唇形轮廓描述,达到唇形识别的目的。

In order to describe the lip contours in a lipreading system automatically and quickly, Fourier descriptors are applied to describe and recognize the lip contours. After movement detection and morphological processing, boundary Fourier transform is used to get the Fourier descriptors of lip contours in unsymmetrical lip contour model, which is used to extract mouth region and parameters of lip contours from the image sequence. The Fourier descriptor ~p is used as the feature vector in speech synthesis system driven by visual-speech based on hidden Markov model. Experiments based on isolated Chinese words show that the lip contours can be reconstructed effectively only by using the first 15 or 20 Fourier descriptors, which reaches the goal of lip movement recognition.