双模态及语义知识的三维人脸表情识别方法

3D facial expression recognition method based on bimodal and semantic knowledge

目前,人脸表情识别的主要研究对象是二维图像,它所包含的信息有限,而且易受人脸姿态、光照等影响。其次,人脸表情识别方法大多是基于图像低层视觉特征,而人类对图像的理解是基于高层语义知识,这两者之间存在本质上的差异,即"语义鸿沟"。为此,在三维人脸表情图像和语义知识的基础上,创新地提出双模态及语义知识的三维人脸表情识别方法。该方法首先提出一种将三维的局部曲率和二维局部角点进行双模态融合的方法,自动提取准确的三维人脸表情低层视觉特征;然后,采用AHP和G1相结合计算高层语义知识向量;最后,采用K-NN算法将低层视觉特征和高层语义知识融合,缩小低层视觉特征和高层语义知识之间的"语义鸿沟",提高人脸表情的识别率。

At present,the main research object of facial expression recognition is 2D image;it does not have enough information, and is vulnerable to the face pose, illumination and etc. Secondly, the facial expression recognition meth- ods are mostly based on low-level visual features of the image, but the human understanding of image is based on high-level semantic knowledge;there are essential differences between them, i. e. the ＂semantic gap＂. So, based on 3D facial expression image and semantic knowledge, a 3D facial expression recognition method is innovatively pro- posed based on bimoda] and semantic knowledge. Firstly, a method is proposed, which carries out the bimodal fusion of 3D local curvature and 2D local corner;and the method can extract the low-level visual features of 3D facial ex- pression automatically. Then a high-level semantic knowledge vector is calculated by combining AHP and G1. Finally, K-NN algorithm is adopted to fuse the low-level visual features and high-level semantic knowledge, narrow the ＂semantic gap＂ between the low-level visual features and high-level semantic knowledge, and increase the recogni- tion rate of facial expression recognition.