基于注意力机制与多尺度特征融合的人脸表情识别

Facial expression recognition based on attention mechanism and multi-scale feature fusion

人脸表情识别一直是人机交互等领域颇为关注的研究,然而当前研究大多忽略了多尺度特征的融合及表情中间特征的改善。针对这一问题,本文提出了一种基于注意力机制与多尺度特征融合学习的人脸表情识别方法,该方法由浅层特征提取模块和多尺度特征融合模块构成,能从深到浅提取更多有价值的信息,并有效改善表情中间特征。首先输入表情图像到浅层网络和骨干网络,分别获取浅层特征和深层特征;然后在浅层特征提取模块中加入注意力机制,对浅层特征进行加强或抑制;最后融合浅层特征与深层特征,构造人脸表情的多尺度融合特征,并通过分类器将人脸表情图像分为7种表情。该方法在两个公开数据集JAFFE和KDEF上的平均识别准确率分别达到了96.67%和89.29%,表现优于现有的深度学习方法和传统方法。实验表明该方法能获取更丰富的人脸表情信息,且具有更强的泛化能力。

Facial expression recognition has been the research of great concern in fields such as human-computer interaction. However, the fusion of multi-scale features and the improvement of intermediate features of facial expressions have been ignored in most of the current studies. To address this problem, a facial expression recognition method based on attention mechanism and multi-scale feature fusion learning was proposed in this paper. The method composed of a shallow layer feature extraction module and a multi-scale feature fusion module can extract more valuable information from deep layer to shallow layer and effectively improve the intermediate features of expression. First, the expression images are input into shallow network and backbone network to obtain shallow layer features and deep layer features respectively. Then an attention mechanism is added to the shallow layer feature extraction module to intensify or suppress the shallow layer features. The shallow layer features and deep layer features are fused to create the multi-scale fusion features of facial expressions. Finally, the facial expression images are divided into 7 expressions by classifier. The average recognition accuracy of this method on two public datasets, JAFFE and KDEF, reaches 96.67% and 89.29%, respectively. The performance outperforms the current deep learning methods and traditional methods. The experimental results demonstrate that the method can bring about more abundant facial expression information with stronger generalization ability.