基于通道注意的可变形金字塔表情识别网络

Channel-attention-based Deformable Pyramid Network for Facial Expression Recognition

人脸表情识别是计算机视觉领域里一项热门且有挑战性的任务。由于人脸表情的特性相对固定,从宏观上针对人脸表情的特性进行方法设计能有效提高人脸表情识别的性能。基于这一观点,针对人脸表情特征的不规则特性和空间多尺度互补特性,提出了基于通道注意的可变形金字塔网络。该网络主要由可变形卷积块、空间金字塔池化块和通道注意块构成,其中可变形卷积块有助于网络对人脸表情的不规则特征进行采样;而空间金字塔池化块则加强了网络学习多尺度空间上下文情绪信息的能力;通道注意块则促使网络动态关注更具判别性的情绪特征。该方法在CK+、JAFFE以及Oulu-CASIA三个实验室环境的人脸表情数据集和FER2013以及RAF-DB两个野外环境的人脸表情数据集上进行了对比实验和消融实验并取得了有竞争力的结果。从可视化结果上看,该方法提取的特征及关注的人脸区域符合不同表情的呈现特性和人们日常判断表情的规律。

Facial expression recognition is a popular and challenging task in the field of computer vision.Since the characteristics of facial expressions are relatively fixed,designing methods for the characteristics of facial expressions from a macro perspective can effectively improve the performance of facial expression recognition.Based on this view,a deformable pyramid network based on channel attention is proposed for the irregular characteristics of facial expression features and the complementarity of spatial multi-scale.The network mainly consists of deformable convolutional blocks,spatial pyramid pooling blocks and channel attention blocks.Specifically,the deformable convolution block helps the network to sample irregular features of facial expressions,while the spatial pyramid pooling block enhances the network’s ability to learn multi-scale spatial contextual emotion information,and the channel attention block motivates the network to dynamically focus on more discriminative feature maps to improve the contribution of discriminative emotion information to the facial expression recognition task.Experimental results on both three in-the-lab facial expression datasets(including CK+,JAFFE,and Oulu-CASIA)and two in-the-wild facial expression datasets(including FER2013 and RAF-DB)demonstrate the effectiveness of the proposed method.From the visualization results,the features extracted by the proposed method and the facial regions of interest are consistent with the presentation characteristics of different expressions and the patterns of people's daily judgment of expressions.