基于注意力机制多尺度网络的自然场景情绪识别

Emotion recognition of the natural scenes based on attention mechanism and multi-scale network

情绪识别作为计算机视觉的一项基本课题已经取得很大进展,然而在无约束自然场景中的情绪识别仍具挑战性.现有方法主要是利用人脸、姿态以及场景信息识别情绪,但是忽略了人物个体在场景中的不确定性,以及不能很好地挖掘场景中的情绪线索.针对现有研究存在的问题,提出了基于人物与场景线索的双分支网络结构,两个分支独立学习,通过早期融合得到情绪分类结果.对于人物在场景中的不确定性,引入身体注意力机制预判人物情绪置信度进而获得人体的特征表示,场景中引入空间注意力机制和特征金字塔以便充分获得场景中不同粒度的情绪线索.实验结果表明,此方法有效融合人物与场景信息,在EMOTIC数据集下能够明显提高情绪识别率.

Emotion recognition as a basic subject in computer vision has made tremendous progress,yet emotion recognition in natural,unconstrained environments is still challenging.Existing methods mainly use face,posture,and contextual information to recognize emotions,but these methods ignore the uncertainty of individuals in the context,and do not tap the emotional cues in the scene well.Aiming at the problems in existing research,a dual-branch network structure based on individual’s body and contextual information is proposed,in which two branches are learning independently to get the result of emotion classification through early fusion.For uncertainties of people in the scenes,the body gesture attention mechanism is utilized to estimate the confidence coefficient the body’s feature representation is extracted.For context branch,spatial attention mechanism and feature pyramid network are employed to fully obtain the emotional cues of different granularities in the scene.The experiment results demonstrated that the effectiveness of the proposed method in the EMOTIC dataset.