基于层级注意力模型的视频序列表情识别

Video Emotion Recognition Based on Hierarchical Attention Model

长短期记忆网络(LSTM)广泛应用于视频序列的人脸表情识别,针对单层LSTM表达能力有限,在解决复杂问题时其泛化能力易受制约的不足,提出一种层级注意力模型:使用堆叠LSTM学习时间序列数据的分层表示,利用自注意力机制构建差异化的层级关系,并通过构造惩罚项,进一步结合损失函数优化网络结构,提升网络性能.在CK+和MMI数据集上的实验结果表明,由于构建了良好的层次级别特征,时间序列上的每一步都从更感兴趣的特征层级上挑选信息,相较于普通的单层LSTM,层级注意力模型能够更加有效地表达视频序列的情感信息.

LSTM network is widely used in facial expression recognition of video sequences.In view of the limited representation ability of single-layer LSTM and the limitation of its generalization ability when solving complex problems,a hierarchical attention model is proposed.Hierarchical representation of time series data is learned by stacking LSTM,self-attention mechanism is used to construct differentiated hierarchical relationships,and a penalty term is constructed and further combined with the loss function to optimize the network performance.Experiments on CK+and MMI datasets,demonstrate that due to the construction of good hierarchical features,each step in time series can select information from the more interesting feature hierarchy.Compared with ordinary single-layer LSTM,hierarchical attention model can express the emotional information of video sequences more effectively.