ON-LSTM和自注意力机制的方面情感分析

Aspect Sentiment Analysis Combining ON-LSTM and Self-attention Mechanism

方面情感分析是更细粒度的文本情感分析,传统的方法是采用长短时记忆神经网络和注意力机制相结合,但实际并未考虑到方面情感特征项与句子上下文之间的联系,并且在预训练阶段通常使用静态语言模型,无法根据需要调整输入词向量.针对以上两个问题,本文提出一种基于有序神经元长短时记忆和自注意力机制的方面情感分析模型(ON-LSTM-SA).首先,利用深层语境化词表征(ELMo)进行语料的预训练.其次,在隐藏层采用ON-LSTM神经网络模型从上下文的左右两个方向同时进行训练,获取方面情感特征项与句子之间的层级结构关系.最后,根据自注意力机制计算内部的词依赖关系.该模型通过在SemEval2014和SemEval2017中的Laptop、Restaurant和Twitter三个数据集上进行实验,与传统LSTM模型相比分别提升了2.1%、5.9%和6.5%.

Aspect sentiment analysis is a more granular text sentiment analysis.The traditional method is to combine the Long Short Term Memory(LSTM)neural network with the Attention mechanism,but it is not considered the connection between the aspect terms and the sentence context actually;In the pre-training phase,the static language model is usually used,and the input word vector cannot be adjusted as needed.Aiming at the above two problems,this paper proposes an Ordered Neurons Long Short-Term Memory-based and Self-Attention Mechanism-based Aspect Sentiment Analysis(ON-LSTM-SA)model.First,the pre-training of corpus is performed by using the deep contextualized word representation-Embeddings from Language Models(ELMo).Secondly,the ON-LSTM neural network model is used in the hidden layer to train from the left and right directions of the context to obtain the hierarchical structure relationship between the aspect terms and the sentences.Finally,the internal word dependencies are calculated according to the Self-Attention mechanism.The model was tested on the three datasets of Laptop,Restaurant and Twitter in SemEval2014 and SemEval 2017,which increased by 2.1%,5.9%and 6.5%,respectively,compared to the traditional LSTM model.