基于BERT和层次化Attention的微博情感分析研究

Microblog Sentiment Analysis Based on BERT and Hierarchical Attention

微博情感分析旨在挖掘网民对特定事件的观点和看法,是网络舆情监测的重要内容。目前的微博情感分析模型一般使用Word2Vector或GloVe等静态词向量方法,不能很好地解决一词多义问题;另外,使用的单一词语层Attention机制未能充分考虑文本层次结构的重要性,对句间关系捕获不足。针对这些问题,提出一种基于BERT和层次化Attention的模型BERT-HAN(bidirectional encoder representations from transformers-hierarchical Attention networks)。通过BERT生成蕴含上下文语意的动态字向量;通过两层BiGRU分别得到句子表示和篇章表示,在句子表示层引入局部Attention机制捕获每句话中重要的字,在篇章表示层引入全局Attention机制以区分不同句子的重要性;通过Softmax对情感进行分类。实验结果表明,提出的BERT-HAN模型能有效提升微博情感分析的Macro F1和Micro F1值,具有较大的实用价值。

Microblog sentiment analysis aims to dig out netizens’views and opinions on specific events,which is an important content of online public opinion monitoring.The current microblog sentiment analysis models generally static word embedding methods such as Word2Vector or GloVe,which cannot solve the polysemy problems well.In addition,the Attention mechanism at word level fails to fully consider the importance of text hierarchy and fails to capture the relationship between sentences.Aiming at these problems,it proposes a model named BERT-HAN(bidirectional encoder representations from transformers-hierarchical Attention networks),which is based on BERT and hierarchical Attention mechanism.Firstly,the dynamic character vector containing the context semantics is generated by BERT.Then,two levels of BiGRU are used to obtain sentence representation and document representation respectively.Local Attention mechanism is introduced in the sentence representation layer to capture the important characters in each sentence,and the global Attention mechanism is introduced in the document representation layer to distinguish the importance of different sentences.Finally,the emotions are classified by Softmax.The experimental results show that the BERT-HAN model proposed in this paper can effectively improve the Macro F1 and Micro F1 values of microblog sentiment analysis,which has great practical value.