DA-Transformer:基于“门”注意力的文本情感分析方法

DA-Transformer:Text Sentiment Analysis Method Based on"Gate"Attention

为了解决Transformer对中文文本词语建模时容易造成信息冗余这一问题,提出了一种“门”注意力结合Transformer(DA-Transformer)的情感分析模型.该模型通过在Transformer模型中的编码和解码过程中插入一种基于自注意力的“门”注意力(DA)来建立文本的长远距离依赖,加速模型学习深层特征与浅层特征的权重比值.本模型在ChnSentiCorp_htl_al和weibo_senti数据集上得到验证.实验表明,本模型的准确率比BLSTM的准确率高1.8%,比BLSTM-Attention模型的准确率高0.9%,表明本模型具有一定的优异性与可实行性.

In order to solve the problem that Transformer is easy to cause information redundancy when modeling Chinese text words,a sentiment analysis model combining"gate"attention with Transformer(DA Transformer)is proposed in this paper.This model establishes the long-term distance dependence of text by inserting a self attention based"gate"attention(DA)during the encoding and decoding process in the Transformer model,accelerating the model’s learning of the weight ratio between deep and shallow features.This model has been validated on the ChnSentiCorp_htl_al and weibo_senti datasets.The experiment shows that the accuracy of this model is 1.8%higher than that of BLSTM,and 0.9%higher than that of BLSTM-Attention model,indicating that this model has certain advantages and feasibility.