基于XLNet+BiGRU+Att(Label)的文本分类模型

Text Classification Model Based on XLNet+BiGRU+Att(Label)

传统的词向量嵌入模型,如Word2Vec、GloVe等模型无法实现一词多义表达;传统的文本分类模型也未能很好地利用标签词的语义信息。基于此,提出一种基于XLNet+BiGRU+Att(Label)的文本分类模型。首先用XLNet生成文本序列与标签序列的动态词向量表达;然后将文本向量输入到双向门控循环单元(BiGRU)中提取文本特征信息;最后将标签词与注意力机制结合,选出文本的倾向标签词,计算倾向标签词与文本向量的注意力得分,根据注意力得分更新文本向量。通过对比实验,本文模型比传统模型在文本分类任务中的准确率更高。使用XLNet作为词嵌入模型,在注意力计算时结合标签词能够提升模型的分类性能。

Traditional word vector embedding models,such as Word2 Vec and GloVe,cannot realize polysemy expression.Traditional text classification models also fail to make good use of the semantic information of label words.Based on this,a classification model based on XLNet+BiGRU+Att(Label)is proposed.Firstly,the dynamic word vector expression of text sequence and label sequence is generated by XLNet.Then,the text vector is input into the Bidirectional Gated Recurrent Unit(BiGRU)to extract the text feature information.At last,the label words are combined with the attention mechanism to select the tendency label words of the text,calculate the attention score of the tendency label words and the text vector,and update the text vector according to the attention score.Through comparative experiments,the model in this paper has higher accuracy than the traditional model in text classification tasks.Using XLNet as the word embedding model and combining label words in attention calculation can improve the classification performance of the model.