融合BERT词嵌入和注意力机制的中文文本分类

Chinese Text Classification Based on BERT and Attention

文本分类是自然语言处理的一个重要领域.近年来,深度学习的方法被广泛应用于文本分类任务中.在处理大规模的数据时,为了兼顾分类的精度和处理效率,本文使用BERT训练词向量作为嵌入层,进一步优化输入语句的词向量,然后用双层的GRU网络作为主体网络,充分提取文本的上下文特征,最后使用注意力机制,将目标语句重点突出,进行文本分类.实验证明,BERT作为嵌入层输入时,有效优化了词向量.同时,文本提出的BBGA模型具有高效的处理能力,在处理THUCNews数据集时,达到了94.34%的精确度,比TextCNN高出5.20%,比BERT;NN高出1.01%.

Text classification is a classic problem in natural language processing.In recent years, deep learning models have drawn a lot of attention in such fields.When it comes to large amounts of data, it is important that we take both accuracy and efficiency into account, so we use the BERT pre-trained language model to obtain word vector, and which is later the embedding layer used to further optimize the word vector of the input sentence.Then the Bidirectional GRU network, which is proved to be efficient in processing text, is utilized to extract the contextual feature of the text.Finally, the attention mechanism is applied to highlight the target sentence and perform text classification.Experiments show that when the BERT pre-trained language model is used as the input of the embedding layer, it effectively optimizes the word vector.At the same time, the method used in the text has reached an accuracy of 94.34% when processing the THUCNews data set, which is 5.19% higher than TextCNN and 1.01% higher than BERT;NN.