基于GRU-Attention的中文文本分类

Chinese Text Classification Based on GRU-Attention

传统的GRU分类模型是一种基于LSTM的模型变体,它在LSTM的基础上将遗忘门和输入门合并成更新门,使得最终的模型比标准的LSTM模型更简单。可是LSTM和GRU均没有体现每个隐层输出的重要程度。为了得到每个输出的重要程度,本文在GRU的基础上加入了注意力(Attention)机制,设计了GRU-Attention分类模型,并通过对比实验的方式说明了增加了Attention的GRU模型在分类效果上有了一定程度的提升。

The traditional GRU classification model is a model variant based on LSTM. It combines forgetting gate and input gate into update gate on the basis of LSTM,which makes the final model simpler than the standard LSTM model. However,LSTM and GRU do not reflect the importance of each hidden layer output. In order to get the importance of each output,Attention mechanism is added to GRU,and GRU-Attention classification model is designed,and the comparison experiment shows that the GRU model with Attention improves the classification effect to a certain extent.