自注意力机制的短文本分类算法

Short text classification algorithm of self-attention mechanism

分析目前的短文本分类算法没有综合考虑文本中隐含的依赖关系和局部关键信息这一问题,提出基于自注意力机制(self-attention mechanism)的堆叠双向长短时记忆网络(stack bidirectional long short term memory)模型(简称Att-BLSTMs)。利用stack Bi-LSTMs捕获上下文隐藏依赖关系,优化短文本特征稀疏的问题;利用自注意力机制加大对短文本中局部关键信息的注意力,优化文本表示。在公开AG-news网页新闻的语料和DBpedia分类数据集中,进行丰富的对比实验。实验结果表明,该模型将文本中隐含依赖关系与局部关键信息综合考虑后,有效提高了短文本分类的准确性。

The implicit dependencies and local key information in the current short text classification algorithm are not comprehensively considered.A stack bidirectional long short term memory model based on the self-attention mechanism was proposed.Stack Bi-LSTMs was used to mine the contextual semantic dependencies information to optimize feature representation.The attention mechanism was used to focus on key information of text to optimize the text representation.The public corpus of the AG-news web news and DBpedia were used to conduct a rich comparative experiment.It is pointed out that the accuracy of the short text classification is improved a lot by considering the implicit dependencies and local key information.