基于隐含主题协同注意力网络的领域分类方法

Latent Topic-Enriched Co-Attention Networks for Domain Classification

基于注意力机制的神经网络模型在文本分类任务中显示出了很好的效果。然而当训练数据的规模有限,或者测试数据与训练数据的分布有较大差异时,一些有价值的信息词很难在训练中被模型捕捉到。为此,该文提出了一种新的基于协同注意力(co-attention)网络的领域分类方法。该文利用隐含主题模型学习隐含主题注意力,并将其引入到文本分类常用的双向长短时记忆网络(BiLSTM)中,与软或硬注意力(soft-or hard-attention)机制一起构成协同注意力。在中文话语领域分类基准语料SMP-ECDT上的实验结果表明,隐含主题协同注意力网络取得了显著优于注意力机制的领域分类效果,比基线注意力机制Soft att、Hard att以及单独的隐含主题注意力机制BTM att分别提高了2.85%、1.86%和1.74%的分类正确率。此外,实验结果还验证了,在额外的未标记数据上训练隐含主题,可以进一步提高该文方法的领域分类性能。

Attention-based bidirectional long short-term memory network(BiLSTM) models have recently shown promising results in text classification tasks.However, when the amount of training data is restricted, or the distribution of the test data is quite different from the training data, some potential informative words are hard to be captured in training.In this work, we propose a new method to learn co-attention for domain classification.Unlike the past attention mechanism guided only by domain tags of training data, we leveroge using the latent topics in the data set to learn topic attention mechanism, and employ it for BiLSTM.Then the co-attention is obtained by combining the topic attention and the network attention.Experiments on the SMP-ECDT benchmark corpus show that the proposed co-attention mechanism outperforms the state-of-the-art soft mechanism, hard attention mechanism and topic attention mechanism in domain classification, by 2.85%, 1.86% and 1.74% accuracy improvement, respectively.