基于注意力与神经图灵机的语义关系抽取模型

Semantic relation extraction model via attention based neural Turing machine

针对语义关系抽取(语义关系分类)中长语句效果不佳和核心词表现力弱的问题,提出了一种基于词级注意力的双向神经图灵机(Ab-NTM)模型。首先,使用神经图灵机(NTM)作为循环神经网络(RNN)的改进,使用长短时记忆(LSTM)网络作为控制器,其互不干扰的存储特性可加强模型在长语句上的记忆能力;然后,构建注意力层组织词级上下文信息,使模型可以加强句中核心词的表现力;最后,输入分类器得到语义关系标签。在SemEval 2010 Task 8公共数据集上的实验表明,该模型获得了86.2%的得分,优于其他方法。

Focusing on the problem of poor memory in long sentences and the lack of core words＇ influence in semantic relation extraction, an Attention based bidirectional Neural Turing Machine （Ab-NTM） model was proposed. Instead of a Recurrent Neural Network （RNN）, a Neural Turing Machine （NTM） was used firstly, and a Long Short-Term Memory （LSTM） network was acted as a controller, which contained larger and non-interfering storage, and it could hold longer memories than the RNN. Secondly, an attention layer was used to organize the context information on the word level so that the model could pay attention to the core words in sentences. Finally, the labels were gotten through the classifier. Experiments on the SemEval-2010 Task 8 dataset show that the proposed model outperforms most state-of-the-art methods with an 86.2% F1-score.