融合高低层语义信息的自然语言句子匹配方法

Natural language sentence matching method fusion of high-level and low-level semantic information

针对目前自然语言句子匹配方法未能融合公共语义信息以及难以捕获深层次语义信息的问题,提出了一种融合高低层语义信息的自然语言句子匹配算法。该算法首先采用预训练词向量GloVe和字符级别的词向量获取句子P和句子Q的词嵌入表示;其次,使用双向LSTM(long-short term memory)对P和Q进行编码,再初步融合P和Q获取低层语义信息;然后,计算P与Q的双向注意力,拼接得到语义表征,再计算其自注意力获取高层语义信息;最后,采用启发式的融合函数将低层语义信息和高层语义信息进行融合,得到最终的语义表征,使用卷积神经网络预测得到答案。在文本蕴涵识别和释义识别两个任务上评估了该模型。在文本蕴涵识别任务SNLI数据集和释义识别任务Quora数据集上进行了实验,实验结果表明该算法在SNLI测试集上的准确率为87.1%,在Quora测试集上的准确率为86.8%,验证了算法在自然语言句子匹配任务上的有效性。

This paper proposed a natural language sentence matching method that combined high-level and low-level semantic information to solve the problems about current natural language sentence matching method fail to integrate common semantic information and it is difficult to capture deep-semantic information.First of all,the method used pre-trained word vector GloVe and character-level word vector to obtained the word embedding representation of sentence P and sentence Q.Secondly,this paper encodered P and Q with bidirectional LSTM,then it contained low-level semantic information through preliminary fusion of P and Q.Thirdly,this paper calculated bidirectional attention between P and Q,then spliced them together to get semantic representation,afterwards it calculated its self-attention to obtained high-level semantic information.Finally,this paper used a heuristic fusion function to fuse the low-level semantic information with the high-level semantic information to obtain the final semantic representation,and it used a convolutional neural network to prediction answers.This paper evaluated the proposed model on two tasks,such as recognition textual entailment,paraphrase recognition.This paper conducted experiments on the SNLI dataset and the Quora dataset.The results show that the accuracy of the proposed algorithm on the SNLI test set is 87.1%,and the accuracy of the Quora test set is 86.8%,which verifies the effectiveness of the algorithm in the task of natural language sentence matching.