一种轻量级文本蕴含模型

A lightweight text entailment model

现有主流文本蕴含模型大多采用循环神经网络编码,并采用各种注意力推理机制或辅以手工提取的特征来提升蕴含关系识别准确率,由于复杂的网络结构和RNNs网络串行机制导致这些模型训练和推理速度较慢.本文提出轻量级文本蕴含模型,采用自注意力编码器编码文本向量,点积注意力交互两段文本,再采用卷积神经网络对交互特征推理,整个结构可根据不同数据的推理难度叠加不同模块数量.在多个文本蕴含数据集实验表明,本文模型在保持较高识别准确率情况下仅用一个块参数仅为665 K,模型推理速度相比其他主流文本蕴含模型至少提升一倍.

Most of the existing mainstream textual entailment models adopt recurrent neutral network to encode text,and various complex attention mechanisms or manually extracted text features are used to improve the accuracy of textual entailment recognition.The training and inference speed of the models is usually slow due to the complex network structure and the sequential nature of RNNs.In this paper,Lightweight Text Entailment Model is proposed.In the proposed model,the self-attentional encoder is adopted to encode text vectors;the dot product attention mechanism is adopted to interact two texts;the convolutional neural network is adopted to deduce interactive features,and the module number of the structure can be adjusted according to the reasoning difficulty of data.Experiments on multiple datasets show that the parameter size of single module in the model is only 665 K,and the inference speed of the model is at least twice as high as that of other mainstream models,under the condition of high accuracy.