基于迁移学习和预训练技术的推理机器阅读理解模型

Reasoning Machine Reading Comprehension Model Based on Transfer Learning and Pre-Training Technology

机器阅读理解旨在评估计算机的自然语言理解能力,当前的大多数机器阅读理解模型缺乏逻辑推理能力,最先进的模型在推理阅读理解数据集上表现也较差。本文提出了一个具有逻辑推理能力的机器阅读理解模型。利用预训练技术,首先在多个不同粒度的域外数据集上微调语言模型赋予模型泛化能力,再使用数据扩充技术和滑动窗口方法解决目标数据集样本数量不足的问题,最终在需要逻辑推理能力的数据集LogiQA上取得了最佳效果,证明了方法的有效性。

Machine reading comprehension aims to evaluate the natural language comprehension ability of computers.Most current machine reading comprehension models lack the ability of logical reasoning,and the most advanced models perform poorly on the reading comprehension task which solely requires the ability of logical reasoning.A machine reading comprehension model with logical reasoning ability is proposed.Using pre-training techniques,we first fine-tune the language model on multiple out-of-domain data sets of different granularity to give the model generalization capability,and then use data augmentation technology and sliding window method to solve the problem of insufficiency in the target data set.Finally,the best results have been achieved on the dataset LogiQA which requires the ability of logical reasoning.Experimental results show that the proposed method is effective.