中文机器阅读理解的鲁棒性研究

Robustness of Chinese Machine Reading Comprehension

为了更好地评价阅读理解模型的鲁棒性,基于Dureader数据集,通过自动抽取和人工标注的方法,对过敏感、过稳定和泛化3个问题分别构建测试数据集。还提出基于答案抽取和掩码位置预测的多任务学习方法。实验结果表明,所提方法能显著地提高阅读理解模型的鲁棒性,所构建的测试集能够对模型的鲁棒性进行有效评估。

In order to better evaluate the robustness of Machine Reading Comprehension(MRC)models,this paper builds three test sets from Dureader by automatically extracting and manually annotating,consisting of oversensitivity,over-stability,and generalization.In addition,this paper proposes a multi-task learning framework with answer extraction task and masked position prediction task.Experimental results demonstrate that proposed method gains significant robustness improvements and show the effectiveness of the three test sets on evaluating the robustness of MRC models.