基于复述模型的词语替代方法

Lexical Substitution Based on Paraphrase Modeling

词语替代任务旨在为句子中的目标词寻找合适的替代词。基于预训练语言模型BERT的词语替代方法直接利用目标词的上下文信息生成替代候选词。由于标注数据资源的缺乏使得研究人员通常采用无监督的方法,这也限制了预训练模型在此任务上的适用性。考虑到现有的大规模复述语料中包含了大量的词语替代规则,该文提出一种通过复述模型生成替代候选词的方法。具体的做法是:利用复述语料训练一个神经复述模型;提出了一种只关注目标词变化的解码策略,用于从复述模型中生成替代词;根据文本生成评估指标计算替代词对原句意思的改变程度,对替代词排序。相对已有的词语替代方法,在两个广泛使用的数据集LS07和CoInCo上进行评估,该文提出的方法取得了显著的提高。

Lexical substitution(LS)aims at finding an appropriate substitute for a target word in a sentence.In contrast to the BERT-based LS,this paper proposes a method to generate substitution candidates base on paraphrase to utilize the existing large-scale paraphrase corpus which contains a large number of rules of word substitution.Specifically,we first employ a paraphrase dataset to train a neural paraphrase model.Then,we propose a special decoding method to focus only on the variation of the target word to extract substitute candidates.Finally,we rank substitute candidates for choosing the most appropriate substitution without modifying the meaning of the original sentence based on text generation evaluation metrics.Compared with existing state-of-the-art methods,experimental results show that our proposed methods achieve the best results on two widely used benchmarks(LSo7 and ColnCo).