基于对比学习和焦点损失的持续关系抽取

Continual Relation Extraction Based on Contrastive Learning and Focal Loss

持续关系抽取旨在训练模型从不断变化的数据流中学习新关系,同时保持对旧关系的准确分类.然而,由于神经网络的灾难性遗忘问题,模型在学习完新关系之后,对旧关系的识别能力往往会大幅度降低.为了缓解灾难性遗忘对模型性能的影响,本文提出了一种基于对比学习和焦点损失的持续关系抽取方法.首先,在训练集与其增强样本集的并集上训练模型,以学习新任务;其次,从训练集中,为每个新关系选取并存储记忆样本;然后,将激活集中的示例与所有已知关系原型进行对比,以学习新旧关系;最后,利用关系原型进行记忆再巩固,并引入焦点损失提高模型对相似关系的区分.在TACRED数据集上进行实验,结果表明本文方法能够进一步缓解灾难性遗忘问题,提升模型的分类能力.

Continual relation extraction aims to train models to learn new relations from evolving data streams while maintaining accurate classification of previously learned relations.However,due to the catastrophic forgetting problem of neural networks,the model’s ability to recognize old relations tends to decrease drastically after being trained on new relations.To mitigate the impact of catastrophic forgetting on model performance,this study proposes a continual relation extraction method based on contrastive learning and focal loss.First,the model is trained on a concatenated set of the original training set and its augmented samples to learn a new task.Second,from the training set,memory samples are selected and stored for each new relation.Then,instances from the activation set are contrasted with all known relation prototypes to learn the old and the new relations.Finally,memory reconsolidation is performed using the relation prototypes and focal loss is introduced to improve the model’s distinction between similar relations.Experiments are conducted on the TACRED dataset,and the results show that the method proposed can further alleviate catastrophic forgetting and improve the model’s classification ability.