基于去偏对比学习的多模态命名实体识别

Debiased Contrastive Learning for Multimodal Named Entity Recognition

命名实体识别作为信息抽取的关键环节,在自然语言处理领域有着广泛应用。随着互联网上多模态信息的不断涌现,研究发现视觉信息有助于文本实现更加准确的命名实体识别。现有工作通常将图像视为视觉对象的集合,试图将图像中的视觉对象与文本中的实体显式对齐。然而,当二者在数量或语义上不一致时,这些方法往往不能很好地应对模态偏差,从而难以实现图像和文本之间的准确语义对齐。针对此问题,该文提出了一种基于去偏对比学习的多模态命名实体识别方法(DebiasCL),利用视觉对象密度指导视觉语境丰富的图文作为扩充样本,通过去偏对比学习优化图文共享的潜在语义空间学习,实现图像与文本间的隐式对齐。在Twitter-2015和Twitter-2017上进行实验,DebiasCL的F1值分别达到75.04%和86.51%,在“PER.”和“MISC.”类别数据中F1分别提升了5.23%和5.2%。实验结果表明,该方法可以有效缓解模态偏差,从而提升多模态命名实体识别系统性能。

Recent studies show that visual information can help text achieve more accurate named entity recognition.However,most of the exiting work treats an image as a collection of visual objects and attempts to explicitly align visual objects with entities in text,fails to cope with modal bias well when visual objects and the entities are quantitatively and semantically inconsistent.To deal with this problem,we propose a debiased contrastive learning approach(DebiasCL)for multimodal named entity recognition.We utilize the visual objects density to guide visual context-rich sample mining,which enhances debiased contrastive learning to achieve better implicit alignment by optimizing the latent semantic space learning between visual and textual representations.Empirical results shows that the DebiasCL achieves a F1-value of 75.04%and 86.51%,with 5.23%and 5.2%increased on"PER"and"MISC"entity type data in Twitter-2015 and Twitter-2017,respectively.