基于跨模态交互与特征融合网络的假新闻检测方法

Fake News Detection Based on Cross-modal Interaction and Feature Fusion Network

近年来,假新闻的激增对人们的决策过程产生了不利影响。现有的假新闻检测方法大多强调对多模态信息(如文本和图像)的探索和利用。然而,如何为检测任务生成有鉴别性的特征并有效地聚合不同模态的特征以进行假新闻检测,仍然是一个开放性问题。文中提出了一种新颖的假新闻检测模型,即跨模态交互与特征融合网络(Cross-modal Interaction and Feature Fusion Network,CMIFFN)。为了生成有鉴别性的特征,所提方法设计了一个基于监督对比学习的特征学习模块,通过同时进行模态内和模态间的监督对比学习,来确保异类特征相似度更小,同类特征相似度更大。此外,为了挖掘更多有用的多模态信息,所提方法设计了多阶段跨模态交互模块,通过多阶段的跨模态交互,学习带有图结构信息的跨模态交互特征。所提方法引入基于一致性评估的注意力机制,通过学习多模态一致性权重,来有效聚合模态特定特征和跨模态交互特征。在两个基准数据集Weibo和Twitter上的实验表明,CMIFFN明显优于现有的多模态假新闻检测方法。

In recent years,the surge in fake news has adversely affected people's decision-making process.Many existing fake news detection methods emphasize the exploration and utilization of multimodal information,such as text and image.However,how to generate discriminative features for the detection task and effectively aggregate features of different modalities for fake news detection remains an open question.In this paper,we propose a novel fake news detection model,i.e.,cross-modal interaction and feature fusion network(CMIFFN).To generate discriminant features,a supervised contrastive learning-based feature learning module is designed.By performing intra-modality and inter-modality supervised contrastive learning simultaneously,it ensures that the similarity of heterogeneous features is smaller and the similarity of similar features is greater.In addition,in order to mine more useful multi-modal information,this paper designs a multi-stage cross-modal interaction module to learn cross-modal interaction features with graph structure information.The method introduces consistency evaluation-based attention me-chanism to effectively aggregate modality-specific features and cross-modal interaction features by learning multi-modal consistency weight.Experiments on two benchmark datasets Weibo and Twitter show that CMIFFN is significantly superior to the state-of-the-art multimodal fake news detection methods.