基于多模态异质图的社交媒体谣言检测模型

Detecting Social Media Rumors Based on Multimodal Heterogeneous Graph

【目的】验证谣言不同模态之间存在关联性,以提高谣言检测准确率,进而提出一种基于多模态异质图的社交媒体谣言检测模型。【方法】以社交平台上多模态的帖子为研究对象,首先通过预处理提取文本、图片两种模态信息及用户属性信息的特征表示,按照文本、图片、用户三者间的关联关系构建异质图,然后按照指定的元路径提取文本类型节点的嵌入表示,最后将其输入分类器中,判断其是否是谣言。【结果】在公开的数据集上进行实验,结果表明,所提模型在两个数据集上的准确率分别达到91.3%和93.8%,其他评价指标也高于基线模型。【局限】由于共享多模态谣言的三类节点会使所构建的异质图存在较大的稀疏性,所提模型更适用于小型的话题社区。【结论】谣言的不同模态之间存在关联性,所提模型利用该特征在多模态谣言检测中表现出良好的效果。

[Objective]This paper proposes a social media rumor detection model based on the multimodal heterogeneous graph,aiming to verify the correlation between different rumor modalities and improve the accuracy of rumor detection.[Methods]First,we retrieved multimodal posts from social platforms.Then,we extracted feature representations of texts,pictures,and user attributes through preprocessing.Third,we constructed a heterogeneous graph based on the correlation between texts,pictures,and users.Fourth,we extracted the embeddings of text-type nodes according to their specified meta path.Finally,we input the embedding into the classifier to determine whether or not it is a rumor.[Results]We examined the proposed model with two open data sets.The accuracy of our model reached 91.3%and 93.8%,which were also higher than the baseline models.[Limitations]The three types of nodes from the sharing multimodal rumors will make the heterogeneous graph sparse.The proposed model is more suitable for small topic communities.[Conclusions]There is a correlation between different modalities of rumors,which helps the proposed model effectively detect multimodal rumors.