基于多模态深度融合的假消息检测

Multi-modal deep fusion based fake news detection method

智能检测虚假信息是社交网络中需要解决的重要任务之一。本文旨在识别同时包含图像和文字的多模态虚假消息。目前,针对多模态的虚假消息检测已有一些成果,但现有模型通过直接拼接各模态特征方式实现多模态利用,忽略了图像和文件之间的关系,无法有效地学习消息中文字和图像的深度融合表示,导致该种类型的虚假消息检测方法表现不佳。本文提出基于预训练模型的多模态融合假消息检测方法,充分利用社交媒体中大量的含有多模态数据的消息,实现对假消息的有效检测,通过不同的训练任务加强模型融合多模态信息的能力,最终学习一个多模态信息的表示辅助假消息识别。在新浪微博真实数据集上的实验结果表明,本文提出的基于预训练的检测模型取得了比当前主流方法更优的效果,同时,本文采用的模型能够缓解训练集和测试集分布不均衡导致的检测准确率下降问题。

Intelligent detection on fake news has become one of most important tasks in social networks.This work aims to identify multi-modal fake news that contains both image and text information.To date some existing efforts have been devoted to addressing this issue,however,they realize multi-modal utilization by simply splicing various modal features,ignoring the multi-modal integration(fusion representation)in proper manner,which leads to poor performance on detection.To circumvent the above problems,a pre-train-oriented multi-modal fusion method to detect fake news is proposed,which makes full use of large numbers of messages containing multi-modal data in social media to achieve effective detection of fake news.For the method,a representation for multi-modal information is learned to enhance the capability of multi-modal fusion through training different tasks,in this way,fake news can be effectively identified in a high accuracy.The multi-facets experiments using real-world dataset SinaWeibo show that the proposed method is rational and outperforms other baselines.Furthermore,the model used in this paper can effectively alleviate the problem of fast degradation of detection accuracy caused by uneven distribution of training set and test set.