基于多模态自适应融合的短视频虚假新闻检测

Multimodal Adaptive Fusion Based Detection of Fake News in Short Videos

随着互联网和社交媒体的迅速发展,新闻的传播途径不再局限于传统的媒体渠道。语义丰富的多模态数据成为新闻的载体,虚假新闻也随之得到了广泛的传播。由于虚假新闻的泛滥会对个人以及社会产生难以预估的影响,针对虚假新闻的检测已经成为目前的研究热点。现有的多模态虚假新闻检测方法仅针对文本和图像数据,无法充分利用短视频中的多模态信息,且忽略了不同模态间的一致性和差异性特征,难以充分发挥多种模态融合的优势。为解决该问题,提出一种基于多模态自适应融合的短视频虚假新闻检测模型。首先对短视频中多模态数据进行特征提取,采用跨模态对齐融合获取不同模态间的一致性和互补性特征;然后根据不同模态特征对最终融合结果的贡献实现自适应融合;最后利用分类器实现虚假新闻检测。在公开的短视频数据集上的实验结果表明,该模型的准确率、精确率、召回率和F1分数都高于当前的先进基线模型。

With the rapid development of Internet and social media,the dissemination route of news is no longer limited to traditional media channels.Semantically rich multimodal data becomes the carrier of news while fake news has been widely spread.As the proliferation of false news will have an unpredictable impact on individuals and society,the detection of false news has become a current research hotspot.Existing multimodal false news detection methods only focus on text and image data,which not only fail to fully utilize the multimodal information in short videos but also ignore the consistency and difference features between different modalities.As a result,it is difficult for them to give full play to the advantages of multimodal fusion.To solve this pro-blem,a fake news detection model for short videos based on multimodal adaptive fusion is proposed.This model extracts features from multimodal data in short videos,uses cross-modal alignment fusion to obtain the consistency and complementarity features among different modalities,and then achieves adaptive fusion based on the contribution of different modal features to the final fusion result.Finally,a classifier is used to achieve fake news detection.The results of the experiment conducted on a publicly avai-lable short video dataset demonstrate that the accuracy,precision,recall,and F1-score of the proposed model are higher than those of the state-of-the-art models.