改进的跨模态关联歧义学习的虚假信息检测方法研究

Study on Improved Fake Information Detection Method Based on Cross-modal Correlation Ambiguity Learning

近年来,随着互联网及多媒体技术的迅猛发展,人们获取信息更加方便快捷,然而虚假信息在网络上的传播也日益严重,负面影响不断扩大。为了增强信息的可信度和欺骗性,虚假信息呈现多模态发展趋势,使得检测工作面临更大挑战。现有的多模态虚假信息检测方法大多关注多模态特征的形成,对于跨模态歧义和不同模态特征在检测中的贡献率的研究尚不完善,忽略了不同模态特征间固有差异性对虚假信息检测的影响。为解决该问题,提出了构建改进的跨模态关联歧义学习的虚假信息检测模型,通过对文本和图像特征进行跨模态歧义学习,利用歧义得分更新单模态与融合特征的权重,自适应地拼接单模态与融合特征;同时采用网格搜索动态分配文本、图像特征权重,提高检测准确率。在Twitter数据集上对该模型的有效性进行验证,其相比基线模型准确率提高了6%,相比未进行动态权重分配的检测方法性能提升了1.6%。

In recent years,with the rapid development of the Internet and multimedia technology,it is more convenient for people to obtain information,but the spread of fake information on the Internet is also increasingly serious,and the negative impact is constantly expanding.In order to enhance the credibility and deception,fake information presents a multi-modal development trend,which makes the detection work face greater challenges.The existing multi-modal fake information detection methods pay more attention to the formation of multi-modal features.The research on the contribution rate of cross-modal ambiguity and different modal features in detection is not perfect,ignoring the impact of inherent differences among different modal features on fake information detection.To solve the problem,this paper proposes to construct an improved fake information detection model based on cross-modal correlation ambiguity learning.Through cross-modal ambiguity learning of text and image features,the weights of unimodal features and fused features are updated by the ambiguity score.The unimodal features and fused features are combined adaptively,and the weights of text and image features are dynamically assigned by grid search to improve the detection accuracy.The effectiveness of the model is verified by experiments on the Twitter dataset.The accuracy is improved by 6%compared with the baseline model and 1.6%compared with the detection without dynamic weight assignment.