基于注意力机制的多模态在线评论有用性预测研究

Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism

在电子商务时代,在线评论被视为一类重要的商品评价,深刻影响着消费者的决策过程。但是指数级增长的评论数量和非结构化的评论数据给评论有用性预测模型的特征选择和精确度提升带来了挑战。此外,目前的研究主要集中于浅层特征和评论文本的特征提取,往往忽略了评论照片所包含的图像信息,同时评论文本、照片、浅层特征这些多模态的信息需要应用多模态融合方法进行信息的提炼融合。基于此,文中将评论照片和评论文本作为影响在线评论有用性的潜在特征,并根据KAM知识采纳理论设计浅层特征集合。对于3种模态的数据,提出了一种基于协同注意力机制的三模态评论有用性预测模型(TMCAM),用于实现跨模态信息的交互和融合。实验结果检验了TMCAM模型的优越性能,证明了图像和文本信息的互补能够达到比单一模态信息更好的效果;浅层特征能够辅助预测评论有用性;相比简单的模态特征拼接,利用协同注意力机制进行跨模态信息交互有助于提升对评论有用性的感知。

In the e-commerce era,online reviews are regarded as important product evaluations,which profoundly influence consumers'decision-making process.However,the exponentially increasing number of reviews and unstructured review data pose challenges to feature selection and accuracy improvement of review helpfulness prediction.In addition,current research mainly focuses on shallow features and feature extraction of review texts,the image information contained in review photos is often ignored.Besides,multi-modal information such as review text,photos,and shallow features needs to be refined and fused by app-lying multi-modal fusion methods.Based on these,this paper regards review photos and review text as a latent feature affecting the helpfulness of online reviews,and designs a shallow feature set according to the KAM knowledge adoption theory.For the data of three modalities,a deep prediction model,i.e.,three-modal review helpfulness prediction based on co-attention mechanism(TMCAM)is proposed,which can achieve the interaction and fusion of cross-modal information.The superior performance of the TMCAM model is tested through experiments,and it is proved that the complementation of image and text information can achieve better results than single modal information.Besides,shallow features can help predict the reviews helpfulness.Moreover,compared with simple modal features splicing,using collaborative attention mechanism for cross-modal information interaction helps to improve the perception of reviews helpfulness.