融合外部知识图谱的多模态知识共创价值识别研究

Research on Multi-modal Knowledge Co-creation Value Recognition by Integrating External Knowledge Graph

[目的/意义]快速且精准地识别海量多模态数据中的价值性内容,对于促进知识传播、提升产出质量具有重要的意义。[方法/过程]基于可解释性视角聚焦知识共创中“用户+多模态知识”的双重推进机制。首先,依托BERT+BiLSTM与ResNet模型分别提炼文本与图片特征以获取多模态知识向量表示;其次,依据社会认知理论剖析用户行为,采用DeepFM捕捉交互特征间的关联生成用户向量表示;再次,借助K-BERT对文本数据嵌入知识图谱得到外部知识向量表示;最后,基于多头注意力机制融合各维度特征向量,通过动态调整权重完成价值内容的识别。[结果/结论]通过使用魅族Flyme社区数据进行实验,所构建的融合模型准确率达到88.31%,相较于其他基线模型与组合模型,评价指标均有不同程度的提升,证明嵌入外部知识并融合文本、图片与用户属性可以有效提升价值的识别效果。

[Purpose/significance]Quickly and accurately identifying valuable content from massive multi-modal data is of great significance for promoting knowledge dissemination and improving output quality.[Method/process]Focusing on the dual promotion mechanism of“user+multi-modal knowledge”in knowledge co-creation from the perspective of interpretability.Firstly,relying on BERT+BiLSTM and ResNet models to extract text and image features separately to obtain multi-modal knowledge vector representations.Secondly,based on social cognitive theory,user behavior is analyzed,and DeepFM is used to capture the correlation between interactive features and generate user vector representations.Again,using K-BERT to embed knowledge graphs into text data,external knowledge vector representations are obtained.Finally,based on the multi-head attention mechanism,various dimensional feature vectors are fused,and the recognition of value content is completed by dynamically adjusting weights.[Result/conclusion]Through experiments using Meizu Flyme community data,the accuracy of the constructed fusion model reached 88.31%.Compared with other baseline models and combination models,the evaluation indicators have improved to varying degrees,proving that embedding external knowledge and integrating text,images,and user attributes can effectively enhance the recognition effect of value.