跨模态数据融合综述

Overview of Cross-Modal Data Fusion

大数据时代下,人们热衷于在社交媒体上以图文等结合的方式发布状态数据。为了更好地整合海量社交媒体数据,首先,介绍了单模态、多模态、跨模态数据融合的最新研究进展,并在此基础上比较其优势与存在的问题;其次,在跨模态应用方面,对跨模态图文检索、跨模态推荐系统、跨模态情感分析、跨模态人机对话系统及其目前存在的问题进行了分析与总结;最后,得出跨模态数据融合能够实现数据合理有效利用的结论,并提出对跨模态数据融合未来发展的展望。

In the big data era,people are keen to release status data by means of pictures and texts on social media.In order to better integrate massive social media data,this paper first introduces the latest research progress of single-mode,multi-mode and cross-mode data fusion,and compares their advantages and existing problems on this basis.Secondly,in terms of cross-modal application,the cross-modal image and text retrieval,recommendation system,sentiment analysis,human-machine dialogue system,and the existing problems are analyzed and summarized.Finally,it is concluded that cross-modal data fusion can realize reasonable and effective data utilization,and the future development prospect of cross-modal data fusion is proposed.