基于哈希算法的异构多模态数据检索研究

Study on Heterogeneous Multimodal Data Retrieval Based on Hash Algorithm

随着大数据时代的发展,网络上的文本、图像、视频、音频等异构多模态数据呈指数级增长。在海量数据中进行异构多模态数据的检索,成为了热门的研究方向。但是,异构多模态数据检索面临两大挑战:1)数据存在“语义鸿沟”,即如何表达异构多模态数据之间的相似性;2)在海量数据中,如何进行准确高效的检索。针对哈希检索算法忽略了异构多模态数据之间语义一致性的问题,文中提出了一种基于CCA(典型相关性分析)语义一致性的哈希检索算法(CCA-SCH)。该算法为了保持模态内的语义一致性,分别生成文本和图像数据的语义模型;为了保持模态间的语义一致性,通过CCA算法融合文本和图像语义,生成最大相关矩阵;同时引入2,ρ范式来减少原始数据集的噪声和冗余信息,使哈希函数具有更好的鲁棒性。实验结果表明,CCA-SCH算法在实验数据集上的均值平均准确率(Map)相较于基准算法提升了10%以上,体现了该算法更好的检索性能。

The development of the era of big data has resulted in an exponentially growing of Internet heterogeneous multimodal data including text,images,video and audio.Therefore,heterogeneous multimodal data retrieval has become a hot direction in big data research.However,heterogeneous multimodal data retrieval encounters two major challenges.The first challenge is how to express the similarity between heterogeneous data while there is a “semantic gap”.The second challenge is how to achieve accurate and efficient retrieval in massive data.To solve the problem that the hash retrieval algorithm ignores semantic similarity of heterogeneous multimodal data,this paper proposed a hash retrieval algorithm based on canonical correlation analysis-semantic consistency,named CCA-SCH.In order to keep semantic consistency within the modality,the CCA-SCH algorithm separately generates semantic models of text and image data.In order to keep semantic consistency between modalities,the CCA algorithm is used to fuse semantics of text and image data to generate the maximum correlation matrix.At the same time,the paradigm 2,ρ is introduced to overcome the noise and redundant information of original datasets,so that the hash function has better robustness.Experiment results show that the mean average precision(Map) of CCA-SCH algorithm is increased by over 10% compared to benchmark algorithms’ performances on experimental data sets,which embodies the better retrieval ability of proposed algorithm.