基于语义一致与多级相似性的跨模态哈希检索

Semantic-Consistent and Multilayer Similarity Based Cross-Modal Hashing Retrieval

【目的】通过关联标签学习丰富的语义表示,在哈希码中保留更多辨别信息,同时考虑到跨模态语义相似性,保持不同模态间的相关性,更好地弥合模态差距。【方法】在多标签的关联约束下,挖掘不同模态的公共语义信息以及隐藏的类语义结构,采用高级语义与低级语义联合相似性度量的非对称学习框架,进而量化获得更具强鉴别性的哈希码。【结果】在MIRFlickr-25K、IAPR TC-12和NUS-WIDE三个多模态基准数据集上,本文方法与7种方法进行实验对比,在5种不同码长情况下,本文方法的平均MAP值比基准模型的最高值分别提升2.1%、5.8%和2.1%。【局限】所提出方法对多标签数据集更具适用性,对单标签数据的语义相关性挖掘尚有欠缺。【结论】所提方法保持样本和类语义结构的一致性,并且充分挖掘内在模态特征,有效提高了检索性能。

[Objective]This paper proposes a retrieval method for learning the rich semantic representations through associated labels and retaining more discriminative information in hash codes.It considers cross-modal semantic similarity,maintains relevance between different modalities,and better bridges the modal gaps.[Methods]Under the constraint of multi-label association,we explored the common semantic information and the hidden class semantic structure of different modalities.Then,we adopted the asymmetric learning framework for joint similarity measurement of high-level and low-level semantics,thereby quantifying to obtain more discriminative hash codes.[Results]We conducted experiments on three multi-modal benchmark datasets:MIRFlickr-25K,IAPR TC-12,and NUS-WIDE,comparing the proposed method with seven other methods.Under five different code lengths,the average MAP values of the proposed method were 2.1%,5.8%,and 2.1%higher than the baseline's maximum value,respectively.[Limitations]The proposed method is more applicable to multi-label datasets and has some deficiencies in mining the semantic relevance of single-label data.[Conclusions]The proposed method maintains the consistency of sample and class semantic structures,fully explores the inherent modal features,and effectively improves retrieval performance.