摘要
【目的】通过关联标签学习丰富的语义表示,在哈希码中保留更多辨别信息,同时考虑到跨模态语义相似性,保持不同模态间的相关性,更好地弥合模态差距。【方法】在多标签的关联约束下,挖掘不同模态的公共语义信息以及隐藏的类语义结构,采用高级语义与低级语义联合相似性度量的非对称学习框架,进而量化获得更具强鉴别性的哈希码。【结果】在MIRFlickr-25K、IAPR TC-12和NUS-WIDE三个多模态基准数据集上,本文方法与7种方法进行实验对比,在5种不同码长情况下,本文方法的平均MAP值比基准模型的最高值分别提升2.1%、5.8%和2.1%。【局限】所提出方法对多标签数据集更具适用性,对单标签数据的语义相关性挖掘尚有欠缺。【结论】所提方法保持样本和类语义结构的一致性,并且充分挖掘内在模态特征,有效提高了检索性能。
[Objective]This paper proposes a retrieval method for learning the rich semantic representations through associated labels and retaining more discriminative information in hash codes.It considers cross-modal semantic similarity,maintains relevance between different modalities,and better bridges the modal gaps.[Methods]Under the constraint of multi-label association,we explored the common semantic information and the hidden class semantic structure of different modalities.Then,we adopted the asymmetric learning framework for joint similarity measurement of high-level and low-level semantics,thereby quantifying to obtain more discriminative hash codes.[Results]We conducted experiments on three multi-modal benchmark datasets:MIRFlickr-25K,IAPR TC-12,and NUS-WIDE,comparing the proposed method with seven other methods.Under five different code lengths,the average MAP values of the proposed method were 2.1%,5.8%,and 2.1%higher than the baseline's maximum value,respectively.[Limitations]The proposed method is more applicable to multi-label datasets and has some deficiencies in mining the semantic relevance of single-label data.[Conclusions]The proposed method maintains the consistency of sample and class semantic structures,fully explores the inherent modal features,and effectively improves retrieval performance.
作者
刘媛媛
王晓燕
张雨欣
朱路
Liu Yuanyuan;Wang Xiaoyan;Zhang Yuxin;Zhu Lu(School of Information Engineering,East China Jiaotong University,Nanchang 330013,China)
出处
《数据分析与知识发现》
EI
CSCD
北大核心
2024年第7期89-102,共14页
Data Analysis and Knowledge Discovery
基金
江西省教育科学“十四五”规划课题基金项目(项目编号:22YB067)的研究成果之一。
关键词
跨模态检索
监督学习
多级相似
哈希
Cross-Modal Retrieval
Supervised Learning
Multi-Level Similarity
Hashing