标签与样本双语义增强的跨模态检索

The Cross-Modal Hash with Tag and Sample Semantic Enhancements

针对目前大多数跨模态哈希检索方法无法捕获多标签信息和特征语义更深层的语义关系信息问题,该文提出了一种标签与样本双语义增强的跨模态检索框架.首先,该框架将不同模态的高维数据映射到低维共享特征语义空间中,进行样本语义学习;其次,引入松弛变量到标签语义制约的哈希码学习函数中,通过最小化标签成对距离强化样本语义相似性哈希码学习,这样既保持了跨模态对应样本语义的关系,强化了哈希码的标签语义学习,又解决了实对称矩阵的求解及算法的收敛性问题;再次,进一步应用样本特征语义和标签语义增强哈希码的语义学习;最后,在3个常用的数据集上的实验结果表明该方法优于目前的方法.

Aiming at the problem that most cross-modal hash methods cannot capture the multi-tag information and the deeper semantic relationship information of feature semantics,a cross-modal retrieval framework with bilingual enhancement of tag and sample is proposed.The framework first decomposes different high-dimensional modal data into low-dimensional shared feature semantic space.Secondly,the hash code learning function that relaxes the variables into the tag semantic constraints is introduced to strengthen the sample semantic similarity hash code learning by minimizing the tag pair distance,which not only maintains the relationship between the cross-modal corresponding sample semantics,strengthens the tag semantic learning of the hash code,but also solves the problem of solving the real symmetric matrix and the convergence of the algorithm.Thirdly,further apply sample feature semantics and tag semantics to enhance the semantic learning of hash codes.Finally,the experimental results on three commonly used data sets show that this method is superior to the current advanced methods.