基于多头注意力网络的无监督跨媒体哈希检索

Unsupervised cross-media Hashing retrieval based on multi-head attention network

跨媒体哈希检索将不同媒体数据编码到公共二值哈希空间中,从而可以有效地测量不同模态样本之间的相关性.为了进一步提高检索性能,提出基于多头注意力网络的无监督跨媒体哈希检索方法.首先,利用多头注意力网络生成哈希码矩阵,使图像和文本能获得更好的匹配.其次,构造一个辅助相似度矩阵,用以整合来自不同模态的原始邻域信息.通过辅助相似度矩阵与哈希码矩阵的协同学习,能够捕获不同模态之间和相同模态内部的潜在联系.此外,设计了两种损失函数训练网络模型,并使用批量归一化和更换哈希码生成函数的策略对模型进行优化,使模型的训练速度得到大幅提升.在3个数据集上的实验表明,本方法的平均性能比目前国际上先进的无监督方法有显著提升,充分证明了本方法的有效性和优越性.

The cross-media Hash retrieval encodes different media data into a common binary Hash space,which can effectively measure the correlation between different modal samples.In order to further improve the retrieval performance,this paper proposes an unsupervised cross-media Hash retrieval method based on multihead attention network.First,we use a multi-head attention network to generate a Hash code matrix,which makes the images and texts match better.Second,an auxiliary similarity matrix is constructed to integrate the original neighborhood information from different modalities.Through the collaborative learning of auxiliary similarity matrix and Hash code matrix,our method can capture the potential correlations between different modalities and within the same modality.In addition,we design two loss functions to train the model,and adopt strategies of batch normalization and replacing Hash code generation functions to optimize the model,which greatly improves the training speed of the model.Experiments on three datasets show that the average performance of our method is significantly higher than many state-of-the-art unsupervised methods,which fully proves the effectiveness and superiority of our method.