基于关联约束的对抗跨模态检索方法

Adversarial Cross-Modal Retrieval Based on Association Constraint

现有跨模态检索方法主要使用某一指标约束得到一个子空间,检索结果往往有差异.为了提高公共子空间的鲁棒性,文中提出基于关联约束的对抗跨模态检索方法.对抗约束通过混淆判别器使其无法分辨子空间特征来自哪个模态,从而提升不同模态特征的一致性.关联约束用于增强投影子空间关联程度.三元组约束同时考虑不同模态同一语义、相同模态不同语义样本之间的结构信息.在数据集上的实验表明文中方法的检索性能得到有效提升.

In the existing cross-modal retrieval methods,retrieval results are obtained via the subspace acquired by a certain index constraint such as distance or similarity.Since the subspaces are learned with different index constraints,retrieval results are different.To improve the robustness of common subspace,a method for adversarial cross-modal retrieval based on association constraint is proposed.The consistency of different modality features is improved by the adversarial constraint to make the discriminator in the constraint unable to distinguish which modality the subspace features come from.The association of different modality features is enhanced by the association constraint.The structural information between example pairs with the same semantics of different modalities and different semantics of the same modality is taken into account by the triple loss constraint.Experimental results on datasets show that the proposed method is more effective than other cross-modal retrieval methods.