混合跨域神经网络的草图检索算法

Hybrid cross-domain joint network for sketch-based image retrieval

基于草图的跨域图像检索任务以手绘草图为输入,从彩色图像数据库中检索得到最相似的图像。为了在基于草图的图像检索任务中,更好地融合来自草图和彩色图像的特征,本文提出了用于草图检索任务的混合跨域神经网络,由草图特征提取分支与异构特征融合的彩色图像网络分支组成。该网络提取获得手绘草图、正负样本彩色图像及其边缘轮廓的特征表示,并将彩色图像及其草图近似图(即彩色图像的边缘轮廓)进行特征融合,作为彩色图像特征,弥补了手绘草图与彩色图像直接匹配的跨域差距。通过对网络模型的参数与网络结构等方面探索,进一步优化草图检索算法。在Flickr15K草图检索数据集上的实验结果表明,本文提出的方法优于当前其他先进的草图检索算法,在检索平均精确度这个客观指标上达到了0.5848,相比其他方法中指标最优的值提升了0.0522。

Sketch-based cross-domain image retrieval(SBIR)uses a sketch as query to retrieve the most similar image from the color image database.In this study,in order to better fuse the features from sketch and color image,a hybrid cross-domain joint network for sketch-based image retrieval was proposed,consisting of a sketch feature extraction branch and a color image heterogeneous feature fusion network branch.The network extracts the feature representations of sketch,positive and negative color image,and corresponding edge outline,and fuses the features of the color image and its sketch approximation(the edge outline of the color image)as the color image feature,which bridges the cross-domain gap between sketches and images.The network model parameters and network structure were further explored to optimize the purposed algorithm.Experiment on Flickr15K dataset shows that the proposed method performed better than other advanced image retrieval methods.The mean average retrieval accuracy of the proposed method was 0.5848,which was 0.0522 higher than the optimal value in other methods.