基于注意力模型的手绘图像检索方法

Sketch-based Image Retrieval Based on Attention Model

针对手绘图像检索领域中手绘图像的特征稀疏、手绘本身易于形变等问题,文中提出了一种基于注意力模型的特征提取方法,通过精确提取手绘图像中的语义特征来获得高效准确的检索结果。首先使用卷积神经网络作为提取语义特征的基础框架;然后在有监督训练的过程中引入了注意力模型机制,通过在卷积神经网络的最后一层卷积层后引入注意力结构块的方法来定位出有效的语义特征,其中注意力结构块由空间注意力结构和通道注意力结构联合组成;最后通过融合不同层次的语义特征形成最终的特征描述子,达到高精度的检索,在基准数据库Flickr15k上的实验结果表明所提方法是可行有效的。此外,在手绘图像分类任务中,提出的注意力机制大幅提高了分类精度。

To solve the problems of the sparse features and the geometric distortion of hand-drawn images in the research field of SBIR(sketch based image retrieval),a new feature extraction method based on attention model is proposed in this paper.The retrieval results can be obtained efficiently and accurately by accurately extracting the semantic features of hand-drawn images.Firstly,convolutional neural network is used as the basic framework for extracting semantic features,and then the supervised training process is carried out.Attention model mechanism is introduced to locate effective semantic features by adding attention block after the last convolution layer of the convolution neural network,and the attention block is composed of spatial attention structure and channel attention structure.Finally,the final feature descriptor is formed by the fusion of semantic features in different layers,to realize high retrieval accuracy.The experimental results on benchmark Flickr15k dataset proves the feasibility and effectiveness of the proposed method.In addition,the proposed attention model can greatly improve the classification accuracy in the task of sketch classification.