基于手绘轮廓图的移动端图像检索

Hand-Sketching Contour Based Image Retrieval on Mobile Device

针对传统利用图像特征信息进行图像检索中难以从语义层次上理解图像相似性的问题,基于深度学习框架,提出一种结合类别分类和精确特征匹配的基于手绘轮廓图的移动端图像检索方法.首先在预处理阶段建立具有输入层、隐藏层以及Softmax输出层的神经网络分类模型,并利用训练数据集对模型进行训练,使其不断优化网络结构权值,实现输入图像的分类预测并提取分类图像标签;然后利用VGG16模型与ResNet50模型分别提取各个分类图像集下的精确特征,得到精确特征向量;最后将归一化并经组合后的特征向量与各个分类图像标签建立映射关系,实现移动端图像检索.采用移动端-服务器架构,用户在移动端输入手绘轮廓图后,系统进行自动预处理并与图像服务器实现交互,图像服务器进行分类预测和精确特征匹配得到检索结果,移动端展示最终检索结果.基于Keras深度学习开发框架,结合VGG16模型与ResNet50模型,实验结果表明,该方法能够根据手绘轮廓图高效、便捷地检索得到目标图像.

Traditional low level features based image retrieval techniques usually have some difficulties on understanding the image similarity from the high level semantic information.To overcome this issue,under the deep-learning framework,a novel hand-sketching contour based image retrieval method on mobile devices is presented in this paper by combining image classification and exact retrieval steps.Firstly,a neural network of image classification is built including the input layer,the hidden layer and the Softmax output layer,which would be trained by image dataset.It will tell which class the input contour image belongs to after training and gets the classification label.Secondly,the VGG16 model and ResNet50 model can be loaded,by which the exact image features of each class can be extracted.Finally,a map between the combinational feature vectors and the image classification labels can be built for the purpose of image retrieval on mobile devices.Based on the C/S structure,the proposed image retrieval system would exchange data with server automatically after mobile device got the contours of input hand-sketching images.And according to the feature index and network model,the server would return the retrieval results.Using the VGG16 model and ResNet50 model loaded with Keras framework,our approach can retrieve images generated by hand-sketching contours efficiently and conveniently.