基于多特征融合的多尺度服装图像精准化检索

Accurate Retrieval of Multi scale Clothing Images Based on Multi feature Fusion

为了充分挖掘服装图像从全局到局部的多级尺度特征,同时发挥深度学习与传统特征各自在提取服装图像深层语义特征和底层特征上的优势,从而实现聚焦服装本身与服装全面特征的提取,提出基于多特征融合的多尺度服装图像精准化检索算法.首先,为了不同类型特征的有效融合,本文设计了基于特征相似性的融合公式FSF(Feature Similarity Fusion).其次,基于YOLOv3模型同时提取服装全局、主体和款式部件区域构成三级尺度图像,极大减弱背景等干扰因素的影响,聚焦服装本身.之后全局、主体和款式部件三级尺度图像分别送入三路卷积神经网络(Convolutional Neural Network,CNN)进行特征提取,每路CNN均依次进行过服装款式属性分类训练和度量学习训练,分别提高了CNN对服装款式属性特征的提取能力,以及对不同服装图像特征的辨识能力.提取的三路CNN特征使用FSF公式进行特征融合,得到的多尺度CNN融合特征则包含了服装图像从全局到主体,再到款式部件的全面特征.然后,加入款式属性预测优化特征间欧氏距离,同时抑制语义漂移,得到初步检索结果.最后,由于底层特征可以很好的对CNN提取的深层语义特征进行补充,故引入传统特征对初步检索结果的纹理、颜色等特征进行约束,通过FSF公式将多尺度CNN融合特征与传统特征相结合,进一步优化初步检索结果的排序.实验结果表明,该算法可以实现对服装从全局到款式部件区域多尺度CNN特征的充分提取,同时结合传统特征有效优化排序结果,提升检索准确率.在返回Top-20的实验中,相比于FashionNet模型准确率提升了16.4%."

In order to fully exploit the multi-scale features of clothing image from global to local,at the same time,the advantages of deep learning in extracting the deep semantic features of clothing images,and the advantages of traditional features in extracting the low-level features of clothing images should be given full play.So as to achieve the focus on clothing itself,and the extraction of comprehensive clothing features,an accurate retrieval of multi-scale clothing images based on multifeature fusion algorithm was proposed.Firstly,in order to effectively fuse different types of features,this paper designed a fusion formula FSF(Feature Similarity Fusion)based on feature similarity.Secondly,using annotation tools LableImg to annotate data set,and training YOLOv3 model,based on YOLOv3 model after training,three-level scale images of the global,main body and style parts were extracted simultaneously,the global area retained the whole clothing and human body area for background removal;The main body area was the upper-body clothing or lower-body clothing area to be retrieved,excluding the non-retrieved clothing area;The style parts area was collar,sleeve and other local areas.The influence of background and other interference factors are weakened in threelevel scale images to varying degrees,and focus on the clothing itself.After that,the three-level scale images of the global,main body and style parts were respectively sent to three Convolutional Neural Network(CNN)for feature extraction,each CNN has successively carried out clothing style attribute classification training and metric learning training,classification training of clothing style attributes improves CNN's ability to extract clothing style attributes,the metric learning by reducing the distance between features of the same category and increase the distance between features of different categories,so as to further improves the ability of different clothing image feature identification.The CNN features of three-level scale image extracted by three-channel CNN were fused using FSF formula,and the multi-scale CNN fusion features includes the overall features of the clothing image from the global to the main body,then to the style parts.Then,the style attributes prediction was added to optimize the Euclidean distance between features,in the case that the prediction results of 5 types of style attributes were more identical,the Euclidean distance between clothing images was reduced proportionally;If the prediction results of 5 types of style attributes were less the same,the Euclidean distance between clothing images should be increased proportionally.while restraining the semantic drift,and the preliminary retrieval results were obtained.Finally,because the low-level features can complement the deep semantic features extracted by CNN well,traditional features were introduced to restrict the texture and color features of the preliminary retrieval results.Through FSF formula,multi-scale CNN fusion features were combined with traditional features to further optimize the ranking of the preliminary retrieval results.The experimental results show that the algorithm can fully extract the multi-scale CNN features from the global to the style parts area,and at the same time,combine the traditional features to effectively optimize the sorting results and improve the retrieval accuracy.In the experiment of returning to Top 20,the accuracy was improved by 16.4%compared with the FashionNet model.