Combining Generative/Discriminative Learning for Automatic Image Annotation and Retrieval

In order to bridge the semantic gap exists in image retrieval, this paper propose an approach combining generative and discriminative learning to accomplish the task of automatic image annotation and retrieval. We firstly present continuous probabilistic latent semantic analysis (PLSA) to model continuous quantity. Furthermore, we propose a hybrid framework which employs continuous PLSA to model visual features of images in generative learning stage and uses ensembles of classifier chains to classify the multi-label data in discriminative learning stage. Since the framework combines the advantages of generative and discriminative learning, it can predict semantic annotation precisely for unseen images. Finally, we conduct a series of experiments on a standard Corel dataset. The experiment results show that our approach outperforms many state-of-the-art approaches.

Personalized Web Image Retrieval Based on User Interest Model

The traditional search engines don’t consider that the users interest are different, and they don’t provide personalized retrieval service, so the retrieval efficiency is not high. In order to solve the problem, a method for personalized web image retrieval based on user interest model is proposed. Firstly, the formalized definition of user interest model is provided. Then the user interest model combines the methods of explicit tracking and implicit tracking to improve user’s interest information and provide personalized web image retrieval. Experimental results show that the user interest model can be successfully applied in web image retrieval.

Semi-supervised learning based probabilistic latent semantic analysis for automatic image annotation

In recent years,multimedia annotation problem has been attracting significant research attention in multimedia and computer vision areas,especially for automatic image annotation,whose purpose is to provide an efficient and effective searching environment for users to query their images more easily. In this paper,a semi-supervised learning based probabilistic latent semantic analysis( PLSA) model for automatic image annotation is presenred. Since it's often hard to obtain or create labeled images in large quantities while unlabeled ones are easier to collect,a transductive support vector machine( TSVM) is exploited to enhance the quality of the training image data. Then,different image features with different magnitudes will result in different performance for automatic image annotation. To this end,a Gaussian normalization method is utilized to normalize different features extracted from effective image regions segmented by the normalized cuts algorithm so as to reserve the intrinsic content of images as complete as possible. Finally,a PLSA model with asymmetric modalities is constructed based on the expectation maximization( EM) algorithm to predict a candidate set of annotations with confidence scores. Extensive experiments on the general-purpose Corel5k dataset demonstrate that the proposed model can significantly improve performance of traditional PLSA for the task of automatic image annotation.

Semantic image annotation based on GMM and random walk model

Automatic image annotation has been an active topic of research in computer vision and pattern recognition for decades.A two stage automatic image annotation method based on Gaussian mixture model(GMM) and random walk model(abbreviated as GMM-RW) is presented.To start with,GMM fitted by the rival penalized expectation maximization(RPEM) algorithm is employed to estimate the posterior probabilities of each annotation keyword.Subsequently,a random walk process over the constructed label similarity graph is implemented to further mine the potential correlations of the candidate annotations so as to capture the refining results,which plays a crucial role in semantic based image retrieval.The contributions exhibited in this work are multifold.First,GMM is exploited to capture the initial semantic annotations,especially the RPEM algorithm is utilized to train the model that can determine the number of components in GMM automatically.Second,a label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity of images associated with the corresponding labels,which is able to avoid the phenomena of polysemy and synonym efficiently during the image annotation process.Third,the random walk is implemented over the constructed label graph to further refine the candidate set of annotations generated by GMM.Conducted experiments on the standard Corel5 k demonstrate that GMM-RW is significantly more effective than several state-of-the-arts regarding their effectiveness and efficiency in the task of automatic image annotation.

Exploiting PLSA model and conditional random field for refining image annotation

This paper presents a new method for refining image annotation by integrating probabilistic la- tent semantic analysis （PLSA） with conditional random field （CRF）. First a PLSA model with asymmetric modalities is constructed to predict a candidate set of annotations with confidence scores, and then model semantic relationship among the candidate annotations by leveraging conditional ran- dom field. In CRF, the confidence scores generated lay the PLSA model and the Fliekr distance be- tween pairwise candidate annotations are considered as local evidences and contextual potentials re- spectively. The novelty of our method mainly lies in two aspects ： exploiting PLSA to predict a candi- date set of annotations with confidence scores as well as CRF to further explore the semantic context among candidate annotations for precise image annotation. To demonstrate the effectiveness of the method proposed in this paper, an experiment is conducted on the standard Corel dataset and its re- sults are ＇compared favorably with several state-of-the-art approaches.

Fusing PLSA model and Markov random fields for automatic image annotation

A novel image auto-annotation method is presented based on probabilistic latent semantic analysis(PLSA) model and multiple Markov random fields(MRF).A PLSA model with asymmetric modalities is first constructed to estimate the joint probability between images and semantic concepts,then a subgraph is extracted served as the corresponding structure of Markov random fields and inference over it is performed by the iterative conditional modes so as to capture the final annotation for the image.The novelty of our method mainly lies in two aspects:exploiting PLSA to estimate the joint probability between images and semantic concepts as well as multiple MRF to further explore the semantic context among keywords for accurate image annotation.To demonstrate the effectiveness of this approach,an experiment on the Corel5 k dataset is conducted and its results are compared favorably with the current state-of-the-art approaches.

Learning a hierarchical image manifold for Web image classification

Image classification is an essential task in content-based image retrieval.However,due to the semantic gap between low-level visual features and high-level semantic concepts,and the diversification of Web images,the performance of traditional classification approaches is far from users' expectations.In an attempt to reduce the semantic gap and satisfy the urgent requirements for dimensionality reduction,high-quality retrieval results,and batch-based processing,we propose a hierarchical image manifold with novel distance measures for calculation.Assuming that the images in an image set describe the same or similar object but have various scenes,we formulate two kinds of manifolds,object manifold and scene manifold,at different levels of semantic granularity.Object manifold is developed for object-level classification using an algorithm named extended locally linear embedding(ELLE) based on intra-and inter-object difference measures.Scene manifold is built for scene-level classification using an algorithm named locally linear submanifold extraction(LLSE) by combining linear perturbation and region growing.Experimental results show that our method is effective in improving the performance of classifying Web images.

Simultaneous image classification and annotation based on probabilistic model

The paper proposes a novel probabilistic generative model for simultaneous image classification and annotation. The model considers the fact that the category information can provide valuable information for image annotation. Once the category of an image is ascertained, the scope of annotation words can be narrowed, and the probability of generating irrelevant annotation words can be reduced. To this end, the idea that annotates images according to class is introduced in the model. Using variational methods, the approximate inference and parameters estimation algorithms of the model are derived, and efficient approximations for classifying and annotating new images are also given. The power of our model is demonstrated on two real world datasets： a 1 600-images LabelMe dataset and a 1 791-images UIUC-Sport dataset. The experiment results show that the classification performance is on par with several state-of-the-art classification models, while the annotation performance is better than that of several state-of-the-art annotation models.

Effective and Efficient Multi-Facet Web Image Annotation

The vast amount of images available on the Web request for an effective and efficient search service to help users find relevant images. The prevalent way is to provide a keyword interface for users to submit queries. However, the amount of images without any tags or annotations are beyond the reach of manual efforts. To overcome this, automatic image annotation techniques emerge, which are generally a process of selecting a suitable set of tags for a given image without user intervention. However, there are three main challenges with respect to Web-scale image annotation： scalability, noise- resistance and diversity. Scalability has a twofold meaning： first an automatic image annotation system should be scalable with respect to billions of images on the Web; second it should be able to automatically identify several relevant tags among a huge tag set for a given image within seconds or even faster. Noise-resistance means that the system should be robust enough against typos and ambiguous terms used in tags. Diversity represents that image content may include both scenes and objects, which are further described by multiple different image features constituting different facets in annotation. In this paper, we propose a unified framework to tackle the above three challenges for automatic Web image annotation. It mainly involves two components： tag candidate retrieval and multi-facet annotation. In the former content-based indexing and concept-based eodebook are leveraged to solve scalability and noise-resistance issues. In the latter the joint feature map has been designed to describe different facets of tags in annotations and the relations between these facets. Tag graph is adopted to represent tags in the entire annotation and the structured learning technique is employed to construct a learning model on top of the tag graph based on the generated joint feature map. Millions of images from Flickr are used in our evaluation. Experimental results show that we have achieved 33% performance improvements compared with those single facet approaches in terms of three metrics： precision, recall and F1 score.

Use Genetic Programming to Rank Web Images

Web image retrieval is a challenging task. One central problem of web image retrieval is to rank a set of images according to how well they meet the user information need. The problem of learning to rank has inspired numerous approaches to resolve it in the text information retrieval, related work for web image retrieval, however, are still limited. We focus on the problem of learning to rank images for web image retrieval, and propose a novel ranking model, which employs a genetic programming architecture to automatically generate an effective ranking function, by combining various types of evidences in web image retrieval, including text information, image visual content features, link structure analysis and temporal information. The experimental results show that the proposed algorithms are capable of learning effective ranking functions for web image retrieval. Significant improvement in relevancy obtained, in comparison to some other well-known ranking techniques, in terms of MAP, NDCG@n and D@n.

Triplet Label Based Image Retrieval Using Deep Learning in Large Database

Recent days,Image retrieval has become a tedious process as the image database has grown very larger.The introduction of Machine Learning(ML)and Deep Learning(DL)made this process more comfortable.In these,the pair-wise label similarity is used tofind the matching images from the database.But this method lacks of limited propose code and weak execution of misclassified images.In order to get-rid of the above problem,a novel triplet based label that incorporates context-spatial similarity measure is proposed.A Point Attention Based Triplet Network(PABTN)is introduced to study propose code that gives maximum discriminative ability.To improve the performance of ranking,a corre-lating resolutions for the classification,triplet labels based onfindings,a spatial-attention mechanism and Region Of Interest(ROI)and small trial information loss containing a new triplet cross-entropy loss are used.From the experimental results,it is shown that the proposed technique exhibits better results in terms of mean Reciprocal Rank(mRR)and mean Average Precision(mAP)in the CIFAR-10 and NUS-WIPE datasets.

Indexing of Content-Based Image Retrieval System with Image Understanding Approach

This paper presents a novel efficient semantic image classification algorithm for high-level feature indexing of high-dimension image database. Experiments show that the algorithm performs well. The size of the train set and the test set is 7 537 and 5 000 respectively. Based on this theory, another ground is built with 12,000 images, which are divided into three classes: city, landscape and person, the total result of the classifications is 88.92%, meanwhile, some preliminary results are presented for image understanding based on semantic image classification and low level features. The groundtruth for the experiments is built with the images from Corel database, photos and some famous face databases.

Towards More Efficient Image Web Search

With the flood of information on the Web, it has become increasingly necessary for users to utilize automated tools in order to find, extract, filter, and evaluate the desired information and knowledge discovery. In this research, we will present a preliminary discussion about using the dominant meaning technique to improve Google Image Web search engine. Google search engine analyzes the text on the page adjacent to the image, the image caption and dozens of other factors to determine the image content. To improve the results, we looked for building a dominant meaning classification model. This paper investigated the influence of using this model to retrieve more efficient images, through sequential procedures to formulate a suitable query. In order to build this model, the specific dataset related to an application domain was collected;K-means algorithm was used to cluster the dataset into K-clusters, and the dominant meaning technique is used to construct a hierarchy model of these clusters. This hierarchy model is used to reformulate a new query. We perform some experiments on Google and validate the effectiveness of the proposed approach. The proposed approach is improved for in precision, recall and F1-measure by 57%, 70%, and 61% respectively.

Multimodal Fused Deep Learning Networks for Domain Specific Image Similarity Search

The exponential increase in data over the past fewyears,particularly in images,has led to more complex content since visual representation became the new norm.E-commerce and similar platforms maintain large image catalogues of their products.In image databases,searching and retrieving similar images is still a challenge,even though several image retrieval techniques have been proposed over the decade.Most of these techniques work well when querying general image databases.However,they often fail in domain-specific image databases,especially for datasets with low intraclass variance.This paper proposes a domain-specific image similarity search engine based on a fused deep learning network.The network is comprised of an improved object localization module,a classification module to narrow down search options and finally a feature extraction and similarity calculation module.The network features both an offline stage for indexing the dataset and an online stage for querying.The dataset used to evaluate the performance of the proposed network is a custom domain-specific dataset related to cosmetics packaging gathered from various online platforms.The proposed method addresses the intraclass variance problem with more precise object localization and the introduction of top result reranking based on object contours.Finally,quantitative and qualitative experiment results are presented,showing improved image similarity search performance.

Searching of Images Based on Content Using Blobs

Now a day’s image searching is still a challenging problem in content based image retrieval (CBIR) system. Most system operates on all images without pre-sorting the images. The image search result contains many unrelated image. The aim of this research is to propose a new method for content based image indexing and research based on blobs feature extraction and existing edges in the image and classification of image to different type and to search image which are similar the given research.

一种自适应的Web图像语义自动标注方法

提出了一种自适应的Web图像语义自动标注方法:首先利用Web标签资源自动获取训练数据;然后通过带约束的分段惩罚加权回归模型将关联文本权重分布自适应学习和先验知识约束有机地结合在一起,实现Web图像语义的自动标注.在4000幅从Web获得的图像数据集上的实验结果验证了该文自动获取训练集方法以及Web图像语义标注方法的有效性.

应用Web标注技术的建筑图像语义采集方法

为解决建筑师难以快速地从互联网中检索到符合创作需求的建筑图像的问题,提出了应用Web标注技术的建筑图像语义采集方法.首先,从建筑学角度界定了建筑图像及建筑图像语义的概念和类型;其次,给出了该方法的总体框架和操作流程;最后,以著名建筑网站为例进行案例演示,验证了该方法的可行性和有效性.操作流程细分为3个步骤,以人工添加和在线学习的方式建立建筑语义词典;运用数据采集软件,从建筑图像所在网页中分别采集图像名称、图像注释、图像周围文本、所在网页标题、所在网页正文、图像超链接网页标题6项图像相关文本;根据图像语义提取规则,从上述文本中提取建筑图像语义,与图像文件建立关联后存储到建筑图像数据库.案例检验结果表明,该方法是可行的,具有较强的操作性,能够自动、批量地从互联网中下载建筑图像,并采集图像名称、图像类别、图像主题、项目名称、项目类型等30多项特征,有效地克服了建筑图像查询效率较低的问题,进而提升了建筑师运用互联网图像进行创作的能力.

使用加权图像标注改进Web图像检索

为了提高Web图像的检索质量,提出了一种融合文本关键字和图像视觉内容的Web图像检索方法.通过改进的图像自动标注模型,将Web图像本身所蕴含的低层视觉特征映射到图像高层语义特征,即图像文本标注;再将词汇相似性计算技术作为语义信息的度量手段,将图像文本标注转换成带有权重的文本标注;利用贝叶斯推理网检索模型内在的多信息融合能力,将带权重的Web图像文本标注特征和Web文档中的文本信息无缝地融合在一起实现Web图像检索.实验结果表明,将Web中的文本关键字和Web图像视觉内容融合起来可在一定程度上提高Web图像检索质量.

结合Web背景知识的图像语义标注

针对基于内容的图像语义标注方法中,相同或相近视觉特征对应语义可能不同的情况,提出了一个结合Web背景知识的图像语义关联模型,利用从Web页面中提取的与图像相关的属性,计算Web图像与标注关键词间的语义相关性,确定待标注Web图像的语义,实验表明该方法具有较好的性能。

一种适合Web图像检索的图像降维算法研究

本文对现有 Web 图像检索技术现状进行了归纳,分析阐明了图像降维算法在基于内容的 Web 图像检索技术中的地位和作用。在介绍了几种经典图像降维方法后,重点介绍了国外近来提出的基于拉普拉斯特征值映射(LE)的图像降维算法。针对 Web 环境下图像检索必须进行剧烈降维的特点,本文进而对基于 KL 变换的主成分分析(PCA)算法和基于 LE 的图像降维算法进行了实验分析和比较。实验结果表明:对于需要进行维数剧烈缩减的 Web图像检索来说,基于 LE 的图像降维算法可以获得最佳的效果。