Image Tagging by Semantic Neighbor Learning Using User-Contributed Social Image Datasets

The explosive increase in the number of images on the Internet has brought with it the great challenge of how to effectively index, retrieve, and organize these resources. Assigning proper tags to the visual content is key to the success of many applications such as image retrieval and content mining. Although recent years have witnessed many advances in image tagging, these methods have limitations when applied to high-quality and large-scale training data that are expensive to obtain. In this paper, we propose a novel semantic neighbor learning method based on user-contributed social image datasets that can be acquired from the Web＇s inexhaustible social image content. In contrast to existing image tagging approaches that rely on high-quality image-tag supervision, we acquire weak supervision of our neighbor learning method by progressive neighborhood retrieval from noisy and diverse user-contributed image collections. The retrieved neighbor images are not only visually alike and partially correlated but also semantically related. We offer a step-by-step and easy-to-use implementation for the proposed method. Extensive experimentation on several datasets demonstrates that the performance of the proposed method significantly outperforms others.

Multi-task multi-label multiple instance learning

For automatic object detection tasks,large amounts of training images are usually labeled to achieve more reliable training of the object classifiers;this is cost-expensive since it requires hiring professionals to label large-scale training images.When a large number of object classes come into view,the issue of obtaining a large enough amount of the labeled training images becomes more critical.There are three potential solutions to reduce the burden for image labeling:(1) allowing people to provide the object labels loosely at the image level rather than at the object level(e.g.,loosely-tagged images without identifying the exact object locations in the images) ;(2) harnessing large-scale collaboratively-tagged images that are available on the Internet;and,(3) developing new machine learning algorithms that can directly leverage large-scale collaboratively-or loosely-tagged images for achieving more eective training of a large number of object classifiers.Based on these observations,a multi-task multi-label multiple instance learning(MTML-MIL) algorithm is developed in this paper by leveraging both inter-object correlations and large-scale loosely-labeled images for object classifier training.By seamlessly integrating multi-task learning,multi-label learning,and multiple instance learning,our MTML-MIL algorithm can achieve more accurate training of a large number of inter-related object classifiers(where an object network is constructed for determining the inter-related learning tasks directly in the feature space rather than in the label space) .Our experimental results have shown that our MTML-MIL algorithm can achieve higher detection accuracy rates for automatic object detection.