Image Retrieval with Text Manipulation by Local Feature Modification

The demand for image retrieval with text manipulation exists in many fields, such as e-commerce and Internet search. Deep metric learning methods are used by most researchers to calculate the similarity between the query and the candidate image by fusing the global feature of the query image and the text feature. However, the text usually corresponds to the local feature of the query image rather than the global feature. Therefore, in this paper, we propose a framework of image retrieval with text manipulation by local feature modification(LFM-IR) which can focus on the related image regions and attributes and perform modification. A spatial attention module and a channel attention module are designed to realize the semantic mapping between image and text. We achieve excellent performance on three benchmark datasets, namely Color-Shape-Size(CSS), Massachusetts Institute of Technology(MIT) States and Fashion200K(+8.3%, +0.7% and +4.6% in R@1).

Research on the Pedestrian Re-Identification Method Based on Local Features and Gait Energy Images

The appearance of pedestrians can vary greatly from image to image,and different pedestrians may look similar in a given image.Such similarities and variabilities in the appearance and clothing of individuals make the task of pedestrian re-identification very challenging.Here,a pedestrian re-identification method based on the fusion of local features and gait energy image(GEI)features is proposed.In this method,the human body is divided into four regions according to joint points.The color and texture of each region of the human body are extracted as local features,and GEI features of the pedestrian gait are also obtained.These features are then fused with the local and GEI features of the person.Independent distance measure learning using the cross-view quadratic discriminant analysis(XQDA)method is used to obtain the similarity of the metric function of the image pairs,and the final similarity is acquired by weight matching.Evaluation of experimental results by cumulative matching characteristic(CMC)curves reveals that,after fusion of local and GEI features,the pedestrian re-identification effect is improved compared with existing methods and is notably better than the recognition rate of pedestrian re-identification with a single feature.

Object tracking method based on joint global and local feature descriptor of 3D LIDAR point cloud

To fully describe the structure information of the point cloud when the LIDAR-object distance is long,a joint global and local feature(JGLF)descriptor is constructed.Compared with five typical descriptors,the object recognition rate of JGLF is higher when the LIDAR-object distances change.Under the situation that airborne LIDAR is getting close to the object,the particle filtering(PF)algorithm is used as the tracking frame.Particle weight is updated by comparing the difference between JGLFs to track the object.It is verified that the proposed algorithm performs 13.95%more accurately and stably than the basic PF algorithm.

A review of object representation based on local features

Object representation based on local features is a topical subject in the domain of image understanding and computer vision. We discuss the defects of global features in present methods and the advantages of local features in object recognition, and briefly explore state-of-the-art recognition methods using local features, especially the main approaches of local feature extraction and object representation. To clearly explain these methods, the problem of local feature extraction is divided into feature region detection, feature region description, and feature space optimization. The main components and merits of these steps are presented. Technologies for object presentation are classified into three types: vector space, sliding window, and structure relationship models. Future development trends are discussed briefly.

Local features and manifold ranking coupled method for sketch-based 3D model retrieval

3D model retrieval virtual reality applications. In can benefit many downstream this paper, we propose a new sketch-based 3D model retrieval framework by coupling local features and manifold ranking. At technical fronts, we exploit spatial pyramids based local structures to facilitate the efficient construction of feature descriptors. Meanwhile, we propose an improved manifold ranking method, wherein all the categories between arbitrary model pairs will be taken into account. Since the smooth and detail-preserving line drawings of 3D model are important for sketch-based 3D model retrieval, the Difference of Gaussians （DOG） method is employed to extract the line drawings over the projected depth images of 3D model, and Bezier Curve is then adopted to further optimize the extracted line drawing. On that basis, we develop a 3D model retrieval engine to verify our method. We have conducted extensive experiments over various public benchmarks, and have made comprehensive comparisons with some state-of-the-art 3D retrieval methods. All the evaluation results based on the widely-used indicators prove the superiority of our method in accuracy, reliability, robustness, and versatility.

Volumetric magnetic resonance imaging classification for Alzheimer＇s disease based on kernel density estimation of local features

Background The classification of Alzheimer＇s disease （AD） from magnetic resonance imaging （MRI） has been challenged by lack of effective and reliable biomarkers due to inter-subject variability. This article presents a classification method for AD based on kernel density estimation （KDE） of local features. Methods First, a large number of local features were extracted from stable image blobs to represent various anatomical patterns for potential effective biomarkers. Based on distinctive descriptors and locations, the local features were robustly clustered to identify correspondences of the same underlying patterns. Then, the KDE was used to estimate distribution parameters of the correspondences by weighting contributions according to their distances. Thus, biomarkers could be reliably quantified by reducing the effects of further away correspondences which were more likely noises from inter-subject variability. Finally, the Bayes classifier was applied on the distribution parameters for the classification of AD. Results Experiments were performed on different divisions of a publicly available database to investigate the accuracy and the effects of age and AD severity. Our method achieved an equal error classification rate of 0.85 for subject aged 60-80 years exhibiting mild AD and outperformed a recent local feature-based work regardless of both effects. Conclusions We proposed a volumetric brain MRI classification method for neurodegenerative disease based on statistics of local features using KDE. The method may be potentially useful for the computer-aided diagnosis in clinical settings.

A monocular visual SLAM system augmented by lightweight deep local feature extractor using in-house and low-cost LIDAR-camera integrated device

Simultaneous Localization and Mapping(SLAM)has been widely used in emergency response,self-driving and city-scale 3D mapping and navigation.Recent deep-learning based feature point extractors have demonstrated superior performance in dealing with the complex environmental challenges(e.g.extreme lighting)while the traditional extractors are struggling.In this paper,we have successfully improved the robustness and accuracy of a monocular visual SLAM system under various complex scenes by adding a deep learning based visual localization thread as an augmentation to the visual SLAM framework.In this thread,our feature extractor with an efficient lightweight deep neural network is used for absolute pose and scale estimation in real time using the highly accurate georeferenced prior map database at 20cm geometric accuracy created by our in-house and low-cost LiDAR and camera integrated device.The closed-loop error provided by our SLAM system with and without this enhancement is 1.03m and 18.28m respectively.The scale estimation of the monocular visual SLAM is also significantly improved(0.01 versus 0.98).In addition,a novel camera-LiDAR calibration workflow is also provided for large-scale 3D mapping.This paper demonstrates the application and research potential of deep-learning based vision SLAM with image and LiDAR sensors.

Robust Image Watermarking Using Local Invariant Features and Independent Component Analysis

This paper proposes a novel robust image watermarking scheme for digital images using local invariant features and Independent Component Analysis （ICA）. Most present watermarking algorithms are unable to resist geometric distortions that desynchronize the location. The method we propose here is robust to geometric attacks. In order to resist geometric distortions, we use a local invariant feature of the image called the scale invariant feature transform, which is invariant to translation and scaling distortions. The watermark is inserted into the circular patches generated by scale-invariant key point extractor. Rotation invariance is achieved using the translation property of the polar-mapped circular patches. Our method belongs to the blind watermark category, because we use Independent Component Analysis for detection that does not need the original image during detection. Experimental results show that our method is robust against geometric distortion attacks as well as signal-processing attacks.

Tire Defect Detection Using Local and Global Features

In this paper, we present a tire defect detection algorithm based on sparse representation. The dictionary learned from reference images can efficiently represent the test image. As the representation coefficients of normal images have a specific distribution, the local feature can be estimate by comparing representation coefficient distribution. Meanwhile, a coding length is used to measure the global features of representation coefficients. The tire defect is located by both these local and global features. Experimental results demonstrate that the proposed method can accurately detect and locate the tire defects.

High speed robust image registration and localization using optimized algorithm and its performances evaluation

Local invariant algorithm applied in downward-looking image registration,usually computes the camera＇s pose relative to visual landmarks.Generally,there are three requirements in the process of image registration when using these approaches.First,the algorithm is apt to be influenced by illumination.Second,algorithm should have less computational complexity.Third,the depth information of images needs to be estimated without other sensors.This paper investigates a famous local invariant feature named speeded up robust feature（SURF）,and proposes a highspeed and robust image registration and localization algorithm based on it.With supports from feature tracking and pose estimation methods,the proposed algorithm can compute camera poses under different conditions of scale,viewpoint and rotation so as to precisely localize object＇s position.At last,the study makes registration experiment by scale invariant feature transform（SIFT）,SURF and the proposed algorithm,and designs a method to evaluate their performances.Furthermore,this study makes object retrieval test on remote sensing video.For there is big deformation on remote sensing frames,the registration algorithm absorbs the Kanade-Lucas-Tomasi（KLT） 3-D coplanar calibration feature tracker methods,which can localize interesting targets precisely and efficiently.The experimental results prove that the proposed method has a higher localization speed and lower localization error rate than traditional visual simultaneous localization and mapping（vSLAM） in a period of time.

PRODUCT IMAGE RETRIEVAL BASED ON CO-FEATURES OF THE OBJECT

In this paper, we propose a product image retrieval method based on the object contour corners, image texture and color. The product image mainly highlights the object and its background is very simple. According to these characteristics, we represent the object using its contour, and detect the corners of contour to reduce the number of pixels. Every corner is described using its approximate curvature based on distance. In addition, the Block Difference of Inverse Probabilities (BDIP) and Block Variation of Local Correlation (BVLC) texture features and color moment are extracted from image's HIS color space. Finally, dynamic time warping method is used to match features with different length. In order to demonstrate the effect of the proposed method, we carry out experiments in Mi-crosoft product image database, and compare it with other feature descriptors. The retrieval precision and recall curves show that our method is feasible.

A Review of Research on Person Re-identification in Surveillance Video

Person re-identification has emerged as a hotspot for computer vision research due to the growing demands of social public safety requirements and the quick development of intelligent surveillance networks.Person re-identification(Re-ID)in video surveillance system can track and identify suspicious people,track and statistically analyze persons.The purpose of person re-identification is to recognize the same person in different cameras.Deep learning-based person re-identification research has produced numerous remarkable outcomes as a result of deep learning's growing popularity.The purpose of this paperis to help researchers better understand where person re-identification research is at the moment and where it is headed.Firstly,this paper arranges the widely used datasets and assessment criteria in person re-identification and reviews the pertinent research on deep learning-based person re-identification techniques conducted in the last several years.Then,the commonly used method techniques are also discussed from four aspects:appearance features,metric learning,local features,and adversarial learning.Finally,future research directions in the field of person re-identification are outlooked.

Cycle GAN-MF:A Cycle-consistent Generative Adversarial Network Based on Multifeature Fusion for Pedestrian Re-recognition

In pedestrian re-recognition,the traditional pedestrian re-recognition method will be affected by the changes of background,veil,clothing and so on,which will make the recognition effect decline.In order to reduce the impact of background,veil,clothing and other changes on the recognition effect,this paper proposes a pedestrian re-recognition method based on the cycle-consistent generative adversarial network and multifeature fusion.By comparing the measured distance between two pedestrians,pedestrian re-recognition is accomplished.Firstly,this paper uses Cycle GAN to transform and expand the data set,so as to reduce the influence of pedestrian posture changes as much as possible.The method consists of two branches:global feature extraction and local feature extraction.Then the global feature and local feature are fused.The fused features are used for comparison measurement learning,and the similarity scores are calculated to sort the samples.A large number of experimental results on large data sets CUHK03 and VIPER show that this new method reduces the influence of background,veil,clothing and other changes on the recognition effect.

LF-CNN:Deep Learning-Guided Small Sample Target Detection for Remote Sensing Classification

Target detection of small samples with a complex background is always difficult in the classification of remote sensing images.We propose a new small sample target detection method combining local features and a convolutional neural network(LF-CNN)with the aim of detecting small numbers of unevenly distributed ground object targets in remote sensing images.The k-nearest neighbor method is used to construct the local neighborhood of each point and the local neighborhoods of the features are extracted one by one from the convolution layer.All the local features are aggregated by maximum pooling to obtain global feature representation.The classification probability of each category is then calculated and classified using the scaled expected linear units function and the full connection layer.The experimental results show that the proposed LF-CNN method has a high accuracy of target detection and classification for hyperspectral imager remote sensing data under the condition of small samples.Despite drawbacks in both time and complexity,the proposed LF-CNN method can more effectively integrate the local features of ground object samples and improve the accuracy of target identification and detection in small samples of remote sensing images than traditional target detection methods.

TWO-STAGE OCCLUDED OBJECT RECOGNITION METHOD FOR MICROASSEMBLY

A two-stage object recognition algorithm with the presence of occlusion is presented for microassembly. Coarse localization determines whether template is in image or not and approximately where it is, and fine localization gives its accurate position. In coarse localization, local feature, which is invariant to translation, rotation and occlusion, is used to form signatures. By comparing signature of template with that of image, approximate transformation parameter from template to image is obtained, which is used as initial parameter value for fine localization. An objective function, which is a function of transformation parameter, is constructed in fine localization and minimized to realize sub-pixel localization accuracy. The occluded pixels are not taken into account in objective function, so the localization accuracy will not be influenced by the occlusion.

Matching DSIFT Descriptors Extracted from CSLM Images

The matching of local descriptors represents at this moment a key tool in computer vision, with a wide variety of methods designed for tasks such as image classification, object recognition and tracking, image stitching, or data mining relying on it. Local feature description techniques are usually developed so as to provide invariance to photometric variations specific to the acquisition of natural images, but are nonetheless used in association with biomedical imaging as well. It has been previously shown that the matching of gradient based descriptors is affected by image modifications specific to Confocal Scanning Laser Microscopy (CSLM). In this paper we extend our previous work in this direction and show how specific acquisition or post-processing methods alleviate or accentuate this problem.

Automated detection and identification of white-backed planthoppers in paddy fields using image processing

A survey of the population densities of rice planthoppers is important for forecasting decisions and efficient control. Tra- ditional manual surveying of rice planthoppers is time-consuming, fatiguing, and subjective. A new three-layer detection method was proposed to detect and identify white-backed planthoppers （WBPHs, Sogatella furcifera （Horvath）） and their developmental stages using image processing. In the first two detection layers, we used an AdaBoost classifier that was trained on a histogram of oriented gradient （HOG） features and a support vector machine （SVM） classifier that was trained on Gabor and Local Binary Pattern （LBP） features to detect WBPHs and remove impurities. We achieved a detection rate of 85.6% and a false detection rate of 10.2%. In the third detection layer, a SVM classifier that was trained on the HOG features was used to identify the different developmental stages of the WBPHs, and we achieved an identification rate of 73.1%, a false identification rate of 23.3%, and a 5.6% false detection rate for the images without WBPHs. The proposed three-layer detection method is feasible and effective for the identification of different developmental stages of planthoppers on rice plants in paddy fields.

Detection of egg stains based on local texture feature clustering

The quality of egg is mainly influenced by the dirt adhering to its shell.Even with good farm-management practices and careful handling,a small percentage of dirty eggs will be produced.The purpose of this research was to detect the egg stains by using image processing technique.Compared to the color values,the local texture was found to be much more adept at accurately segmenting of the complex and miscellaneous dirt stains on the egg shell.Firstly,the global threshold of the image was obtained by two-peak method.The irrelevant background was removed by using the global threshold and the interested region was acquired.The local texture information extracted from the interested region was taken as the input of fuzzy C-means clustering for segmentation of the dirt stains.According to the principle of projection,the area of dirt stains on the curved egg surface was accurately calculated.The validation experimental results showed that the proposed method for classifying eggs in terms of stain has the specificity of 91.4%for white eggs and 89.5%for brown eggs.

Automated Facial Expression Recognition and Age Estimation Using Deep Learning

With the advancement of computer vision techniques in surveillance systems,the need for more proficient,intelligent,and sustainable facial expressions and age recognition is necessary.The main purpose of this study is to develop accurate facial expressions and an age recognition system that is capable of error-free recognition of human expression and age in both indoor and outdoor environments.The proposed system first takes an input image pre-process it and then detects faces in the entire image.After that landmarks localization helps in the formation of synthetic face mask prediction.A novel set of features are extracted and passed to a classifier for the accurate classification of expressions and age group.The proposed system is tested over two benchmark datasets,namely,the Gallagher collection person dataset and the Images of Groups dataset.The system achieved remarkable results over these benchmark datasets about recognition accuracy and computational time.The proposed system would also be applicable in different consumer application domains such as online business negotiations,consumer behavior analysis,E-learning environments,and emotion robotics.

An Effective Image Retrieval Mechanism Using Family-based Spatial Consistency Filtration with Object Region

How to construct an appropriate spatial consistent measurement is the key to improving image retrieval performance. To address this problem, this paper introduces a novel image retrieval mechanism based on the family filtration in object region. First, we supply an object region by selecting a rectangle in a query image such that system returns a ranked list of images that contain the same object, retrieved from the corpus based on 100 images, as a result of the first rank. To further improve retrieval performance, we add an efficient spatial consistency stage, which is named family-based spatial consistency filtration, to re-rank the results returned by the first rank. We elaborate the performance of the retrieval system by some experiments on the dataset selected from the key frames of ＂TREC Video Retrieval Evaluation 2005 （TRECVID2005）＂. The results of experiments show that the retrieval mechanism proposed by us has vast major effect on the retrieval quality. The paper also verifies the stability of the retrieval mechanism by increasing the number of images from 100 to 2000 and realizes generalized retrieval with the object outside the dataset.