Point Cloud Classification Using Content-Based Transformer via Clustering in Feature Space

Recently, there have been some attempts of Transformer in 3D point cloud classification. In order to reduce computations, most existing methods focus on local spatial attention,but ignore their content and fail to establish relationships between distant but relevant points. To overcome the limitation of local spatial attention, we propose a point content-based Transformer architecture, called PointConT for short. It exploits the locality of points in the feature space(content-based), which clusters the sampled points with similar features into the same class and computes the self-attention within each class, thus enabling an effective trade-off between capturing long-range dependencies and computational complexity. We further introduce an inception feature aggregator for point cloud classification, which uses parallel structures to aggregate high-frequency and low-frequency information in each branch separately. Extensive experiments show that our PointConT model achieves a remarkable performance on point cloud shape classification. Especially, our method exhibits 90.3% Top-1 accuracy on the hardest setting of ScanObjectN N. Source code of this paper is available at https://github.com/yahuiliu99/PointC onT.

Research on color image matching method based on feature point compensation in dark light environment

Image matching refers to the process of matching two or more images obtained at different time,different sensors or different conditions through a large number of feature points in the image.At present,image matching is widely used in target recognition and tracking,indoor positioning and navigation.Local features missing,however,often occurs in color images taken in dark light,making the extracted feature points greatly reduced in number,so as to affect image matching and even fail the target recognition.An unsharp masking(USM)based denoising model is established and a local adaptive enhancement algorithm is proposed to achieve feature point compensation by strengthening local features of the dark image in order to increase amount of image information effectively.Fast library for approximate nearest neighbors(FLANN)and random sample consensus(RANSAC)are image matching algorithms.Experimental results show that the number of effective feature points obtained by the proposed algorithm from images in dark light environment is increased,and the accuracy of image matching can be improved obviously.

A Hybrid Features Based Detection Method for Inshore Ship Targets in SAR Imagery

Convolutional Neural Networks(CNNs)have recently attracted much attention in the ship detection from Synthetic Aperture Radar(SAR)images.However,compared with optical images,SAR ones are hard to understand.Moreover,due to the high similarity between the man-made targets near shore and inshore ships,the classical methods are unable to achieve effective detection of inshore ships.To mitigate the influence of onshore ship-like objects,this paper proposes an inshore ship detection method in SAR images by using hybrid features.Firstly,the sea-land segmentation is applied in the pre-processing to exclude obvious land regions from SAR images.Then,a CNN model is designed to extract deep features for identifying potential ship targets in both inshore and offshore water.On this basis,the high-energy point number of amplitude spectrum is further introduced as an important and delicate feature to suppress false alarms left.Finally,to verify the effectiveness of the proposed method,numerical and comparative studies are carried out in experiments on Sentinel-1 SAR images.

Automatic Feature Point Detection and Tracking of Human Actions in Time-of-flight Videos

Detecting feature points on the human body in video frames is a key step for tracking human movements. There have been methods developed that leverage models of human pose and classification of pixels of the body image. Yet, occlusion and robustness are still open challenges. In this paper, we present an automatic, model-free feature point detection and action tracking method using a time-of-flight camera. Our method automatically detects feature points for movement abstraction. To overcome errors caused by miss-detection and occlusion, a refinement method is devised that uses the trajectory of the feature points to correct the erroneous detections. Experiments were conducted using videos acquired with a Microsoft Kinect camera and a publicly available video set and comparisons were conducted with the state-of-the-art methods. The results demonstrated that our proposed method delivered improved and reliable performance with an average accuracy in the range of 90 %.The trajectorybased refinement also demonstrated satisfactory effectiveness that recovers the detection with a success rate of 93.7 %. Our method processed a frame in an average time of 71.1 ms.

Effective Self-calibration for Camera Parameters and Hand-eye Geometry Based on Two Feature Points Motions

A novel and effective self-calibration approach for robot vision is presented, which can effectively estimate both the camera intrinsic parameters and the hand-eye transformation at the same time. The proposed calibration procedure is based on two arbitrary feature points of the environment, and three pure translational motions and two rotational motions of robot endeffector are needed. New linear solution equations are deduced,and the calibration parameters are finally solved accurately and effectively. The proposed algorithm has been verified by simulated data with different noise and disturbance. Because of the need of fewer feature points and robot motions, the proposed method greatly improves the efficiency and practicality of the calibration procedure.

Image Feature Extraction and Matching of Augmented Solar Images in Space Weather

Augmented solar images were used to research the adaptability of four representative image extraction and matching algorithms in space weather domain.These include the scale-invariant feature transform algorithm,speeded-up robust features algorithm,binary robust invariant scalable keypoints algorithm,and oriented fast and rotated brief algorithm.The performance of these algorithms was estimated in terms of matching accuracy,feature point richness,and running time.The experiment result showed that no algorithm achieved high accuracy while keeping low running time,and all algorithms are not suitable for image feature extraction and matching of augmented solar images.To solve this problem,an improved method was proposed by using two-frame matching to utilize the accuracy advantage of the scale-invariant feature transform algorithm and the speed advantage of the oriented fast and rotated brief algorithm.Furthermore,our method and the four representative algorithms were applied to augmented solar images.Our application experiments proved that our method achieved a similar high recognition rate to the scale-invariant feature transform algorithm which is significantly higher than other algorithms.Our method also obtained a similar low running time to the oriented fast and rotated brief algorithm,which is significantly lower than other algorithms.

FeatureMatching Combining Variable Velocity Model with Reverse Optical Flow

The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an inaccurate pose estimation or failed tracking.To address the challenge above,a new method of feature point matching is proposed in this paper,which combines the variable velocity model with the reverse optical flow method.First,the constant velocity model is extended to a new variable velocity model,and the expanded variable velocity model is used to provide the initial pixel shifting for the reverse optical flow method.Then the search range of feature points is accurately determined according to the results of the reverse optical flow method,thereby improving the accuracy and reliability of feature matching,with strengthened interframe tracking effects.Finally,we tested on TUM data set based on the RGB-D camera.Experimental results show that this method can reduce the probability of tracking failure and improve localization accuracy on SLAM(Simultaneous Localization and Mapping)systems.Compared with the traditional ORB-SLAM2,the test error of this method on each sequence in the TUM data set is significantly reduced,and the root mean square error is only 63.8%of the original system under the optimal condition.

Point Reg Net: Invariant Features for Point Cloud Registration Using in Image-Guided Radiation Therapy

In image-guided radiation therapy, extracting features from medical point cloud is the key technique for multimodality registration. This novel framework, denoted Control Point Net (CPN), provides an alternative to the common applications of manually designed keypoint descriptors for coarse point cloud registration. The CPN directly consumes a point cloud, divides it into equally spaced 3D voxels and transforms the points within each voxel into a unified feature representation through voxel feature encoding (VFE) layer. Then all volumetric representations are aggregated by Weighted Extraction Layer which selectively extracts features and synthesize into global descriptors and coordinates of control points. Utilizing global descriptors instead of local features allows the available geometrical data to be better exploited to improve the robustness and precision. Specifically, CPN unifies feature extraction and clustering into a single network, omitting time-consuming feature matching procedure. The algorithm is tested on point cloud datasets generated from CT images. Experiments and comparisons with the state-of-the-art descriptors demonstrate that CPN is highly discriminative, efficient, and robust to noise and density changes.

Robust Radiometric Normalization of the near Equatorial Satellite Images Using Feature Extraction and Remote Sensing Analysis

Relative radiometric normalization (RRN) minimizes radiometric differences among images caused by inconsistencies of acquisition conditions rather than changes in surface. Scale invariant feature transform (SIFT) has the ability to automatically extract control points (CPs) and is commonly used for remote sensing images. However, its results are mostly inaccurate and sometimes contain incorrect matching caused by generating a small number of false CP pairs. These CP pairs have high false alarm matching. This paper presents a modified method to improve the performance of SIFT CPs matching by applying sum of absolute difference (SAD) in a different manner for the new optical satellite generation called near-equatorial orbit satellite and multi-sensor images. The proposed method, which has a significantly high rate of correct matches, improves CP matching. The data in this study were obtained from the RazakSAT satellite a new near equatorial satellite system. The proposed method involves six steps: 1) data reduction, 2) applying the SIFT to automatically extract CPs, 3) refining CPs matching by using SAD algorithm with empirical threshold, and 4) calculation of true CPs intensity values over all image’ bands, 5) preforming a linear regression model between the intensity values of CPs locate in reverence and sensed image’ bands, 6) Relative radiometric normalization conducting using regression transformation functions. Different thresholds have experimentally tested and used in conducting this study (50 and 70), by followed the proposed method, and it removed the false extracted SIFT CPs to be from 775, 1125, 883, 804, 883 and 681 false pairs to 342, 424, 547, 706, 547, and 469 corrected and matched pairs, respectively.

A Novel Airborne 3D Laser Point Cloud Hole Repair Algorithm Considering Topographic Features

Hole repair processing is an important part of point cloud data processing in airborne 3-dimensional(3D)laser scanning technology.Due to the fragmentation and irregularity of the surface morphology,when applying the 3D laser scanning technology to mountain mapping,the conventional mathematical cloud-based point cloud hole repair method is not ideal in practical applications.In order to solve this problem,we propose to repair the valley and ridge line first,and then repair the point cloud hole.The main technical steps of the method include the following points:First,the valley and ridge feature lines are extracted by the GIS slope analysis method;Then,the valley and ridge line missing from the hole are repaired by the mathematical interpolation method,and the repaired results are edited and inserted to the original point cloud;Finally,the traditional repair method is used to repair the point cloud hole whose valley line and ridge line have been repaired.Three experiments were designed and implemented in the east bank of the Xiaobaini River to test the performance of the proposed method.The results showed that compared with the direct point cloud hole repair method in Geomagic Studio software,the average repair accuracy of the proposed method,in the 16 m buffer zone of valley line and ridge line,is increased from 56.31 cm to 31.49 cm.The repair performance is significantly improved.

Feature detection on point clouds via Gabriel Triangles creation and l1 normal reconstruction

In this paper, we present a robust subneighborhoods selection technique for feature detection on point clouds scattered over a piecewise smooth surface. The proposed method first identifies all potential features using covariance analysis of the local- neighborhoods. To further extract the accurate features from potential features, Gabriel triangles are created in local neighborhoods of each potential feature vertex. These triangles tightly attach to underlying surface and effectively reflect the local geometry struc- ture. Applying a shared nearest neighbor clustering algorithm on ~ 1 reconstructed normals of created triangle set, we classify the lo- cal neighborhoods of the potential feature vertex into multiple subneighborhoods. Each subneighborhood indicates a piecewise smooth surface. The final feature vertex is identified by checking whether it is locating on the intersection of the multiple surfaces. An advantage of this framework is that it is not only robust to noise, but also insensitive to the size of selected neighborhoods. Ex- perimental results on a variety of models are used to illustrate the effectiveness and robustness of our method.

Image Relaxation Matching Based on Feature Points for DSM Generation

In photogrammetry and remote sensing, image matching is a basic and crucial process for automatic DEM generation. In this paper we presented a image relaxation matching method based on feature points. This method can be considered as an extention of regular grid point based matching. It avoids the shortcome of grid point based matching. For example, with this method, we can avoid low or even no texture area where errors frequently appear in cross correlaton matching. In the mean while, it makes full use of some mature techniques such as probability relaxation, image pyramid and the like which have already been successfully used in grid point matching process. Application of the technique to DEM generaton in different regions proved that it is more reasonable and reliable.

Loop Closure Detection of Visual SLAM Based on Point and Line Features

For traditional loop closure detection algorithm,only using the vectorization of point features to build visual dictionary is likely to cause perceptual ambiguity.In addition,when scene lacks texture information,the number of point features extracted from it will be small and cannot describe the image effectively.Therefore,this paper proposes a loop closure detection algorithm which combines point and line features.To better recognize scenes with hybrid features,the building process of traditional dictionary tree is improved in the paper.The features with different flag bits were clustered separately to construct a mixed dictionary tree and word vectors that can represent the hybrid features,which can better describe structure and texture information of scene.To ensure that the similarity score between images is more reasonable,different similarity coefficients were set in different scenes,and the candidate frame with the highest similarity score was selected as the candidate closed loop.Experiments show that the point line comprehensive feature was superior to the single feature in the structured scene and the strong texture scene,the recall rate of the proposed algorithm was higher than the state of the art methods when the accuracy is 100%,and the algorithm can be applied to more diverse environments.

Arm-Root Curve Fitting Based on Body Surface Feature Points for Young Male

This paper is aiming to obtain an arm-root curve function performing the human arm-root size and shape realistically. A gypsum replica of upper arm for young male was made and scanned for extracting the 3D coordinates of 4 feature points of shoulder point, the anterior/posterior armpit point and the axillary point describing the real arm-root shape under the normalized definitions, and the 5 landmarks were confirmed additionally for improving the fitting precision. Then, the wholly and piecewise fitting of arm-root curve with 9 feature points and mark points in total were generated respectively based on least square polynomial fitting method. Comparing to the wholly fitting, the piecewise fitted function segmented by the line between anterior and posterior axillary points showed a high fitting degree of arm-root morphology with R-square of 1, the length difference between fitted curve and gypsum curve is 0.003 cm within error range. And it provided a basic curve model with standard feature points to simulate arm-root morphology realistically by curve fitting for accurate body measurement extraction.

GRADIENT OF REFERENCE DIFFERENCE BASED MATCHING ALGORITHM FOR IMAGE FEATURE POINT

During matching on feature point, gray correlation matching technology is utilized to extract multi-peaks as a coarse matching set. A pair of given corresponding reference points within the left and right images is used to calculate gradients of reference difference between the reference points and each feature point within the multi-peaks set. The unique correspondence is determined by criterion of minimal gradients of reference difference. The obtained correspondence is taken as a new pair of reference points to update the reference points continuously until all feature points in the left (or right) image being matched with the right (or left) image. The gradients of reference difference can be calculated easily by means of pre-setting a pair of obvious feature points in the left and right images as a pair of corresponding reference points. Besides, the efficiency of matching can be improved greatly by taking the obtained matching point as a new pair of reference points, and by updating the reference point continuously. It is proved that the proposed algorithm is valid and reliable by 3D reconstruction on two pairs of actual natural images with abundant and weak texture, respectively.

STATISTICAL PROBABILITY BASED ALGORITHM FOR EXTRACTING FEATURE POINTS IN 2-DIMENSIONAL IMAGE

An algorithm for automatically extracting feature points is developed after the area of feature points in 2-dimensional (2D) imagebeing located by probability theory, correlated methods and criterion for abnormity. Feature points in 2D image can be extracted only by calculating standard deviation of gray within sampled pixels area in our approach statically. While extracting feature points, the limitation to confirm threshold by tentative method according to some a priori information on processing image can be avoided. It is proved that the proposed algorithm is valid and reliable by extracting feature points on actual natural images with abundant and weak texture, including multi-object with complex background, respectively. It can meet the demand of extracting feature points of 2D image automatically in machine vision system.

Coherent Point Drift Registration Combined with Image Feature and its Application

A key step of constructing active appearance model is requiring a set of appropriate training shapes with well-defined correspondences.In this paper,we introduce a novel point correspondence method(FB-CPD),which can improve the accuracy of coherent point drift(CPD) by using the information of image feature.The objective function of the proposed method is defined by both of geometric spatial information and image feature information,and the origin Gaussian mixture model in CPD is modified according to the image feature of points.FB-CPD is tested on the 3D prostate and liver point sets through the simulation experiments.The registration error can be reduced efficiently by FB-CPD.Moreover,the active appearance model constructed by FB-CPD can obtain fine segmentation in 3D CT prostate image.Compared with the original CPD,the overlap ratio of voxels was improved from 88.7% to 90.2% by FB-CPD.

Spectral matching algorithm based on nonsubsampled contourlet transform and scale-invariant feature transform

A new spectral matching algorithm is proposed by us- ing nonsubsampled contourlet transform and scale-invariant fea- ture transform. The nonsubsampled contourlet transform is used to decompose an image into a low frequency image and several high frequency images, and the scale-invariant feature transform is employed to extract feature points from the low frequency im- age. A proximity matrix is constructed for the feature points of two related images. By singular value decomposition of the proximity matrix, a matching matrix （or matching result） reflecting the match- ing degree among feature points is obtained. Experimental results indicate that the proposed algorithm can reduce time complexity and possess a higher accuracy.

A Feature-based Robust Digital Image Watermarking Against Desynchronization Attacks

In this paper, a new content-based image watermarking scheme is proposed. The Harris-Laplace detector is adopted to extract feature points, which can survive a variety of attacks. The local characteristic regions （LCRs） are adaptively constructed based on scale-space theory. Then, the LCRs are mapped to geometrically invariant space by using image normalization technique. Finally, several copies of the digital watermark are embedded into the nonoverlapped LCRs by quantizing the magnitude vectors of discrete Fourier transform （DFT） coefficients. By binding a watermark with LCR, resilience against desynchronization attacks can be readily obtained. Simulation results show that the proposed scheme is invisible and robust against various attacks which includes common signals processing and desynchronization attacks.