Feature Matching via Topology-Aware Graph Interaction Model

Feature matching plays a key role in computer vision. However, due to the limitations of the descriptors, the putative matches are inevitably contaminated by massive outliers.This paper attempts to tackle the outlier filtering problem from two aspects. First, a robust and efficient graph interaction model,is proposed, with the assumption that matches are correlated with each other rather than independently distributed. To this end, we construct a graph based on the local relationships of matches and formulate the outlier filtering task as a binary labeling energy minimization problem, where the pairwise term encodes the interaction between matches. We further show that this formulation can be solved globally by graph cut algorithm. Our new formulation always improves the performance of previous localitybased method without noticeable deterioration in processing time,adding a few milliseconds. Second, to construct a better graph structure, a robust and geometrically meaningful topology-aware relationship is developed to capture the topology relationship between matches. The two components in sum lead to topology interaction matching(TIM), an effective and efficient method for outlier filtering. Extensive experiments on several large and diverse datasets for multiple vision tasks including general feature matching, as well as relative pose estimation, homography and fundamental matrix estimation, loop-closure detection, and multi-modal image matching, demonstrate that our TIM is more competitive than current state-of-the-art methods, in terms of generality, efficiency, and effectiveness. The source code is publicly available at http://github.com/YifanLu2000/TIM.

CMMCAN:Lightweight Feature Extraction and Matching Network for Endoscopic Images Based on Adaptive Attention

In minimally invasive surgery,endoscopes or laparoscopes equipped with miniature cameras and tools are used to enter the human body for therapeutic purposes through small incisions or natural cavities.However,in clinical operating environments,endoscopic images often suffer from challenges such as low texture,uneven illumination,and non-rigid structures,which affect feature observation and extraction.This can severely impact surgical navigation or clinical diagnosis due to missing feature points in endoscopic images,leading to treatment and postoperative recovery issues for patients.To address these challenges,this paper introduces,for the first time,a Cross-Channel Multi-Modal Adaptive Spatial Feature Fusion(ASFF)module based on the lightweight architecture of EfficientViT.Additionally,a novel lightweight feature extraction and matching network based on attention mechanism is proposed.This network dynamically adjusts attention weights for cross-modal information from grayscale images and optical flow images through a dual-branch Siamese network.It extracts static and dynamic information features ranging from low-level to high-level,and from local to global,ensuring robust feature extraction across different widths,noise levels,and blur scenarios.Global and local matching are performed through a multi-level cascaded attention mechanism,with cross-channel attention introduced to simultaneously extract low-level and high-level features.Extensive ablation experiments and comparative studies are conducted on the HyperKvasir,EAD,M2caiSeg,CVC-ClinicDB,and UCL synthetic datasets.Experimental results demonstrate that the proposed network improves upon the baseline EfficientViT-B3 model by 75.4%in accuracy(Acc),while also enhancing runtime performance and storage efficiency.When compared with the complex DenseDescriptor feature extraction network,the difference in Acc is less than 7.22%,and IoU calculation results on specific datasets outperform complex dense models.Furthermore,this method increases the F1 score by 33.2%and accelerates runtime by 70.2%.It is noteworthy that the speed of CMMCAN surpasses that of comparative lightweight models,with feature extraction and matching performance comparable to existing complex models but with faster speed and higher cost-effectiveness.

Loop Closure Detection via Locality Preserving Matching With Global Consensus

A critical component of visual simultaneous localization and mapping is loop closure detection(LCD),an operation judging whether a robot has come to a pre-visited area.Concretely,given a query image(i.e.,the latest view observed by the robot),it proceeds by first exploring images with similar semantic information,followed by solving the relative relationship between candidate pairs in the 3D space.In this work,a novel appearance-based LCD system is proposed.Specifically,candidate frame selection is conducted via the combination of Superfeatures and aggregated selective match kernel(ASMK).We incorporate an incremental strategy into the vanilla ASMK to make it applied in the LCD task.It is demonstrated that this setting is memory-wise efficient and can achieve remarkable performance.To dig up consistent geometry between image pairs during loop closure verification,we propose a simple yet surprisingly effective feature matching algorithm,termed locality preserving matching with global consensus(LPM-GC).The major objective of LPM-GC is to retain the local neighborhood information of true feature correspondences between candidate pairs,where a global constraint is further designed to effectively remove false correspondences in challenging sceneries,e.g.,containing numerous repetitive structures.Meanwhile,we derive a closed-form solution that enables our approach to provide reliable correspondences within only a few milliseconds.The performance of the proposed approach has been experimentally evaluated on ten publicly available and challenging datasets.Results show that our method can achieve better performance over the state-of-the-art in both feature matching and LCD tasks.We have released our code of LPM-GC at https://github.com/jiayi-ma/LPM-GC.

FeatureMatching Combining Variable Velocity Model with Reverse Optical Flow

The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an inaccurate pose estimation or failed tracking.To address the challenge above,a new method of feature point matching is proposed in this paper,which combines the variable velocity model with the reverse optical flow method.First,the constant velocity model is extended to a new variable velocity model,and the expanded variable velocity model is used to provide the initial pixel shifting for the reverse optical flow method.Then the search range of feature points is accurately determined according to the results of the reverse optical flow method,thereby improving the accuracy and reliability of feature matching,with strengthened interframe tracking effects.Finally,we tested on TUM data set based on the RGB-D camera.Experimental results show that this method can reduce the probability of tracking failure and improve localization accuracy on SLAM(Simultaneous Localization and Mapping)systems.Compared with the traditional ORB-SLAM2,the test error of this method on each sequence in the TUM data set is significantly reduced,and the root mean square error is only 63.8%of the original system under the optimal condition.

Image Feature Extraction and Matching of Augmented Solar Images in Space Weather

Augmented solar images were used to research the adaptability of four representative image extraction and matching algorithms in space weather domain.These include the scale-invariant feature transform algorithm,speeded-up robust features algorithm,binary robust invariant scalable keypoints algorithm,and oriented fast and rotated brief algorithm.The performance of these algorithms was estimated in terms of matching accuracy,feature point richness,and running time.The experiment result showed that no algorithm achieved high accuracy while keeping low running time,and all algorithms are not suitable for image feature extraction and matching of augmented solar images.To solve this problem,an improved method was proposed by using two-frame matching to utilize the accuracy advantage of the scale-invariant feature transform algorithm and the speed advantage of the oriented fast and rotated brief algorithm.Furthermore,our method and the four representative algorithms were applied to augmented solar images.Our application experiments proved that our method achieved a similar high recognition rate to the scale-invariant feature transform algorithm which is significantly higher than other algorithms.Our method also obtained a similar low running time to the oriented fast and rotated brief algorithm,which is significantly lower than other algorithms.

Outliers rejection in similar image matching

Background Image matching is crucial in numerous computer vision tasks such as 3D reconstruction and simultaneous visual localization and mapping.The accuracy of the matching significantly impacted subsequent studies.Because of their local similarity,when image pairs contain comparable patterns but feature pairs are positioned differently,incorrect recognition can occur as global motion consistency is disregarded.Methods This study proposes an image-matching filtering algorithm based on global motion consistency.It can be used as a subsequent matching filter for the initial matching results generated by other matching algorithms based on the principle of motion smoothness.A particular matching algorithm can first be used to perform the initial matching;then,the rotation and movement information of the global feature vectors are combined to effectively identify outlier matches.The principle is that if the matching result is accurate,the feature vectors formed by any matched point should have similar rotation angles and moving distances.Thus,global motion direction and global motion distance consistencies were used to reject outliers caused by similar patterns in different locations.Results Four datasets were used to test the effectiveness of the proposed method.Three datasets with similar patterns in different locations were used to test the results for similar images that could easily be incorrectly matched by other algorithms,and one commonly used dataset was used to test the results for the general image-matching problem.The experimental results suggest that the proposed method is more accurate than other state-of-the-art algorithms in identifying mismatches in the initial matching set.Conclusions The proposed outlier rejection matching method can significantly improve the matching accuracy for similar images with locally similar feature pairs in different locations and can provide more accurate matching results for subsequent computer vision tasks.

ANTI-ROTATION AND ANTI-SCALE IMAGE MATCHING ALGORITHM FOR NAVIGATION SYSTEM

Based on the inertial navigation system, the influences of the excursion of the inertial navigation system and the measurement error of the wireless pressure altimeter on the rotation and scale of the real image are quantitatively analyzed in scene matching. The log-polar transform （LPT） is utilized and an anti-rotation and anti- scale image matching algorithm is proposed based on the image edge feature point extraction. In the algorithm, the center point is combined with its four-neighbor points, and the corresponding computing process is put forward. Simulation results show that in the image rotation and scale variation range resulted from the navigation system error and the measurement error of the wireless pressure altimeter, the proposed image matching algo- rithm can satisfy the accuracy demands of the scene aided navigation system and provide the location error-correcting information of the system.

Illumination Adaptive Identification Algorithm of a Reconfigurable Modular Robot

Reconfigurable modular robots feature high mobility due to their unconstrained connection manners.Inspired by the snake multi-joint crawling principle,a chain-type reconfigurable modular robot(CRMR)is designed,which could reassemble into various configurations through the compound joint motion.Moreover,an illumination adaptive modular robot identification(IAMRI)algorithm is proposed for CRMR.At first,an adaptive threshold is applied to detect oriented FAST features in the robot image.Then,the effective detection of features in non-uniform illumination areas is achieved through an optimized quadtree decomposition method.After matching features,an improved random sample consensus algorithm is employed to eliminate the mismatched features.Finally,the reconfigurable robot module is identified effectively through the perspective transformation.Compared with ORB,MA,Y-ORB,and S-ORB algorithms,the IAMRI algorithm has an improvement of over 11.6%in feature uniformity,and 13.7%in the comprehensive indicator,respectively.The IAMRI algorithm limits the relative error within 2.5 pixels,efficiently completing the CRMR identification under complex environmental changes.

A Visual Indoor Localization Method Based on Efficient Image Retrieval

The task of indoor visual localization, utilizing camera visual information for user pose calculation, was a core component of Augmented Reality (AR) and Simultaneous Localization and Mapping (SLAM). Existing indoor localization technologies generally used scene-specific 3D representations or were trained on specific datasets, making it challenging to balance accuracy and cost when applied to new scenes. Addressing this issue, this paper proposed a universal indoor visual localization method based on efficient image retrieval. Initially, a Multi-Layer Perceptron (MLP) was employed to aggregate features from intermediate layers of a convolutional neural network, obtaining a global representation of the image. This approach ensured accurate and rapid retrieval of reference images. Subsequently, a new mechanism using Random Sample Consensus (RANSAC) was designed to resolve relative pose ambiguity caused by the essential matrix decomposition based on the five-point method. Finally, the absolute pose of the queried user image was computed, thereby achieving indoor user pose estimation. The proposed indoor localization method was characterized by its simplicity, flexibility, and excellent cross-scene generalization. Experimental results demonstrated a positioning error of 0.09 m and 2.14° on the 7Scenes dataset, and 0.15 m and 6.37° on the 12Scenes dataset. These results convincingly illustrated the outstanding performance of the proposed indoor localization method.

A fast, accurate and dense feature matching algorithm for aerial images

Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mismatching and sparse feature pairs using traditional algorithms.Therefore,an algorithm is proposed to realize fast,accurate and dense feature matching.The algorithm consists of four steps.Firstly,we achieve a balance between the feature matching time and the number of matching pairs by appropriately reducing the image resolution.Secondly,to realize further screening of the mismatches,a feature screening algorithm based on similarity judgment or local optimization is proposed.Thirdly,to make the algorithm more widely applicable,we combine the results of different algorithms to get dense results.Finally,all matching feature pairs in the low-resolution images are restored to the original images.Comparisons between the original algorithms and our algorithm show that the proposed algorithm can effectively reduce the matching time,screen out the mismatches,and improve the number of matches.

Image matching algorithm based on SIFT using color and exposure information

Image matching based on scale invariant feature transform（SIFT） is one of the most popular image matching algorithms, which exhibits high robustness and accuracy. Grayscale images rather than color images are generally used to get SIFT descriptors in order to reduce the complexity. The regions which have a similar grayscale level but different hues tend to produce wrong matching results in this case. Therefore, the loss of color information may result in decreasing of matching ratio. An image matching algorithm based on SIFT is proposed, which adds a color offset and an exposure offset when converting color images to grayscale images in order to enhance the matching ratio. Experimental results show that the proposed algorithm can effectively differentiate the regions with different colors but the similar grayscale level, and increase the matching ratio of image matching based on SIFT. Furthermore, it does not introduce much complexity than the traditional SIFT.

Trifocal Tensor Based Feature Matching Algorithm

Feature matching is of significance in the field of computer vision.In this paper,a trifocal tensor based feature matching algorithm is proposed for three views,including a trinocular vision system.Initial matching point-pairs can be determined according to generic matching algorithms,on which an initial trifocal tensor of three views can be confirmed.Then the initial matching point-pairs should be re-selected.Meanwhile,the trifocal tensor will be recomputed.Iteratively,the optimized trifocal tensor can be obtained.Compatible fundamental matrix of every two views can be determined.Furthermore,in the trinocular vision sensor,the trifocal tensor can be calculated based on the intrinsic parameter matrix of each camera.With the strict constraint provided by the trifocal tensor,feature matching results will be optimized.Experiments show that our proposed algorithm has the characteristics of feasibility and precision.

A calculation method for low dynamic vehicle velocity based on fusion of optical flow and feature point matching

Aming at the problem of the low accuracy of low dynamic vehicle velocity under the environment of uneven distribution of light intensity,an improved adaptive Kalman filter method for the velocity error estimate by the fusion of optical flow tracking and scale mvaiant feature transform(SIFT)is proposed.The algorithm introduces anonlinear fuzzy membership function and the filter residual for the noise covariance matrix in the adaptive adjustment process.In the process of calculating the velocity of the vehicle,the tracking and matching of the inter-frame displacement a d the vehicle velocity calculation a e carried out by using the optical fow tracing and the SIF'T methods,respectively.Meanwhile,the velocity difference between theoutputs of thesetwo methods is used as the observation of the improved adaptive Kalman filter.Finally,the velocity calculated by the optical fow method is corrected by using the velocity error estimate of the output of the modified adaptive Kalman filter.The results of semi-physical experiments show that the maximum velocityeror of the fusion algorithm is decreased by29%than that of the optical fow method,and the computation time is reduced by80%compared with the SIFT method.

Method of weld recognition based on textural feature matching

In this paper an automatic visual method of seam recognizing and seam tracking based on textural feature matching was proposed, in order to recognize the weld of multi-layer or multi-pass welding in which the weld is difficult to be recognized by conventional visual methods. This method focuses on the obvious difference of image textural feature between the weld region and the base metal region, as well as the similarity of the textural features along the welding direction. The method consists of the following steps ： setting image template and choosing the edge region as ROI （ region of interest ）, extracting the image textural feature of the template and the edge region, feature matching, and recognition of weld region. Experiment showed that the method proposed was effective for weld seam recognition in multi-layer welding.

Individual Identification of Dairy Cows Based on Deep Feature Extrac-tion and Matching

Individual identification of dairy cows is the prerequisite for automatic analysis and intelligent perception of dairy cows'behavior.At present,individual identification of dairy cows based on deep convolutional neural network had the disadvantages in prolonged training at the additions of new cows samples.Therefore,a cow individual identification framework was proposed based on deep feature extraction and matching,and the individual identification of dairy cows based on this framework could avoid repeated training.Firstly,the trained convolutional neural network model was used as the feature extractor;secondly,the feature extraction was used to extract features and stored the features into the template feature library to complete the enrollment;finally,the identifies of dairy cows were identified.Based on this framework,when new cows joined the herd,enrollment could be completed quickly.In order to evaluate the application performance of this method in closed-set and open-set individual identification of dairy cows,back images of 524 cows were collected,among which the back images of 150 cows were selected as the training data to train feature extractor.The data of the remaining 374 cows were used to generate the template data set and the data to be identified.The experiment results showed that in the closed-set individual identification of dairy cows,the highest identification accuracy of top-1 was 99.73%,the highest identification accuracy from top-2 to top-5 was 100%,and the identification time of a single cow was 0.601 s,this method was verified to be effective.In the open-set individual identification of dairy cows,the recall was 90.38%,and the accuracy was 89.46%.When false accept rate(FAR)=0.05,true accept rate(TAR)=84.07%,this method was verified that the application had certain research value in open-set individual identification of dairy cows,which provided a certain idea for the application of individual identification in the field of intelligent animal husbandry.

Matching DSIFT Descriptors Extracted from CSLM Images

The matching of local descriptors represents at this moment a key tool in computer vision, with a wide variety of methods designed for tasks such as image classification, object recognition and tracking, image stitching, or data mining relying on it. Local feature description techniques are usually developed so as to provide invariance to photometric variations specific to the acquisition of natural images, but are nonetheless used in association with biomedical imaging as well. It has been previously shown that the matching of gradient based descriptors is affected by image modifications specific to Confocal Scanning Laser Microscopy (CSLM). In this paper we extend our previous work in this direction and show how specific acquisition or post-processing methods alleviate or accentuate this problem.

English-Chinese Neural Machine Translation Based on Self-organizing Mapping Neural Network and Deep Feature Matching

The traditional Chinese-English translation model tends to translate some source words repeatedly,while mistakenly ignoring some words.Therefore,we propose a novel English-Chinese neural machine translation based on self-organizing mapping neural network and deep feature matching.In this model,word vector,two-way LSTM,2D neural network and other deep learning models are used to extract the semantic matching features of question-answer pairs.Self-organizing mapping(SOM)is used to classify and identify the sentence feature.The attention mechanism-based neural machine translation model is taken as the baseline system.The experimental results show that this framework significantly improves the adequacy of English-Chinese machine translation and achieves better results than the traditional attention mechanism-based English-Chinese machine translation model.

Feature-based sequential partial vision measurement method for large scale machine parts

To realize the high-precision vision measurement for large scale machine parts, a new vision measurement method based on dimension features of sequential partial images is proposed. Instead of mosaicking the partial images, extracting the dimension features of the sequential partial images and deriving the part size according to the relationships between the sequential images is a novel method to realize the high- precision and fast measurement of machine parts. To overcome the corresponding problems arising from the relative rotation between two sequential partial images, a rectifying method based on texture features is put forward to effectively improve the processing speed. Finally, a case study is provided to demonstrate the analysis procedure and the effectiveness of the proposed method. The experiments show that the relative error is less than 0. 012% using the sequential image measurement method to gauge large scale straight-edge parts. The measurement precision meets the needs of precise measurement for sheet metal parts.

A method of automatic plane detection without random search

Plane detection is a prerequisite for many computer vision tasks. This paper proposes a new method which can automatically detect planes from two projective images. Firstly, we modify Scott’s feature point matching method by post-processing its result with the concept of similarity, and then get the lines matching according to feature points matching based on the approximate invariance of the features’ distribution between two images. Finally, we group all feature points into subsets in terms of their geometric relations with feature lines as initial sets to estimate homography rather than by a random search strategy (like RANSAC) as in most existing methods. The proposed method is especially suitable to detecting planes in man-made scenes. This method is validated on real images.

Automatic Road Change Detection and GIS Updating from High Spatial Remotely-Sensed Imagery

This paper presents a framework for road network change detection in order to update the Canadian National Topographic DataBase (NTDB). The methodology has been developed on the basis of road extraction from IRS\|pan images (with a 5.8 m spatial resolution) by using a wavelet approach. The feature matching and conflation techniques are used to road change detection and updating. Elementary experiments have showed that the proposed framework could be used for developing an operational road database updating system.