In feature-based visual localization for small-scale scenes,local descriptors are used to estimate the camera pose of a query image.For large and ambiguous environments,learning-based hierarchical networks that employ...In feature-based visual localization for small-scale scenes,local descriptors are used to estimate the camera pose of a query image.For large and ambiguous environments,learning-based hierarchical networks that employ local as well as global descriptors to reduce the search space of database images into a smaller set of reference views have been introduced.However,since global descriptors are generated using visual features,reference images with some of these features may be erroneously selected.In order to address this limitation,this paper proposes two clustering methods based on how often features appear as well as their covisibility.For both approaches,the scene is represented by voxels whose size and number are computed according to the size of the scene and the number of available 3Dpoints.In the first approach,a voxel-based histogram representing highly reoccurring scene regions is generated from reference images.A meanshift is then employed to group the most highly reoccurring voxels into place clusters based on their spatial proximity.In the second approach,a graph representing the covisibility-based relationship of voxels is built.Local matching is performed within the reference image clusters,and a perspective-n-point is employed to estimate the camera pose.The experimental results showed that camera pose estimation using the proposed approaches was more accurate than that of previous methods.展开更多
Visual localization is a crucial component in the application of mobile robot and autonomous driving.Image retrieval is an efficient and effective technique in image-based localization methods.Due to the drastic varia...Visual localization is a crucial component in the application of mobile robot and autonomous driving.Image retrieval is an efficient and effective technique in image-based localization methods.Due to the drastic variability of environmental conditions,e.g.,illumination changes,retrievalbased visual localization is severely affected and becomes a challenging problem.In this work,a general architecture is first formulated probabilistically to extract domain-invariant features through multi-domain image translation.Then,a novel gradientweighted similarity activation mapping loss(Grad-SAM)is incorporated for finer localization with high accuracy.We also propose a new adaptive triplet loss to boost the contrastive learning of the embedding in a self-supervised manner.The final coarse-to-fine image retrieval pipeline is implemented as the sequential combination of models with and without Grad-SAM loss.Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMU-Seasons dataset.The strong generalization ability of our approach is verified with the RobotCar dataset using models pre-trained on urban parts of the CMU-Seasons dataset.Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision,especially under challenging environments with illumination variance,vegetation,and night-time images.Moreover,real-site experiments have been conducted to validate the efficiency and effectiveness of the coarse-to-fine strategy for localization.展开更多
When a vehicle travels in urban areas, onboard global positioning system (GPS) signals may be obstructed by high-rise buildings and thereby cannot provide accurate positions. It is proposed to perform localization b...When a vehicle travels in urban areas, onboard global positioning system (GPS) signals may be obstructed by high-rise buildings and thereby cannot provide accurate positions. It is proposed to perform localization by registering ground images to a 2D building boundary map which is generated from aerial images. Multilayer feature graphs (MFG) is employed to model building facades from the ground images. MFG was reported in the previous work to facilitate the robot scene understand- ing in urhan areas. By constructing MFG, the 2D/3D positions of features can be obtained, inclu- cling line segments, ideal lines, and all primary vertical planes. Finally, a voting-based feature weighted localization method is developed based on MFGs and the 2D building boundary map. The proposed method has been implemented and validated in physical experiments. In the proposed ex- periments, the algorithm has achieved an overall localization accuracy of 2.2m, which is better than commercial GPS working in open environments.展开更多
To overcome the shortcomings of existing robot localization sensors, such as low accuracy and poor robustness, a high precision visual localization system based on infrared-reflective artificial markers is designed an...To overcome the shortcomings of existing robot localization sensors, such as low accuracy and poor robustness, a high precision visual localization system based on infrared-reflective artificial markers is designed and illustrated in detail in this paper. First, the hardware system of the localization sensor is developed. Secondly, we design a novel kind of infrared-reflective artificial marker whose characteristics can be extracted by the acquisition and processing of the infrared image. In addition, a confidence calculation method for marker identification is proposed to obtain the probabilistic localization results. Finally, the autonomous localization of the robot is achieved by calculating the relative pose relation between the robot and the artificial marker based on the perspective-3-point(P3P) visual localization algorithm. Numerous experiments and practical applications show that the designed localization sensor system is immune to the interferences of the illumination and observation angle changes. The precision of the sensor is ±1.94 cm for position localization and ±1.64° for angle localization. Therefore, it satisfies perfectly the requirements of localization precision for an indoor mobile robot.展开更多
Indoor visual localization,i.e.,6 Degree-of-Freedom camera pose estimation for a query image with respect to a known scene,is gaining increased attention driven by rapid progress of applications such as robotics and a...Indoor visual localization,i.e.,6 Degree-of-Freedom camera pose estimation for a query image with respect to a known scene,is gaining increased attention driven by rapid progress of applications such as robotics and augmented reality.However,drastic visual discrepancies between an onsite query image and prerecorded indoor images cast a significant challenge for visual localization.In this paper,based on the key observation of the constant existence of planar surfaces such as floors or walls in indoor scenes,we propose a novel system incorporating geometric information to address issues using only pixelated images.Through the system implementation,we contribute a hierarchical structure consisting of pre-scanned images and point cloud,as well as a distilled representation of the planar-element layout extracted from the original dataset.A view synthesis procedure is designed to generate synthetic images as complementary to that of a sparsely sampled dataset.Moreover,a global image descriptor based on the image statistic modality,called block mean,variance,and color(BMVC),was employed to speed up the candidate pose identification incorporated with a traditional convolutional neural network(CNN)descriptor.Experimental results on a popular benchmark demonstrate that the proposed method outperforms the state-of-the-art approaches in terms of visual localization validity and accuracy.展开更多
A localization method based on distance function of projected features is presented to solve the accuracy reduction or failure problem due to occlusion and blurring caused by smog, when dealing with vision based local...A localization method based on distance function of projected features is presented to solve the accuracy reduction or failure problem due to occlusion and blurring caused by smog, when dealing with vision based localization for target oil and gas wellhead (OGWH). Firstly, the target OGWH is modeled as a cylinder with marker, and a vector with redundant parameter is used to describe its pose. Secondly, the explicit mapping relationship between the pose vector with redundant parameter and projected features is derived. Then, a 2D-point-to-feature distance function is proposed, as well as its derivative. Finally, based on this distance function and its derivative, an algorithm is proposed to estimate the pose of target OGWH directly according to the 2D image information, and the validity of the method is verified by both synthetic data and real image experiments. The results show that this method is able to accomplish the localization in the case of occlusion and blurring, and its anti-noise ability is good especially with noise ratio of less than 70%.展开更多
With the rapid popularization of mobile devices and the wide application of various sensors,scene perception methods applied to mobile devices occupy an important position in location-based services such as navigation...With the rapid popularization of mobile devices and the wide application of various sensors,scene perception methods applied to mobile devices occupy an important position in location-based services such as navigation and augmented reality(AR).The development of deep learning technologies has greatly improved the visual perception ability of machines to scenes.The basic framework of scene visual perception,related technologies and the specific process applied to AR navigation are introduced,and future technology development is proposed.An application(APP)is designed to improve the application effect of AR navigation.The APP includes three modules:navigation map generation,cloud navigation algorithm,and client design.The navigation map generation tool works offline.The cloud saves the navigation map and provides navigation algorithms for the terminal.The terminal realizes local real-time positioning and AR path rendering.展开更多
Purified terephthalic acid(PTA) is an important chemical raw material. P-xylene(PX) is transformed to terephthalic acid(TA) through oxidation process and TA is refined to produce PTA. The PX oxidation reaction is a co...Purified terephthalic acid(PTA) is an important chemical raw material. P-xylene(PX) is transformed to terephthalic acid(TA) through oxidation process and TA is refined to produce PTA. The PX oxidation reaction is a complex process involving three-phase reaction of gas, liquid and solid. To monitor the process and to improve the product quality, as well as to visualize the fault type clearly, a fault diagnosis method based on selforganizing map(SOM) and high dimensional feature extraction method, local tangent space alignment(LTSA),is proposed. In this method, LTSA can reduce the dimension and keep the topology information simultaneously,and SOM distinguishes various states on the output map. Monitoring results of PX oxidation reaction process indicate that the LTSA–SOM can well detect and visualize the fault type.展开更多
The extraction and description of image features are very important for visual simultaneous localization and mapping(V-SLAM).A rotated boosted efficient binary local image descriptor(BEBLID)SLAM(RB-SLAM)algorithm base...The extraction and description of image features are very important for visual simultaneous localization and mapping(V-SLAM).A rotated boosted efficient binary local image descriptor(BEBLID)SLAM(RB-SLAM)algorithm based on improved oriented fast and rotated brief(ORB)feature description is proposed in this paper,which can solve the problems of low localization accuracy and time efficiency of the current ORB-SLAM3 algorithm.Firstly,it uses the BEBLID to replace the feature point description algorithm of the original ORB to enhance the expressiveness and description efficiency of the image.Secondly,it adds rotational invariance to the BEBLID using the orientation information of the feature points.It also selects the rotationally stable bits in the BEBLID to further enhance the rotational invariance of the BEBLID.Finally,it retrains the binary visual dictionary based on the BEBLID to reduce the cumulative error of V-SLAM and improve the loading speed of the visual dictionary.Experiments show that the dictionary loading efficiency is improved by more than 10 times.The RB-SLAM algorithm improves the trajectory accuracy by 24.75%on the TUM dataset and 26.25%on the EuRoC dataset compared to the ORB-SLAM3 algorithm.展开更多
Simultaneous Localization and Mapping(SLAM)has been widely used in emergency response,self-driving and city-scale 3D mapping and navigation.Recent deep-learning based feature point extractors have demonstrated superio...Simultaneous Localization and Mapping(SLAM)has been widely used in emergency response,self-driving and city-scale 3D mapping and navigation.Recent deep-learning based feature point extractors have demonstrated superior performance in dealing with the complex environmental challenges(e.g.extreme lighting)while the traditional extractors are struggling.In this paper,we have successfully improved the robustness and accuracy of a monocular visual SLAM system under various complex scenes by adding a deep learning based visual localization thread as an augmentation to the visual SLAM framework.In this thread,our feature extractor with an efficient lightweight deep neural network is used for absolute pose and scale estimation in real time using the highly accurate georeferenced prior map database at 20cm geometric accuracy created by our in-house and low-cost LiDAR and camera integrated device.The closed-loop error provided by our SLAM system with and without this enhancement is 1.03m and 18.28m respectively.The scale estimation of the monocular visual SLAM is also significantly improved(0.01 versus 0.98).In addition,a novel camera-LiDAR calibration workflow is also provided for large-scale 3D mapping.This paper demonstrates the application and research potential of deep-learning based vision SLAM with image and LiDAR sensors.展开更多
This paper presents a hierarchical simultaneous localization and mapping(SLAM) system for a small unmanned aerial vehicle(UAV) using the output of an inertial measurement unit(IMU) and the bearing-only observati...This paper presents a hierarchical simultaneous localization and mapping(SLAM) system for a small unmanned aerial vehicle(UAV) using the output of an inertial measurement unit(IMU) and the bearing-only observations from an onboard monocular camera.A homography based approach is used to calculate the motion of the vehicle in 6 degrees of freedom by image feature match.This visual measurement is fused with the inertial outputs by an indirect extended Kalman filter(EKF) for attitude and velocity estimation.Then,another EKF is employed to estimate the position of the vehicle and the locations of the features in the map.Both simulations and experiments are carried out to test the performance of the proposed system.The result of the comparison with the referential global positioning system/inertial navigation system(GPS/INS) navigation indicates that the proposed SLAM can provide reliable and stable state estimation for small UAVs in GPS-denied environments.展开更多
In this paper,we present a novel algorithm for odometry estimation based on ceiling vision.The main contribution of this algorithm is the introduction of principal direction detection that can greatly reduce error acc...In this paper,we present a novel algorithm for odometry estimation based on ceiling vision.The main contribution of this algorithm is the introduction of principal direction detection that can greatly reduce error accumulation problem in most visual odometry estimation approaches.The principal direction is defned based on the fact that our ceiling is flled with artifcial vertical and horizontal lines which can be used as reference for the current robot s heading direction.The proposed approach can be operated in real-time and it performs well even with camera s disturbance.A moving low-cost RGB-D camera(Kinect),mounted on a robot,is used to continuously acquire point clouds.Iterative closest point(ICP) is the common way to estimate the current camera position by registering the currently captured point cloud to the previous one.However,its performance sufers from data association problem or it requires pre-alignment information.The performance of the proposed principal direction detection approach does not rely on data association knowledge.Using this method,two point clouds are properly pre-aligned.Hence,we can use ICP to fne-tune the transformation parameters and minimize registration error.Experimental results demonstrate the performance and stability of the proposed system under disturbance in real-time.Several indoor tests are carried out to show that the proposed visual odometry estimation method can help to signifcantly improve the accuracy of simultaneous localization and mapping(SLAM).展开更多
Sensitivity analysis is a powerful method for discovering the significant factors that contribute to understanding the interaction between variables in multivariate datasets. A number of sensitivity analysis methods f...Sensitivity analysis is a powerful method for discovering the significant factors that contribute to understanding the interaction between variables in multivariate datasets. A number of sensitivity analysis methods fall into the class of local analysis, in which the sensitivity is defined as the partial derivatives of a target variable with respect to a group of independent variables. In a recent paper, we presented a novel pointwise local pattern exploration system for visual sensitivity analysis. Using this system, analysts are able to explore local patterns and the sensitivity at individual data points, which reveals the relationships between a focal point and its neighbors this paper we present several evaluations of the system, including case studies with real datasets, user studies the effectiveness of the visualizations and interactions, and a detailed description of the experience of a user In on展开更多
Pulse diagnosis equipment used in Traditional Chinese Medicine(TCM)has long been developed for collecting pulse information and in TCM research.However,it is still difficult to implement pulse taking automatically or ...Pulse diagnosis equipment used in Traditional Chinese Medicine(TCM)has long been developed for collecting pulse information and in TCM research.However,it is still difficult to implement pulse taking automatically or efficiently in clinical practice.Here,we present a digital protocol for TCM pulse information collection based on bionic pulse diagnosis equipment,which ensures high efficiency,reliability and data integrity of pulse diagnosis information.A four-degree-of-freedom pulse taking platform together with a wrist bracket can satisfy the spatial positioning and angle requirements for individually adaptive pulse acquisition.Three-dimensional reconstruction of a wrist surface and an image localization model are combined to provide coordinates of the acquisition position and detection direction automatically.Three series elastic joints can not only simulate the TCM pulse taking method that“Three fingers in a straight line,the middle finger determining the‘Guan’location and finger pulp pressing on the radial artery,”but also simultaneously carry out the force-controlled multi-gradient pressing process.In terms of pulse information integrity,this proposed protocol can generate rich pulse information,including basic individual information,pulse localization distribution,multi-gradient dynamic pulse force time series,and objective pulse parameters,which can help establish the fundamental data sets that are required as the pulse phenotype for subsequent comprehensive analysis of pulse diagnosis.The implementation of this scheme is beneficial to promote the standardization of the digitalized collection of pulse information,the effectiveness of detecting abnormal health status,and the promotion of the fundamental and clinical research of TCM,such as TCM pulse phenomics.展开更多
To use the benefits of Advanced Driver Assistance Systems(ADAS)-Tests in simulation and reality a new approach for using Augmented Reality(AR)in an automotive vehicle for testing ADAS is presented in this paper.Our pr...To use the benefits of Advanced Driver Assistance Systems(ADAS)-Tests in simulation and reality a new approach for using Augmented Reality(AR)in an automotive vehicle for testing ADAS is presented in this paper.Our procedure provides a link between simulation and reality and should enable a faster development process for future increasingly complex ADAS tests and future mobility solutions.Test fields for ADAS offer a small number of orientation points.Furthermore,these must be detected and processed at high vehicle speeds.That requires high computational power both for developing our method and its subsequent use in testing.Using image segmentation(IS),artificial intelligence(AI)for object recognition,and visual simultaneous localization and mapping(vSLAM),we aim to create a three-dimensional model with accurate information about the test site.It is expected that using AI and IS will significantly improve performance as computational speed and accuracy for AR applications in automobiles.展开更多
Simultaneous localization and mapping(SLAM) technology becomes more and more important in robot localization. The purpose of this paper is to improve the robustness of visual features to lighting changes and increase ...Simultaneous localization and mapping(SLAM) technology becomes more and more important in robot localization. The purpose of this paper is to improve the robustness of visual features to lighting changes and increase the recall rate of map re-localization under different lighting environments by optimizing the image transformation model. An image transformation method based on matches and photometric error(name the method as MPT) is proposed in this paper, and it is seamlessly integrated into the pre-processing stage of the feature-based visual SLAM framework. The results of the experiment show that the MPT method has a better matching effect on different visual features. In addition, the image transformation module encapsulated by a robot operating system(ROS) can be used with multiple visual SLAM systems and improve its re-localization effect under different lighting environments.展开更多
A new visual measurement method is proposed to estimate three-dimensional (3D) position of the object on the floor based on a single camera. The camera fixed on a robot is in an inclined position with respect to the...A new visual measurement method is proposed to estimate three-dimensional (3D) position of the object on the floor based on a single camera. The camera fixed on a robot is in an inclined position with respect to the floor. A measurement model with the camera's extrinsic parameters such as the height and pitch angle is described. Single image of a chessboard pattern placed on the floor is enough to calibrate the camera's extrinsic parameters after the camera's intrinsic parameters are calibrated. Then the position of object on the floor can be computed with the measurement model. Furthermore, the height of object can be calculated with the paired-points in the vertical line sharing the same position on the floor. Compared to the conventional method used to estimate the positions on the plane, this method can obtain the 3D positions. The indoor experiment testifies the accuracy and validity of the proposed method.展开更多
基金supported by the Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Ministry of Education(NRF-2018R1D1A1B07049932).
文摘In feature-based visual localization for small-scale scenes,local descriptors are used to estimate the camera pose of a query image.For large and ambiguous environments,learning-based hierarchical networks that employ local as well as global descriptors to reduce the search space of database images into a smaller set of reference views have been introduced.However,since global descriptors are generated using visual features,reference images with some of these features may be erroneously selected.In order to address this limitation,this paper proposes two clustering methods based on how often features appear as well as their covisibility.For both approaches,the scene is represented by voxels whose size and number are computed according to the size of the scene and the number of available 3Dpoints.In the first approach,a voxel-based histogram representing highly reoccurring scene regions is generated from reference images.A meanshift is then employed to group the most highly reoccurring voxels into place clusters based on their spatial proximity.In the second approach,a graph representing the covisibility-based relationship of voxels is built.Local matching is performed within the reference image clusters,and a perspective-n-point is employed to estimate the camera pose.The experimental results showed that camera pose estimation using the proposed approaches was more accurate than that of previous methods.
文摘Visual localization is a crucial component in the application of mobile robot and autonomous driving.Image retrieval is an efficient and effective technique in image-based localization methods.Due to the drastic variability of environmental conditions,e.g.,illumination changes,retrievalbased visual localization is severely affected and becomes a challenging problem.In this work,a general architecture is first formulated probabilistically to extract domain-invariant features through multi-domain image translation.Then,a novel gradientweighted similarity activation mapping loss(Grad-SAM)is incorporated for finer localization with high accuracy.We also propose a new adaptive triplet loss to boost the contrastive learning of the embedding in a self-supervised manner.The final coarse-to-fine image retrieval pipeline is implemented as the sequential combination of models with and without Grad-SAM loss.Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMU-Seasons dataset.The strong generalization ability of our approach is verified with the RobotCar dataset using models pre-trained on urban parts of the CMU-Seasons dataset.Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision,especially under challenging environments with illumination variance,vegetation,and night-time images.Moreover,real-site experiments have been conducted to validate the efficiency and effectiveness of the coarse-to-fine strategy for localization.
基金Supported by the National High Technology Research and Development Program of China(No.2012AA041403)National Natural Science Foundation of China(No.60905061,61305107)+1 种基金the Fundamental Research Funds for the Central Universities(No.ZXH2012N003)the Scientific Research Funds for Civil Aviation University of China(No.2012QD23x)
文摘When a vehicle travels in urban areas, onboard global positioning system (GPS) signals may be obstructed by high-rise buildings and thereby cannot provide accurate positions. It is proposed to perform localization by registering ground images to a 2D building boundary map which is generated from aerial images. Multilayer feature graphs (MFG) is employed to model building facades from the ground images. MFG was reported in the previous work to facilitate the robot scene understand- ing in urhan areas. By constructing MFG, the 2D/3D positions of features can be obtained, inclu- cling line segments, ideal lines, and all primary vertical planes. Finally, a voting-based feature weighted localization method is developed based on MFGs and the 2D building boundary map. The proposed method has been implemented and validated in physical experiments. In the proposed ex- periments, the algorithm has achieved an overall localization accuracy of 2.2m, which is better than commercial GPS working in open environments.
基金supported by the National High-Tech R&D Program(863)of China(No.2009AA04Z220)the National Natural Science Foundation of China(No.61375084)the Key Program of Shandong Provincial Natural Science Foundation,China(No.ZR2015QZ08)
文摘To overcome the shortcomings of existing robot localization sensors, such as low accuracy and poor robustness, a high precision visual localization system based on infrared-reflective artificial markers is designed and illustrated in detail in this paper. First, the hardware system of the localization sensor is developed. Secondly, we design a novel kind of infrared-reflective artificial marker whose characteristics can be extracted by the acquisition and processing of the infrared image. In addition, a confidence calculation method for marker identification is proposed to obtain the probabilistic localization results. Finally, the autonomous localization of the robot is achieved by calculating the relative pose relation between the robot and the artificial marker based on the perspective-3-point(P3P) visual localization algorithm. Numerous experiments and practical applications show that the designed localization sensor system is immune to the interferences of the illumination and observation angle changes. The precision of the sensor is ±1.94 cm for position localization and ±1.64° for angle localization. Therefore, it satisfies perfectly the requirements of localization precision for an indoor mobile robot.
基金supported by the National Natural Science Foundation of China under Grant Nos.62072284 and 61772318the Special Project of Science and Technology Innovation Base of Key Laboratory of Shandong Province for Software Engineering under Grant No.11480004042015。
文摘Indoor visual localization,i.e.,6 Degree-of-Freedom camera pose estimation for a query image with respect to a known scene,is gaining increased attention driven by rapid progress of applications such as robotics and augmented reality.However,drastic visual discrepancies between an onsite query image and prerecorded indoor images cast a significant challenge for visual localization.In this paper,based on the key observation of the constant existence of planar surfaces such as floors or walls in indoor scenes,we propose a novel system incorporating geometric information to address issues using only pixelated images.Through the system implementation,we contribute a hierarchical structure consisting of pre-scanned images and point cloud,as well as a distilled representation of the planar-element layout extracted from the original dataset.A view synthesis procedure is designed to generate synthetic images as complementary to that of a sparsely sampled dataset.Moreover,a global image descriptor based on the image statistic modality,called block mean,variance,and color(BMVC),was employed to speed up the candidate pose identification incorporated with a traditional convolutional neural network(CNN)descriptor.Experimental results on a popular benchmark demonstrate that the proposed method outperforms the state-of-the-art approaches in terms of visual localization validity and accuracy.
基金supported by National Natural Science Foundation of China(No.61403226)the State Key Laboratory of Tribology of China(No.SKLT09A03)
文摘A localization method based on distance function of projected features is presented to solve the accuracy reduction or failure problem due to occlusion and blurring caused by smog, when dealing with vision based localization for target oil and gas wellhead (OGWH). Firstly, the target OGWH is modeled as a cylinder with marker, and a vector with redundant parameter is used to describe its pose. Secondly, the explicit mapping relationship between the pose vector with redundant parameter and projected features is derived. Then, a 2D-point-to-feature distance function is proposed, as well as its derivative. Finally, based on this distance function and its derivative, an algorithm is proposed to estimate the pose of target OGWH directly according to the 2D image information, and the validity of the method is verified by both synthetic data and real image experiments. The results show that this method is able to accomplish the localization in the case of occlusion and blurring, and its anti-noise ability is good especially with noise ratio of less than 70%.
基金supported by ZTE Industry⁃University⁃Institute Coopera⁃tion Funds under Grant No.HC⁃CN⁃20210707004.
文摘With the rapid popularization of mobile devices and the wide application of various sensors,scene perception methods applied to mobile devices occupy an important position in location-based services such as navigation and augmented reality(AR).The development of deep learning technologies has greatly improved the visual perception ability of machines to scenes.The basic framework of scene visual perception,related technologies and the specific process applied to AR navigation are introduced,and future technology development is proposed.An application(APP)is designed to improve the application effect of AR navigation.The APP includes three modules:navigation map generation,cloud navigation algorithm,and client design.The navigation map generation tool works offline.The cloud saves the navigation map and provides navigation algorithms for the terminal.The terminal realizes local real-time positioning and AR path rendering.
基金Supported by the Major State Basic Research Development Program of China(2012CB720500)the National Natural Science Foundation of China(6133301021276078)+3 种基金the National Science Fund for Outstanding Young Scholars(61222303)the Fundamental Research Funds for the Central Universities,Shanghai Rising-Star Program(13QH1401200)the Program for New Century Excellent Talents in University(NCET-10-0885)Shanghai R&D Platform Construction Program(13DZ2295300)
文摘Purified terephthalic acid(PTA) is an important chemical raw material. P-xylene(PX) is transformed to terephthalic acid(TA) through oxidation process and TA is refined to produce PTA. The PX oxidation reaction is a complex process involving three-phase reaction of gas, liquid and solid. To monitor the process and to improve the product quality, as well as to visualize the fault type clearly, a fault diagnosis method based on selforganizing map(SOM) and high dimensional feature extraction method, local tangent space alignment(LTSA),is proposed. In this method, LTSA can reduce the dimension and keep the topology information simultaneously,and SOM distinguishes various states on the output map. Monitoring results of PX oxidation reaction process indicate that the LTSA–SOM can well detect and visualize the fault type.
文摘The extraction and description of image features are very important for visual simultaneous localization and mapping(V-SLAM).A rotated boosted efficient binary local image descriptor(BEBLID)SLAM(RB-SLAM)algorithm based on improved oriented fast and rotated brief(ORB)feature description is proposed in this paper,which can solve the problems of low localization accuracy and time efficiency of the current ORB-SLAM3 algorithm.Firstly,it uses the BEBLID to replace the feature point description algorithm of the original ORB to enhance the expressiveness and description efficiency of the image.Secondly,it adds rotational invariance to the BEBLID using the orientation information of the feature points.It also selects the rotationally stable bits in the BEBLID to further enhance the rotational invariance of the BEBLID.Finally,it retrains the binary visual dictionary based on the BEBLID to reduce the cumulative error of V-SLAM and improve the loading speed of the visual dictionary.Experiments show that the dictionary loading efficiency is improved by more than 10 times.The RB-SLAM algorithm improves the trajectory accuracy by 24.75%on the TUM dataset and 26.25%on the EuRoC dataset compared to the ORB-SLAM3 algorithm.
基金supported by the National Key Research and Development Program of China under[Grant number 2019YFC1511304]supported by the Pilot Fund of Frontier Science and Disruptive Technology of Aerospace Information Research Institute,Chinese Academy of Sciences under[Grant number E0Z21101].
文摘Simultaneous Localization and Mapping(SLAM)has been widely used in emergency response,self-driving and city-scale 3D mapping and navigation.Recent deep-learning based feature point extractors have demonstrated superior performance in dealing with the complex environmental challenges(e.g.extreme lighting)while the traditional extractors are struggling.In this paper,we have successfully improved the robustness and accuracy of a monocular visual SLAM system under various complex scenes by adding a deep learning based visual localization thread as an augmentation to the visual SLAM framework.In this thread,our feature extractor with an efficient lightweight deep neural network is used for absolute pose and scale estimation in real time using the highly accurate georeferenced prior map database at 20cm geometric accuracy created by our in-house and low-cost LiDAR and camera integrated device.The closed-loop error provided by our SLAM system with and without this enhancement is 1.03m and 18.28m respectively.The scale estimation of the monocular visual SLAM is also significantly improved(0.01 versus 0.98).In addition,a novel camera-LiDAR calibration workflow is also provided for large-scale 3D mapping.This paper demonstrates the application and research potential of deep-learning based vision SLAM with image and LiDAR sensors.
基金supported by National High Technology Research Development Program of China (863 Program) (No.2011AA040202)National Science Foundation of China (No.51005008)
文摘This paper presents a hierarchical simultaneous localization and mapping(SLAM) system for a small unmanned aerial vehicle(UAV) using the output of an inertial measurement unit(IMU) and the bearing-only observations from an onboard monocular camera.A homography based approach is used to calculate the motion of the vehicle in 6 degrees of freedom by image feature match.This visual measurement is fused with the inertial outputs by an indirect extended Kalman filter(EKF) for attitude and velocity estimation.Then,another EKF is employed to estimate the position of the vehicle and the locations of the features in the map.Both simulations and experiments are carried out to test the performance of the proposed system.The result of the comparison with the referential global positioning system/inertial navigation system(GPS/INS) navigation indicates that the proposed SLAM can provide reliable and stable state estimation for small UAVs in GPS-denied environments.
文摘In this paper,we present a novel algorithm for odometry estimation based on ceiling vision.The main contribution of this algorithm is the introduction of principal direction detection that can greatly reduce error accumulation problem in most visual odometry estimation approaches.The principal direction is defned based on the fact that our ceiling is flled with artifcial vertical and horizontal lines which can be used as reference for the current robot s heading direction.The proposed approach can be operated in real-time and it performs well even with camera s disturbance.A moving low-cost RGB-D camera(Kinect),mounted on a robot,is used to continuously acquire point clouds.Iterative closest point(ICP) is the common way to estimate the current camera position by registering the currently captured point cloud to the previous one.However,its performance sufers from data association problem or it requires pre-alignment information.The performance of the proposed principal direction detection approach does not rely on data association knowledge.Using this method,two point clouds are properly pre-aligned.Hence,we can use ICP to fne-tune the transformation parameters and minimize registration error.Experimental results demonstrate the performance and stability of the proposed system under disturbance in real-time.Several indoor tests are carried out to show that the proposed visual odometry estimation method can help to signifcantly improve the accuracy of simultaneous localization and mapping(SLAM).
基金Supported by the National Science Foundation of USA (No.IIS-0812027)
文摘Sensitivity analysis is a powerful method for discovering the significant factors that contribute to understanding the interaction between variables in multivariate datasets. A number of sensitivity analysis methods fall into the class of local analysis, in which the sensitivity is defined as the partial derivatives of a target variable with respect to a group of independent variables. In a recent paper, we presented a novel pointwise local pattern exploration system for visual sensitivity analysis. Using this system, analysts are able to explore local patterns and the sensitivity at individual data points, which reveals the relationships between a focal point and its neighbors this paper we present several evaluations of the system, including case studies with real datasets, user studies the effectiveness of the visualizations and interactions, and a detailed description of the experience of a user In on
基金supported by the Shanghai 2021 Science and Technology Innovation Action Plan Project(Grant No.21S31902500)the Independent Deployment of Scientific Research Projects of Jihua Laboratory(Grant No.X190051TB190)the National Natural Science Foundation of China(Grant No.U1913216).
文摘Pulse diagnosis equipment used in Traditional Chinese Medicine(TCM)has long been developed for collecting pulse information and in TCM research.However,it is still difficult to implement pulse taking automatically or efficiently in clinical practice.Here,we present a digital protocol for TCM pulse information collection based on bionic pulse diagnosis equipment,which ensures high efficiency,reliability and data integrity of pulse diagnosis information.A four-degree-of-freedom pulse taking platform together with a wrist bracket can satisfy the spatial positioning and angle requirements for individually adaptive pulse acquisition.Three-dimensional reconstruction of a wrist surface and an image localization model are combined to provide coordinates of the acquisition position and detection direction automatically.Three series elastic joints can not only simulate the TCM pulse taking method that“Three fingers in a straight line,the middle finger determining the‘Guan’location and finger pulp pressing on the radial artery,”but also simultaneously carry out the force-controlled multi-gradient pressing process.In terms of pulse information integrity,this proposed protocol can generate rich pulse information,including basic individual information,pulse localization distribution,multi-gradient dynamic pulse force time series,and objective pulse parameters,which can help establish the fundamental data sets that are required as the pulse phenotype for subsequent comprehensive analysis of pulse diagnosis.The implementation of this scheme is beneficial to promote the standardization of the digitalized collection of pulse information,the effectiveness of detecting abnormal health status,and the promotion of the fundamental and clinical research of TCM,such as TCM pulse phenomics.
文摘To use the benefits of Advanced Driver Assistance Systems(ADAS)-Tests in simulation and reality a new approach for using Augmented Reality(AR)in an automotive vehicle for testing ADAS is presented in this paper.Our procedure provides a link between simulation and reality and should enable a faster development process for future increasingly complex ADAS tests and future mobility solutions.Test fields for ADAS offer a small number of orientation points.Furthermore,these must be detected and processed at high vehicle speeds.That requires high computational power both for developing our method and its subsequent use in testing.Using image segmentation(IS),artificial intelligence(AI)for object recognition,and visual simultaneous localization and mapping(vSLAM),we aim to create a three-dimensional model with accurate information about the test site.It is expected that using AI and IS will significantly improve performance as computational speed and accuracy for AR applications in automobiles.
文摘Simultaneous localization and mapping(SLAM) technology becomes more and more important in robot localization. The purpose of this paper is to improve the robustness of visual features to lighting changes and increase the recall rate of map re-localization under different lighting environments by optimizing the image transformation model. An image transformation method based on matches and photometric error(name the method as MPT) is proposed in this paper, and it is seamlessly integrated into the pre-processing stage of the feature-based visual SLAM framework. The results of the experiment show that the MPT method has a better matching effect on different visual features. In addition, the image transformation module encapsulated by a robot operating system(ROS) can be used with multiple visual SLAM systems and improve its re-localization effect under different lighting environments.
基金supported by National Natural Science Foundation of China(Nos.61273352 and 61473295)National High Technology Research and Development Program of China(863 Program)(No.2015AA042307)Beijing Natural Science Foundation(No.4161002)
文摘A new visual measurement method is proposed to estimate three-dimensional (3D) position of the object on the floor based on a single camera. The camera fixed on a robot is in an inclined position with respect to the floor. A measurement model with the camera's extrinsic parameters such as the height and pitch angle is described. Single image of a chessboard pattern placed on the floor is enough to calibrate the camera's extrinsic parameters after the camera's intrinsic parameters are calibrated. Then the position of object on the floor can be computed with the measurement model. Furthermore, the height of object can be calculated with the paired-points in the vertical line sharing the same position on the floor. Compared to the conventional method used to estimate the positions on the plane, this method can obtain the 3D positions. The indoor experiment testifies the accuracy and validity of the proposed method.