Visual simultaneous localization and mapping(SLAM)is crucial in robotics and autonomous driving.However,traditional visual SLAM faces challenges in dynamic environments.To address this issue,researchers have proposed ...Visual simultaneous localization and mapping(SLAM)is crucial in robotics and autonomous driving.However,traditional visual SLAM faces challenges in dynamic environments.To address this issue,researchers have proposed semantic SLAM,which combines object detection,semantic segmentation,instance segmentation,and visual SLAM.Despite the growing body of literature on semantic SLAM,there is currently a lack of comprehensive research on the integration of object detection and visual SLAM.Therefore,this study aims to gather information from multiple databases and review relevant literature using specific keywords.It focuses on visual SLAM based on object detection,covering different aspects.Firstly,it discusses the current research status and challenges in this field,highlighting methods for incorporating semantic information from object detection networks into mileage measurement,closed-loop detection,and map construction.It also compares the characteristics and performance of various visual SLAM object detection algorithms.Lastly,it provides an outlook on future research directions and emerging trends in visual SLAM.Research has shown that visual SLAM based on object detection has significant improvements compared to traditional SLAM in dynamic point removal,data association,point cloud segmentation,and other technologies.It can improve the robustness and accuracy of the entire SLAM system and can run in real time.With the continuous optimization of algorithms and the improvement of hardware level,object visual SLAM has great potential for development.展开更多
A great number of visual simultaneous localization and mapping(VSLAM)systems need to assume static features in the environment.However,moving objects can vastly impair the performance of a VSLAM system which relies on...A great number of visual simultaneous localization and mapping(VSLAM)systems need to assume static features in the environment.However,moving objects can vastly impair the performance of a VSLAM system which relies on the static-world assumption.To cope with this challenging topic,a real-time and robust VSLAM system based on ORB-SLAM2 for dynamic environments was proposed.To reduce the influence of dynamic content,we incorporate the deep-learning-based object detection method in the visual odometry,then the dynamic object probability model is added to raise the efficiency of object detection deep neural network and enhance the real-time performance of our system.Experiment with both on the TUM and KITTI benchmark dataset,as well as in a real-world environment,the results clarify that our method can significantly reduce the tracking error or drift,enhance the robustness,accuracy and stability of the VSLAM system in dynamic scenes.展开更多
This article presents a brief survey to visual simultaneous localization and mapping (SLAM) systems applied to multiple independently moving agents, such as a team of ground or aerial vehicles, a group of users holdin...This article presents a brief survey to visual simultaneous localization and mapping (SLAM) systems applied to multiple independently moving agents, such as a team of ground or aerial vehicles, a group of users holding augmented or virtual reality devices. Such visual SLAM system, name as collaborative visual SLAM, is different from a typical visual SLAM deployed on a single agent in that information is exchanged or shared among different agents to achieve better robustness, efficiency, and accuracy. We review the representative works on this topic proposed in the past ten years and describe the key components involved in designing such a system including collaborative pose estimation and mapping tasks, as well as the emerging topic of decentralized architecture. We believe this brief survey could be helpful to someone who are working on this topic or developing multi-agent applications, particularly micro-aerial vehicle swarm or collaborative augmented/virtual reality.展开更多
Feature selection is always an important issue in the visual SLAM （simultaneous location and mapping） literature. Considering that the location estimation can be improved by tracking features with larger value of vi...Feature selection is always an important issue in the visual SLAM （simultaneous location and mapping） literature. Considering that the location estimation can be improved by tracking features with larger value of visible time, a new feature selection method based on motion estimation is proposed. First, a k-step iteration algorithm is presented for visible time estimation using an affme motion model; then a delayed feature detection method is introduced for efficiently detecting features with the maximum visible time. As a means of validation for the proposed method, both simulation and real data experiments are carded out. Results show that the proposed method can improve both the estimation performance and the computational performance compared with the existing random feature selection method.展开更多
This paper presents a modified Rao-Blackwellized Particle Filter (RBPF) approach for the bearing-only monocular SLAM problem. While FastSLAM 2.0 is known to be one of the most computationally efficient SLAM approaches...This paper presents a modified Rao-Blackwellized Particle Filter (RBPF) approach for the bearing-only monocular SLAM problem. While FastSLAM 2.0 is known to be one of the most computationally efficient SLAM approaches;it is not applicable to certain formulations of the SLAM problem in which some of the states are not explicitly expressed in the measurement equation. This constraint impacts the versatility of the FastSLAM 2.0 in dealing with partially ob-servable systems, especially in dynamic environments where inclusion of higher order but unobservable states such as velocity and acceleration in the filtering process is highly desirable. In this paper, the formulation of an enhanced RBPF-based SLAM with proper sampling and importance weights calculation for resampling distributions is presented. As an example, the new formulation uses the higher order states of the pose of a monocular camera to carry out SLAM for a mobile robot. The results of the experiments on the robot verify the improved performance of the higher order RBPF under low parallax angles conditions.展开更多
The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an...The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an inaccurate pose estimation or failed tracking.To address the challenge above,a new method of feature point matching is proposed in this paper,which combines the variable velocity model with the reverse optical flow method.First,the constant velocity model is extended to a new variable velocity model,and the expanded variable velocity model is used to provide the initial pixel shifting for the reverse optical flow method.Then the search range of feature points is accurately determined according to the results of the reverse optical flow method,thereby improving the accuracy and reliability of feature matching,with strengthened interframe tracking effects.Finally,we tested on TUM data set based on the RGB-D camera.Experimental results show that this method can reduce the probability of tracking failure and improve localization accuracy on SLAM(Simultaneous Localization and Mapping)systems.Compared with the traditional ORB-SLAM2,the test error of this method on each sequence in the TUM data set is significantly reduced,and the root mean square error is only 63.8%of the original system under the optimal condition.展开更多
Visual SLAM(Simultaneously Localization and Mapping)is a solution to achieve localization and mapping of robots simultaneously.Significant achievements have been made during the past decades,geography-based methods ar...Visual SLAM(Simultaneously Localization and Mapping)is a solution to achieve localization and mapping of robots simultaneously.Significant achievements have been made during the past decades,geography-based methods are becoming more and more successful in dealing with static environments.However,they still cannot handle a challenging environment.With the great achievements of deep learning methods in the field of computer vision,there is a trend of applying deep learning methods to visual SLAM.In this paper,the latest research progress of deep learning applied to the field of visual SLAM is reviewed.The outstanding research results of deep learning visual odometry and deep learning loop closure detect are summarized.Finally,future development directions of visual SLAM based on deep learning is prospected.展开更多
The entorhinal-hippocampus structure in the mammalian brain is the core area for realizing spatial cognition.However,the visual perception and loop detection methods in the current biomimetic robot navigation model st...The entorhinal-hippocampus structure in the mammalian brain is the core area for realizing spatial cognition.However,the visual perception and loop detection methods in the current biomimetic robot navigation model still rely on traditional visual SLAM schemes and lack the process of exploring and applying biological visual methods.Based on this,we propose amap constructionmethod thatmimics the entorhinal-hippocampal cognitive mechanismof the rat brain according to the response of entorhinal cortex neurons to eye saccades in recent related studies.That is,when mammals are free to watch the scene,the entorhinal cortex neurons will encode the saccade position of the eyeball to realize the episodicmemory function.The characteristics of thismodel are as follows:1)A scenememory algorithmthat relies on visual saccade vectors is constructed to imitate the biological brain’s memory of environmental situation information matches the current scene information with the memory;2)According to the information transmission mechanism formed by the hippocampus and the activation theory of spatial cells,a localization model based on the grid cells of the entorhinal cortex and the place cells of the hippocampus was constructed;3)Finally,the scene memory algorithm is used to correct the errors of the positioning model and complete the process of constructing the cognitive map.The model was subjected to simulation experiments on publicly available datasets and physical experiments using a mobile robot platform to verify the environmental adaptability and robustness of the algorithm.The algorithm will provide a basis for further research into bionic robot navigation.展开更多
This paper presents a hierarchical simultaneous localization and mapping（SLAM） system for a small unmanned aerial vehicle（UAV） using the output of an inertial measurement unit（IMU） and the bearing-only observati...This paper presents a hierarchical simultaneous localization and mapping（SLAM） system for a small unmanned aerial vehicle（UAV） using the output of an inertial measurement unit（IMU） and the bearing-only observations from an onboard monocular camera.A homography based approach is used to calculate the motion of the vehicle in 6 degrees of freedom by image feature match.This visual measurement is fused with the inertial outputs by an indirect extended Kalman filter（EKF） for attitude and velocity estimation.Then,another EKF is employed to estimate the position of the vehicle and the locations of the features in the map.Both simulations and experiments are carried out to test the performance of the proposed system.The result of the comparison with the referential global positioning system/inertial navigation system（GPS/INS） navigation indicates that the proposed SLAM can provide reliable and stable state estimation for small UAVs in GPS-denied environments.展开更多
基金the National Natural Science Foundation of China(No.62063006)to the Natural Science Foundation of Guangxi Province(No.2023GXNS-FAA026025)+3 种基金to the Innovation Fund of Chinese Universities Industry-University-Research(ID:2021RYC06005)to the Research Project for Young and Middle-aged Teachers in Guangxi Universities(ID:2020KY15013)to the Special Research Project of Hechi University(ID:2021GCC028)supported by the Project of Outstanding Thousand Young Teachers’Training in Higher Education Institutions of Guangxi,Guangxi Colleges and Universities Key Laboratory of AI and Information Processing(Hechi University),Education Department of Guangxi Zhuang Autonomous Region.
文摘Visual simultaneous localization and mapping(SLAM)is crucial in robotics and autonomous driving.However,traditional visual SLAM faces challenges in dynamic environments.To address this issue,researchers have proposed semantic SLAM,which combines object detection,semantic segmentation,instance segmentation,and visual SLAM.Despite the growing body of literature on semantic SLAM,there is currently a lack of comprehensive research on the integration of object detection and visual SLAM.Therefore,this study aims to gather information from multiple databases and review relevant literature using specific keywords.It focuses on visual SLAM based on object detection,covering different aspects.Firstly,it discusses the current research status and challenges in this field,highlighting methods for incorporating semantic information from object detection networks into mileage measurement,closed-loop detection,and map construction.It also compares the characteristics and performance of various visual SLAM object detection algorithms.Lastly,it provides an outlook on future research directions and emerging trends in visual SLAM.Research has shown that visual SLAM based on object detection has significant improvements compared to traditional SLAM in dynamic point removal,data association,point cloud segmentation,and other technologies.It can improve the robustness and accuracy of the entire SLAM system and can run in real time.With the continuous optimization of algorithms and the improvement of hardware level,object visual SLAM has great potential for development.
基金the National Natural Science Foundation of China(No.61671470).
文摘A great number of visual simultaneous localization and mapping(VSLAM)systems need to assume static features in the environment.However,moving objects can vastly impair the performance of a VSLAM system which relies on the static-world assumption.To cope with this challenging topic,a real-time and robust VSLAM system based on ORB-SLAM2 for dynamic environments was proposed.To reduce the influence of dynamic content,we incorporate the deep-learning-based object detection method in the visual odometry,then the dynamic object probability model is added to raise the efficiency of object detection deep neural network and enhance the real-time performance of our system.Experiment with both on the TUM and KITTI benchmark dataset,as well as in a real-world environment,the results clarify that our method can significantly reduce the tracking error or drift,enhance the robustness,accuracy and stability of the VSLAM system in dynamic scenes.
基金Project Grant JZX7Y2-0190258055601National Natural Science Foundation of China(61402283).
文摘This article presents a brief survey to visual simultaneous localization and mapping (SLAM) systems applied to multiple independently moving agents, such as a team of ground or aerial vehicles, a group of users holding augmented or virtual reality devices. Such visual SLAM system, name as collaborative visual SLAM, is different from a typical visual SLAM deployed on a single agent in that information is exchanged or shared among different agents to achieve better robustness, efficiency, and accuracy. We review the representative works on this topic proposed in the past ten years and describe the key components involved in designing such a system including collaborative pose estimation and mapping tasks, as well as the emerging topic of decentralized architecture. We believe this brief survey could be helpful to someone who are working on this topic or developing multi-agent applications, particularly micro-aerial vehicle swarm or collaborative augmented/virtual reality.
文摘Feature selection is always an important issue in the visual SLAM （simultaneous location and mapping） literature. Considering that the location estimation can be improved by tracking features with larger value of visible time, a new feature selection method based on motion estimation is proposed. First, a k-step iteration algorithm is presented for visible time estimation using an affme motion model; then a delayed feature detection method is introduced for efficiently detecting features with the maximum visible time. As a means of validation for the proposed method, both simulation and real data experiments are carded out. Results show that the proposed method can improve both the estimation performance and the computational performance compared with the existing random feature selection method.
文摘This paper presents a modified Rao-Blackwellized Particle Filter (RBPF) approach for the bearing-only monocular SLAM problem. While FastSLAM 2.0 is known to be one of the most computationally efficient SLAM approaches;it is not applicable to certain formulations of the SLAM problem in which some of the states are not explicitly expressed in the measurement equation. This constraint impacts the versatility of the FastSLAM 2.0 in dealing with partially ob-servable systems, especially in dynamic environments where inclusion of higher order but unobservable states such as velocity and acceleration in the filtering process is highly desirable. In this paper, the formulation of an enhanced RBPF-based SLAM with proper sampling and importance weights calculation for resampling distributions is presented. As an example, the new formulation uses the higher order states of the pose of a monocular camera to carry out SLAM for a mobile robot. The results of the experiments on the robot verify the improved performance of the higher order RBPF under low parallax angles conditions.
基金This work was supported by The National Natural Science Foundation of China under Grant No.61304205 and NO.61502240The Natural Science Foundation of Jiangsu Province under Grant No.BK20191401 and No.BK20201136Postgraduate Research&Practice Innovation Program of Jiangsu Province under Grant No.SJCX21_0364 and No.SJCX21_0363.
文摘The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an inaccurate pose estimation or failed tracking.To address the challenge above,a new method of feature point matching is proposed in this paper,which combines the variable velocity model with the reverse optical flow method.First,the constant velocity model is extended to a new variable velocity model,and the expanded variable velocity model is used to provide the initial pixel shifting for the reverse optical flow method.Then the search range of feature points is accurately determined according to the results of the reverse optical flow method,thereby improving the accuracy and reliability of feature matching,with strengthened interframe tracking effects.Finally,we tested on TUM data set based on the RGB-D camera.Experimental results show that this method can reduce the probability of tracking failure and improve localization accuracy on SLAM(Simultaneous Localization and Mapping)systems.Compared with the traditional ORB-SLAM2,the test error of this method on each sequence in the TUM data set is significantly reduced,and the root mean square error is only 63.8%of the original system under the optimal condition.
文摘Visual SLAM(Simultaneously Localization and Mapping)is a solution to achieve localization and mapping of robots simultaneously.Significant achievements have been made during the past decades,geography-based methods are becoming more and more successful in dealing with static environments.However,they still cannot handle a challenging environment.With the great achievements of deep learning methods in the field of computer vision,there is a trend of applying deep learning methods to visual SLAM.In this paper,the latest research progress of deep learning applied to the field of visual SLAM is reviewed.The outstanding research results of deep learning visual odometry and deep learning loop closure detect are summarized.Finally,future development directions of visual SLAM based on deep learning is prospected.
基金This research was funded by the National Science Foundation of China,Grant No.62076014as well as the Beijing Natural Science Foundation under Grant No.4162012.
文摘The entorhinal-hippocampus structure in the mammalian brain is the core area for realizing spatial cognition.However,the visual perception and loop detection methods in the current biomimetic robot navigation model still rely on traditional visual SLAM schemes and lack the process of exploring and applying biological visual methods.Based on this,we propose amap constructionmethod thatmimics the entorhinal-hippocampal cognitive mechanismof the rat brain according to the response of entorhinal cortex neurons to eye saccades in recent related studies.That is,when mammals are free to watch the scene,the entorhinal cortex neurons will encode the saccade position of the eyeball to realize the episodicmemory function.The characteristics of thismodel are as follows:1)A scenememory algorithmthat relies on visual saccade vectors is constructed to imitate the biological brain’s memory of environmental situation information matches the current scene information with the memory;2)According to the information transmission mechanism formed by the hippocampus and the activation theory of spatial cells,a localization model based on the grid cells of the entorhinal cortex and the place cells of the hippocampus was constructed;3)Finally,the scene memory algorithm is used to correct the errors of the positioning model and complete the process of constructing the cognitive map.The model was subjected to simulation experiments on publicly available datasets and physical experiments using a mobile robot platform to verify the environmental adaptability and robustness of the algorithm.The algorithm will provide a basis for further research into bionic robot navigation.
基金supported by National High Technology Research Development Program of China (863 Program) (No.2011AA040202)National Science Foundation of China (No.51005008)
文摘This paper presents a hierarchical simultaneous localization and mapping（SLAM） system for a small unmanned aerial vehicle（UAV） using the output of an inertial measurement unit（IMU） and the bearing-only observations from an onboard monocular camera.A homography based approach is used to calculate the motion of the vehicle in 6 degrees of freedom by image feature match.This visual measurement is fused with the inertial outputs by an indirect extended Kalman filter（EKF） for attitude and velocity estimation.Then,another EKF is employed to estimate the position of the vehicle and the locations of the features in the map.Both simulations and experiments are carried out to test the performance of the proposed system.The result of the comparison with the referential global positioning system/inertial navigation system（GPS/INS） navigation indicates that the proposed SLAM can provide reliable and stable state estimation for small UAVs in GPS-denied environments.
