In this paper a semi-direct visual odometry and mapping system is proposed with a RGB-D camera,which combines the merits of both feature based and direct based methods.The presented system directly estimates the camer...In this paper a semi-direct visual odometry and mapping system is proposed with a RGB-D camera,which combines the merits of both feature based and direct based methods.The presented system directly estimates the camera motion of two consecutive RGB-D frames by minimizing the photometric error.To permit outliers and noise,a robust sensor model built upon the t-distribution and an error function mixing depth and photometric errors are used to enhance the accuracy and robustness.Local graph optimization based on key frames is used to reduce the accumulative error and refine the local map.The loop closure detection method,which combines the appearance similarity method and spatial location constraints method,increases the speed of detection.Experimental results demonstrate that the proposed approach achieves higher accuracy on the motion estimation and environment reconstruction compared to the other state-of-the-art methods. Moreover,the proposed approach works in real-time on a laptop without a GPU,which makes it attractive for robots equipped with limited computational resources.展开更多
An improved method with better selection capability using a single camera was presented in comparison with previous method. To improve performance, two methods were applied to landmark selection in an unfamiliar indoo...An improved method with better selection capability using a single camera was presented in comparison with previous method. To improve performance, two methods were applied to landmark selection in an unfamiliar indoor environment. First, a modified visual attention method was proposed to automatically select a candidate region as a more useful landmark. In visual attention, candidate landmark regions were selected with different characteristics of ambient color and intensity in the image. Then, the more useful landmarks were selected by combining the candidate regions using clustering. As generally implemented, automatic landmark selection by vision-based simultaneous localization and mapping(SLAM) results in many useless landmarks, because the features of images are distinguished from the surrounding environment but detected repeatedly. These useless landmarks create a serious problem for the SLAM system because they complicate data association. To address this, a method was proposed in which the robot initially collected landmarks through automatic detection while traversing the entire area where the robot performed SLAM, and then, the robot selected only those landmarks that exhibited high rarity through clustering, which enhanced the system performance. Experimental results show that this method of automatic landmark selection results in selection of a high-rarity landmark. The average error of the performance of SLAM decreases 52% compared with conventional methods and the accuracy of data associations increases.展开更多
vSLAM(visual Simultaneous Localization and Mapping)是一种基于视觉传感器实现同时定位与建图的技术,不仅可为地面机器人提供服务,同时在无人机的定位导航中也有着非常重要的应用。对基于无人机的vSLAM发展概况进行整理研究,就其中...vSLAM(visual Simultaneous Localization and Mapping)是一种基于视觉传感器实现同时定位与建图的技术,不仅可为地面机器人提供服务,同时在无人机的定位导航中也有着非常重要的应用。对基于无人机的vSLAM发展概况进行整理研究,就其中几大关键方向的研究现状予以介绍,主要包括结合IMU、结合光流传感器的vSLAM,同时总结目前研究中仍存在的一些问题和不足之处。结合经典理论与最新研究动态,对基于无人机的vSLAM重点研究内容和未来发展方向提出了新的展望。展开更多
Aiming at the problem of system error and noise in simultaneous localization and mapping(SLAM) technology, we propose a calibration model based on Project Tango device and a loop closure detection algorithm based on v...Aiming at the problem of system error and noise in simultaneous localization and mapping(SLAM) technology, we propose a calibration model based on Project Tango device and a loop closure detection algorithm based on visual vocabulary with memory management. The graph optimization is also combined to achieve a running application. First, the color image and depth information of the environment are collected to establish the calibration model of system error and noise. Second, with constraint condition provided by loop closure detection algorithm, speed up robust feature is calculated and matched. Finally, the motion pose model is solved, and the optimal scene model is determined by graph optimization method. This method is compared with Open Constructor for reconstruction on several experimental scenarios. The results show the number of model's points and faces are larger than Open Constructor's, and the scanning time is less than Open Constructor's. The experimental results show the feasibility and efficiency of the proposed algorithm.展开更多
There are about 253 million people with visual impairment worldwide.Many of them use a white cane and/or a guide dog as the mobility tool for daily travel.Despite decades of efforts,electronic navigation aid that can ...There are about 253 million people with visual impairment worldwide.Many of them use a white cane and/or a guide dog as the mobility tool for daily travel.Despite decades of efforts,electronic navigation aid that can replace white cane is still research in progress.In this paper,we propose an RGB-D camera based visual positioning system(VPS)for real-time localization of a robotic navigation aid(RNA)in an architectural floor plan for assistive navigation.The core of the system is the combination of a new 6-DOF depth-enhanced visual-inertial odometry(DVIO)method and a particle filter localization(PFL)method.DVIO estimates RNA’s pose by using the data from an RGB-D camera and an inertial measurement unit(IMU).It extracts the floor plane from the camera’s depth data and tightly couples the floor plane,the visual features(with and without depth data),and the IMU’s inertial data in a graph optimization framework to estimate the device’s 6-DOF pose.Due to the use of the floor plane and depth data from the RGB-D camera,DVIO has a better pose estimation accuracy than the conventional VIO method.To reduce the accumulated pose error of DVIO for navigation in a large indoor space,we developed the PFL method to locate RNA in the floor plan.PFL leverages geometric information of the architectural CAD drawing of an indoor space to further reduce the error of the DVIO-estimated pose.Based on VPS,an assistive navigation system is developed for the RNA prototype to assist a visually impaired person in navigating a large indoor space.Experimental results demonstrate that:1)DVIO method achieves better pose estimation accuracy than the state-of-the-art VIO method and performs real-time pose estimation(18 Hz pose update rate)on a UP Board computer;2)PFL reduces the DVIO-accrued pose error by 82.5%on average and allows for accurate wayfinding(endpoint position error≤45 cm)in large indoor spaces.展开更多
A fundamental task for mobile robots is simultaneous localization and mapping(SLAM).Moreover,long-term robustness is an important property for SLAM.When vehicles or robots steer fast or steer in certain scenarios,such...A fundamental task for mobile robots is simultaneous localization and mapping(SLAM).Moreover,long-term robustness is an important property for SLAM.When vehicles or robots steer fast or steer in certain scenarios,such as low-texture environments,long corridors,tunnels,or other duplicated structural environments,most SLAM systems might fail.In this paper,we propose a novel robust visual inertial light detection and ranging(Li Da R)navigation(VILN)SLAM system,including stereo visual-inertial Li Da R odometry and visual-Li Da R loop closure.The proposed VILN SLAM system can perform well with low drift after long-term experiments,even when the Li Da R or visual measurements are degraded occasionally in complex scenes.Extensive experimental results show that the robustness has been greatly improved in various scenarios compared to state-of-the-art SLAM systems.展开更多
随着实际应用需求的不断提高,传统视觉同步定位与地图构建(visual simultaneous localization and mapping,VSLAM)难以建立符合要求的环境地图。语义VSLAM在传统VSLAM基础框架上引入深度学习获取环境图像的语义信息,并且构建环境的3D点...随着实际应用需求的不断提高,传统视觉同步定位与地图构建(visual simultaneous localization and mapping,VSLAM)难以建立符合要求的环境地图。语义VSLAM在传统VSLAM基础框架上引入深度学习获取环境图像的语义信息,并且构建环境的3D点云语义地图,能为更高层次的交互任务提供有效支持。通过分析传统的VSLAM框架中视觉里程计、回环与优化以及建图上的不足,阐述了近年来使用语义信息改善VSLAM系统性能及为环境交互任务提供助力的方法。最后,结合研究现状结论及课题组正在开展的工作对语义VSLAM未来研究方向及应用进行展望。展开更多
基金Supported by the National Natural Science Foundation of China(61501034)
文摘In this paper a semi-direct visual odometry and mapping system is proposed with a RGB-D camera,which combines the merits of both feature based and direct based methods.The presented system directly estimates the camera motion of two consecutive RGB-D frames by minimizing the photometric error.To permit outliers and noise,a robust sensor model built upon the t-distribution and an error function mixing depth and photometric errors are used to enhance the accuracy and robustness.Local graph optimization based on key frames is used to reduce the accumulative error and refine the local map.The loop closure detection method,which combines the appearance similarity method and spatial location constraints method,increases the speed of detection.Experimental results demonstrate that the proposed approach achieves higher accuracy on the motion estimation and environment reconstruction compared to the other state-of-the-art methods. Moreover,the proposed approach works in real-time on a laptop without a GPU,which makes it attractive for robots equipped with limited computational resources.
文摘An improved method with better selection capability using a single camera was presented in comparison with previous method. To improve performance, two methods were applied to landmark selection in an unfamiliar indoor environment. First, a modified visual attention method was proposed to automatically select a candidate region as a more useful landmark. In visual attention, candidate landmark regions were selected with different characteristics of ambient color and intensity in the image. Then, the more useful landmarks were selected by combining the candidate regions using clustering. As generally implemented, automatic landmark selection by vision-based simultaneous localization and mapping(SLAM) results in many useless landmarks, because the features of images are distinguished from the surrounding environment but detected repeatedly. These useless landmarks create a serious problem for the SLAM system because they complicate data association. To address this, a method was proposed in which the robot initially collected landmarks through automatic detection while traversing the entire area where the robot performed SLAM, and then, the robot selected only those landmarks that exhibited high rarity through clustering, which enhanced the system performance. Experimental results show that this method of automatic landmark selection results in selection of a high-rarity landmark. The average error of the performance of SLAM decreases 52% compared with conventional methods and the accuracy of data associations increases.
文摘vSLAM(visual Simultaneous Localization and Mapping)是一种基于视觉传感器实现同时定位与建图的技术,不仅可为地面机器人提供服务,同时在无人机的定位导航中也有着非常重要的应用。对基于无人机的vSLAM发展概况进行整理研究,就其中几大关键方向的研究现状予以介绍,主要包括结合IMU、结合光流传感器的vSLAM,同时总结目前研究中仍存在的一些问题和不足之处。结合经典理论与最新研究动态,对基于无人机的vSLAM重点研究内容和未来发展方向提出了新的展望。
基金Supported by the National Natural Science Foundation of China(61772379)
文摘Aiming at the problem of system error and noise in simultaneous localization and mapping(SLAM) technology, we propose a calibration model based on Project Tango device and a loop closure detection algorithm based on visual vocabulary with memory management. The graph optimization is also combined to achieve a running application. First, the color image and depth information of the environment are collected to establish the calibration model of system error and noise. Second, with constraint condition provided by loop closure detection algorithm, speed up robust feature is calculated and matched. Finally, the motion pose model is solved, and the optimal scene model is determined by graph optimization method. This method is compared with Open Constructor for reconstruction on several experimental scenarios. The results show the number of model's points and faces are larger than Open Constructor's, and the scanning time is less than Open Constructor's. The experimental results show the feasibility and efficiency of the proposed algorithm.
基金supported by the NIBIB and the NEI of the National Institutes of Health(R01EB018117)。
文摘There are about 253 million people with visual impairment worldwide.Many of them use a white cane and/or a guide dog as the mobility tool for daily travel.Despite decades of efforts,electronic navigation aid that can replace white cane is still research in progress.In this paper,we propose an RGB-D camera based visual positioning system(VPS)for real-time localization of a robotic navigation aid(RNA)in an architectural floor plan for assistive navigation.The core of the system is the combination of a new 6-DOF depth-enhanced visual-inertial odometry(DVIO)method and a particle filter localization(PFL)method.DVIO estimates RNA’s pose by using the data from an RGB-D camera and an inertial measurement unit(IMU).It extracts the floor plane from the camera’s depth data and tightly couples the floor plane,the visual features(with and without depth data),and the IMU’s inertial data in a graph optimization framework to estimate the device’s 6-DOF pose.Due to the use of the floor plane and depth data from the RGB-D camera,DVIO has a better pose estimation accuracy than the conventional VIO method.To reduce the accumulated pose error of DVIO for navigation in a large indoor space,we developed the PFL method to locate RNA in the floor plan.PFL leverages geometric information of the architectural CAD drawing of an indoor space to further reduce the error of the DVIO-estimated pose.Based on VPS,an assistive navigation system is developed for the RNA prototype to assist a visually impaired person in navigating a large indoor space.Experimental results demonstrate that:1)DVIO method achieves better pose estimation accuracy than the state-of-the-art VIO method and performs real-time pose estimation(18 Hz pose update rate)on a UP Board computer;2)PFL reduces the DVIO-accrued pose error by 82.5%on average and allows for accurate wayfinding(endpoint position error≤45 cm)in large indoor spaces.
基金Project supported by the National Key R&D Program of China(No.2018YFB1305500)the National Natural Science Foundation of China(No.U1813219)。
文摘A fundamental task for mobile robots is simultaneous localization and mapping(SLAM).Moreover,long-term robustness is an important property for SLAM.When vehicles or robots steer fast or steer in certain scenarios,such as low-texture environments,long corridors,tunnels,or other duplicated structural environments,most SLAM systems might fail.In this paper,we propose a novel robust visual inertial light detection and ranging(Li Da R)navigation(VILN)SLAM system,including stereo visual-inertial Li Da R odometry and visual-Li Da R loop closure.The proposed VILN SLAM system can perform well with low drift after long-term experiments,even when the Li Da R or visual measurements are degraded occasionally in complex scenes.Extensive experimental results show that the robustness has been greatly improved in various scenarios compared to state-of-the-art SLAM systems.
文摘随着实际应用需求的不断提高,传统视觉同步定位与地图构建(visual simultaneous localization and mapping,VSLAM)难以建立符合要求的环境地图。语义VSLAM在传统VSLAM基础框架上引入深度学习获取环境图像的语义信息,并且构建环境的3D点云语义地图,能为更高层次的交互任务提供有效支持。通过分析传统的VSLAM框架中视觉里程计、回环与优化以及建图上的不足,阐述了近年来使用语义信息改善VSLAM系统性能及为环境交互任务提供助力的方法。最后,结合研究现状结论及课题组正在开展的工作对语义VSLAM未来研究方向及应用进行展望。
