An RGB-D Camera Based Visual Positioning System for Assistive Navigation by a Robotic Navigation Aid

There are about 253 million people with visual impairment worldwide.Many of them use a white cane and/or a guide dog as the mobility tool for daily travel.Despite decades of efforts,electronic navigation aid that can replace white cane is still research in progress.In this paper,we propose an RGB-D camera based visual positioning system(VPS)for real-time localization of a robotic navigation aid(RNA)in an architectural floor plan for assistive navigation.The core of the system is the combination of a new 6-DOF depth-enhanced visual-inertial odometry(DVIO)method and a particle filter localization(PFL)method.DVIO estimates RNA’s pose by using the data from an RGB-D camera and an inertial measurement unit(IMU).It extracts the floor plane from the camera’s depth data and tightly couples the floor plane,the visual features(with and without depth data),and the IMU’s inertial data in a graph optimization framework to estimate the device’s 6-DOF pose.Due to the use of the floor plane and depth data from the RGB-D camera,DVIO has a better pose estimation accuracy than the conventional VIO method.To reduce the accumulated pose error of DVIO for navigation in a large indoor space,we developed the PFL method to locate RNA in the floor plan.PFL leverages geometric information of the architectural CAD drawing of an indoor space to further reduce the error of the DVIO-estimated pose.Based on VPS,an assistive navigation system is developed for the RNA prototype to assist a visually impaired person in navigating a large indoor space.Experimental results demonstrate that:1)DVIO method achieves better pose estimation accuracy than the state-of-the-art VIO method and performs real-time pose estimation(18 Hz pose update rate)on a UP Board computer;2)PFL reduces the DVIO-accrued pose error by 82.5%on average and allows for accurate wayfinding(endpoint position error≤45 cm)in large indoor spaces.

Design of Evolvable Hardware for Robotic Navigation

This paper presents an integrated on line learning system to evolve programmable logic array (PLA) controllers for navigating an autonomous robot in a two dimensional environment. The integrated on line learning system consists of two learning modules: one is the module of reinforcement learning based on temporal difference learning based on genetic algorithms, and the other is the module of evolutionary learning based on genetic algorithms. The control rules extracted from the module of reinforcement learning can be used as input to the module of evolutionary learning, and quickly implemented by the PLA through on line evolution. The on line evolution has shown promise as a method of learning systems in complex environment. The evolved PLA controllers can successfully navigate the robot to a target in the two dimensional environment while avoiding collisions with randomly positioned obstacles.

Behavior of Delivery Robot in Human-Robot Collaborative Spaces During Navigation

Navigation is an essential skill for robots.It becomes a cumbersome task for the robot in a human-populated environment,and Industry 5.0 is an emerging trend that focuses on the interaction between humans and robots.Robot behavior in a social setting is the key to human acceptance while ensuring human comfort and safety.With the advancement in robotics technology,the true use cases of robots in the tourism and hospitality industry are expanding in number.There are very few experimental studies focusing on how people perceive the navigation behavior of a delivery robot.A robotic platform named“PI”has been designed,which incorporates proximity and vision sensors.The robot utilizes a real-time object recognition algorithm based on the You Only Look Once(YOLO)algorithm to detect objects and humans during navigation.This study is aimed towards evaluating human experience,for which we conducted a study among 36 participants to explore the perceived social presence,role,and perception of a delivery robot exhibiting different behavior conditions while navigating in a hotel corridor.The participants’responses were collected and compared for different behavior conditions demonstrated by the robot and results show that humans prefer an assistant role of a robot enabled with audio and visual aids exhibiting social behavior.Further,this study can be useful for developers to gain insight into the expected behavior of a delivery robot.

Accuracy study of a binocular-stereo-vision-based navigation robot for minimally invasive interventional procedures

BACKGROUND Medical robot is a promising surgical tool,but no specific one has been designed for interventional treatment of chronic pain.We developed a computed tomography-image based navigation robot using a new registration method with binocular vision.This kind of robot is appropriate for minimal invasive interventional procedures and easy to operate.The feasibility,accuracy and stability of this new robot need to be tested.AIM To assess quantitatively the feasibility,accuracy and stability of the binocularstereo-vision-based navigation robot for minimally invasive interventional procedures.METHODS A box model was designed for assessing the accuracy for targets at different distances.Nine(three sets)lead spheres were embedded in the model as puncture goals.The entry-to-target distances were set 50 mm(short-distance),100 mm(medium-distance)and 150 mm(long-distance).Puncture procedure was repeated three times for each goal.The Euclidian error of each puncture was calculated and statistically analyzed.Three head phantoms were used to explore the clinical feasibility and stability.Three independent operators conducted foramen ovale placement on head phantoms(both sides)by freehand or under the guidance of robot(18 punctures with each method).The operation time,adjustment time and one-time success rate were recorded,and the two guidancemethods were compared.RESULTS On the box model,the mean puncture errors of navigation robot were 1.7±0.9 mm for the short-distance target,2.4±1.0 mm for the moderate target and 4.4±1.4 mm for the long-distance target.On the head phantom,no obvious differences in operation time and adjustment time were found among the three performers(P>0.05).The median adjustment time was significantly less under the guidance of the robot than under free hand.The one-time success rate was significantly higher with the robot(P<0.05).There was no obvious difference in operation time between the two methods(P>0.05).CONCLUSION In the laboratory environment,accuracy of binocular-stereo-vision-based navigation robot is acceptable for target at 100 mm depth or less.Compared with freehand,foramen ovale placement accuracy can be improved with robot guidance.

PATH FOLLOWING GPS-BASED CONTROL OF SMALL-SIZE ROBOTIC UNMANNED BLIMP

Robotic unmanned blimps own an enormous potential for applications in low-speed and low-altitude exploration, surveillance, and monitoring, as well as telecommunication relay platforms. To make lighter-than-air platform a robotic blimp with significant levels of autonomy, the decoupled longitude and latitude dynamic model is developed, and the hardware and software of the flight control system are designed and detailed. Flight control and navigation strategy and algorithms for waypoint flight problem are discussed. A result of flight experiment is also presented, which validates that the flight control system is applicable and initial machine intelligence of robotic blimp is achieved.

Binocular Visual Navigation and Obstacle Avoidance of Mobile Robots Based on Speeded-Up Robust Features

This article presents a good robust and real-time system scheme of the mobile robot obstacle detection and navigation, which principle of work is based on the feature descriptor SURF. In this scheme, firstly, the image information of the mobile robot path was captured by the binocular camera; then the feature points were extracted and corresponding matched using SURF to the binocular images as the undetected obstacles; finally fixed the position of the objective by the parallax between the matching points combining with the binocular vision calibration model. Theoretical derivation and experimental results show that this scheme is more accurate for the detection and navigation of the interest points. It has fast matching speed and high accuracy and low error. So, it has certain practical effect and popularizing value for the mobile robot real-time obstacle avoidance and navigation.

Application of reinforcement learning and neural network in robot navigation

Presents the navigation based on reinforcement learning and an algorithm, and disscusses the combination of the neural network with Q learning.

Mobile Robot Indoor Autonomous Navigation with Position Estimation Using RF Signal Triangulation

In the mobile robotic systems a precise estimate of the robot pose (Cartesian [x y] position plus orientation angle theta) with the intention of the path planning optimization is essential for the correct performance, on the part of the robots, for tasks that are destined to it, especially when intention is for mobile robot autonomous navigation. This work uses a ToF (Time-of-Flight) of the RF digital signal interacting with beacons for computational triangulation in the way to provide a pose estimative at bi-dimensional indoor environment, where GPS system is out of range. It’s a new technology utilization making good use of old ultrasonic ToF methodology that takes advantage of high performance multicore DSP processors to calculate ToF of the order about ns. Sensors data like odometry, compass and the result of triangulation Cartesian estimative, are fused in a Kalman filter in the way to perform optimal estimation and correct robot pose. A mobile robot platform with differential drive and nonholonomic constraints is used as base for state space, plants and measurements models that are used in the simulations and for validation the experiments.

Deep Learning Implemented Visualizing City Cleanliness Level by Garbage Detection

In an urban city,the daily challenges of managing cleanliness are the primary aspect of routine life,which requires a large number of resources,the manual process of labour,and budget.Street cleaning techniques include street sweepers going away to different metropolitan areas,manually verifying if the street required cleaning taking action.This research presents novel street garbage recognizing robotic navigation techniques by detecting the city’s street-level images and multi-level segmentation.For the large volume of the process,the deep learning-based methods can be better to achieve a high level of classifica-tion,object detection,and accuracy than other learning algorithms.The proposed Histogram of Oriented Gradients(HOG)is used to features extracted while using the deep learning technique to classify the ground-level segmentation process’s images.In this paper,we use mobile edge computing to process street images in advance andfilter out pictures that meet our needs,which significantly affect recognition efficiency.To measure the urban streets’cleanliness,our street clean-liness assessment approach provides a multi-level assessment model across differ-ent layers.Besides,with ground-level segmentation using a deep neural network,a novel navigation strategy is proposed for robotic classification.Single Shot Mul-tiBox Detector(SSD)approaches the output space of bounding boxes into a set of default boxes over different feature ratios and scales per attribute map location from the dataset.The SSD can classify and detect the garbage’s accurately and autonomously by using deep learning for garbage recognition.Experimental results show that accurate street garbage detection and navigation can reach approximately the same cleaning effectiveness as traditional methods.

Reinforcement Learning Navigation for Robots Based on Hippocampus Episode Cognition

Artificial intelligence is currently achieving impressive success in all fields.However,autonomous navigation remains a major challenge for AI.Reinforcement learning is used for target navigation to simulate the interaction between the brain and the environment at the behavioral level,but the Artificial Neural Network trained by reinforcement learning cannot match the autonomous mobility of humans and animals.The hippocampus–striatum circuits are considered as key circuits for target navigation planning and decision-making.This paper aims to construct a bionic navigation model of reinforcement learning corresponding to the nervous system to improve the autonomous navigation performance of the robot.The ventral striatum is considered to be the behavioral evaluation region,and the hippocampal–striatum circuit constitutes the position–reward association.In this paper,a set of episode cognition and reinforcement learning system simulating the mechanism of hippocampus and ventral striatum is constructed,which is used to provide target guidance for the robot to perform autonomous tasks.Compared with traditional methods,this system reflects the high efficiency of learning and better Environmental Adaptability.Our research is an exploration of the intersection and fusion of artificial intelligence and neuroscience,which is conducive to the development of artificial intelligence and the understanding of the nervous system.

Computer-Vision Based Object Detection and Recognition for Service Robot in Indoor Environment

The near future has been envisioned as a collaboration of humans with mobile robots to help in the day-to-day tasks.In this paper,we present a viable approach for a real-time computer vision based object detection and recognition for efficient indoor navigation of a mobile robot.The mobile robotic systems are utilized mainly for home assistance,emergency services and surveillance,in which critical action needs to be taken within a fraction of second or real-time.The object detection and recognition is enhanced with utilization of the proposed algorithm based on the modification of You Look Only Once(YOLO)algorithm,with lesser computational requirements and relatively smaller weight size of the network structure.The proposed computer-vision based algorithm has been compared with the other conventional object detection/recognition algorithms,in terms of mean Average Precision(mAP)score,mean inference time,weight size and false positive percentage.The presented framework also makes use of the result of efficient object detection/recognition,to aid the mobile robot navigate in an indoor environment with the utilization of the results produced by the proposed algorithm.The presented framework can be further utilized for a wide variety of applications involving indoor navigation robots for different services.

Study of A New Method for Vision Based Robot Target-Tracking Problem

Focused on several problems during robot target tracking, and proposed a new kind of scheme and algorithm for it. The hybrid systematic structure reduces the control complexity and guarantees the tracking effectiveness as well as the control stability. The convergence and the feasibility of the algorithm are analyzed and proofed thoroughly. An on-line updating method for navigation coefficient is presented. Finally, the control scheme and proposed algorithm is applied to the real robotic system. The simulation and experimental results show its effectiveness.

ARCHITECTURE AND ITS IMPLEMENTATION FOR ROBOTS TO NAVIGATE IN UNKNOWN INDOOR ENVIRONMENTS

It is discussed with the design and implementation of an architecture for a mobile robot to navigate in dynamic and anknown indoor environments. The architecture is based on the framework of Open Robot Control Software at KTH （OROCOS@KTH）, which is also discussed and evaluated to navigate indoor efficiently, a new algorithm named door-like-exit detection is proposed which employs 2D feature oft. door and extracts key points of pathway from the raw data of a laser scanner. As a hybrid architecture, it is decomposed into several basic components which can be classified as either deliberative or reactive. Each component can concurrently execute and communicate with another. It is expansible and transferable and its components are reusable.

Deep Reinforcement Learning Based Mobile Robot Navigation:A Review

Navigation is a fundamental problem of mobile robots,for which Deep Reinforcement Learning(DRL)has received significant attention because of its strong representation and experience learning abilities.There is a growing trend of applying DRL to mobile robot navigation.In this paper,we review DRL methods and DRL-based navigation frameworks.Then we systematically compare and analyze the relationship and differences between four typical application scenarios:local obstacle avoidance,indoor navigation,multi-robot navigation,and social navigation.Next,we describe the development of DRL-based navigation.Last,we discuss the challenges and some possible solutions regarding DRL-based navigation.

Navigation of Non-holonomic Mobile Robot Using Neuro-fuzzy Logic with Integrated Safe Boundary Algorithm

In the present work, autonomous mobile robot（AMR） system is intended with basic behaviour, one is obstacle avoidance and the other is target seeking in various environments. The AMR is navigated using fuzzy logic, neural network and adaptive neurofuzzy inference system（ANFIS） controller with safe boundary algorithm. In this method of target seeking behaviour, the obstacle avoidance at every instant improves the performance of robot in navigation approach. The inputs to the controller are the signals from various sensors fixed at front face, left and right face of the AMR. The output signal from controller regulates the angular velocity of both front power wheels of the AMR. The shortest path is identified using fuzzy, neural network and ANFIS techniques with integrated safe boundary algorithm and the predicted results are validated with experimentation. The experimental result has proven that ANFIS with safe boundary algorithm yields better performance in navigation, in particular with curved/irregular obstacles.