A digital twins enabled underwater intelligent internet vehicle path planning system via reinforcement learning and edge computing

The Autonomous Underwater Glider(AUG)is a kind of prevailing underwater intelligent internet vehicle and occupies a dominant position in industrial applications,in which path planning is an essential problem.Due to the complexity and variability of the ocean,accurate environment modeling and flexible path planning algorithms are pivotal challenges.The traditional models mainly utilize mathematical functions,which are not complete and reliable.Most existing path planning algorithms depend on the environment and lack flexibility.To overcome these challenges,we propose a path planning system for underwater intelligent internet vehicles.It applies digital twins and sensor data to map the real ocean environment to a virtual digital space,which provides a comprehensive and reliable environment for path simulation.We design a value-based reinforcement learning path planning algorithm and explore the optimal network structure parameters.The path simulation is controlled by a closed-loop model integrated into the terminal vehicle through edge computing.The integration of state input enriches the learning of neural networks and helps to improve generalization and flexibility.The task-related reward function promotes the rapid convergence of the training.The experimental results prove that our reinforcement learning based path planning algorithm has great flexibility and can effectively adapt to a variety of different ocean conditions.

A Reverse Path Planning Approach for Enhanced Performance of Multi-Degree-of-Freedom Industrial Manipulators

In the domain of autonomous industrial manipulators,precise positioning and appropriate posture selection in path planning are pivotal for tasks involving obstacle avoidance,such as handling,heat sealing,and stacking.While Multi-Degree-of-Freedom(MDOF)manipulators offer kinematic redundancy,aiding in the derivation of optimal inverse kinematic solutions to meet position and posture requisites,their path planning entails intricate multiobjective optimization,encompassing path,posture,and joint motion optimization.Achieving satisfactory results in practical scenarios remains challenging.In response,this study introduces a novel Reverse Path Planning(RPP)methodology tailored for industrial manipulators.The approach commences by conceptualizing the manipulator’s end-effector as an agent within a reinforcement learning(RL)framework,wherein the state space,action set,and reward function are precisely defined to expedite the search for an initial collision-free path.To enhance convergence speed,the Q-learning algorithm in RL is augmented with Dyna-Q.Additionally,we formulate the cylindrical bounding box of the manipulator based on its Denavit-Hartenberg(DH)parameters and propose a swift collision detection technique.Furthermore,the motion performance of the end-effector is refined through a bidirectional search,and joint weighting coefficients are introduced to mitigate motion in high-power joints.The efficacy of the proposed RPP methodology is rigorously examined through extensive simulations conducted on a six-degree-of-freedom(6-DOF)manipulator encountering two distinct obstacle configurations and target positions.Experimental results substantiate that the RPP method adeptly orchestrates the computation of the shortest collision-free path while adhering to specific posture constraints at the target point.Moreover,itminimizes both posture angle deviations and joint motion,showcasing its prowess in enhancing the operational performance of MDOF industrial manipulators.

Heuristic Expanding Disconnected Graph:A Rapid Path Planning Method for Mobile Robots

Existing mobile robots mostly use graph search algorithms for path planning,which suffer from relatively low planning efficiency owing to high redundancy and large computational complexity.Due to the limitations of the neighborhood search strategy,the robots could hardly obtain the most optimal global path.A global path planning algorithm,denoted as EDG*,is proposed by expanding nodes using a well-designed expanding disconnected graph operator(EDG)in this paper.Firstly,all obstacles are marked and their corners are located through the map pre-processing.Then,the EDG operator is designed to find points in non-obstruction areas to complete the rapid expansion of disconnected nodes.Finally,the EDG*heuristic iterative algorithm is proposed.It selects the candidate node through a specific valuation function and realizes the node expansion while avoiding collision with a minimum offset.Path planning experiments were conducted in a typical indoor environment and on the public dataset CSM.The result shows that the proposed EDG*reduced the planning time by more than 90%and total length of paths reduced by more than 4.6%.Compared to A*,Dijkstra and JPS,EDG*does not show an exponential explosion effect in map size.The EDG*showed better performance in terms of path smoothness,and collision avoidance.This shows that the EDG*algorithm proposed in this paper can improve the efficiency of path planning and enhance path quality.

Multi-UAVs Collaborative Path Planning in the Cramped Environment

Due to its flexibility and complementarity, the multiUAVs system is well adapted to complex and cramped workspaces, with great application potential in the search and rescue(SAR) and indoor goods delivery fields. However, safe and effective path planning of multiple unmanned aerial vehicles(UAVs)in the cramped environment is always challenging: conflicts with each other are frequent because of high-density flight paths, collision probability increases because of space constraints, and the search space increases significantly, including time scale, 3D scale and model scale. Thus, this paper proposes a hierarchical collaborative planning framework with a conflict avoidance module at the high level and a path generation module at the low level. The enhanced conflict-base search(ECBS) in our framework is improved to handle the conflicts in the global path planning and avoid the occurrence of local deadlock. And both the collision and kinematic models of UAVs are considered to improve path smoothness and flight safety. Moreover, we specifically designed and published the cramped environment test set containing various unique obstacles to evaluating our framework performance thoroughly. Experiments are carried out relying on Rviz, with multiple flight missions: random, opposite, and staggered, which showed that the proposed method can generate smooth cooperative paths without conflict for at least 60 UAVs in a few minutes.The benchmark and source code are released in https://github.com/inin-xingtian/multi-UAVs-path-planner.

Ground threat prediction-based path planning of unmanned autonomous helicopter using hybrid enhanced artificial bee colony algorithm

Unmanned autonomous helicopter(UAH)path planning problem is an important component of the UAH mission planning system.Aiming to reduce the influence of non-complete ground threat information on UAH path planning,a ground threat prediction-based path planning method is proposed based on artificial bee colony(ABC)algorithm by collaborative thinking strategy.Firstly,a dynamic threat distribution probability model is developed based on the characteristics of typical ground threats.The dynamic no-fly zone of the UAH is simulated and established by calculating the distribution probability of ground threats in real time.Then,a dynamic path planning method for UAH is designed in complex environment based on the real-time prediction of ground threats.By adding the collision warning mechanism to the path planning model,the flight path could be dynamically adjusted according to changing no-fly zones.Furthermore,a hybrid enhanced ABC algorithm is proposed based on collaborative thinking strategy.The proposed algorithm applies the leader-member thinking mechanism to guide the direction of population evolution,and reduces the negative impact of local optimal solutions caused by collaborative learning update strategy,which makes the optimization performance of ABC algorithm more controllable and efficient.Finally,simulation results verify the feasibility and effectiveness of the proposed ground threat prediction path planning method.

Optimal search path planning of UUV in battlefeld ambush scene

Aiming at the practical application of Unmanned Underwater Vehicle(UUV)in underwater combat,this paper proposes a battlefield ambush scene with UUV considering ocean current.Firstly,by establishing these mathematical models of ocean current environment,target movement,and sonar detection,the probability calculation methods of single UUV searching target and multiple UUV cooperatively searching target are given respectively.Then,based on the Hybrid Quantum-behaved Particle Swarm Optimization(HQPSO)algorithm,the path with the highest target search probability is found.Finally,through simulation calculations,the influence of different UUV parameters and target parameters on the target search probability is analyzed,and the minimum number of UUVs that need to be deployed to complete the ambush task is demonstrated,and the optimal search path scheme is obtained.The method proposed in this paper provides a theoretical basis for the practical application of UUV in the future combat.

Research on Evacuation Path Planning Based on Improved Sparrow Search Algorithm

Reducing casualties and property losses through effective evacuation route planning has been a key focus for researchers in recent years.As part of this effort,an enhanced sparrow search algorithm(MSSA)was proposed.Firstly,the Golden Sine algorithm and a nonlinear weight factor optimization strategy were added in the discoverer position update stage of the SSA algorithm.Secondly,the Cauchy-Gaussian perturbation was applied to the optimal position of the SSA algorithm to improve its ability to jump out of local optima.Finally,the local search mechanism based on the mountain climbing method was incorporated into the local search stage of the SSA algorithm,improving its local search ability.To evaluate the effectiveness of the proposed algorithm,the Whale Algorithm,Gray Wolf Algorithm,Improved Gray Wolf Algorithm,Sparrow Search Algorithm,and MSSA Algorithm were employed to solve various test functions.The accuracy and convergence speed of each algorithm were then compared and analyzed.The results indicate that the MSSA algorithm has superior solving ability and stability compared to other algorithms.To further validate the enhanced algorithm’s capabilities for path planning,evacuation experiments were conducted using different maps featuring various obstacle types.Additionally,a multi-exit evacuation scenario was constructed according to the actual building environment of a teaching building.Both the sparrow search algorithm and MSSA algorithm were employed in the simulation experiment for multiexit evacuation path planning.The findings demonstrate that the MSSA algorithm outperforms the comparison algorithm,showcasing its greater advantages and higher application potential.

Real-time UAV path planning based on LSTM network

To address the shortcomings of single-step decision making in the existing deep reinforcement learning based unmanned aerial vehicle(UAV)real-time path planning problem,a real-time UAV path planning algorithm based on long shortterm memory(RPP-LSTM)network is proposed,which combines the memory characteristics of recurrent neural network(RNN)and the deep reinforcement learning algorithm.LSTM networks are used in this algorithm as Q-value networks for the deep Q network(DQN)algorithm,which makes the decision of the Q-value network has some memory.Thanks to LSTM network,the Q-value network can use the previous environmental information and action information which effectively avoids the problem of single-step decision considering only the current environment.Besides,the algorithm proposes a hierarchical reward and punishment function for the specific problem of UAV real-time path planning,so that the UAV can more reasonably perform path planning.Simulation verification shows that compared with the traditional feed-forward neural network(FNN)based UAV autonomous path planning algorithm,the RPP-LSTM proposed in this paper can adapt to more complex environments and has significantly improved robustness and accuracy when performing UAV real-time path planning.

Improving path planning efficiency for underwater gravity-aided navigation based on a new depth sorting fast search algorithm

This study focuses on the improvement of path planning efficiency for underwater gravity-aided navigation.Firstly,a Depth Sorting Fast Search(DSFS)algorithm was proposed to improve the planning speed of the Quick Rapidly-exploring Random Trees*(Q-RRT*)algorithm.A cost inequality relationship between an ancestor and its descendants was derived,and the ancestors were filtered accordingly.Secondly,the underwater gravity-aided navigation path planning system was designed based on the DSFS algorithm,taking into account the fitness,safety,and asymptotic optimality of the routes,according to the gravity suitability distribution of the navigation space.Finally,experimental comparisons of the computing performance of the ChooseParent procedure,the Rewire procedure,and the combination of the two procedures for Q-RRT*and DSFS were conducted under the same planning environment and parameter conditions,respectively.The results showed that the computational efficiency of the DSFS algorithm was improved by about 1.2 times compared with the Q-RRT*algorithm while ensuring correct computational results.

Distributed collaborative complete coverage path planning based on hybrid strategy

Collaborative coverage path planning(CCPP) refers to obtaining the shortest paths passing over all places except obstacles in a certain area or space. A multi-unmanned aerial vehicle(UAV) collaborative CCPP algorithm is proposed for the urban rescue search or military search in outdoor environment.Due to flexible control of small UAVs, it can be considered that all UAVs fly at the same altitude, that is, they perform search tasks on a two-dimensional plane. Based on the agents’ motion characteristics and environmental information, a mathematical model of CCPP problem is established. The minimum time for UAVs to complete the CCPP is the objective function, and complete coverage constraint, no-fly constraint, collision avoidance constraint, and communication constraint are considered. Four motion strategies and two communication strategies are designed. Then a distributed CCPP algorithm is designed based on hybrid strategies. Simulation results compared with patternbased genetic algorithm(PBGA) and random search method show that the proposed method has stronger real-time performance and better scalability and can complete the complete CCPP task more efficiently and stably.

A path planning method for robot patrol inspection in chemical industrial parks

Safety patrol inspection in chemical industrial parks is a complex multi-objective task with multiple degrees of freedom.Traditional pointer instruments with advantages like high reliability and strong adaptability to harsh environment,are widely applied in such parks.However,they rely on manual readings which have problems like heavy patrol workload,high labor cost,high false positives/negatives and poor timeliness.To address the above problems,this study proposes a path planning method for robot patrol in chemical industrial parks,where a path optimization model based on improved iterated local search and random variable neighborhood descent(ILS-RVND)algorithm is established by integrating the actual requirements of patrol tasks in chemical industrial parks.Further,the effectiveness of the model and algorithm is verified by taking real park data as an example.The results show that compared with GA and ILS-RVND,the improved algorithm reduces quantification cost by about 24%and saves patrol time by about 36%.Apart from shortening the patrol time of robots,optimizing their patrol path and reducing their maintenance loss,the proposed algorithm also avoids the untimely patrol of robots and enhances the safety factor of equipment.

Path Planning for Robotic Arms Based on an Improved RRT Algorithm

The burgeoning robotics industry has catalyzed significant strides in the development and deployment of industrial and service robotic arms, positioning path planning as a pivotal facet for augmenting their operational safety and efficiency. Existing path planning algorithms, while capable of delineating feasible trajectories, often fall short of achieving optimality, particularly concerning path length, search duration, and success likelihood. This study introduces an enhanced Rapidly-Exploring Random Tree (RRT) algorithm, meticulously designed to rectify the issues of node redundancy and the compromised path quality endemic to conventional RRT approaches. Through the integration of an adaptive pruning mechanism and a dynamic elliptical search strategy within the Informed RRT* framework, our algorithm efficiently refines the search tree by discarding branches that surpass the cost of the optimal path, thereby refining the search space and significantly boosting efficiency. Extensive comparative analysis across both two-dimensional and three-dimensional simulation settings underscores the algorithm’s proficiency in markedly improving path precision and search velocity, signifying a breakthrough in the domain of robotic arm path planning.

Research on Shortest Path BFS Strategy in Multi-AGV Scheduling System

With the increasing maturity of automated guided vehicles(AGV)technology and the widespread application of flexible manufacturing systems,enhancing the efficiency of AGVs in complex environments has become crucial.This paper analyzes the challenges of path planning and scheduling in multi-AGV systems,introduces a map-based path search algorithm,and proposes the BFS algorithm for shortest path planning.Through optimization using the breadth-first search(BFS)algorithm,efficient scheduling of multiple AGVs in complex environments is achieved.In addition,this paper validated the effectiveness of the proposed method in a production workshop experiment.The experimental results show that the BFS algorithm can quickly search for the shortest path,reduce the running time of AGVs,and significantly improve the performance of multi-AGV scheduling systems.

Path Planning of UAV by Combing Improved Ant Colony System and Dynamic Window Algorithm

A fusion algorithm is proposed to enhance the search speed of an ant colony system(ACS)for the global path planning and overcome the challenges of the local path planning in an unmanned aerial vehicle(UAV).The ACS search efficiency is enhanced by adopting a 16-direction 24-neighborhood search way,a safety grid search way,and an elite hybrid strategy to accelerate global convergence.Quadratic planning is performed using the moving average(MA)method.The fusion algorithm incorporates a dynamic window approach(DWA)to deal with the local path planning,sets a retracement mechanism,and adjusts the evaluation function accordingly.Experimental results in two environments demonstrate that the improved ant colony system(IACS)achieves superior planning efficiency.Additionally,the optimized dynamic window approach(ODWA)demonstrates its ability to handle multiple dynamic situations.Overall,the fusion optimization algorithm can accomplish the mixed path planning effectively.

Energy-Efficient UAVs Coverage Path Planning Approach

Unmanned aerial vehicles(UAVs),commonly known as drones,have drawn significant consideration thanks to their agility,mobility,and flexibility features.They play a crucial role in modern reconnaissance,inspection,intelligence,and surveillance missions.Coverage path planning(CPP)which is one of the crucial aspects that determines an intelligent system’s quality seeks an optimal trajectory to fully cover the region of interest(ROI).However,the flight time of the UAV is limited due to a battery limitation and may not cover the whole region,especially in large region.Therefore,energy consumption is one of the most challenging issues that need to be optimized.In this paper,we propose an energy-efficient coverage path planning algorithm to solve the CPP problem.The objective is to generate a collision-free coverage path that minimizes the overall energy consumption and guarantees covering the whole region.To do so,the flight path is optimized and the number of turns is reduced to minimize the energy consumption.The proposed approach first decomposes the ROI into a set of cells depending on a UAV camera footprint.Then,the coverage path planning problem is formulated,where the exact solution is determined using the CPLEX solver.For small-scale problems,the CPLEX shows a better solution in a reasonable time.However,the CPLEX solver fails to generate the solution within a reasonable time for large-scale problems.Thus,to solve the model for large-scale problems,simulated annealing forCPP is developed.The results show that heuristic approaches yield a better solution for large-scale problems within amuch shorter execution time than the CPLEX solver.Finally,we compare the simulated annealing against the greedy algorithm.The results show that simulated annealing outperforms the greedy algorithm in generating better solution quality.

Improved RRT^(∗)Algorithm for Automatic Charging Robot Obstacle Avoidance Path Planning in Complex Environments

A new and improved RRT∗algorithm has been developed to address the low efficiency of obstacle avoidance planning and long path distances in the electric vehicle automatic charging robot arm.This algorithm enables the robot to avoid obstacles,find the optimal path,and complete automatic charging docking.It maintains the global completeness and path optimality of the RRT algorithmwhile also improving the iteration speed and quality of generated paths in both 2D and 3D path planning.After finding the optimal path,the B-sample curve is used to optimize the rough path to create a smoother and more optimal path.In comparison experiments,the new algorithmyielded reductions of 35.5%,29.2%,and 11.7%in search time and 22.8%,19.2%,and 9%in path length for the 3D environment.Finally,experimental validation of the automatic charging of electric vehicles was conducted to further verify the effectiveness of the algorithm.The simulation experimental validation was carried out by kinematic modeling and building an experimental platform.The error between the experimental results and the simulation results is within 10%.The experimental results show the effectiveness and practicality of the algorithm.

Real-Time Indoor Path Planning Using Object Detection for Autonomous Flying Robots

Unknown closed spaces are a big challenge for the navigation of robots since there are no global and pre-defined positioning options in the area.One of the simplest and most efficient algorithms,the artificial potential field algorithm(APF),may provide real-time navigation in those places but fall into local mini-mum in some cases.To overcome this problem and to present alternative escape routes for a robot,possible crossing points in buildings may be detected by using object detection and included in the path planning algorithm.This study utilized a proposed sensor fusion method and an improved object classification method for detecting windows,doors,and stairs in buildings and these objects were classified as valid or invalid for the path planning algorithm.The performance of the approach was evaluated in a simulated environment with a quadrotor that was equipped with camera and laser imaging detection and ranging(LIDAR)sensors to navigate through an unknown closed space and reach a desired goal point.Inclusion of crossing points allows the robot to escape from areas where it is con-gested.The navigation of the robot has been tested in different scenarios based on the proposed path planning algorithm and compared with other improved APF methods.The results showed that the improved APF methods and the methods rein-forced with other path planning algorithms were similar in performance with the proposed method for the same goals in the same room.For the goals outside the current room,traditional APF methods were quite unsuccessful in reaching the goals.Even though improved methods were able to reach some outside targets,the proposed method gave approximately 17%better results than the most success-ful example in achieving targets outside the current room.The proposed method can also work in real-time to discover a building and navigate between rooms.

Local Path Planning and Tracking Control of Vehicle Collision Avoidance System

Automotive collision avoidance technology can effectively avoid the accidents caused by dangerous traffic conditions or driver's manipulation errors.Moreover,it can promote the development of autonomous driving for intelligent vehicle in intelligent transportation.We present a collision avoidance system,which is composed of an evasive trajectory planner and a path following controller.Considering the stability of the vehicle in the conflict-free process,the evasive trajectory planner is designed by polynomial parametric method and optimized by genetic algorithm.The path following controller is proposed to make the car drive along the designed path by controlling the vehicle's lateral movement.Simulation results show that the vehicle with the proposed controller has good stability in the collision process,and it can ensure the vehicle driving in accordance with the planned trajectory at different speeds.The research results can provide a certain basis for the research and development of automotive collision avoidance technology.

Improved lazy theta algorithm based on octree map for path planning of UAV

This paper investigates the path planning method of unmanned aerial vehicle(UAV)in threedimensional map.Firstly,in order to keep a safe distance between UAV and obstacles,the obstacle grid in the map is expanded.By using the data structure of octree,the octree map is constructed,and the search nodes is significantly reduced.Then,the lazy theta*algorithm,including neighbor node search,line-of-sight algorithm and heuristics weight adjustment is improved.In the process of node search,UAV constraint conditions are considered to ensure the planned path is actually flyable.The redundant nodes are reduced by the line-of-sight algorithm through judging whether visible between two nodes.Heuristic weight adjustment strategy is employed to control the precision and speed of search.Finally,the simulation results show that the improved lazy theta*algorithm is suitable for path planning of UAV in complex environment with multi-constraints.The effectiveness and flight ability of the algorithm are verified by comparing experiments and real flight.

Minimum dose path planning for facility inspection based on the discrete Rao-combined ABC algorithm in radioactive environments with obstacles

Workers who conduct regular facility inspections in radioactive environments will inevitably be affected by radiation.Therefore,it is important to optimize the inspection path to ensure that workers are exposed to the least amount of radiation.This study proposes a discrete Rao-combined artificial bee colony(ABC)algorithm for planning inspection paths with minimum exposure doses in radioactive environments with obstacles.In this algorithm,retaining the framework of the traditional ABC algorithm,we applied the directional solution update rules of Rao algorithms at the employed bee stage and onlooker bee stage to increase the exploitation ability of the algorithm and implement discretion using the swap operator and swap sequence.To increase the randomness of solution generation,the chaos algorithm was used at the initialization stage.The K-opt operation technique was introduced at the scout bee stage to increase the exploration ability of the algorithm.For path planning in an environment with complex structural obstacles,an obstacle detour technique using a recursive algorithm was applied.To evaluate the performance of the proposed algorithm,we performed experimental simulations in three hypothetical environments and compared the results with those of improved particle swarm optimization,chaos particle swarm optimization,improved ant colony optimization,and discrete Rao’s algorithms.The experimental results show the high performance of the proposed discrete Rao-combined ABC algorithm and its obstacle detour capability.