基于Safe-PPO算法的安全优先路径规划方法

Safety priority path planning method based on Safe-PPO algorithm

现有的路径规划算法对路径规划过程中的路径安全性问题考虑较少,并且传统的近端策略优化(PPO)算法存在一定的方差适应性问题。为解决这些问题,提出一种融合进化策略思想和安全奖励函数的安全近端策略优化(Safe-PPO)算法,所提算法以安全优先进行路径规划。采用协方差自适应调整的进化策略(CMA-ES)的思想对PPO算法进行改进,并引入危险系数与动作因子来评估路径的安全性。使用二维栅格地图进行仿真实验,采用传统的PPO算法和Safe-PPO算法进行对比;采用六足机器人在搭建的场景中进行实物实验。仿真实验结果表明:所提算法在安全优先导向的路径规划方面具有合理性与可行性:在训练时Safe-PPO算法相比传统的PPO算法收敛速度提升了18%,获得的奖励提升了5.3%;在测试时采用融合危险系数与动作因子的方案能使机器人学会选择更加安全的道路而非直观上最快速的道路。实物实验结果表明:机器人可以在现实环境中选择更加安全的路径到达目标点。

The existing path planning algorithms seldom consider the problem of security,and the traditional proximal policy optimization(PPO)algorithm has a variance adaptability problem.To solve these problems,the Safe-PPO algorithm combining evolutionary strategy and safety reward function was proposed.The algorithm is safety-oriented for path planning.CMA-ES was used to improve the PPO algorithm.The hazard coefficient and movement coefficient were introduced to evaluate the safety of the path.Used a grid map for simulation experiments,and compared the traditional PPO algorithm with the Safe-PPO algorithm;The hexapod robot was used to carry out the physical experiment in the constructed scene.The simulation results show that the Safe-PPO algorithm is reasonable and feasible in safety-oriented path planning.When compared to the conventional PPO algorithm,the Safe-PPO algorithm increased the rate of convergence during training by 18%and the incentive received by 5.3%.Using the algorithm that combined the Hazard coefficient and movement coefficient during testing enabled the robot to learn to choose the safer path rather than the fastest one.The outcomes of the physical testing demonstrated that the robot could select a more secure route to the objective in the created setting.