基于改进TD3算法的机器人路径规划

Robot path planning based on improved TD3 algorithm

针对机器人路径规划中深度强化学习方法训练时间长、收敛速度慢的问题,提出一种改进双延迟深度确定性策略梯度算法(twin delayed deep deterministic policy gradient,TD3)。该算法引入人工势场中的引力场和斥力场优化TD3的奖励函数,引导机器人合理的避开障碍物前往目标点,从而提高算法的收敛速度和准确率;同时运用运动约束规则对机器人的运动方向进行约束,使得运动轨迹更加平滑流畅。仿真实验结果表明,在多障碍物环境中,所提算法能有效地使机器人避开障碍物并规划出合理的路径。与其他方法相比,改进后的算法规划成功率更高,规划路径更短。

Aiming at the problems of long training time and slow convergence speed of deep reinforcement learning methods in robot path planning,an improved Twin Delayed Deep Deterministic Policy Gradient(TD3)algorithm is proposed.The algorithm introduces the gravitational field and repulsive field in the artificial potential field to optimize the reward function of TD3,which guides the robot to avoid obstacles to the target point reasonably,so as to improve the convergence speed and accuracy of the algorithm;at the same time,it applies motion constraints rules to constrain the direction of the robot s motion,which makes the trajectory smoother and more fluent.The results of simulation experiments show that in a multi-obstacle environment,the proposed algorithm can effectively make the robot avoid obstacles and plan a reasonable path,and the improved algorithm has a higher planning success rate and a shorter planning path compared with other methods.