基于DDPG算法的无人车辆防碰撞控制策略

Anti Collision Control Strategy of Unmanned Vehicle Based on DDPG Algorithm

目前,强化学习在无人驾驶领域得到了广泛应用,但是如何提高无人车辆的稳定性并满足在不同工况中同时完成路径跟踪和车辆避障的要求依旧是一个难题。针对无人车辆路径跟踪与避障功能需求,提出一种基于深度确定梯度策略(Deep Deterministic Policy Gradient,DDPG)算法的无人车辆防碰撞控制策略。首先,根据DDPG算法原理和车辆控制模型得到控制系统的输入输出量,并提出一种基于sin函数的变道轨迹规划方式,来提高车辆避障能力。其次,根据控制系统输入输出量设计神经网络控制器以及研究其策略探索方案,并提出一种基于对数函数的奖励塑造方案,以解决奖励稀疏问题。最后,通过仿真实验证明,基于DDPG算法的无人车辆控制策略能够更加安全、稳定地控制车辆完成路径跟踪与避障任务,且控制精度更高。

At present,reinforcement learning has been widely used in the field of unmanned driving,but how to improve the stability of unmanned vehicles and meet the requirements of path tracking and vehicle obstacle avoidance under different working conditions is still a difficult problem.Aiming at the functional requirements of path tracking and obstacle avoidance of unmanned vehicles,an anti-collision control strategy of unmanned vehicles based on deep deterministic policy gradient(DDPG)algorithm was proposed in this paper.Firstly,according to the principle of DDPG algorithm and vehicle control model,the input and output of the control system were obtained,and a lane change trajectory planning method based on sin function was proposed to improve the vehicle obstacle avoidance ability.Secondly,according to the input and output of the control system,the neural network controller was designed and its strategy exploration scheme was studied,and a reward shaping scheme based on logarithmic function was proposed to solve the problem of sparse reward.Finally,the simulation results show that the unmanned vehicle control strategy based on DDPG algorithm can control the vehicle to complete the path tracking and obstacle avoidance tasks more safely and stably,and the control accuracy is higher.