基于分层强化学习的自动驾驶车辆掉头问题研究

Research on autonomous vehicle U-turn problem based on hierarchical reinforcement learning

调头任务是自动驾驶研究的内容之一,大多数在城市规范道路下的方案无法在非规范道路上实施。针对这一问题,建立了一种车辆掉头动力学模型,并设计了一种多尺度卷积神经网络提取特征图作为智能体的输入。另外还针对调头任务中的稀疏奖励问题,结合分层强化学习和近端策略优化算法提出了分层近端策略优化算法。在简单和复杂场景的实验中,该算法相比于其他算法能够更快地学习到策略,并且具有更高的掉头成功率。

The U-turn task is one of the contents of autonomous driving research,and most of the solutions under the standard roads in cities cannot be implemented on non-standard roads.Aiming at solving this problem,this paper established a vehicle U-turn dynamical model and designed a multi-scale convolutional neural network to extract feature maps as the input of the agent.In addition,for the sparse reward problem in the U-turn task,this paper proposed a hierarchical proximal policy optimization algorithm that combined hierarchical reinforcement learning and proximal policy optimization algorithm.In experiments with simple and complex scena-rios,this algorithm learns policies faster and has a higher success rate of U-turn compared to other algorithms.