基于深度强化学习的智能车辆行为决策研究

Intelligent Vehicles Behavior Decision-making Based on Deep Reinforcement Learning

自动驾驶车辆决策系统直接影响车辆综合行驶性能,是实现自动驾驶技术需要解决的关键难题之一。基于深度强化学习算法DDPG(deep deterministic policy gradient),针对此问题提出了一种端到端驾驶行为决策模型。首先,结合驾驶员模型选取自车、道路、干扰车辆等共64维度状态空间信息作为输入数据集对决策模型进行训练,决策模型输出合理的驾驶行为以及控制量,为解决训练测试中的奖励和控制量突变问题,改进DDPG决策模型对决策控制效果进行优化,并在TORCS(the open racing car simulator)平台进行仿真实验验证。结果表明:所提出的决策模型可以根据车辆和环境实时状态信息输出合理的驾驶行为以及控制量,与DDPG模型相比,改进的模型具有更好的控制精度,且车辆横向速度显著减小,车辆舒适性以及车辆稳定性明显改善。

Autonomous driving vehicle decision-making system has direct influence on driving performance.It is one of the key challenges to be addressed to realize fully autonomous driving.To solve this problem,a driving decision-making system based on deep reinforcement learning algorithm deep deterministic policy gradient(DDPG)was proposed.Firstly,a total of 64 dimensions of state spaces information such as ego vehicle information,road information and obstacle vehicle information on the basis of a driver model were selected as input variables of the constructed model.Then the decision-making was trained and outputs reasonable driving behaviors and control variable values.Finally,aiming at the problems of reward value and control variable values saltation,the DDPG decision model was improved to optimize decision control effect.To verify the performance of the proposed decision making model,simulation experiments were conducted on the open racing car simulator(TORCS)platform.The results show that the proposed decision-making model can output reasonable driving behaviors and accurate control quantities based on real-time state information of vehicles and environment.Compared with the DDPG model,the improved decision-making model has better control accuracy,significantly reduces vehicle lateral speed,improves vehicle comfort and stability.