基于深度强化学习的网联车辆队列纵向控制

Longitudinal control of connected vehicle platoonn based on deep reinforcement learning

针对车辆队列中多目标控制优化问题,研究基于强化学习的车辆队列控制方法.控制器输入为队列各车辆状态信息以及车辆间状态误差,输出为基于车辆纵向动力学的期望加速度,实现在V2X通信下的队列单车稳定行驶和队列稳定行驶.根据队列行驶场景以及采用的间距策略、通信拓扑结构等特性,建立队列马尔科夫决策过程(Markov decision process,MDP)模型.同时根据队列多输入-多输出高维样本特性,引入优先经验回放策略,提高算法收敛效率.为贴近实际车辆队列行驶工况,仿真基于PreScan构建多自由度燃油车动力学模型,联合Matlab/Simulink搭建仿真环境,同时引入噪声对队列控制器中动作网络和评价网络进行训练.仿真结果表明基于强化学习的车辆队列控制燃油消耗更低,且控制器实时性更高,对车辆的控制更为平滑.

This paper presents a vehicle platoon control method based on reinforcement learning(RL)to solve the multi-objective optimization problem.The actor network is designed to receive the state information of each vehicle in the platoon and the inter-vehicle state error,and outputs the desired acceleration based on the longitudinal dynamics of the vehicle.The proposed approach ensures both the individual vehicle stability and the string stability of the platoon under V2X communication.To model the platoon driving scenario with the spacing policy and communication topology,the Markov decision process(MDP)model of the platoon is established.In addition,considering the multi-input and multi-output high-level sample characteristics of the platoon,the deep deterministic policy gradient(DDPG)algorithm is adopted with the priority experience replay strategy to improve the convergence efficiency.To better approximate the actual platoon vehicle fuel consumption,the simulation is based on PreScan to build a high-degree fuel vehicle dynamics model.A co-simulation environment is created using Matlab/Simulink to train the actor network and critic network in the platoon controller by adding noise.The simulation results demonstrate that the reinforcement learning-based vehicle platoon control approach reduces fuel consumption and achieves faster and smoothervehiclecontrol.