基于强化学习的交叉口智能网联车多目标通行控制方法

A Multi-objective Traffic Control Method for Connected and Automated Vehicle at Signalized Intersection Based on Reinforcement Learning

针对传统控制方法下的智能网联车辆(connected and autonomous vehicle,CAV)在动态交通环境中通行能耗较高且效率较低等问题,研究了基于强化学习的CAV通行控制方法,旨在降低车辆能源消耗,提升车辆通行效率以及行驶舒适度。通过考虑CAV与交叉口信控系统的信息交互和物理环境,收集信号相位和信号配时(SPaT)以及前车速度和位置等信息,构建强化学习框架的状态空间。以电池能量回收的上限作为边界条件,建立CAV的行驶能耗模型,并基于车辆行驶的关键特征指标,如单位时间电能能耗、通行距离以及加速度变化率,设计多目标加权奖励函数。利用层次分析法确定各指标的权重,进而采用深度确定性策略梯度算法对模型进行训练,并通过梯度下降方法对算法参数进行调整和更新。采用SUMO平台开展仿真实验,实验结果表明:在设计的算法控制下的CAV各方面行驶性能最为均衡,相较于DQN算法电能消耗和加速度变化率均值分别降低了9.22%和18.77%;相较于Krauss跟驰模型行程时间缩短了8.39%。本研究提出的CAV通行控制方法在降低车辆能耗、提高行驶效率和舒适性等方面具有较好的可行性和有效性。

To address the issue of high energy consumption and low efficiency of connected and autonomous vehicles(CAV)in dynamic traffic environments under traditional control methods,a reinforcement learning-based control approach for CAV is proposed,aiming at reducing energy consumption,improving travel efficiency,and enhancing driving comfort.By considering the interactions between CAV and traffic signal control systems,as well as physical environmental factors,we collect signal phase and timing(SPaT),preceding vehicle speed and position,and other information to establish the state space of the reinforcement learning framework.Furthermore,an energy consumption model is established with the limit of battery energy recovery,and a multi-objective weighted reward function is designed based on key performance indicators such as energy consumption per unit time,travel distance,and acceleration change rate.The optimal weights for each performance indicator are determined using the analytic hierarchy process,and the model is trained using a deep deterministic policy gradient algorithm,with the algorithm parameters optimized through gradient descent.Simulation experiments were carried out using the SUMO platform the results demonstrate that the proposed algorithm achieves the most balanced travel performance,with a 9.22% reduction in energy consumption and an 18.77% reduction in change rate of acceleration compared to the DQN algorithm,as well as an 8.39% reduction in travel time compared to the Krauss car-following model.In conclusion,the results validate the feasibility and effectiveness of the proposed CAV control approach in reducing energy consumption,improving travel efficiency,and enhancing driving comfort.