基于Double DQN的双模式多目标信号配时方法

A Dual-mode Multi-objective Signal Timing Method Based on Double DQN

近年来深度强化学习作为一种高效可靠的机器学习方法被广泛应用在交通信号控制领域。目前,现有交通信号配时方法通常忽略了特殊车辆(例如救护车、消防车等)的优先通行;此外,基于传统深度强化学习的信号配时方法优化目标较为单一,导致其在复杂交通场景中性能不佳。针对上述问题,基于Double DQN提出一种融合特殊车辆优先通行的双模式多目标信号配时方法(Dual-mode Multi-objective signal timing method based on Double DQN,DMDD),以提高不同交通场景下路口的通行效率。该方法首先基于路口的饱和状态选择信号控制模式,特殊车辆在紧急控制模式下被赋予更高的通行权重,有利于其更快通过路口;接着针对等待时长、队列长度和CO 2排放量3个指标分别设计神经网络进行奖励计算;最后利用Double DQN进行最优信号相位的选择,通过灵活切换信号相位以提升通行效率。基于SUMO的实验结果表明,DMDD与对比方法相比能有效缩短路口处特殊车辆的等待时长、队列长度和CO 2排放量,特殊车辆能够更快通过路口,有效地提高了通行效率。

In recent years,deep reinforcement learning has been widely used as an efficient and reliable machine learning method in the field of traffic signal control.Currently,existing traffic signal timing methods usually ignore the priority of special vehicles(e.g.,ambulances,fire engines,etc.);in addition,the optimization objectives of signal timing methods based on traditional deep reinforcement learning are often relatively single,resulting in poor performance in complex traffic scenarios.To address the above problems,we propose a Dual-mode Multi-objective signal timing method based on Double DQN(DMDD)that incorporates the priority of special vehicles for improving the traffic efficiency of intersections under different scenarios.The method first decides the signal control mode based on the saturation state of the intersection and gives higher weights to special vehicles when in emergency control mode so that they can pass through the intersection faster.Then,neural networks are designed to calculate the rewards for the three metrics of waiting time,queue length and CO 2 emission.Finally,Double DQN is utilized to select the optimal signal phase,and the signal phase is flexibly switched to improve the traffic efficiency.The experimental results based on SUMO show that the DMDD can effectively reduce the waiting time,queue length and CO 2 emission of special vehicles at the intersection compared with other methods,and special vehicles can pass through the intersection faster,which effectively improves the efficiency of traffic.