基于深度强化学习的大型活动关键交叉口信号控制

Signal Control of Key Intersections in Large-scale Events Based on Deep Reinforcement Learning

大型活动举办时期,场馆周边路网的交通压力与日常交通运行状态存在差异,活动场馆周边关键交叉口的正常运行是保证大型活动顺利举办的重要因素之一,应采取动态的管控方式以达到提高关键交叉口通行效率、满足参与大型活动出行者交通需求的目的。为此,基于A2C(advantage actor critic)的强化学习算法,考虑大型活动背景下出行者数量大且大多采用公共交通出行的特点,在奖励函数构建过程中将车辆排队时间细分为出行者不同出行方式的车辆等待时间,通过引入参数,修正不同车型的奖励计算方法,使智能体在信号配时优化的过程中优先考虑大型活动参与者的出行需求。最后,以北京市首都体育馆周边大型交叉口为例,借助交通流仿真软件SUMO进行仿真实验,仿真实验结果证明,修改奖励函数结构后的A2C信号控制方法在控制效果上优于定时信号控制以及基于DQN(deep-Q-network)算法的控制方法,可以达到提高交叉口公共交通以及整体车流通行效率的目的。

The normal operation of key intersections around the event site is one of the most important factors in ensuring the smooth running of the event.A dynamic control method should be adopted to improve the efficiency of key intersections and meet the traffic demand of the event participants.Based on the reinforcement learning algorithm of A2C(advantage actor critic),the characteristics of a large number of travelers was considered in the context of large events and most of them used public transport to travel,and subdivided the vehicle waiting time into the vehicle waiting time of different travel modes of travelers in the process of reward function construction.The priority to the travel demand of large event participants was given by agent in the process of signal timing optimization.Finally,simulation experiments were conducted by using the SUMO traffic flow simulation software at the large intersection around the Beijing Capital Stadium.The simulation experiment results prove that the A2C signal control method with modified reward function structure is better than the timing signal control method and the control method based on DQN(deep-Q-network)algorithm in terms of control effect,and the goal of improving public transportation at the intersection and the overall traffic flow at intersections is achieved.