基于SARSA学习的单交叉口配时优化仿真与设计

Simulation and design of timing optimization at a single intersection based on SARSA learning

现如今我国城市道路网络迅速扩大,城市道路交通拥挤状况加剧。传统的定时控制和感应控制不再适应交通环境的多变性、随机性及不确定性。针对Q学习算法只考虑新的状态下获得的最大奖赏,而不考虑新状态带来的风险,将Sarsa学习应用于单交叉口配时优化方法中,Sarsa学习在目标选择策略中有着既考虑最优值,又考虑探索作用值的优点。以单交叉口平均延误最小为优化目标,确定最优策略,并在VisSim中进行仿真,在仿真检验阶段1 000个步长后,Sarsa学习的车辆平均延误相比于Q学习减少了1.277s。结果表明Sarsa学习算法在配时优化延误指标上优于Q学习。

Nowadays,China’s urban road network is rapidly expanding,and urban road traffic congestion is intensifying.Traditional timing control and induction control no longer adapt to the variability,randomness and uncertainty of the traffic environment.For the Q learning algorithm,only the maximum reward obtained in the new state is considered,and the risk brought by the new state is not considered.This design applies Sarsa learning to the single intersection timing optimization method.Sarsa learning has both advantages and disadvantages in the target selection strategy.Consider the optimal value,and consider the advantages of exploring the action value.This design takes the minimum average delay at a single intersection as the optimization goal,determines the optimal strategy,and performs simulation in VisSim.After 1000 steps in the simulation verification stage,the average delay of the vehicle learned by Sarsa is reduced by 1.277 seconds compared to Q learning.The results show that Sarsa learning algorithm is better than Q learning in timing optimization delay index.