基于绿灯时间等饱和度的TD学习配时优化模型

The Optimization Model of TD Learning Timing Based on the Green Time Equi-saturation

首先对传统的绿灯时间等饱和度概念进行了扩展,提出了分级绿灯时间等饱和度.在此基础上,针对分级绿灯时间等饱和度目标,构造了奖赏函数,采用了模糊方法解决流量状态空间维数爆炸问题,建立了定周期和变周期两种模式下的四种离线TD学习配时优化模型.通过Matlab编程,开发了这四种模型的计算程序,相对于在线TD学习模型,离线TD学习模型更适合交叉口信号配时优化.以一个两相位控制的单交叉口配时优化作为算例,对比分析了四种模型的性能.总体上变周期模式的离线TD学习模型可以获得解的结构、最优解的分布,这是传统配时理论不具备的.定周期条件下,奖赏分级的效果不明显;变周期条件下,奖赏分级效果明显,交通性能更优.

We propose the multi-level green time saturation.On this basis,for the classification of green time saturation target,the study constructs a reward function,uses the fuzzy method to solve the traffic state space dimension explosion problem,and establishes four optimization models of offline TD learning under fixed period and variable cycle two modes.Using a two-phase control of a single intersection as an example,the study comparatively analyzes the performance of four models.Generally speaking,offline TD learning model of variable cycle mode can obtain the structure of solutions and the optimal solutions distribution,which does not belong to the traditional timing theory.Under the fixed period condition,reward grading effect is not obvious,while under the variable cycle condi-tion,reward grading effect is obvious and the traffic has better performance.