CTS:基于拥堵溯源算法的信号灯多智能体强化学习组织方案

CTS:traffic lights multi-agent reinforcement learning organization scheme based on congestion trace source algorithm

在交通路网的运行中红绿灯起着至关重要的调度作用,随着目前交通的飞速发展,道路越来越复杂、车辆越来越繁多,导致红绿灯的调度压力越来越大、调节能力却越来越弱。为了解决这一问题,建立了CTS(congestion trace source)方案,将交通疏导的主体对象红绿灯作为智能体进行强化学习以优化其对交通的疏导控制能力,通过构建拥堵链和拥堵环综合分析路网拥堵情况,佐以红绿灯相位及其配时数据以达到对红绿灯智能体对象状态的综合判断;CTS方案设计了红绿灯排队长度算法将拥堵情况数字化作为智能体奖励对优化效果进行评判。使用SUMO仿真环境进行实验,设计交通优化指标路口平均排队长度并进行对比,最终该方案的路口平均排队长度相较于原始数据提升了40%。

Traffic lights play a vital role in the operation of the traffic network.However,with the rapid development of traffic,roads are becoming more and more complex,and vehicles are becoming more and more numerous,which leads to the increasing pressure of traffic lights scheduling,but the regulation ability is becoming weaker and weaker.In order to solve this problem,this paper established the convergence trace source(CTS)scheme.This scheme used the traffic light,the main object of traffic diversion,as an agent for reinforcement learning to optimize its ability to control traffic diversion.It comprehensively analyzed the congestion situation of the road network by constructing the congestion chain and congestion ring,and used the traffic light phase and its timing data to achieve the comprehensive judgment of the object state of the traffic light agent.This scheme designed the traffic light queue length algorithm,and used the digitization of congestion as an agent reward to evaluate the optimization effect.This paper used the SUMO simulation environment for experiments,designed and compared the average queue length at the intersection of the traffic optimization index.Finally,the average queue length at the intersection of this scheme is increased by 40%compared with the original data.