NSGA-Ⅱ based traffic signal control optimization algorithm for over-saturated intersection group

In order to improve the efficiency of traffic signal control for an over-saturated intersection group, a nondominated sorting genetic algorithm Ⅱ（NSGA-Ⅱ） based traffic signal control optimization algorithm is proposed. The throughput maximum and average queue ratio minimum for the critical route of the intersection group are selected as the optimization objectives of the traffic signal control for the over-saturated condition. The consequences of the efficiency between traffic signal timing plans generated by the proposed algorithm and a commonly utilized signal timing optimization software Synchro are compared in a VISSIM signal control application programming interfaces （SCAPI） simulation environment by using real filed observed traffic data. The simulation results indicate that the signal timing plan generated by the proposed algorithm is more efficient in managing oversaturated flows at intersection groups, and, thus, it has the capability of optimizing signal timing under the over-saturated conditions.

Fuzzy traffic signal control with DNA evolutionary algorithm

In order to optimize the signal control system, this paper proposes a method to design an optimized fuzzy logic controller （FLC） with the DNA evolutionary algorithm. Inspired by the DNA molecular operation characteristics, the DNA evolutionary algorithm modifies the corresponding genetic operators. Compared with the traditional genetic algorithm （GA）, the DNA evolutionary algorithm can overcome weak local search capability and premature convergence. The parameters of membership functions are optimized by adopting the quaternary encoding method and performing corresponding DNA genetic operators. The relevant optimized parameters are combined with the FLC for single intersection traffic signal control. Simulation experiments shows the better performance of the FLC with the DNA evolutionary algorithm optimization. The experimental results demonstrate the efficiency of the nrotmsed method.

Tri-level programming model for combined urban traffic signal control and traffic flow guidance

In order to balance the temporal-spatial distribution of urban traffic flow, a model is established for combined urban traffic signal control and traffic flow guidance. With consideration of the wide use of fixed signal control at intersections, traffic assignment under traffic flow guidance, and dynamic characteristics of urban traffic management, a tri-level programming model is presented. To reflect the impact of intersection delay on traffic assignment, the lower level model is set as a modified user equilibrium model. The middle level model, which contains several definitional constraints for different phase modes, is built for the traffic signal control optimization. To solve the problem of tide lane management, the upper level model is built up based on nonlinear 0-1 integer programming. A heuristic iterative optimization algorithm(HIOA) is set up to solve the tri-level programming model. The lower level model is solved by method of successive averages(MSA), the middle level model is solved by non-dominated sorting genetic algorithm II(NSGA II), and the upper level model is solved by genetic algorithm(GA). A case study is raised to show the efficiency and applicability of the proposed modelling and computing method.

FL-FN-MOGA Based Traffic Signal Control

In this paper, a traffic signal control method based on fuzzy logic (FL), fuzzy-neuro (FN) and multi-objective genetic algorithms (MOGA) for an isolated four-approach intersection with through and left-turning movements is presented. This method has an adaptive signal timing ability, and can make adjustments to signal timing in response to observed changes.The 'urgency degree' term, which can describe the different user's demand for green time is used in decision-making by which strategy of signal timing can be determined. Using a fuzzy logic controller, we can determine whether to extend or terminate the current signal phase and select the sequences of phases. In this paper, a method based on fuzzy-neuro can be used to predict traffic parameters used in fuzzy logic controller. The feasibility of using a multi-objective genetic algorithm ( MOGA) to find a group of optimizing sets of parameters for fuzzy logic controller depending on different objects is also demonstrated. Simulation results show that the proposed methed is effecfive to adjust the signal timing in response to changing traffic conditions on a real-time basis, and the controller can produce lower vehicle delays and percentage of stopped vehicles than a traffic-actuated controller.

An integrated and cooperative architecture for multi-intersection traffic signal control

Traffic signal control(TSC)systems are one essential component in intelligent transport systems.However,relevant studies are usually independent of the urban traffic simulation environment,collaborative TSC algorithms and traffic signal communication.In this paper,we propose(1)an integrated and cooperative Internet-of-Things architecture,namely General City Traffic Computing System(GCTCS),which simultaneously leverages an urban traffic simulation environment,TSC algorithms,and traffic signal communication;and(2)a general multi-agent reinforcement learning algorithm,namely General-MARL,considering cooperation and communication between traffic lights for multi-intersection TSC.In experiments,we demonstrate that the integrated and cooperative architecture of GCTCS is much closer to the real-life traffic environment.The General-MARL increases the average movement speed of vehicles in traffic by 23.2%while decreases the network latency by 11.7%.

A Survey of Model Predictive Control Methods for Traffic Signal Control

Enhancing traffic efficiency and alleviating(even circumventing) traffic congestion with advanced traffic signal control(TSC) strategies are always the main issues to be addressed in urban transportation systems. Since model predictive control(MPC) has a lot of advantages in modeling complex dynamic systems, it has been widely studied in traffic signal control over the past 20 years. There is a need for an in-depth understanding of MPC-based TSC methods for traffic networks. Therefore, this paper presents the motivation of using MPC for TSC and how MPC-based TSC approaches are implemented to manage and control the dynamics of traffic flows both in urban road networks and freeway networks. Meanwhile, typical performance evaluation metrics, solution methods, examples of simulations,and applications related to MPC-based TSC approaches are reported. More importantly, this paper summarizes the recent developments and the research trends in coordination and control of traffic networks with MPC-based TSC approaches. Remaining challenges and open issues are discussed towards the end of this paper to discover potential future research directions.

Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control

This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system.A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency.Firstly a regional multi-agent Q-learning framework is proposed,which can equivalently decompose the global Q value of the traffic system into the local values of several regions Based on the framework and the idea of human-machine cooperation,a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to realtime traffic flow densities.In order to achieve better cooperation inside each region,a lightweight spatio-temporal fusion feature extraction network is designed.The experiments in synthetic real-world and city-level scenarios show that the proposed RegionS TLight converges more quickly,is more stable,and obtains better asymptotic performance compared to state-of-theart models.

Adaptive green traffic signal controlling using vehicular communication

The importance of using adaptive traffic signal control for figuring out the unpredictable traffic congestion in today's metropolitan life cannot be overemphasized. The vehicular ad hoc network(VANET), as an integral component of intelligent transportation systems(ITSs), is a new potent technology that has recently gained the attention of academics to replace traditional instruments for providing information for adaptive traffic signal controlling systems(TSCSs). Meanwhile, the suggestions of VANET-based TSCS approaches have some weaknesses:(1) imperfect compatibility of signal timing algorithms with the obtained VANET-based data types, and(2) inefficient process of gathering and transmitting vehicle density information from the perspective of network quality of service(Qo S). This paper proposes an approach that reduces the aforementioned problems and improves the performance of TSCS by decreasing the vehicle waiting time, and subsequently their pollutant emissions at intersections. To achieve these goals, a combination of vehicle-to-vehicle(V2V) and vehicle-to-infrastructure(V2I) communications is used. The V2 V communication scheme incorporates the procedure of density calculation of vehicles in clusters, and V2 I communication is employed to transfer the computed density information and prioritized movements information to the road side traffic controller. The main traffic input for applying traffic assessment in this approach is the queue length of vehicle clusters at the intersections. The proposed approach is compared with one of the popular VANET-based related approaches called MC-DRIVE in addition to the traditional simple adaptive TSCS that uses the Webster method. The evaluation results show the superiority of the proposed approach based on both traffic and network Qo S criteria.

Urban Intersection Traffic Signal Control Based on Fuzzy Logic

This paper presents a fuzzy logic adaptive traffic signal control method for an isolated four-approach intersection with through and left-turning movements. In the proposed method, the fuzzy logic controller can make adjustments to signal timing in response to observed changes. The 'urgency degree' term that can describe different user's demands for a green light is used in the fuzzy logic decision-making. In addition, a three-level fuzzy controller model decides whether to extend or terminate the current signal phase and the sequence of phases. Simulation results show that the fuzzy controller can adjust its signal timing in response to changing traffic conditions on a real-time basis and that the proposed fuzzy logic controller leads to less vehicle delays and a lower percentage of stopped vehicles.

Image-based traffic signal control via world models

Traffic signal control is shifting from passive control to proactive control, which enables the controller to direct current traffic flow to reach its expected destinations. To this end, an effective prediction model is needed for signal controllers. What to predict, how to predict, and how to leverage the prediction for control policy optimization are critical problems for proactive traffic signal control. In this paper, we use an image that contains vehicle positions to describe intersection traffic states. Then, inspired by a model-based reinforcement learning method, DreamerV2,we introduce a novel learning-based traffic world model. The traffic world model that describes traffic dynamics in image form is used as an abstract alternative to the traffic environment to generate multi-step planning data for control policy optimization. In the execution phase, the optimized traffic controller directly outputs actions in real time based on abstract representations of traffic states, and the world model can also predict the impact of different control behaviors on future traffic conditions. Experimental results indicate that the traffic world model enables the optimized real-time control policy to outperform common baselines, and the model achieves accurate image-based prediction, showing promising applications in futuristic traffic signal control.

Traffic Signals Control with Adaptive Fuzzy Controller in Urban Road Network

An adaptive fuzzy logic controller (AFC) is presented for the signal control of the urban traffic network. The AFC is composed of the signal control system-oriented control level and the signal controller-oriented fuzzy rules regulation level. The control level decides the signal timings in an intersection with a fuzzy logic controller. The regulation level optimizes the fuzzy rules by the Adaptive Rule Module in AFC according to both the system performance index in current control period and the traffic flows in the last one. Consequently the system performances are improved. A weight coefficient controller (WCC) is also developed to describe the interactions of traffic flow among the adjacent intersections. So the AFC combined with the WCC can be applied in a road network for signal timings. Simulations of the AFC on a real traffic scenario have been conducted. Simulation results indicate that the adaptive controller for traffic control shows better performance than the actuated one.

A multi process value-based reinforcement learning environment framework for adaptive traffic signal control

Realising adaptive traffic signal control(ATSC)through reinforcement learning(RL)is an important means to easetraffic congestion.This paper finds the computing power of the central processing unit(CPU)cannot fully usedwhen Simulation of Urban MObility(SUMO)is used as an environment simulator for RL.We propose a multi-process framework under value-basedRL.First,we propose a shared memory mechanism to improve exploration efficiency.Second,we use the weight sharing mechanism to solve the problem of asynchronous multi-process agents.We also explained the reason shared memory in ATSC does not lead to early local optima of the agent.Wehave verified in experiments the sampling efficiency of the 10-process method is 8.259 times that of the single process.The sampling efficiency of the 20-process method is 13.409 times that of the single process.Moreover,the agent can also converge to the optimal solution.

Traffic signal control in mixed traffic environment based on advance decision and reinforcement learning

Reinforcement learning-based traffic signal control systems (RLTSC) can enhance dynamic adaptability, save vehicle travelling timeand promote intersection capacity. However, the existing RLTSC methods do not consider the driver’s response time requirement, sothe systems often face efficiency limitations and implementation difficulties.We propose the advance decision-making reinforcementlearning traffic signal control (AD-RLTSC) algorithm to improve traffic efficiency while ensuring safety in mixed traffic environment.First, the relationship between the intersection perception range and the signal control period is established and the trust region state(TRS) is proposed. Then, the scalable state matrix is dynamically adjusted to decide the future signal light status. The decision will bedisplayed to the human-driven vehicles (HDVs) through the bi-countdown timer mechanism and sent to the nearby connected automatedvehicles (CAVs) using the wireless network rather than be executed immediately. HDVs and CAVs optimize the driving speedbased on the remaining green (or red) time. Besides, the Double Dueling Deep Q-learning Network algorithm is used for reinforcementlearning training;a standardized reward is proposed to enhance the performance of intersection control and prioritized experiencereplay is adopted to improve sample utilization. The experimental results on vehicle micro-behaviour and traffic macro-efficiencyshowed that the proposed AD-RLTSC algorithm can simultaneously improve both traffic efficiency and traffic flow stability.

Adaptive signal control and coordination for urban traffic control in a connected vehicle environment: A review

Existing signal control systems for urban traffic are usually based on traffic flow data from fixed location detectors.Because of rapid advances in emerging vehicular communication,connected vehicle(CV)-based signal control demonstrates significant improvements over existing conventional signal control systems.Though various CV-based signal control systems have been investigated in the past decades,these approaches still have many issues and drawbacks to overcome.We summarize typical components and structures of these existing CV-based urban traffic signal control systems and digest several important issues from the summarized vital concepts.Last,future research directions are discussed with some suggestions.We hope this survey can facilitate the connected and automated vehicle and transportation research community to efficiently approach next-generation urban traffic signal control methods and systems.

Evolution of Road Traffic Congestion Control:A Survey from Perspective of Sensing,Communication,and Computation

Road traffic congestion can inevitably de-grade road infrastructure and decrease travel efficiency in urban traffic networks,which can be relieved by employing appropriate congestion control.Accord-ing to different developmental driving forces,in this paper,the evolution of road traffic congestion control is divided into two stages.The ever-growing num-ber of advanced sensing techniques can be seen as the key driving force of the first stage,called the sens-ing stage,in which congestion control strategies ex-perienced rapid growth owing to the accessibility of traffic data.At the second stage,i.e.,the communica-tion stage,communication and computation capabil-ity can be regarded as the identifying symbols for this stage,where the ability of collecting finer-grained in-sight into transportation and mobility reality improves dramatically with advances in vehicular networks,Big Data,and artificial intelligence.Specifically,as the pre-requisite for congestion control,in this paper,ex-isting congestion detection techniques are first elab-orated and classified.Then,a comprehensive survey of the recent advances for current congestion control strategies with a focus on traffic signal control,vehi-cle route guidance,and their combined techniques is provided.In this regard,the evolution of these strate-gies with continuous development of sensing,com-munication,and computation capability are also intro-duced.Finally,the paper concludes with several re-search challenges and trends to fully promote the in-tegration of advanced techniques for traffic congestion mitigation in transportation systems.

Efficacy of decentralized traffic signal controllers on stabilizing heterogeneous urban grid network

Macroscopic Fundamental Diagrams(MFDs)are valuable for designing and evaluating network-wide traffic management schemes.Since obtaining empirical MFDs can be expensive,analytical methodologies are crucial to estimate variations in MFD shapes under different control strategies and predict their efficacy in mitigating congestion.Analyses of urban grid networks'abstractions can provide an inexpensive methodology to obtain a qualitative understanding of impacts of control policies.However,existing abstractions are valid only for simple intersection layouts with unidirectional and single-lane links and two conflicting movement groups.Naturally,the real intersections are more complex,with multiple incoming and outgoing lanes,heterogeneous incoming links'capacities and several conflicting movement groups.To this end,we consider a grid network with differences in capacities of horizontal and vertical directions,allowing us to investigate the characteristics of control policies that can avoid pernicious gridlock in heterogeneous networks.We develop a new,more comprehensive network abstraction of such grid networks to analyze and compare the impacts of two families of decentralized Traffic Signal Controllers(TSCs)on the network's stability.The obtained theoretical insights are verified using microsimulation results of grid networks with multiple signalized intersections.The analyses suggest that considering both upstream and downstream congestion information in deciding signal plans can encourage more evenly distributed traffic in the network,making them more robust and effective at all congestion levels.The study provides a framework to understand general expectations from decentralized control policies when network inhomogeneity arises due to variations in incoming link capacities and turning directions.

Real-Time Traffic Signal Timing for Urban Road Multi-Intersection

this paper develops a real-time traffic signal timing model which is to be integrated into a single intersection for urban road, thereby solving the problem of traffic congestion. We analyze the current situation of the traffic flow with release matrix firstly, and then put forward the basic models to minimize total delay time of vehicles at the intersection. The optimal real-time signal timing model (non-fixed cycle and non-fixed split) is built with the Webster split optimal model. At last, the simulated results, which are compared with conventional model, manifest the promising properties of proposed model.

An ATMS data-driven method for signalized arterial coordination

In order to minimize the delays and stops caused by the early started coordinated green phase of the vehicle- actuated signal systems, a stochastic offsets calculation method based on the new types of advanced traffic management system （ATMS） data is proposed. As the mainline green starts randomly in vehicle-actuated signal systems, the random theory is applied to obtain the distribution of the unused green time at side streets based on the green gap-out mechanism. Then, the green start time of the mainline can be selected at the point with maximum probability to minimize the delays or stops caused by the randomly started mainline green. A case study in Maine, USA, whose traffic conditions are similar to those of the middle-size Chinese cities, proves that the proposed method can significantly reduce the travel time and delays.

基于全局状态预测与公平经验重放的交通信号控制算法

为了应对交通拥堵而设计的高效交通信号控制算法能提升现有交通网络下的车辆通行效率。尽管深度强化学习算法在单路口交通信号控制问题上已展现出卓越的性能,然而这些算法在多路口环境下的应用仍然面临着重大的挑战——多智能体强化学习(MARL)算法产生的时间和空间的部分可观测性引发的非平稳性问题会导致这些算法无法稳定的收敛。因此,提出一种基于全局状态预测与公平经验重放的多路口交通信号控制算法IS-DQN。一方面,基于不同车道的车流历史信息预测多路口的全局状态,从而扩展IS-DQN的状态空间,以避免算法产生空间部分可观测性而带来非平稳性问题;另一方面,为应对传统经验重放策略的时间部分可观测性,采用蓄水池抽样算法来保证经验重放池的公正性,进而避免其中的非平稳性问题。在复杂的多路口环境下应用IS-DQN算法进行3种不同的交通压力仿真实验的结果表明:在不同交通流情况下,尤其是在中低交通流量下,相较于独立的深度强化学习算法,ISDQN算法能得到更短的车辆平均行驶时间,并表现出了更优的收敛性能与收敛稳定性。

基于改进D3QN的单点交叉口信号控制研究

近年交通拥堵已成为制约城市经济发展的重要问题,利用深度强化学习(Deep Reinforcement Learning,DRL)对交通信号灯进行自适应控制是缓解交通拥堵的研究热点。针对决斗双重深度Q网络(Dueling Double Deep Q-Network,D3QN)算法在交通信号控制中存在的样本利用率低、学习速度慢,以及路网状态信息复杂且灵活性差等问题,基于非均匀划分道路的离散交通状态编码(Discrete Traffic State Encode,DTSE)方法,提出一种D3PQN2交通信号控制算法。该算法在D3QN算法基础上引入噪声网络、优先级经验回放技术来提高样本的利用效率以及学习速度,通过噪声扰动代替传统的ε-贪婪策略,使得算法能够更快更好地收敛到全局最优解。以扬州市文昌路和扬子江路交叉口为例,在Weibull分布生成的车流下进行实验,结果表明,改进后的算法相较于对抗深度Q网络(Dueling Deep Q-Network,Dueling DQN)算法和固定配时的控制方法,车辆平均排队长度分别减少了12.11%和67.44%,累计延误时间分别减少了13.89%和42.88%,具有更好的控制效果。