Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control

This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system.A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency.Firstly a regional multi-agent Q-learning framework is proposed,which can equivalently decompose the global Q value of the traffic system into the local values of several regions Based on the framework and the idea of human-machine cooperation,a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to realtime traffic flow densities.In order to achieve better cooperation inside each region,a lightweight spatio-temporal fusion feature extraction network is designed.The experiments in synthetic real-world and city-level scenarios show that the proposed RegionS TLight converges more quickly,is more stable,and obtains better asymptotic performance compared to state-of-theart models.

NSGA-Ⅱ based traffic signal control optimization algorithm for over-saturated intersection group

In order to improve the efficiency of traffic signal control for an over-saturated intersection group, a nondominated sorting genetic algorithm Ⅱ（NSGA-Ⅱ） based traffic signal control optimization algorithm is proposed. The throughput maximum and average queue ratio minimum for the critical route of the intersection group are selected as the optimization objectives of the traffic signal control for the over-saturated condition. The consequences of the efficiency between traffic signal timing plans generated by the proposed algorithm and a commonly utilized signal timing optimization software Synchro are compared in a VISSIM signal control application programming interfaces （SCAPI） simulation environment by using real filed observed traffic data. The simulation results indicate that the signal timing plan generated by the proposed algorithm is more efficient in managing oversaturated flows at intersection groups, and, thus, it has the capability of optimizing signal timing under the over-saturated conditions.

Travel time function for basic link considering signal control in network traffic model

In order to describe the travel time of signalcontrolled roads, a travel time model for urban basic roads based on the cumulative curve is proposed. First, the traffic wave method is used to analyze the formation and dispersion of the vehicle queue. Cumulative curves for road entrances and exits are established. Based on the cumulative curves, the travel time of the one-lane road under stable flow input is derived. And then, the multi-lane road is decomposed into a series of single-lane links based on its topological characteristics. Hence, the travel time function for the basic road is obtained. The travel time is a function of road length, flow and control parameters. Numerical analyses show that the travel time depends on the supply-demand condition, and it has high sensitivity during peak hours.

Fuzzy traffic signal control with DNA evolutionary algorithm

In order to optimize the signal control system, this paper proposes a method to design an optimized fuzzy logic controller （FLC） with the DNA evolutionary algorithm. Inspired by the DNA molecular operation characteristics, the DNA evolutionary algorithm modifies the corresponding genetic operators. Compared with the traditional genetic algorithm （GA）, the DNA evolutionary algorithm can overcome weak local search capability and premature convergence. The parameters of membership functions are optimized by adopting the quaternary encoding method and performing corresponding DNA genetic operators. The relevant optimized parameters are combined with the FLC for single intersection traffic signal control. Simulation experiments shows the better performance of the FLC with the DNA evolutionary algorithm optimization. The experimental results demonstrate the efficiency of the nrotmsed method.

Traffic Signal Timing via Deep Reinforcement Learning

In this paper, we propose a set of algorithms to design signal timing plans via deep reinforcement learning. The core idea of this approach is to set up a deep neural network (DNN) to learn the Q-function of reinforcement learning from the sampled traffic state/control inputs and the corresponding traffic system performance output. Based on the obtained DNN, we can find the appropriate signal timing policies by implicitly modeling the control actions and the change of system states. We explain the possible benefits and implementation tricks of this new approach. The relationships between this new approach and some existing approaches are also carefully discussed. © 2014 Chinese Association of Automation.

A Survey of Model Predictive Control Methods for Traffic Signal Control

Enhancing traffic efficiency and alleviating(even circumventing) traffic congestion with advanced traffic signal control(TSC) strategies are always the main issues to be addressed in urban transportation systems. Since model predictive control(MPC) has a lot of advantages in modeling complex dynamic systems, it has been widely studied in traffic signal control over the past 20 years. There is a need for an in-depth understanding of MPC-based TSC methods for traffic networks. Therefore, this paper presents the motivation of using MPC for TSC and how MPC-based TSC approaches are implemented to manage and control the dynamics of traffic flows both in urban road networks and freeway networks. Meanwhile, typical performance evaluation metrics, solution methods, examples of simulations,and applications related to MPC-based TSC approaches are reported. More importantly, this paper summarizes the recent developments and the research trends in coordination and control of traffic networks with MPC-based TSC approaches. Remaining challenges and open issues are discussed towards the end of this paper to discover potential future research directions.

Joint optimization traffic signal control for an urban arterial road

This paper considers the optimal traffic signal setting for an urban arterial road. By introducing the concepts of synchronization rate and non-synchronization degree, a mathematical model is constructed and an optimization problem is posed. Then, a new iterative algorithm is developed to solve this optimal traffic control signal setting problem. Convergence properties for this iterative algorithm are established. Finally, a numerical example is solved to illustrate the effectiveness of the method.

Connected Vehicle-Based Traffic Signal Coordination

This study presents a connected vehicles(CVs)-based traffic signal optimization framework for a coordinated arterial corridor.The signal optimization and coordination problem are first formulated in a centralized scheme as a mixed-integer nonlinear program(MINLP).The optimal phase durations and offsets are solved together by minimizing fuel consumption and travel time considering an individual vehicle’s trajectories.Due to the complexity of the model,we decompose the problem into two levels:an intersection level to optimize phase durations using dynamic programming(DP),and a corridor level to optimize the offsets of all intersections.In order to solve the two-level model,a prediction-based solution technique is developed.The proposed models are tested using traffic simulation under various scenarios.Compared with the traditional actuated signal timing and coordination plan,the signal timing plans generated by solving the MINLP and the two-level model can reasonably improve the signal control performance.When considering varies vehicle types under high demand levels,the proposed two-level model reduced the total system cost by 3.8%comparing to baseline actuated plan.MINLP reduced the system cost by 5.9%.It also suggested that coordination scheme was beneficial to corridors with relatively high demand levels.For intersections with major and minor street,coordination conducted for major street had little impacts on the vehicles at the minor street.

Modulating Traffic Signal Phases to Realize Real-Time Traffic Control System

This paper proposes innovations to address challenges emanating from road traffic congestion. Improving economies create more car owners resulting in increased automobile manufacturing, increased vehicle population giving rise to higher emission of CO2 resulting in traffic congestion. Congested traffic has idling vehicles which emit higher CO2 and pollution. Besides, traffic congestion increases turnaround time, delivery time, commuting time and related logistical aspects. Commuting time negatively impacts working hours. Unless the traffic congestion is mitigated, the economy will take a beating creating a vicious ecology cycle. Building new roads, bridges or reconditioning of infrastructure is not always the best possible solutions. Efficient traffic management is a key to country’s economic growth. Various analytical models are employed to study, appreciate traffic congestion. The paper studies these models to infer that real time approach is the only solution. Several approaches are being worked on and few commercial systems too are available. These systems provide traffic information for course correction. However, it has latency and hence deviates from real time environment. Traffic congestion being highly dynamic in nature, it necessitates real time solution with real time inputs. It is proposed to integrate Real time traffic data with the traffic signal thus modulating the cycle timings at every junction. Deviation from static asymmetric cycle timing is implemented by assigning green phases based on density of vehicles. With minimalistic infrastructure and negligible incremental cost, the paper not only proposes to address traffic congestion but also paves the way for capturing traffic offenses, vehicle tracking and toll collection. The research is imminently realizable and makes a strong case for a PPP (Public Private Partnership) project.

A new cellular automaton for signal controlled traffic flow based on driving behaviors

The complexity of signal controlled traffic largely stems from the various driving behaviors developed in response to the traffic signal. However, the existing models take a few driving behaviors into account and consequently the traffic dynamics has not been completely explored. Therefore, a new cellular automaton model, which incorporates the driving behaviors typically manifesting during the different stages when the vehicles are moving toward a traffic light, is proposed in this paper. Numerical simulations have demonstrated that the proposed model can produce the spontaneous traffic breakdown and the dissolution of the over-saturated traffic phenomena. Furthermore, the simulation results indicate that the slow-to-start behavior and the inch-forward behavior can foster the traffic breakdown. Particularly, it has been discovered that the over-saturated traffic can be revised to be an under-saturated state when the slow-down behavior is activated after the spontaneous breakdown. Finally, the contributions of the driving behaviors on the traffic breakdown have been examined.

Longitudinal Performance Assessment of Traffic Signal System Impacted by Long-Term Interstate Construction Diversion Using Connected Vehicle Data

Local arterials can be significantly impacted by diversions from adjacent work zones. These diversions often occur on unofficial detour routes due to guidance received on personal navigation devices. Often, these routes do not have sufficient sensing or communication equipment to obtain infrastructure-based traffic signal performance measures, so other data sources are required to identify locations being significantly affected by diversions. This paper examines the network impact caused by the start of an 18-month closure of the I-65/70 interchange (North Split), which usually serves approximately 214,000 vehicles per day in Indianapolis, IN. In anticipation of some proportion of the public diverting from official detour routes to local streets, a connected vehicle monitoring program was established to provide daily performances measures for over 100 intersections in the area without the need for vehicle sensing equipment. This study reports on 13 of the most impacted signals on an alternative arterial to identify locations and time of day where operations are most degraded, so that decision makers have quantitative information to make informed adjustments to the system. Individual vehicle movements at the studied locations are analyzed to estimate changes in volume, split failures, downstream blockage, arrivals on green, and travel times. Over 130,000 trajectories were analyzed in an 11-week period. Weekly afternoon peak period volumes increased by approximately 455%, split failures increased 3%, downstream blockage increased 10%, arrivals on green decreased 16%, and travel time increase 74%. The analysis performed in this paper will serve as a framework for any agency that wants to assess traffic signal performance at hundreds of locations with little or no existing sensing or communication infrastructure to prioritize tactical retiming and/or longer-term infrastructure investments.

Traffic jam in signalized road network

Traffic jam in large signalized road network presents a complex nature.In order to reveal the jam characteristics,two indexes,SVS(speed of virtual signal) and VOS(velocity of spillover),were proposed respectively.SVS described the propagation of queue within a link while VOS reflected the spillover velocity of vehicle queue.Based on the two indexes,network jam simulation was carried out on a regular signalized road network.The simulation results show that:1) The propagation of traffic congestion on a signalized road network can be classified into two stages:virtual split driven stage and flow rate driven stage.The former stage is characterized by decreasing virtual split while the latter only depends on flow rate; 2) The jam propagation rate and direction are dependent on traffic demand distribution and other network parameters.The direction with higher demand gets more chance to be jammed.Our findings can serve as the basis of the prevention of the formation and propagation of network traffic jam.

Tri-level programming model for combined urban traffic signal control and traffic flow guidance

In order to balance the temporal-spatial distribution of urban traffic flow, a model is established for combined urban traffic signal control and traffic flow guidance. With consideration of the wide use of fixed signal control at intersections, traffic assignment under traffic flow guidance, and dynamic characteristics of urban traffic management, a tri-level programming model is presented. To reflect the impact of intersection delay on traffic assignment, the lower level model is set as a modified user equilibrium model. The middle level model, which contains several definitional constraints for different phase modes, is built for the traffic signal control optimization. To solve the problem of tide lane management, the upper level model is built up based on nonlinear 0-1 integer programming. A heuristic iterative optimization algorithm(HIOA) is set up to solve the tri-level programming model. The lower level model is solved by method of successive averages(MSA), the middle level model is solved by non-dominated sorting genetic algorithm II(NSGA II), and the upper level model is solved by genetic algorithm(GA). A case study is raised to show the efficiency and applicability of the proposed modelling and computing method.

A Model with Traffic Routers, Dynamically Managing Signal Phases to Address Traffic Congestion in Real Time

On-road Vehicular traffic congestion has detrimental effect on three lifelines: Economy, Productivity and Pollution (EPP). With ever increasing population of vehicles on road, traffic congestion is a major challenge to the economy, productivity and pollution, notwithstanding continuous developments in alternative fuels, alternative sources of energy. The research develops accurate and precise model in real time which computes congestion detection, dynamic signaling algorithm to evenly distribute vehicle densities while ensuring avoidance of starvation and deadlock situation. The model incorporates road segment length and breadth, quality and achievable average speed to compute road capacity. Vehicles installed with GPS enabled devices provide their location, which enables computing road occupancy. Road occupancy is evaluated based on number of vehicles as well as area occupied by vehicles. Ratio of road occupancy and road capacity provides congestion index important to compute signal phases. The algorithm ensures every direction is serviced once during a signaling cycle ensuring no starvation. Secondly, the definition of minimum and maximum signal timings ensures against dead lock situation. A simulator is developed to validate the proposition and proves it can ease congestion by more than 50% which is better than any of the contemporary approaches offering 15% improvement. In case of higher congestion index, alternate routes are suggested based on evaluation of traffic density graphs for shortest route or knowledge database. The algorithm to compute shortest route is optimized drastically, reducing computation cost to 3*√2N vis-à-vis computation cost of N2 by classical algorithms. The proposal brings down the cost of implementation per traffic junction from USD 30,000 to USD 2000.

Study on Geometric Factors Influencing Saturation Flow Rate at Signalized Intersections under Heterogeneous Traffic Conditions

The main objective of intersection design is to facilitate the convenience, comfort, and safety of people traversing the intersection by enhancing the efficient movement of road users. The intersections on urban roads in India generally cater to heterogeneous motorized traffic, along with slow-moving traffic including pedestrians. It is therefore necessary to consider saturation flow for mixed traffic conditions to evaluate the overall operation of signalized intersections. A proper traffic model must consider varying characteristics of all the road users to effectively design and efficiently manage signalized intersections. This paper presents the results of the study on analyses of saturation flow rate conducted at signalized intersections with mixed traffic conditions in the city of Bangalore, India. Studies were carried out at 15 signalized intersections in the city of Bangalore with varying geometric factors such as width of road (w), gradient of the road (g), and turning radius (r) for right turning vehicles. Saturation flow rate computed as per Highway Capacity manual (HCM: 2000), Indonesian highway capacity manual (IHCM), and IRC SP: 41-1994 was compared with the field observations. The geometric factors, which affect the saturation flow, have been considered in this study and accordingly a new model has been proposed for determining saturation flow. It has been shown that by the introduction of the suggested adjustment factors in this paper, the saturation flow rate can give better picture of the field conditions, especially under heterogeneous traffic conditions of an urban area.

Cell transmission model based traffic signal timing in oversaturated conditions

In order to investigate enhancements to cell transmission model (CTM) as a tool for traffic signal timing in oversaturated conditions, randomly distributed saturation flow rates and arrival rates were used instead of constant values to simulate traffic flow movement, estimate the average delay of the network and search for an optimal traffic signal timing plan. A case study was given to demonstrate that the proposed methodology can capture unique phenomena in oversaturated conditions such as forward wave, spillback and lane entrance blockage. The results show that CTM underestimates travel time by 25% when compared to Simtraffic, while the enhanced CTM underestimates by only 3%. A second case study shows that a dynamic signal timing plan is superior to a fixed signal timing plan in the term of average delay.

Traffic Signals Control with Adaptive Fuzzy Controller in Urban Road Network

An adaptive fuzzy logic controller (AFC) is presented for the signal control of the urban traffic network. The AFC is composed of the signal control system-oriented control level and the signal controller-oriented fuzzy rules regulation level. The control level decides the signal timings in an intersection with a fuzzy logic controller. The regulation level optimizes the fuzzy rules by the Adaptive Rule Module in AFC according to both the system performance index in current control period and the traffic flows in the last one. Consequently the system performances are improved. A weight coefficient controller (WCC) is also developed to describe the interactions of traffic flow among the adjacent intersections. So the AFC combined with the WCC can be applied in a road network for signal timings. Simulations of the AFC on a real traffic scenario have been conducted. Simulation results indicate that the adaptive controller for traffic control shows better performance than the actuated one.

FL-FN-MOGA Based Traffic Signal Control

In this paper, a traffic signal control method based on fuzzy logic (FL), fuzzy-neuro (FN) and multi-objective genetic algorithms (MOGA) for an isolated four-approach intersection with through and left-turning movements is presented. This method has an adaptive signal timing ability, and can make adjustments to signal timing in response to observed changes.The 'urgency degree' term, which can describe the different user's demand for green time is used in decision-making by which strategy of signal timing can be determined. Using a fuzzy logic controller, we can determine whether to extend or terminate the current signal phase and select the sequences of phases. In this paper, a method based on fuzzy-neuro can be used to predict traffic parameters used in fuzzy logic controller. The feasibility of using a multi-objective genetic algorithm ( MOGA) to find a group of optimizing sets of parameters for fuzzy logic controller depending on different objects is also demonstrated. Simulation results show that the proposed methed is effecfive to adjust the signal timing in response to changing traffic conditions on a real-time basis, and the controller can produce lower vehicle delays and percentage of stopped vehicles than a traffic-actuated controller.

Real-Time Traffic Signal Timing for Urban Road Multi-Intersection

this paper develops a real-time traffic signal timing model which is to be integrated into a single intersection for urban road, thereby solving the problem of traffic congestion. We analyze the current situation of the traffic flow with release matrix firstly, and then put forward the basic models to minimize total delay time of vehicles at the intersection. The optimal real-time signal timing model (non-fixed cycle and non-fixed split) is built with the Webster split optimal model. At last, the simulated results, which are compared with conventional model, manifest the promising properties of proposed model.

Delay Analysis and Formulation Inference of Signalized Intersection for Traffic Congestion Conditions

Vehicle delay is an important measure to evaluate the signal timings of signalized intersections.When optimization the signal control parameters, delays of vehicles from all approach directions of an intersection should be considered. Based on the analysis of the vehicle delay on an approach of intersection, directed against the typical condition of a congested intersection-over-saturated condition, the paper has analyzed and inferred the intersection delay dynamic formulation, and has established the relation between intersection delay,the signal timings, vehicle arrival rate and the queue lengths, and that provides useful information for understanding vehicle delay of signalized intersection and for establishing performance index function of signal timing optimization.