Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control

This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system.A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency.Firstly a regional multi-agent Q-learning framework is proposed,which can equivalently decompose the global Q value of the traffic system into the local values of several regions Based on the framework and the idea of human-machine cooperation,a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to realtime traffic flow densities.In order to achieve better cooperation inside each region,a lightweight spatio-temporal fusion feature extraction network is designed.The experiments in synthetic real-world and city-level scenarios show that the proposed RegionS TLight converges more quickly,is more stable,and obtains better asymptotic performance compared to state-of-theart models.

NSGA-Ⅱ based traffic signal control optimization algorithm for over-saturated intersection group

In order to improve the efficiency of traffic signal control for an over-saturated intersection group, a nondominated sorting genetic algorithm Ⅱ（NSGA-Ⅱ） based traffic signal control optimization algorithm is proposed. The throughput maximum and average queue ratio minimum for the critical route of the intersection group are selected as the optimization objectives of the traffic signal control for the over-saturated condition. The consequences of the efficiency between traffic signal timing plans generated by the proposed algorithm and a commonly utilized signal timing optimization software Synchro are compared in a VISSIM signal control application programming interfaces （SCAPI） simulation environment by using real filed observed traffic data. The simulation results indicate that the signal timing plan generated by the proposed algorithm is more efficient in managing oversaturated flows at intersection groups, and, thus, it has the capability of optimizing signal timing under the over-saturated conditions.

Travel time function for basic link considering signal control in network traffic model

In order to describe the travel time of signalcontrolled roads, a travel time model for urban basic roads based on the cumulative curve is proposed. First, the traffic wave method is used to analyze the formation and dispersion of the vehicle queue. Cumulative curves for road entrances and exits are established. Based on the cumulative curves, the travel time of the one-lane road under stable flow input is derived. And then, the multi-lane road is decomposed into a series of single-lane links based on its topological characteristics. Hence, the travel time function for the basic road is obtained. The travel time is a function of road length, flow and control parameters. Numerical analyses show that the travel time depends on the supply-demand condition, and it has high sensitivity during peak hours.

Fuzzy traffic signal control with DNA evolutionary algorithm

In order to optimize the signal control system, this paper proposes a method to design an optimized fuzzy logic controller （FLC） with the DNA evolutionary algorithm. Inspired by the DNA molecular operation characteristics, the DNA evolutionary algorithm modifies the corresponding genetic operators. Compared with the traditional genetic algorithm （GA）, the DNA evolutionary algorithm can overcome weak local search capability and premature convergence. The parameters of membership functions are optimized by adopting the quaternary encoding method and performing corresponding DNA genetic operators. The relevant optimized parameters are combined with the FLC for single intersection traffic signal control. Simulation experiments shows the better performance of the FLC with the DNA evolutionary algorithm optimization. The experimental results demonstrate the efficiency of the nrotmsed method.

A Survey of Model Predictive Control Methods for Traffic Signal Control

Enhancing traffic efficiency and alleviating(even circumventing) traffic congestion with advanced traffic signal control(TSC) strategies are always the main issues to be addressed in urban transportation systems. Since model predictive control(MPC) has a lot of advantages in modeling complex dynamic systems, it has been widely studied in traffic signal control over the past 20 years. There is a need for an in-depth understanding of MPC-based TSC methods for traffic networks. Therefore, this paper presents the motivation of using MPC for TSC and how MPC-based TSC approaches are implemented to manage and control the dynamics of traffic flows both in urban road networks and freeway networks. Meanwhile, typical performance evaluation metrics, solution methods, examples of simulations,and applications related to MPC-based TSC approaches are reported. More importantly, this paper summarizes the recent developments and the research trends in coordination and control of traffic networks with MPC-based TSC approaches. Remaining challenges and open issues are discussed towards the end of this paper to discover potential future research directions.

Tri-level programming model for combined urban traffic signal control and traffic flow guidance

In order to balance the temporal-spatial distribution of urban traffic flow, a model is established for combined urban traffic signal control and traffic flow guidance. With consideration of the wide use of fixed signal control at intersections, traffic assignment under traffic flow guidance, and dynamic characteristics of urban traffic management, a tri-level programming model is presented. To reflect the impact of intersection delay on traffic assignment, the lower level model is set as a modified user equilibrium model. The middle level model, which contains several definitional constraints for different phase modes, is built for the traffic signal control optimization. To solve the problem of tide lane management, the upper level model is built up based on nonlinear 0-1 integer programming. A heuristic iterative optimization algorithm(HIOA) is set up to solve the tri-level programming model. The lower level model is solved by method of successive averages(MSA), the middle level model is solved by non-dominated sorting genetic algorithm II(NSGA II), and the upper level model is solved by genetic algorithm(GA). A case study is raised to show the efficiency and applicability of the proposed modelling and computing method.

Traffic Signals Control with Adaptive Fuzzy Controller in Urban Road Network

An adaptive fuzzy logic controller (AFC) is presented for the signal control of the urban traffic network. The AFC is composed of the signal control system-oriented control level and the signal controller-oriented fuzzy rules regulation level. The control level decides the signal timings in an intersection with a fuzzy logic controller. The regulation level optimizes the fuzzy rules by the Adaptive Rule Module in AFC according to both the system performance index in current control period and the traffic flows in the last one. Consequently the system performances are improved. A weight coefficient controller (WCC) is also developed to describe the interactions of traffic flow among the adjacent intersections. So the AFC combined with the WCC can be applied in a road network for signal timings. Simulations of the AFC on a real traffic scenario have been conducted. Simulation results indicate that the adaptive controller for traffic control shows better performance than the actuated one.

Modeling and Comprehensive Review of Signaling Storms in 3GPP-Based Mobile Broadband Networks:Causes,Solutions,and Countermeasures

Control signaling is mandatory for the operation and management of all types of communication networks,including the Third Generation Partnership Project(3GPP)mobile broadband networks.However,they consume important and scarce network resources such as bandwidth and processing power.There have been several reports of these control signaling turning into signaling storms halting network operations and causing the respective Telecom companies big financial losses.This paper draws its motivation from such real network disaster incidents attributed to signaling storms.In this paper,we present a thorough survey of the causes,of the signaling storm problems in 3GPP-based mobile broadband networks and discuss in detail their possible solutions and countermeasures.We provide relevant analytical models to help quantify the effect of the potential causes and benefits of their corresponding solutions.Another important contribution of this paper is the comparison of the possible causes and solutions/countermeasures,concerning their effect on several important network aspects such as architecture,additional signaling,fidelity,etc.,in the form of a table.This paper presents an update and an extension of our earlier conference publication.To our knowledge,no similar survey study exists on the subject.

FL-FN-MOGA Based Traffic Signal Control

In this paper, a traffic signal control method based on fuzzy logic (FL), fuzzy-neuro (FN) and multi-objective genetic algorithms (MOGA) for an isolated four-approach intersection with through and left-turning movements is presented. This method has an adaptive signal timing ability, and can make adjustments to signal timing in response to observed changes.The 'urgency degree' term, which can describe the different user's demand for green time is used in decision-making by which strategy of signal timing can be determined. Using a fuzzy logic controller, we can determine whether to extend or terminate the current signal phase and select the sequences of phases. In this paper, a method based on fuzzy-neuro can be used to predict traffic parameters used in fuzzy logic controller. The feasibility of using a multi-objective genetic algorithm ( MOGA) to find a group of optimizing sets of parameters for fuzzy logic controller depending on different objects is also demonstrated. Simulation results show that the proposed methed is effecfive to adjust the signal timing in response to changing traffic conditions on a real-time basis, and the controller can produce lower vehicle delays and percentage of stopped vehicles than a traffic-actuated controller.

Adaptive signal control and coordination for urban traffic control in a connected vehicle environment: A review

Existing signal control systems for urban traffic are usually based on traffic flow data from fixed location detectors.Because of rapid advances in emerging vehicular communication,connected vehicle(CV)-based signal control demonstrates significant improvements over existing conventional signal control systems.Though various CV-based signal control systems have been investigated in the past decades,these approaches still have many issues and drawbacks to overcome.We summarize typical components and structures of these existing CV-based urban traffic signal control systems and digest several important issues from the summarized vital concepts.Last,future research directions are discussed with some suggestions.We hope this survey can facilitate the connected and automated vehicle and transportation research community to efficiently approach next-generation urban traffic signal control methods and systems.

An integrated and cooperative architecture for multi-intersection traffic signal control

Traffic signal control(TSC)systems are one essential component in intelligent transport systems.However,relevant studies are usually independent of the urban traffic simulation environment,collaborative TSC algorithms and traffic signal communication.In this paper,we propose(1)an integrated and cooperative Internet-of-Things architecture,namely General City Traffic Computing System(GCTCS),which simultaneously leverages an urban traffic simulation environment,TSC algorithms,and traffic signal communication;and(2)a general multi-agent reinforcement learning algorithm,namely General-MARL,considering cooperation and communication between traffic lights for multi-intersection TSC.In experiments,we demonstrate that the integrated and cooperative architecture of GCTCS is much closer to the real-life traffic environment.The General-MARL increases the average movement speed of vehicles in traffic by 23.2%while decreases the network latency by 11.7%.

Traffic Signal Timing via Deep Reinforcement Learning

In this paper, we propose a set of algorithms to design signal timing plans via deep reinforcement learning. The core idea of this approach is to set up a deep neural network (DNN) to learn the Q-function of reinforcement learning from the sampled traffic state/control inputs and the corresponding traffic system performance output. Based on the obtained DNN, we can find the appropriate signal timing policies by implicitly modeling the control actions and the change of system states. We explain the possible benefits and implementation tricks of this new approach. The relationships between this new approach and some existing approaches are also carefully discussed. © 2014 Chinese Association of Automation.

Dynamic signal control for at-grade intersections under preliminary autonomous vehicle environment

Autonomous vehicle technology will transform fundamentally urban traffic systems.To better enhance the coming era of connected and autonomous vehicles,effective control strategies that interact wisely with these intelligent vehicles for signalized at-grade intersections are indispensable.Vehicle-to-infrastructure communication technology offers unprecedented clues to reduce the delay at signalized intersections by innovative information-based control strategies.This paper proposes a new dynamic control strategy for signalized intersections with vehicle-to-signal information.The proposed strategy is called periodic vehicle holding(PVH)strategy while the traffic signal can provide information for the vehicles that are approaching an intersection.Under preliminary autonomous vehicle(PAV)environment,left-turning and through-moving vehicles will be sorted based on different information they receive.The paper shows how PVH reorganizes traffic to increase the capacity of an intersection without causing severe spillback to the upstream intersection.Results show that PVH can reduce the delay by approximately 15%at a signalized intersection under relatively high traffic demand.

Traffic jam in signalized road network

Traffic jam in large signalized road network presents a complex nature.In order to reveal the jam characteristics,two indexes,SVS(speed of virtual signal) and VOS(velocity of spillover),were proposed respectively.SVS described the propagation of queue within a link while VOS reflected the spillover velocity of vehicle queue.Based on the two indexes,network jam simulation was carried out on a regular signalized road network.The simulation results show that:1) The propagation of traffic congestion on a signalized road network can be classified into two stages:virtual split driven stage and flow rate driven stage.The former stage is characterized by decreasing virtual split while the latter only depends on flow rate; 2) The jam propagation rate and direction are dependent on traffic demand distribution and other network parameters.The direction with higher demand gets more chance to be jammed.Our findings can serve as the basis of the prevention of the formation and propagation of network traffic jam.

Evolution of Road Traffic Congestion Control:A Survey from Perspective of Sensing,Communication,and Computation

Road traffic congestion can inevitably de-grade road infrastructure and decrease travel efficiency in urban traffic networks,which can be relieved by employing appropriate congestion control.Accord-ing to different developmental driving forces,in this paper,the evolution of road traffic congestion control is divided into two stages.The ever-growing num-ber of advanced sensing techniques can be seen as the key driving force of the first stage,called the sens-ing stage,in which congestion control strategies ex-perienced rapid growth owing to the accessibility of traffic data.At the second stage,i.e.,the communica-tion stage,communication and computation capabil-ity can be regarded as the identifying symbols for this stage,where the ability of collecting finer-grained in-sight into transportation and mobility reality improves dramatically with advances in vehicular networks,Big Data,and artificial intelligence.Specifically,as the pre-requisite for congestion control,in this paper,ex-isting congestion detection techniques are first elab-orated and classified.Then,a comprehensive survey of the recent advances for current congestion control strategies with a focus on traffic signal control,vehi-cle route guidance,and their combined techniques is provided.In this regard,the evolution of these strate-gies with continuous development of sensing,com-munication,and computation capability are also intro-duced.Finally,the paper concludes with several re-search challenges and trends to fully promote the in-tegration of advanced techniques for traffic congestion mitigation in transportation systems.

A Model with Traffic Routers, Dynamically Managing Signal Phases to Address Traffic Congestion in Real Time

On-road Vehicular traffic congestion has detrimental effect on three lifelines: Economy, Productivity and Pollution (EPP). With ever increasing population of vehicles on road, traffic congestion is a major challenge to the economy, productivity and pollution, notwithstanding continuous developments in alternative fuels, alternative sources of energy. The research develops accurate and precise model in real time which computes congestion detection, dynamic signaling algorithm to evenly distribute vehicle densities while ensuring avoidance of starvation and deadlock situation. The model incorporates road segment length and breadth, quality and achievable average speed to compute road capacity. Vehicles installed with GPS enabled devices provide their location, which enables computing road occupancy. Road occupancy is evaluated based on number of vehicles as well as area occupied by vehicles. Ratio of road occupancy and road capacity provides congestion index important to compute signal phases. The algorithm ensures every direction is serviced once during a signaling cycle ensuring no starvation. Secondly, the definition of minimum and maximum signal timings ensures against dead lock situation. A simulator is developed to validate the proposition and proves it can ease congestion by more than 50% which is better than any of the contemporary approaches offering 15% improvement. In case of higher congestion index, alternate routes are suggested based on evaluation of traffic density graphs for shortest route or knowledge database. The algorithm to compute shortest route is optimized drastically, reducing computation cost to 3*√2N vis-à-vis computation cost of N2 by classical algorithms. The proposal brings down the cost of implementation per traffic junction from USD 30,000 to USD 2000.

Delay Analysis and Formulation Inference of Signalized Intersection for Traffic Congestion Conditions

Vehicle delay is an important measure to evaluate the signal timings of signalized intersections.When optimization the signal control parameters, delays of vehicles from all approach directions of an intersection should be considered. Based on the analysis of the vehicle delay on an approach of intersection, directed against the typical condition of a congested intersection-over-saturated condition, the paper has analyzed and inferred the intersection delay dynamic formulation, and has established the relation between intersection delay,the signal timings, vehicle arrival rate and the queue lengths, and that provides useful information for understanding vehicle delay of signalized intersection and for establishing performance index function of signal timing optimization.

Real-Time Traffic Signal Timing for Urban Road Multi-Intersection

this paper develops a real-time traffic signal timing model which is to be integrated into a single intersection for urban road, thereby solving the problem of traffic congestion. We analyze the current situation of the traffic flow with release matrix firstly, and then put forward the basic models to minimize total delay time of vehicles at the intersection. The optimal real-time signal timing model (non-fixed cycle and non-fixed split) is built with the Webster split optimal model. At last, the simulated results, which are compared with conventional model, manifest the promising properties of proposed model.

Evolvement regularity of signal control parameters of urban road in cold region

In snow-icy road environment, the survey data indicate that the largest decrease in traffic flow running characters occurs when snow and ice begin to accumulate on the road surface. Saturation flow is decreased by 16% , speed is decreased by 30% , and start-up lost time is increased by 27%. Based on the signal control theory of HCM and Webster, the character values of traffic flow in different urban road environments were investigated, and the evolvement regularity of signal control parameters such as cycle, split, green time, offset, yellow time and red time in snow-icy road environment was analyzed. The impact factors and the changes in the scope of signal control parameters were achieved. Simulation results and practical application show that the signal control plan of road enviromnent without snow and ice will increase the vehicle delay, stop length and traffic congestion in snow-icy road environment. Thus, the traffic signal control system should address a suitable signal control plan based on different road environments.

An ATMS data-driven method for signalized arterial coordination

In order to minimize the delays and stops caused by the early started coordinated green phase of the vehicle- actuated signal systems, a stochastic offsets calculation method based on the new types of advanced traffic management system （ATMS） data is proposed. As the mainline green starts randomly in vehicle-actuated signal systems, the random theory is applied to obtain the distribution of the unused green time at side streets based on the green gap-out mechanism. Then, the green start time of the mainline can be selected at the point with maximum probability to minimize the delays or stops caused by the randomly started mainline green. A case study in Maine, USA, whose traffic conditions are similar to those of the middle-size Chinese cities, proves that the proposed method can significantly reduce the travel time and delays.