Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control

This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system.A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency.Firstly a regional multi-agent Q-learning framework is proposed,which can equivalently decompose the global Q value of the traffic system into the local values of several regions Based on the framework and the idea of human-machine cooperation,a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to realtime traffic flow densities.In order to achieve better cooperation inside each region,a lightweight spatio-temporal fusion feature extraction network is designed.The experiments in synthetic real-world and city-level scenarios show that the proposed RegionS TLight converges more quickly,is more stable,and obtains better asymptotic performance compared to state-of-theart models.

Adaptive signal control and coordination for urban traffic control in a connected vehicle environment: A review

Existing signal control systems for urban traffic are usually based on traffic flow data from fixed location detectors.Because of rapid advances in emerging vehicular communication,connected vehicle(CV)-based signal control demonstrates significant improvements over existing conventional signal control systems.Though various CV-based signal control systems have been investigated in the past decades,these approaches still have many issues and drawbacks to overcome.We summarize typical components and structures of these existing CV-based urban traffic signal control systems and digest several important issues from the summarized vital concepts.Last,future research directions are discussed with some suggestions.We hope this survey can facilitate the connected and automated vehicle and transportation research community to efficiently approach next-generation urban traffic signal control methods and systems.

An integrated and cooperative architecture for multi-intersection traffic signal control

Traffic signal control(TSC)systems are one essential component in intelligent transport systems.However,relevant studies are usually independent of the urban traffic simulation environment,collaborative TSC algorithms and traffic signal communication.In this paper,we propose(1)an integrated and cooperative Internet-of-Things architecture,namely General City Traffic Computing System(GCTCS),which simultaneously leverages an urban traffic simulation environment,TSC algorithms,and traffic signal communication;and(2)a general multi-agent reinforcement learning algorithm,namely General-MARL,considering cooperation and communication between traffic lights for multi-intersection TSC.In experiments,we demonstrate that the integrated and cooperative architecture of GCTCS is much closer to the real-life traffic environment.The General-MARL increases the average movement speed of vehicles in traffic by 23.2%while decreases the network latency by 11.7%.

NSGA-Ⅱ based traffic signal control optimization algorithm for over-saturated intersection group

In order to improve the efficiency of traffic signal control for an over-saturated intersection group, a nondominated sorting genetic algorithm Ⅱ（NSGA-Ⅱ） based traffic signal control optimization algorithm is proposed. The throughput maximum and average queue ratio minimum for the critical route of the intersection group are selected as the optimization objectives of the traffic signal control for the over-saturated condition. The consequences of the efficiency between traffic signal timing plans generated by the proposed algorithm and a commonly utilized signal timing optimization software Synchro are compared in a VISSIM signal control application programming interfaces （SCAPI） simulation environment by using real filed observed traffic data. The simulation results indicate that the signal timing plan generated by the proposed algorithm is more efficient in managing oversaturated flows at intersection groups, and, thus, it has the capability of optimizing signal timing under the over-saturated conditions.

Travel time function for basic link considering signal control in network traffic model

In order to describe the travel time of signalcontrolled roads, a travel time model for urban basic roads based on the cumulative curve is proposed. First, the traffic wave method is used to analyze the formation and dispersion of the vehicle queue. Cumulative curves for road entrances and exits are established. Based on the cumulative curves, the travel time of the one-lane road under stable flow input is derived. And then, the multi-lane road is decomposed into a series of single-lane links based on its topological characteristics. Hence, the travel time function for the basic road is obtained. The travel time is a function of road length, flow and control parameters. Numerical analyses show that the travel time depends on the supply-demand condition, and it has high sensitivity during peak hours.

An ATMS data-driven method for signalized arterial coordination

In order to minimize the delays and stops caused by the early started coordinated green phase of the vehicle- actuated signal systems, a stochastic offsets calculation method based on the new types of advanced traffic management system （ATMS） data is proposed. As the mainline green starts randomly in vehicle-actuated signal systems, the random theory is applied to obtain the distribution of the unused green time at side streets based on the green gap-out mechanism. Then, the green start time of the mainline can be selected at the point with maximum probability to minimize the delays or stops caused by the randomly started mainline green. A case study in Maine, USA, whose traffic conditions are similar to those of the middle-size Chinese cities, proves that the proposed method can significantly reduce the travel time and delays.

Fuzzy traffic signal control with DNA evolutionary algorithm

In order to optimize the signal control system, this paper proposes a method to design an optimized fuzzy logic controller （FLC） with the DNA evolutionary algorithm. Inspired by the DNA molecular operation characteristics, the DNA evolutionary algorithm modifies the corresponding genetic operators. Compared with the traditional genetic algorithm （GA）, the DNA evolutionary algorithm can overcome weak local search capability and premature convergence. The parameters of membership functions are optimized by adopting the quaternary encoding method and performing corresponding DNA genetic operators. The relevant optimized parameters are combined with the FLC for single intersection traffic signal control. Simulation experiments shows the better performance of the FLC with the DNA evolutionary algorithm optimization. The experimental results demonstrate the efficiency of the nrotmsed method.

Notch signaling controls chondrocyte hypertrophy via indirect regulation of Sox9

RBPjk-dependent Notch signaling regulates both the onset of chondrocyte hypertrophy and the progression to terminal chondrocyte maturation during endochondral ossification. It has been suggested that Notch signaling can regulate Sox9 transcription, although how this occurs at the molecular level in chondrocytes and whether this transcriptional regulation mediates Notch control of chondrocyte hypertrophy and cartilage development is unknown or controversial. Here we have provided conclusive genetic evidence linking RBPjk-dependent Notch signaling to the regulation of Sox9 expression and chondrocyte hypertrophy by examining tissuespecific Rbpjk mutant（Prx1Cre;Rbpjkf/f）, Rbpjk mutant/Sox9 haploinsufficient（Prx1Cre;Rbpjkf/f;Sox9f/1）,and control embryos for alterations in SOX9 expression and chondrocyte hypertrophy during cartilage development. These studies demonstrate that Notch signaling regulates the onset of chondrocyte maturation in a SOX9-dependent manner, while Notch-mediated regulation of terminal chondrocyte maturation likely functions independently of SOX9. Furthermore, our in vitro molecular analyses of the Sox9 promoter and Notch-mediated regulation of Sox9 gene expression in chondrogenic cells identified the ability of Notch to induce Sox9 expression directly in the acute setting, but suppresses Sox9 transcription with prolonged Notch signaling that requires protein synthesis of secondary effectors.

Vehicle actuation based short-term traffic flow prediction model for signalized intersections

Traffic flow prediction is an important component for real-time traffic-adaptive signal control in urban arterial networks.By exploring available detector and signal controller information from neighboring intersections,a dynamic data-driven flow prediction model was developed.The model consists of two prediction components based on the signal states(red or green) for each movement at an upstream intersection.The characteristics of each signal state were carefully examined and the corresponding travel time from the upstream intersection to the approach in question at the downstream intersection was predicted.With an online turning proportion estimation method,along with the predicted travel times,the anticipated vehicle arrivals can be forecasted at the downstream intersection.The model performance was tested at a set of two signalized intersections located in the city of Gainesville,Florida,USA,using the CORSIM microscopic simulation package.Analysis results show that the model agrees well with empirical arrival data measured at 10 s intervals within an acceptable range of 10%-20%,and show a normal distribution.It is reasonably believed that the model has potential applicability for use in truly proactive real-time traffic adaptive signal control systems.

Gα_s signaling controls intramembranous ossification during cranial bone development by regulating both Hedgehog and Wnt/β-catenin signaling

How osteoblast cells are induced is a central question for understanding skeletal formation. Abnormal osteoblast differentiation leads to a broad range of devastating craniofacial diseases. Here we have investigated intramembranous ossification during cranial bone development in mouse models of skeletal genetic diseases that exhibit craniofacial bone defects. The GNAS gene encodes Gαs that transduces GPCR signaling. GNAS activation or loss-of-function mutations in humans cause fibrous dysplasia(FD) or progressive osseous heteroplasia(POH) that shows craniofacial hyperostosis or craniosynostosis, respectively. We find here that, while Hh ligand-dependent Hh signaling is essential for endochondral ossification, it is dispensable for intramembranous ossification, where Gαsregulates Hh signaling in a ligand-independent manner. We further show that Gαscontrols intramembranous ossification by regulating both Hh and Wnt/β-catenin signaling. In addition, Gαsactivation in the developing cranial bone leads to reduced ossification but increased cartilage presence due to reduced cartilage dissolution, not cell fate switch. Small molecule inhibitors of Hh and Wnt signaling can effectively ameliorate cranial bone phenotypes in mice caused by loss or gain of Gnas function mutations, respectively. Our work shows that studies of genetic diseases provide invaluable insights in both pathological bone defects and normal bone development, understanding both leads to better diagnosis and therapeutic treatment of bone diseases.

Tri-level programming model for combined urban traffic signal control and traffic flow guidance

In order to balance the temporal-spatial distribution of urban traffic flow, a model is established for combined urban traffic signal control and traffic flow guidance. With consideration of the wide use of fixed signal control at intersections, traffic assignment under traffic flow guidance, and dynamic characteristics of urban traffic management, a tri-level programming model is presented. To reflect the impact of intersection delay on traffic assignment, the lower level model is set as a modified user equilibrium model. The middle level model, which contains several definitional constraints for different phase modes, is built for the traffic signal control optimization. To solve the problem of tide lane management, the upper level model is built up based on nonlinear 0-1 integer programming. A heuristic iterative optimization algorithm(HIOA) is set up to solve the tri-level programming model. The lower level model is solved by method of successive averages(MSA), the middle level model is solved by non-dominated sorting genetic algorithm II(NSGA II), and the upper level model is solved by genetic algorithm(GA). A case study is raised to show the efficiency and applicability of the proposed modelling and computing method.

Traffic Signals Control with Adaptive Fuzzy Controller in Urban Road Network

An adaptive fuzzy logic controller (AFC) is presented for the signal control of the urban traffic network. The AFC is composed of the signal control system-oriented control level and the signal controller-oriented fuzzy rules regulation level. The control level decides the signal timings in an intersection with a fuzzy logic controller. The regulation level optimizes the fuzzy rules by the Adaptive Rule Module in AFC according to both the system performance index in current control period and the traffic flows in the last one. Consequently the system performances are improved. A weight coefficient controller (WCC) is also developed to describe the interactions of traffic flow among the adjacent intersections. So the AFC combined with the WCC can be applied in a road network for signal timings. Simulations of the AFC on a real traffic scenario have been conducted. Simulation results indicate that the adaptive controller for traffic control shows better performance than the actuated one.

FL-FN-MOGA Based Traffic Signal Control

In this paper, a traffic signal control method based on fuzzy logic (FL), fuzzy-neuro (FN) and multi-objective genetic algorithms (MOGA) for an isolated four-approach intersection with through and left-turning movements is presented. This method has an adaptive signal timing ability, and can make adjustments to signal timing in response to observed changes.The 'urgency degree' term, which can describe the different user's demand for green time is used in decision-making by which strategy of signal timing can be determined. Using a fuzzy logic controller, we can determine whether to extend or terminate the current signal phase and select the sequences of phases. In this paper, a method based on fuzzy-neuro can be used to predict traffic parameters used in fuzzy logic controller. The feasibility of using a multi-objective genetic algorithm ( MOGA) to find a group of optimizing sets of parameters for fuzzy logic controller depending on different objects is also demonstrated. Simulation results show that the proposed methed is effecfive to adjust the signal timing in response to changing traffic conditions on a real-time basis, and the controller can produce lower vehicle delays and percentage of stopped vehicles than a traffic-actuated controller.

Evolvement regularity of signal control parameters of urban road in cold region

In snow-icy road environment, the survey data indicate that the largest decrease in traffic flow running characters occurs when snow and ice begin to accumulate on the road surface. Saturation flow is decreased by 16% , speed is decreased by 30% , and start-up lost time is increased by 27%. Based on the signal control theory of HCM and Webster, the character values of traffic flow in different urban road environments were investigated, and the evolvement regularity of signal control parameters such as cycle, split, green time, offset, yellow time and red time in snow-icy road environment was analyzed. The impact factors and the changes in the scope of signal control parameters were achieved. Simulation results and practical application show that the signal control plan of road enviromnent without snow and ice will increase the vehicle delay, stop length and traffic congestion in snow-icy road environment. Thus, the traffic signal control system should address a suitable signal control plan based on different road environments.

Evolution of Road Traffic Congestion Control:A Survey from Perspective of Sensing,Communication,and Computation

Road traffic congestion can inevitably de-grade road infrastructure and decrease travel efficiency in urban traffic networks,which can be relieved by employing appropriate congestion control.Accord-ing to different developmental driving forces,in this paper,the evolution of road traffic congestion control is divided into two stages.The ever-growing num-ber of advanced sensing techniques can be seen as the key driving force of the first stage,called the sens-ing stage,in which congestion control strategies ex-perienced rapid growth owing to the accessibility of traffic data.At the second stage,i.e.,the communica-tion stage,communication and computation capabil-ity can be regarded as the identifying symbols for this stage,where the ability of collecting finer-grained in-sight into transportation and mobility reality improves dramatically with advances in vehicular networks,Big Data,and artificial intelligence.Specifically,as the pre-requisite for congestion control,in this paper,ex-isting congestion detection techniques are first elab-orated and classified.Then,a comprehensive survey of the recent advances for current congestion control strategies with a focus on traffic signal control,vehi-cle route guidance,and their combined techniques is provided.In this regard,the evolution of these strate-gies with continuous development of sensing,com-munication,and computation capability are also intro-duced.Finally,the paper concludes with several re-search challenges and trends to fully promote the in-tegration of advanced techniques for traffic congestion mitigation in transportation systems.

Bus Priority Control for Dynamic Exclusive Bus Lane

One problem with the existing dynamic exclusive bus lane strategies is that bus signal priority strategies with multi-phase priority request at intersections are not adequately considered.The principle of bus signal priority level was designed based on the isolated multi-phase structure principle consideration of the bus signal priority,and a new priority approach for the dynamic exclusive bus lane was proposed.Two types of priority strategies,green extension and red truncation,were proposed for current phase and next phase buses,respectively.The control parameters including minimum green time,green extension time,maximum green time and bus arrival time are calculated.The case studies for this paper were carried out using four consecutive intersections of Huaide Middle Road in Changzhou City.The signal control scheme was designed using the conventional,exclusive bus lane method,the dynamic exclusive bus lane without signal priority method,and the proposed approach,respectively.The authors used the VISSIM simulation platform to evaluate the efficiency of each approach.Results showed that the method of approach can significantly decrease delays caused by social and conventional buses and make up for the negative impact social buses have on the bus rapid transit(BRT)operation,which allows the method to complement the dynamic,exclusive bus lane design.

Control strategies and hardware-in-the-loop simulation for bus signal priority

It is considered as an important and effective means to give priority to the development of public transit which can improve the efficiency of transportation resources utilization and alleviate traffic jams. Public transit signal priority belongs to the "time priority" among the right-of-way priorities. After reviewing the existing bus priority signal control strategies and the advances in related technologies at home and abroad, this article analyzed the breakthrough direction of the bus signal priority design technologies suitable for China's conditions, and then proposed the hardware and software systems and the modules for the bus priority signal control system. Finally, the hardware-in-the-loop simulation (HILS) was introduced to evaluate bus priority signal control programs in order to optimize the control strategies.

A Real-Time Power Controller for Grid-Connected Inverters in LV Smart Microgrids

This paper presents a real-time power flow controller for VSIs （voltage source inverters） interfaced to low voltage microgrids. The proposed controller is modular, flexible, intelligent, inexpensive, portable, adaptive and designed to positively contribute in low voltage microgrids in which the lines R/X ratio is greater than the transmission lines. Therefore, the proposed control strategy is developed for operation in distribution lines. The controller strategy is different from the conventional grid-connected inverters which are designed based on transmission line characteristics. This controller, using a Texas Instrument general purpose DSP （digital signal processor）, is programmed and tuned using MATLAB/SIMULINK in order to enhance self-healing, reliability and stability of the grid. This general purpose controller makes proper decisions using its local measurements as the primary source of data. The controller has the capability of communicating with the adjacent controllers and sharing the information if/when needed. The power flow output of the inverter is tested for both islanded and grid-connected modes of operation. The inverter positively contributes to active and reactive power supply while operating in grid-connected mode. The proposed control method has been implemented on a Texas Instrument DSC （digital signal controller） chip and tested on a hardware test bench at the Alternative Energy Laboratory at WVU1T （West Virginia University Institute of Technology）. The system＇s experimental results veri~ the validity and efficiency of the proposed controller.

Direct Digital Frequency Control Based on the Phase Step Change Characteristic between Signals

We present a new digital phase lock technology to achieve the frequency control and transformation through high precision multi-cycle group synchronization between signals without the frequency transformation circuit. In the case of digital sampling, the passing zero point of the phase of the controlled signal has the phase step characteristic, the phase step of the passing zero point is monotonic continuous with high resolution in the phase lock process, and using the border effect of digital fuzzy area, the gate can synchronize with the two signals, the quantization error is reduced. This technique is quite different from the existing methods of frequency transformation and frequency synthesis, the phase change characteristic between the periodic signals with different nominal is used. The phase change has the periodic phenomenon, and it has the high resolution step value. With the application of the physical law, the noise is reduced because of simplifying frequency transformation circuits, and the phase is locked with high precision. The regular phase change between frequency signals is only used for frequency measurement, and the change has evident randomness, but this randomness is greatly reduced in frequency control, and the certainty of the process result is clear. The experiment shows that the short term frequency stability can reach 10-12/s orders of magnitude.

DEVELOPMENT AND USAGE OF SIGNAL PR OCESSING AND SOUND SITIMULATING SYSTEM CONTROLLED BY COMPUTER

In hearing physiological experiments and clinic tests,we need not only a signal processing system,but also a synchronous sound stimulator’ Most of stimulators we are now using are function generators which are independent to processing units,and can be controlled only by hand. Although some of them have ports through which they can be controlled by computer,but as they are designed for industrial aims,not for hearing research,most of them can’t generate the special waveforms we need. We use the TDT signal processing system and develop a software package have both usage. On the interface of the program we can control the sampling parameters and generate stimulating waveforms’