Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control

This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system.A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency.Firstly a regional multi-agent Q-learning framework is proposed,which can equivalently decompose the global Q value of the traffic system into the local values of several regions Based on the framework and the idea of human-machine cooperation,a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to realtime traffic flow densities.In order to achieve better cooperation inside each region,a lightweight spatio-temporal fusion feature extraction network is designed.The experiments in synthetic real-world and city-level scenarios show that the proposed RegionS TLight converges more quickly,is more stable,and obtains better asymptotic performance compared to state-of-theart models.

Adaptive signal control and coordination for urban traffic control in a connected vehicle environment: A review

Existing signal control systems for urban traffic are usually based on traffic flow data from fixed location detectors.Because of rapid advances in emerging vehicular communication,connected vehicle(CV)-based signal control demonstrates significant improvements over existing conventional signal control systems.Though various CV-based signal control systems have been investigated in the past decades,these approaches still have many issues and drawbacks to overcome.We summarize typical components and structures of these existing CV-based urban traffic signal control systems and digest several important issues from the summarized vital concepts.Last,future research directions are discussed with some suggestions.We hope this survey can facilitate the connected and automated vehicle and transportation research community to efficiently approach next-generation urban traffic signal control methods and systems.

An integrated and cooperative architecture for multi-intersection traffic signal control

Traffic signal control(TSC)systems are one essential component in intelligent transport systems.However,relevant studies are usually independent of the urban traffic simulation environment,collaborative TSC algorithms and traffic signal communication.In this paper,we propose(1)an integrated and cooperative Internet-of-Things architecture,namely General City Traffic Computing System(GCTCS),which simultaneously leverages an urban traffic simulation environment,TSC algorithms,and traffic signal communication;and(2)a general multi-agent reinforcement learning algorithm,namely General-MARL,considering cooperation and communication between traffic lights for multi-intersection TSC.In experiments,we demonstrate that the integrated and cooperative architecture of GCTCS is much closer to the real-life traffic environment.The General-MARL increases the average movement speed of vehicles in traffic by 23.2%while decreases the network latency by 11.7%.

NSGA-Ⅱ based traffic signal control optimization algorithm for over-saturated intersection group

In order to improve the efficiency of traffic signal control for an over-saturated intersection group, a nondominated sorting genetic algorithm Ⅱ（NSGA-Ⅱ） based traffic signal control optimization algorithm is proposed. The throughput maximum and average queue ratio minimum for the critical route of the intersection group are selected as the optimization objectives of the traffic signal control for the over-saturated condition. The consequences of the efficiency between traffic signal timing plans generated by the proposed algorithm and a commonly utilized signal timing optimization software Synchro are compared in a VISSIM signal control application programming interfaces （SCAPI） simulation environment by using real filed observed traffic data. The simulation results indicate that the signal timing plan generated by the proposed algorithm is more efficient in managing oversaturated flows at intersection groups, and, thus, it has the capability of optimizing signal timing under the over-saturated conditions.

Travel time function for basic link considering signal control in network traffic model

In order to describe the travel time of signalcontrolled roads, a travel time model for urban basic roads based on the cumulative curve is proposed. First, the traffic wave method is used to analyze the formation and dispersion of the vehicle queue. Cumulative curves for road entrances and exits are established. Based on the cumulative curves, the travel time of the one-lane road under stable flow input is derived. And then, the multi-lane road is decomposed into a series of single-lane links based on its topological characteristics. Hence, the travel time function for the basic road is obtained. The travel time is a function of road length, flow and control parameters. Numerical analyses show that the travel time depends on the supply-demand condition, and it has high sensitivity during peak hours.

Design and Simulation of an Audio Signal Alerting and Automatic Control System

A large part of our daily lives is spent with audio information. Massive obstacles are frequently presented by the colossal amounts of acoustic information and the incredibly quick processing times. This results in the need for applications and methodologies that are capable of automatically analyzing these contents. These technologies can be applied in automatic contentanalysis and emergency response systems. Breaks in manual communication usually occur in emergencies leading to accidents and equipment damage. The audio signal does a good job by sending a signal underground, which warrants action from an emergency management team at the surface. This paper, therefore, seeks to design and simulate an audio signal alerting and automatic control system using Unity Pro XL to substitute manual communication of emergencies and manual control of equipment. Sound data were trained using the neural network technique of machine learning. The metrics used are Fast Fourier transform magnitude, zero crossing rate, root mean square, and percentage error. Sounds were detected with an error of approximately 17%;thus, the system can detect sounds with an accuracy of 83%. With more data training, the system can detect sounds with minimal or no error. The paper, therefore, has critical policy implications about communication, safety, and health for underground mine.

Fuzzy traffic signal control with DNA evolutionary algorithm

In order to optimize the signal control system, this paper proposes a method to design an optimized fuzzy logic controller （FLC） with the DNA evolutionary algorithm. Inspired by the DNA molecular operation characteristics, the DNA evolutionary algorithm modifies the corresponding genetic operators. Compared with the traditional genetic algorithm （GA）, the DNA evolutionary algorithm can overcome weak local search capability and premature convergence. The parameters of membership functions are optimized by adopting the quaternary encoding method and performing corresponding DNA genetic operators. The relevant optimized parameters are combined with the FLC for single intersection traffic signal control. Simulation experiments shows the better performance of the FLC with the DNA evolutionary algorithm optimization. The experimental results demonstrate the efficiency of the nrotmsed method.

Notch signaling controls chondrocyte hypertrophy via indirect regulation of Sox9

RBPjk-dependent Notch signaling regulates both the onset of chondrocyte hypertrophy and the progression to terminal chondrocyte maturation during endochondral ossification. It has been suggested that Notch signaling can regulate Sox9 transcription, although how this occurs at the molecular level in chondrocytes and whether this transcriptional regulation mediates Notch control of chondrocyte hypertrophy and cartilage development is unknown or controversial. Here we have provided conclusive genetic evidence linking RBPjk-dependent Notch signaling to the regulation of Sox9 expression and chondrocyte hypertrophy by examining tissuespecific Rbpjk mutant（Prx1Cre;Rbpjkf/f）, Rbpjk mutant/Sox9 haploinsufficient（Prx1Cre;Rbpjkf/f;Sox9f/1）,and control embryos for alterations in SOX9 expression and chondrocyte hypertrophy during cartilage development. These studies demonstrate that Notch signaling regulates the onset of chondrocyte maturation in a SOX9-dependent manner, while Notch-mediated regulation of terminal chondrocyte maturation likely functions independently of SOX9. Furthermore, our in vitro molecular analyses of the Sox9 promoter and Notch-mediated regulation of Sox9 gene expression in chondrogenic cells identified the ability of Notch to induce Sox9 expression directly in the acute setting, but suppresses Sox9 transcription with prolonged Notch signaling that requires protein synthesis of secondary effectors.

Gα_s signaling controls intramembranous ossification during cranial bone development by regulating both Hedgehog and Wnt/β-catenin signaling

How osteoblast cells are induced is a central question for understanding skeletal formation. Abnormal osteoblast differentiation leads to a broad range of devastating craniofacial diseases. Here we have investigated intramembranous ossification during cranial bone development in mouse models of skeletal genetic diseases that exhibit craniofacial bone defects. The GNAS gene encodes Gαs that transduces GPCR signaling. GNAS activation or loss-of-function mutations in humans cause fibrous dysplasia(FD) or progressive osseous heteroplasia(POH) that shows craniofacial hyperostosis or craniosynostosis, respectively. We find here that, while Hh ligand-dependent Hh signaling is essential for endochondral ossification, it is dispensable for intramembranous ossification, where Gαsregulates Hh signaling in a ligand-independent manner. We further show that Gαscontrols intramembranous ossification by regulating both Hh and Wnt/β-catenin signaling. In addition, Gαsactivation in the developing cranial bone leads to reduced ossification but increased cartilage presence due to reduced cartilage dissolution, not cell fate switch. Small molecule inhibitors of Hh and Wnt signaling can effectively ameliorate cranial bone phenotypes in mice caused by loss or gain of Gnas function mutations, respectively. Our work shows that studies of genetic diseases provide invaluable insights in both pathological bone defects and normal bone development, understanding both leads to better diagnosis and therapeutic treatment of bone diseases.

Tri-level programming model for combined urban traffic signal control and traffic flow guidance

In order to balance the temporal-spatial distribution of urban traffic flow, a model is established for combined urban traffic signal control and traffic flow guidance. With consideration of the wide use of fixed signal control at intersections, traffic assignment under traffic flow guidance, and dynamic characteristics of urban traffic management, a tri-level programming model is presented. To reflect the impact of intersection delay on traffic assignment, the lower level model is set as a modified user equilibrium model. The middle level model, which contains several definitional constraints for different phase modes, is built for the traffic signal control optimization. To solve the problem of tide lane management, the upper level model is built up based on nonlinear 0-1 integer programming. A heuristic iterative optimization algorithm(HIOA) is set up to solve the tri-level programming model. The lower level model is solved by method of successive averages(MSA), the middle level model is solved by non-dominated sorting genetic algorithm II(NSGA II), and the upper level model is solved by genetic algorithm(GA). A case study is raised to show the efficiency and applicability of the proposed modelling and computing method.

Traffic Signals Control with Adaptive Fuzzy Controller in Urban Road Network

An adaptive fuzzy logic controller (AFC) is presented for the signal control of the urban traffic network. The AFC is composed of the signal control system-oriented control level and the signal controller-oriented fuzzy rules regulation level. The control level decides the signal timings in an intersection with a fuzzy logic controller. The regulation level optimizes the fuzzy rules by the Adaptive Rule Module in AFC according to both the system performance index in current control period and the traffic flows in the last one. Consequently the system performances are improved. A weight coefficient controller (WCC) is also developed to describe the interactions of traffic flow among the adjacent intersections. So the AFC combined with the WCC can be applied in a road network for signal timings. Simulations of the AFC on a real traffic scenario have been conducted. Simulation results indicate that the adaptive controller for traffic control shows better performance than the actuated one.

A Survey of Model Predictive Control Methods for Traffic Signal Control

Enhancing traffic efficiency and alleviating(even circumventing) traffic congestion with advanced traffic signal control(TSC) strategies are always the main issues to be addressed in urban transportation systems. Since model predictive control(MPC) has a lot of advantages in modeling complex dynamic systems, it has been widely studied in traffic signal control over the past 20 years. There is a need for an in-depth understanding of MPC-based TSC methods for traffic networks. Therefore, this paper presents the motivation of using MPC for TSC and how MPC-based TSC approaches are implemented to manage and control the dynamics of traffic flows both in urban road networks and freeway networks. Meanwhile, typical performance evaluation metrics, solution methods, examples of simulations,and applications related to MPC-based TSC approaches are reported. More importantly, this paper summarizes the recent developments and the research trends in coordination and control of traffic networks with MPC-based TSC approaches. Remaining challenges and open issues are discussed towards the end of this paper to discover potential future research directions.

A new approach to detecting weak signal in strong noise based on chaos system control

In this paper, a chaos system and proportional differential control are both used to detect the frequency of an unknown signal. In traditional methods the useful signal is obtained through the Duffing equation or other chaotic oscillators. But these methods are too complex because of using a lot of chaos oscillators. In this paper a new method is presented that uses the Rossler equation and proportional differential control to detect a weak signal frequency. Substituting the detected signal frequency into the RSssler equation leads the Rossler phase state to be considerably changed. The chaos state can be controlled through the proportional differential method. Through its phase diagram and spectrum analysis, the unknown frequency is obtained. The simulation results verify that the presented method is feasible and that the detection accuracy is higher than those of other methods.

Multi-Variable Flocking Control for Multi-Agent Systems via a Received Signal Strength Indicator

A novel flocking control approach is proposed for multi-agent systems by integrating the variables of velocities, motion directions, and positions of agents. A received signal strength indicator （RSSI） is applied as a variable to estimate the inter-distance between agents. A key parameter that contains the local information of agents is defined, and a multi-variable controller is proposed based on the parameter. For the position control of agents, the RSSI is introduced to substitute the distance as a control variable in the systems. The advantages of RSSI include that the relative distance between every two agents can be adjusted through the communication quality under different environments, and it can shun the shortage of the limit of sensors. Simulation studies demonstrate the effectiveness of the proposed control approach.

FL-FN-MOGA Based Traffic Signal Control

In this paper, a traffic signal control method based on fuzzy logic (FL), fuzzy-neuro (FN) and multi-objective genetic algorithms (MOGA) for an isolated four-approach intersection with through and left-turning movements is presented. This method has an adaptive signal timing ability, and can make adjustments to signal timing in response to observed changes.The 'urgency degree' term, which can describe the different user's demand for green time is used in decision-making by which strategy of signal timing can be determined. Using a fuzzy logic controller, we can determine whether to extend or terminate the current signal phase and select the sequences of phases. In this paper, a method based on fuzzy-neuro can be used to predict traffic parameters used in fuzzy logic controller. The feasibility of using a multi-objective genetic algorithm ( MOGA) to find a group of optimizing sets of parameters for fuzzy logic controller depending on different objects is also demonstrated. Simulation results show that the proposed methed is effecfive to adjust the signal timing in response to changing traffic conditions on a real-time basis, and the controller can produce lower vehicle delays and percentage of stopped vehicles than a traffic-actuated controller.

Evolvement regularity of signal control parameters of urban road in cold region

In snow-icy road environment, the survey data indicate that the largest decrease in traffic flow running characters occurs when snow and ice begin to accumulate on the road surface. Saturation flow is decreased by 16% , speed is decreased by 30% , and start-up lost time is increased by 27%. Based on the signal control theory of HCM and Webster, the character values of traffic flow in different urban road environments were investigated, and the evolvement regularity of signal control parameters such as cycle, split, green time, offset, yellow time and red time in snow-icy road environment was analyzed. The impact factors and the changes in the scope of signal control parameters were achieved. Simulation results and practical application show that the signal control plan of road enviromnent without snow and ice will increase the vehicle delay, stop length and traffic congestion in snow-icy road environment. Thus, the traffic signal control system should address a suitable signal control plan based on different road environments.

Control strategies and hardware-in-the-loop simulation for bus signal priority

It is considered as an important and effective means to give priority to the development of public transit which can improve the efficiency of transportation resources utilization and alleviate traffic jams. Public transit signal priority belongs to the "time priority" among the right-of-way priorities. After reviewing the existing bus priority signal control strategies and the advances in related technologies at home and abroad, this article analyzed the breakthrough direction of the bus signal priority design technologies suitable for China's conditions, and then proposed the hardware and software systems and the modules for the bus priority signal control system. Finally, the hardware-in-the-loop simulation (HILS) was introduced to evaluate bus priority signal control programs in order to optimize the control strategies.

Signal averaging for noise reduction in anesthesia monitoring and control with communication channels

This paper investigates impact of noise and signal averaging on patient control in anesthesia applications, especially in networked control system settings such as wireless connected systems, sensor networks, local area networks, or tele-medicine over a wide area network. Such systems involve communication channels which introduce noises due to quantization, channel noises, and have limited communication bandwidth resources. Usually signal averaging can be used effectively in reducing noise effects when remote monitoring and diagnosis are involved. However, when feedback is intended, we show that signal averaging will lose its utility substantially. To explain this phenomenon, we analyze stability margins under signal averaging and derive some optimal strategies for selecting window sizes. A typical case of anesthe-sia depth control problems is used in this development.

INTELLIGENT CONTROL SYSTEM OF PULSED MAG WELDING INVERTER BASED ON DIGITAL SIGNAL PROCESSOR

A fuzzy logic intelligent control system of pulsed MAG welding inverter based on digital signal processor （DSP） is proposed to obtain the consistency of arc length in pulsed MAG welding. The proposed control system combines the merits of intelligent control with DSP digital control. The fuzzy logic intelligent control system designed is a typical two-input-single-output structure, and regards the error and the change in error of peak arc voltage as two inputs and the background time as single output. The fuzzy logic intelligent control system is realized in a look-up table （LUT） method by using MATLAB based fuzzy logic toolbox, and the implement of LUT method based on DSP is also discussed. The pulsed MAG welding experimental results demonstrate that the developed fuzzy logic intelligent control system based on DSP has strong arc length controlling ability to accomplish the stable pulsed MAG welding process and controls pulsed MAG welding inverter digitally and intelligently.

Real-Time Discrete Adaptive Control of Robot Arm Based on Digital Signal Processing

A discrete model reference adaptive controller of robot arm is obtained by integrating the reduced dynamic model of robot, model reference adaptive control （MRAC） and digital signal processing （DSP） computer system into an electromechanical system. With the DSP computer system, the control signal of each joint of the robot arm can be processed in real time and independently. The simulation and experiment results show that with the control strategy, the robot achieved a good trajectory following precision, a good decoupling performance and a high real-time adaptivity.