Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Observer-based dynamic event-triggered control for distributed parameter systems over mobile sensor-plus-actuator networks

We develop a policy of observer-based dynamic event-triggered state feedback control for distributed parameter systems over a mobile sensor-plus-actuator network.It is assumed that the mobile sensing devices that provide spatially averaged state measurements can be used to improve state estimation in the network.For the purpose of decreasing the update frequency of controller and unnecessary sampled data transmission, an efficient dynamic event-triggered control policy is constructed.In an event-triggered system, when an error signal exceeds a specified time-varying threshold, it indicates the occurrence of a typical event.The global asymptotic stability of the event-triggered closed-loop system and the boundedness of the minimum inter-event time can be guaranteed.Based on the linear quadratic optimal regulator, the actuator selects the optimal displacement only when an event occurs.A simulation example is finally used to verify that the effectiveness of such a control strategy can enhance the system performance.

A master-slave generalized predictive synchronization control for preheating process of multi-cavity hot runner system

As a key component of injection molding,multi-cavity hot runner(MCHR)system faces the crucial problem of polymer melt filling imbalance among the cavities.The thermal imbalance in the system has been considered as the leading cause.Hence,the solution may rest with the synchronization of those heating processes in MCHR system.This paper proposes a’Master-Slave’generalized predictive synchronization control(MS-GPSC)method with’Mr.Slowest’strategy for preheating stage of MCHR system.The core of the proposed method is choosing the heating process with slowest dynamics as the’Master’to track the setpoint,while the other heating processes are treated as‘Slaves’tracking the output of’Master’.This proposed method is shown to have the good ability of temperature synchronization.The corresponding analysis is conducted on parameters tuning and stability,simulations and experiments show the strategy is effective.

Achievement of chaotic synchronization trajectories of master-slave manipulators with feedback control strategy

This paper addresses a master-slave synchro- nization strategy for complex dynamic systems based on feedback control. This strategy is applied to 3-DOF pla- nar manipulators in order to obtain synchronization in such complicated as chaotic motions of end-effectors. A chaotic curve is selected from Duffing equation as the trajectory of master end-effector and a piecewise approximation method is proposed to accurately represent this chaotic trajectory of end-effectors. The dynamical equations of master-slave manipulators with synchronization controller are derived, and the Lyapunov stability theory is used to determine the stability of this controlled synchronization system. In numer- ical experiments, the synchronous motions of end-effectors as well as three joint angles and torques of master-slave manipulators are studied under the control of the proposed synchronization strategy. It is found that the positive gain matrix affects the implementation of synchronization con- trol strategy. This synchronization control strategy proves the synchronization＇s feasibility and controllability for com- plicated motions generated by master-slave manipulators.

Virtually coupled train set control subject to space-time separation:A distributed economic MPC approach with emergency braking configuration

The emerging virtual coupling technology aims to operate multiple train units in a Virtually Coupled Train Set(VCTS)at a minimal but safe distance.To guarantee collision avoidance,the safety distance should be calculated using the state-of-the-art space-time separation principle that separates the Emergency Braking(EB)trajectories of two successive units during the whole EB process.In this case,the minimal safety distance is usually numerically calculated without an analytic formulation.Thus,the constrained VCTS control problem is hard to address with space-time separation,which is still a gap in the existing literature.To solve this problem,we propose a Distributed Economic Model Predictive Control(DEMPC)approach with computation efficiency and theoretical guarantee.Specifically,to alleviate the computation burden,we transform implicit safety constraints into explicitly linear ones,such that the optimal control problem in DEMPC is a quadratic programming problem that can be solved efficiently.For theoretical analysis,sufficient conditions are derived to guarantee the recursive feasibility and stability of DEMPC,employing compatibility constraints,tube techniques and terminal ingredient tuning.Moreover,we extend our approach with globally optimal and distributed online EB configuration methods to shorten the minimal distance among VCTS.Finally,experimental results demonstrate the performance and advantages of the proposed approaches.

Hierarchical hybrid control network design based on LON and master-slave RS-422/485 protocol

Aiming at the weaknesses of LON bus, combining the coexistence of fieldbus and DCS (Distribu ted Control Systems) in control networks, the authors introduce a hierarchical hybrid control network design based on LON and master slave RS 422/485 protocol. This design adopts LON as the trunk, master slave RS 422/485 control networks are connected to LON as special subnets by dedicated gateways. It is an implementation method for isomerous control network integration. Data management is ranked according to real time requirements for different network data. The core components, such as control network nodes, router and gateway, are detailed in the paper. The design utilizes both communication advantage of LonWorks technology and the more powerful control ability of universal MCUs or PLCs, thus it greatly increases system response speed and performance cost ratio.

Distributed Generation Islanding Effect on Distribution Networks and End User Loads Using the Master-Slave Islanding Method

This study aims to address the feasibility of planned islanding operation and to investigate the effect of unplanned islanding using the master-slave islanding method for controlling the distributed generation units during grid-connected and islanding operation. Neplan desktop power simulation tool was used for the modelling and simulation of a realistic MV network with four different distributed generation technologies (diesel, gas, hydro and wind) along with their excitation and governor control systems, while an exponential model was used to represent the loads in the network. The dynamic and steady state behavior of the four distributed generation technologies were investigated during grid-connected operation and two transition modes to the islanding situation, planned and unplanned. The obtained results that validated through various case studies have shown that a suitable planned islanding transition could provide support to critical loads at the event of electricity utility outages.

STABILITY ANALYSIS AND COOPERATIVE CONTROL OF DISTRIBUTED MULTI-AGENT SYSTEM WITH SAMPLED COMMUNICATION

The cooperative control and stability analysis problems for the multi-agent system with sampled com- munication are investigated. Distributed state feedback controllers are adopted for the cooperation of networked agents. A theorem in the form of linear matrix inequalities（LMI） is derived to analyze the system stability. An- other theorem in the form of optimization problem subject to LMI constraints is proposed to design the controller, and then the algorithm is presented. The simulation results verify the validity and the effectiveness of the pro- posed approach.

STUDY ON OPTIMAL CONTROL OF MUNICIPAL WATER DISTRIBUTION NETWORK

A systematic investigation is made on the problems which are related to the optimal control of the municipal water distribution network.A mathematical model of forecasting the water short term demand is proposed using the time series trigonometric function analysis method;the service discharge based macroscopic model of network performance is established using the network structuring method;a relatively satisfactory mathematical model for the optimal control of water distribution network is put forward in view of security and economy,and solved by the constrained mixed discrete variable complex arithmetic.The model is applied in many examples and the results are satisfactory.

Distributed Secondary Control and Optimal Power Sharing in Microgrids

We address the control problem of microgrids and present a fully distributed control system which consists of primary controller, secondary controller, and optimal active power sharing controller. Different from the existing control structure in microgrids, all these controllers are implemented as local controllers at each distributed generator. Thus, the requirement for a central controller is obviated. The performance analysis of the proposed control systems is provided, and the finite-time convergence properties for distributed secondary frequency and voltage controllers are achieved. Moreover, the distributed control system possesses the optimal active power sharing property. In the end, a microgrid test system is investigated to validate the effectiveness of the proposed control strategies. © 2014 Chinese Association of Automation.

Size Distributions of Aerosol Sulfates and Nitrates in Beijing during the 2008 Olympic Games： Impacts of Pollution Control Measures and Regional Transport

For the 2008 Olympic Games, drastic control measures were implemented on industrial and urban emissions of sulfur dioxide （SO2）, nitrogen oxides （NOx） and other pollutants to address the issues of poor air quality in Beijing. To investigate the effects of SO2 and NOx reductions on the particulate sulfate and nitrate concentrations as well as their size distributions, size-segregated aerosol samples were collected using micro-orifice uniform deposit impactors （MOUDIs） at urban and downwind rural sites in Beijing before and after full-scale controls. During the sampling period, the mass concentrations of fine particles （PMI.s） at the urban and rural sites were 94.0 and 85.9 p.g m-3, respectively. More than 90% of the sulfates and 60% of nitrates formed as fine particles. Benefiting from the advantageous meteorological conditions and the source controls, sulfates were observed in rather low concentrations and primarily in condensation mode during the Olympics. The effects of the control measures were separately analyzed for the northerly and the southerly air-mass-dominated days to account for any bias. After the control measures were implemented, PM, sulfates, and nitrates were significantly reduced when the northerly air masses prevailed, with a higher percentage of reduction in larger particles. The droplet mode particles, which dominated the sulfates and nitrates before the controls were implemented, were remarkably reduced in mass concentration after the control measures were implemented. Nevertheless, when the polluted southerly air masses prevailed, the local source control measures in Beijing did not effectively reduce the ambient sulfate concentration due to the enormous regional contribution from the North China Plain.

Stress distribution and surrounding rock control of mining near to the overlying coal pillar in the working face

The occurrence of overlying coal pillar(OCP)exerts a strong effect on the stress and strain distribution of the surrounding rock in the stope.In this paper,the stress distribution characteristics are analyzed via the numerical calculation with the account of OCP presence or absence.In addition,this study revealed the joint effect of side pressure relief area of the goaf and stress concentration in OCP on the final stress distribution.Furthermore,the rules of abutment stress distribution affected by three influencing factors,namely horizontal-vertical distances between OCP and working face and buried depth of OCP,are analyzed.The functional model linking the peak stress of surrounding rock with the above influencing factors is developed.The field application of the above results proved that the rib spalling and deformation of a 2.95 m-high and 5.66 m-wide roadway could be efficiently controlled by rationally adjusting working states of the support,and adopting the hydraulic prop coordinated with the p type metal beam and anchor cable to strengthen the surrounding rock of working face and roadway,respectively.The proposed measures are considered appropriate to satisfy the safe operation requirements.

Distribution pattern of marine flagellate and its controlling factors in the central and north part of the Huanghai Sea in early summer

A survey was carried out in the central and north part of the Huanghai Sea (34.5° -37.0°N, 120.5°-124.0°E) during June 12-27, 2000. It was found that the abundance of marine flagellate ranged from 45 to 1 278 cell/ml, 479 cell/ml in average. Flagellate was more abundant in the central part than in the north part of Huanghai Sea, and the abundance decreased with the increasing distance from the coast, showing a similar distribution pattern with isotherm. Vertically, high density of flagellate was always presented in the bottom of thermocline, and formed a dense accumulation in the central area of the Huanghai Sea Cold Water Mass. The effects of physical and biological factors on the distribution of marine flagellate in early summer were discussed. Water temperature (especially the existence of thermocline) rather than salinity showed significant effect on the distribution pattern of marine flagellate in the Huanghai Sea in early summer. When comparing the abundance of marine flagellate with that of other microorganisms, it revealed a comparatively stable relationship among these organ-hisms, with a ratio of heterotrophic bacteria: cyanobacteria: flagellate: dinoflagellate: ciliate being 105 :103:102:101:100.

Distributed cooperative control of autonomous multi-agent UAV systems using smooth control

This paper addresses the cooperative control problem of multiple unmanned aerial vehicles(multi-UAV)systems.First,a new distributed consensus algorithm for second-order nonlinear multi-agent systems(MAS)is formulated under the leader-following approach.The algorithm provides smooth input signals to the agents’control channels,which avoids the chattering effect generated by the conventional sliding mode-based control protocols.Second,a new formation control scheme is developed by integrating smooth distributed consensus control protocols into the geometric pattern model to achieve three-dimensional formation tracking.The Lyapunov theory is used to prove the stability and convergence of both distributed consensus and formation controllers.The effectiveness of the proposed algorithms is demonstrated through simulation results.

Distributed Control of Multiple-Bus Microgrid With Paralleled Distributed Generators

A microgrid is hard to control due to its reduced inertia and increased uncertainties. To overcome the challenges of microgrid control, advanced controllers need to be developed.In this paper, a distributed, two-level, communication-economic control scheme is presented for multiple-bus microgrids with each bus having multiple distributed generators(DGs) connected in parallel. The control objective of the upper level is to calculate the voltage references for one-bus subsystems. The objectives of the lower control level are to make the subsystems' bus voltages track the voltage references and to enhance load current sharing accuracy among the local DGs. Firstly, a distributed consensusbased power sharing algorithm is introduced to determine the power generations of the subsystems. Secondly, a discrete-time droop equation is used to adjust subsystem frequencies for voltage reference calculations. Finally, a Lyapunov-based decentralized control algorithm is designed for bus voltage regulation and proportional load current sharing. Extensive simulation studies with microgrid models of different levels of detail are performed to demonstrate the merits of the proposed control scheme.

Distributed Power Control for Energy Conservation in Hybrid Cellular Network with Device-to-Device Communication

Device-to-Device(D2D) communication has been proposed as a promising implementation of green communication to benefit the existed cellular network.In order to limit cross-tier interference while explore the gain of short-range communication,we devise a series of distributed power control(DPC) schemes for energy conservation(EC)and enhancement of radio resource utilization in the hybrid system.Firstly,a constrained opportunistic power control model is built up to take advantage of the interference avoidance methodology in the presence of service requirement and power constraint.Then,biasing scheme and admission control are added to evade ineffective power consumption and maintain the feasibility of the system.Upon feasibility,a non-cooperative game is further formulated to exploit the profit in EC with minor influence on spectral efficiency(SE).The convergence of the DPC schemes is validated and their performance is confirmed via simulation results.

Leader-following formation control of multi-agent networks based on distributed observers

To investigate the leader-following formation control, in this paper we present the design problem of control protocols and distributed observers under which the agents can achieve and maintain the desired formation from any initial states, while the velocity converges to that of the virtual leader whose velocity cannot be measured by agents in real time. The two cases of switching topologies without communication delay and fixed topology with time-varying communication delay are both considered for multi-agent networks. By using the Lyapunov stability theory, the issue of stability is analysed for multi-agent systems with switching topologies. Then, by considering the time-varying communication delay, the sufficient condition is proposed for the multi-agent systems with fixed topology. Finally, two numerical examples are given to illustrate the effectiveness of the proposed leader-following formation control protocols.

NOVEL STRUCTURE FOR DISTRIBUTED NUMERICAL CONTROL

Because of an unexpected signal noise within the network or an unpredictedfault with personal computers (PCs), many problems emerge in the implementation of distributednumerical control (DNC) with PCs-based network. To solve the problems, an industrial solution ofinvolving the field-bus technology in DNC communicating area is provided. A kind of advancedField-bus, named controller area network (CAN), is originally developed to support cheap and rathersimple automotive applications. However, because of its good performance and low cost, it is alsobeing considered in automated-manufacturing and process control environments to interconnectintelligent devices, such as modem sensors and actuators. Recently it creates a new role for CANBusin DNC that brings new thinking to DNC. CAN is used as the network platform for connecting machinetools to share information with each other reliably. Additionally, thanks to also applying of'plug-in' technology and a special interface of hardware, this solution exhibits some highcompatibility with different pedigree numerical control (NC) systems, such as Fanuc, Siemens,Cincinnati and so on. In order to improve CANBus for DNC application, a communicating competitionmodel of the basic CAN protocol, called CC model, is then highlighted. This model is able to satisfythe requirements that different machine tools share the communicating bandwidth fairly when theyrun concurrently. Finally the novel view of the latest advancement in CANBus-based DNC incombination with the manufacturing paradigm is also presented.

Chattering analysis for discrete sliding mode control ofdistributed control systems

The chattering characteristic of sliding mode control isanalyzed when it is applied in distributed control systems （DCSs）.For a DCS with random time delay and packet dropout, a discreteswitching system model with time varying sampling period isconstructed based on the time delay system method. The reachinglaw based sliding mode controller is applied in the proposedsystem. The exponential stability condition in the form of linearmatrix inequality is figured out based on the multi-Lyaponov functionmethod. Then, the chattering characteristic is analyzed for theswitching system, and a chattering region related with time varyingsampling period and external disturbance is proposed. Finally, numericalexamples are given to illustrate the validity of the analysisresult.

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming(ADP) where only one critic neural network(NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness(UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.