Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Observer-based dynamic event-triggered control for distributed parameter systems over mobile sensor-plus-actuator networks

We develop a policy of observer-based dynamic event-triggered state feedback control for distributed parameter systems over a mobile sensor-plus-actuator network.It is assumed that the mobile sensing devices that provide spatially averaged state measurements can be used to improve state estimation in the network.For the purpose of decreasing the update frequency of controller and unnecessary sampled data transmission, an efficient dynamic event-triggered control policy is constructed.In an event-triggered system, when an error signal exceeds a specified time-varying threshold, it indicates the occurrence of a typical event.The global asymptotic stability of the event-triggered closed-loop system and the boundedness of the minimum inter-event time can be guaranteed.Based on the linear quadratic optimal regulator, the actuator selects the optimal displacement only when an event occurs.A simulation example is finally used to verify that the effectiveness of such a control strategy can enhance the system performance.

Virtually coupled train set control subject to space-time separation:A distributed economic MPC approach with emergency braking configuration

The emerging virtual coupling technology aims to operate multiple train units in a Virtually Coupled Train Set(VCTS)at a minimal but safe distance.To guarantee collision avoidance,the safety distance should be calculated using the state-of-the-art space-time separation principle that separates the Emergency Braking(EB)trajectories of two successive units during the whole EB process.In this case,the minimal safety distance is usually numerically calculated without an analytic formulation.Thus,the constrained VCTS control problem is hard to address with space-time separation,which is still a gap in the existing literature.To solve this problem,we propose a Distributed Economic Model Predictive Control(DEMPC)approach with computation efficiency and theoretical guarantee.Specifically,to alleviate the computation burden,we transform implicit safety constraints into explicitly linear ones,such that the optimal control problem in DEMPC is a quadratic programming problem that can be solved efficiently.For theoretical analysis,sufficient conditions are derived to guarantee the recursive feasibility and stability of DEMPC,employing compatibility constraints,tube techniques and terminal ingredient tuning.Moreover,we extend our approach with globally optimal and distributed online EB configuration methods to shorten the minimal distance among VCTS.Finally,experimental results demonstrate the performance and advantages of the proposed approaches.

STABILITY ANALYSIS AND COOPERATIVE CONTROL OF DISTRIBUTED MULTI-AGENT SYSTEM WITH SAMPLED COMMUNICATION

The cooperative control and stability analysis problems for the multi-agent system with sampled com- munication are investigated. Distributed state feedback controllers are adopted for the cooperation of networked agents. A theorem in the form of linear matrix inequalities（LMI） is derived to analyze the system stability. An- other theorem in the form of optimization problem subject to LMI constraints is proposed to design the controller, and then the algorithm is presented. The simulation results verify the validity and the effectiveness of the pro- posed approach.

Distributed Secondary Control and Optimal Power Sharing in Microgrids

We address the control problem of microgrids and present a fully distributed control system which consists of primary controller, secondary controller, and optimal active power sharing controller. Different from the existing control structure in microgrids, all these controllers are implemented as local controllers at each distributed generator. Thus, the requirement for a central controller is obviated. The performance analysis of the proposed control systems is provided, and the finite-time convergence properties for distributed secondary frequency and voltage controllers are achieved. Moreover, the distributed control system possesses the optimal active power sharing property. In the end, a microgrid test system is investigated to validate the effectiveness of the proposed control strategies. © 2014 Chinese Association of Automation.

Distributed cooperative control of autonomous multi-agent UAV systems using smooth control

This paper addresses the cooperative control problem of multiple unmanned aerial vehicles(multi-UAV)systems.First,a new distributed consensus algorithm for second-order nonlinear multi-agent systems(MAS)is formulated under the leader-following approach.The algorithm provides smooth input signals to the agents’control channels,which avoids the chattering effect generated by the conventional sliding mode-based control protocols.Second,a new formation control scheme is developed by integrating smooth distributed consensus control protocols into the geometric pattern model to achieve three-dimensional formation tracking.The Lyapunov theory is used to prove the stability and convergence of both distributed consensus and formation controllers.The effectiveness of the proposed algorithms is demonstrated through simulation results.

Distributed Control of Multiple-Bus Microgrid With Paralleled Distributed Generators

A microgrid is hard to control due to its reduced inertia and increased uncertainties. To overcome the challenges of microgrid control, advanced controllers need to be developed.In this paper, a distributed, two-level, communication-economic control scheme is presented for multiple-bus microgrids with each bus having multiple distributed generators(DGs) connected in parallel. The control objective of the upper level is to calculate the voltage references for one-bus subsystems. The objectives of the lower control level are to make the subsystems' bus voltages track the voltage references and to enhance load current sharing accuracy among the local DGs. Firstly, a distributed consensusbased power sharing algorithm is introduced to determine the power generations of the subsystems. Secondly, a discrete-time droop equation is used to adjust subsystem frequencies for voltage reference calculations. Finally, a Lyapunov-based decentralized control algorithm is designed for bus voltage regulation and proportional load current sharing. Extensive simulation studies with microgrid models of different levels of detail are performed to demonstrate the merits of the proposed control scheme.

Distributed Power Control for Energy Conservation in Hybrid Cellular Network with Device-to-Device Communication

Device-to-Device(D2D) communication has been proposed as a promising implementation of green communication to benefit the existed cellular network.In order to limit cross-tier interference while explore the gain of short-range communication,we devise a series of distributed power control(DPC) schemes for energy conservation(EC)and enhancement of radio resource utilization in the hybrid system.Firstly,a constrained opportunistic power control model is built up to take advantage of the interference avoidance methodology in the presence of service requirement and power constraint.Then,biasing scheme and admission control are added to evade ineffective power consumption and maintain the feasibility of the system.Upon feasibility,a non-cooperative game is further formulated to exploit the profit in EC with minor influence on spectral efficiency(SE).The convergence of the DPC schemes is validated and their performance is confirmed via simulation results.

Leader-following formation control of multi-agent networks based on distributed observers

To investigate the leader-following formation control, in this paper we present the design problem of control protocols and distributed observers under which the agents can achieve and maintain the desired formation from any initial states, while the velocity converges to that of the virtual leader whose velocity cannot be measured by agents in real time. The two cases of switching topologies without communication delay and fixed topology with time-varying communication delay are both considered for multi-agent networks. By using the Lyapunov stability theory, the issue of stability is analysed for multi-agent systems with switching topologies. Then, by considering the time-varying communication delay, the sufficient condition is proposed for the multi-agent systems with fixed topology. Finally, two numerical examples are given to illustrate the effectiveness of the proposed leader-following formation control protocols.

NOVEL STRUCTURE FOR DISTRIBUTED NUMERICAL CONTROL

Because of an unexpected signal noise within the network or an unpredictedfault with personal computers (PCs), many problems emerge in the implementation of distributednumerical control (DNC) with PCs-based network. To solve the problems, an industrial solution ofinvolving the field-bus technology in DNC communicating area is provided. A kind of advancedField-bus, named controller area network (CAN), is originally developed to support cheap and rathersimple automotive applications. However, because of its good performance and low cost, it is alsobeing considered in automated-manufacturing and process control environments to interconnectintelligent devices, such as modem sensors and actuators. Recently it creates a new role for CANBusin DNC that brings new thinking to DNC. CAN is used as the network platform for connecting machinetools to share information with each other reliably. Additionally, thanks to also applying of'plug-in' technology and a special interface of hardware, this solution exhibits some highcompatibility with different pedigree numerical control (NC) systems, such as Fanuc, Siemens,Cincinnati and so on. In order to improve CANBus for DNC application, a communicating competitionmodel of the basic CAN protocol, called CC model, is then highlighted. This model is able to satisfythe requirements that different machine tools share the communicating bandwidth fairly when theyrun concurrently. Finally the novel view of the latest advancement in CANBus-based DNC incombination with the manufacturing paradigm is also presented.

Chattering analysis for discrete sliding mode control ofdistributed control systems

The chattering characteristic of sliding mode control isanalyzed when it is applied in distributed control systems （DCSs）.For a DCS with random time delay and packet dropout, a discreteswitching system model with time varying sampling period isconstructed based on the time delay system method. The reachinglaw based sliding mode controller is applied in the proposedsystem. The exponential stability condition in the form of linearmatrix inequality is figured out based on the multi-Lyaponov functionmethod. Then, the chattering characteristic is analyzed for theswitching system, and a chattering region related with time varyingsampling period and external disturbance is proposed. Finally, numericalexamples are given to illustrate the validity of the analysisresult.

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming(ADP) where only one critic neural network(NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness(UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.

Direct Yaw Moment Control for Distributed Drive Electric Vehicle Handling Performance Improvement

For a distributed drive electric vehicle（DDEV） driven by four in-wheel motors, advanced vehicle dynamic control methods can be realized easily because motors can be controlled independently, quickly and precisely. And direct yaw-moment control（DYC） has been widely studied and applied to vehicle stability control. Good vehicle handling performance： quick yaw rate transient response, small overshoot, high steady yaw rate gain, etc, is required by drivers under normal conditions, which is less concerned, however. Based on the hierarchical control methodology, a novel control system using direct yaw moment control for improving handling performance of a distributed drive electric vehicle especially under normal driving conditions has been proposed. The upper-loop control system consists of two parts： a state feedback controller, which aims to realize the ideal transient response of yaw rate, with a vehicle sideslip angle observer; and a steering wheel angle feedforward controller designed to achieve a desired yaw rate steady gain. Under the restriction of the effect of poles and zeros in the closed-loop transfer function on the system response and the capacity of in-wheel motors, the integrated time and absolute error（ITAE） function is utilized as the cost function in the optimal control to calculate the ideal eigen frequency and damper coefficient of the system and obtain optimal feedback matrix and feedforward matrix. Simulations and experiments with a DDEV under multiple maneuvers are carried out and show the effectiveness of the proposed method： yaw rate rising time is reduced, steady yaw rate gain is increased, vehicle steering characteristic is close to neutral steer and drivers burdens are also reduced. The control system improves vehicle handling performance under normal conditions in both transient and steady response. State feedback control instead of model following control is introduced in the control system so that the sense of control intervention to drivers is relieved.

Distributed Secondary Control of AC Microgrids With External Disturbances and Directed Communication Topologies:A Full-Order Sliding-Mode Approach

This paper addresses the problem of distributed secondary control for islanded AC microgrids with external disturbances.By using a full-order sliding-mode(FOSM)approach,voltage regulation and frequency restoration are achieved in finite time.For voltage regulation,a distributed observer is proposed for each distributed generator(DG)to estimate a reference voltage level.Different from some conventional observers,the reference voltage level in this paper is accurately estimated under directed communication topologies.Based on the observer,a new nonlinear controller is designed in a backstepping manner such that an FOSM surface is reached in finite time.On the surface,the voltages of DGs are regulated to the reference level in finite time.For frequency restoration,a distributed controller is further proposed such that a constructed FOSM surface is reached in finite time,on which the frequencies of DGs are restored to a reference level in finite time under directed communication topologies.Finally,case studies on a modified IEEE 37-bus test system are conducted to demonstrate the effectiveness,the robustness against load changes,and the plug-and-play capability of the proposed controllers.

Distributed Periodic Event-Triggered Optimal Control of DC Microgrids Based on Virtual Incremental Cost

This article presents a distributed periodic eventtriggered(PET)optimal control scheme to achieve generation cost minimization and average bus voltage regulation in DC microgrids.In order to accommodate the generation constraints of the distributed generators(DGs),a virtual incremental cost is firstly designed,based on which an optimality condition is derived to facilitate the control design.To meet the discrete-time(DT)nature of modern control systems,the optimal controller is directly developed in the DT domain.Afterward,to reduce the communication requirement among the controllers,a distributed event-triggered mechanism is introduced for the DT optimal controller.The event-triggered condition is detected periodically and therefore naturally avoids the Zeno phenomenon.The closed-loop system stability is proved by the Lyapunov synthesis for switched systems.The generation cost minimization and average bus voltage regulation are obtained at the equilibrium point.Finally,switch-level microgrid simulations validate the performance of the proposed optimal controller.

DCS devices based non-linear process control system design for plants with distributed time-delay using particle filter

Remote control process system with distributed time-delay has attracted much attention in different fields.In this paper,non-linear remote control of a single tank process system with wireless network is considered.To deal with the distributed time-delay in a large-scale plant,the time-delay compensation controller based on DCS devices is designed by using operator theory and particle filter.Distributed control system(DCS)device is developed to monitor and control from the central monitoring room to each process.The particle filter is a probabilistic method to estimate unobservable information from observable information.First,remote control system and experimental equipment are introduced.Second,control system based on an operator theory is designed.Then,process system with distributed time-delay using particle filter is carried out.Finally,the actual experiment is conducted by using the proposed time-delay compensation controller.When estimating with the proposed method,the result is close to the case in which the distributed time-delay does not exist.The effectiveness of the proposed control system is confirmed by experiment results.

Resilience Against Replay Attacks:A Distributed Model Predictive Control Scheme for Networked Multi-Agent Systems

In this paper,a resilient distributed control scheme against replay attacks for multi-agent networked systems subject to input and state constraints is proposed.The methodological starting point relies on a smart use of predictive arguments with a twofold aim:1)Promptly detect malicious agent behaviors affecting normal system operations;2)Apply specific control actions,based on predictive ideas,for mitigating as much as possible undesirable domino effects resulting from adversary operations.Specifically,the multi-agent system is topologically described by a leader-follower digraph characterized by a unique leader and set-theoretic receding horizon control ideas are exploited to develop a distributed algorithm capable to instantaneously recognize the attacked agent.Finally,numerical simulations are carried out to show benefits and effectiveness of the proposed approach.

Optimal distributed resource allocation in a wireless sensor network for control systems

Wireless technology is applied increasingly in networked control systems. A new form of wireless network called wireless sensor network can bring control systems some advantages, such as flexibility and feasibility of network deployment at low costs, while it also raises some new challenges. First, the communication resources shared by all the control loops are limited. Second, the wireless and multi-hop character of sensor network makes the resources scheduling more difficult. Thus, how to effectively allocate the limited communication resources for those control loops is an important problem. In this paper, this problem is formulated as an optimal sampling frequency assignment problem, where the objective function is to maximize the utility of control systems, subject to channel capacity constraints. Then an iterative distributed algorithm based on local buffer information is proposed. Finally, the simulation results show that the proposed algorithm can effectively allocate the limited communication resource in a distributed way. It can achieve the optimal quality of the control system and adapt to the network load changes.

SLIDING MODE CONTROL OF A CLASS OF ITO TYPE DISTRIBUTED PARAMETER SYSTEMS WITH DELAY

Sliding mode control problem of a class of Ito^ type partial differential equations with delay is probed. The variable structure controller is designed. The existence of motion of sliding mode is shown. And the character of invariance of sliding control system about uncertainty on the sliding switching surface and stability are analyzed.

Distributed generator-based distribution system service restoration strategy and model-free control methods

The rapid growth of distributed generator(DG)capacities has introduced additional controllable assets to improve the performance of distribution systems in terms of service restoration.Renewable DGs are of particular interest to utility companies,but the stochastic nature of intermittent renewable DGs could have a negative impact on the electric grid if they are not properly handled.In this study,we investigate distribution system service restoration using DGs as the primary power source,and we develop an effective approach to handle the uncertainty of renewable DGs under extreme conditions.The distribution system service restoration problem can be described as a mixed-integer second-order cone programming model by modifying the radial topology constraints and power flow equations.The uncertainty of renewable DGs will be modeled using a chance-constrained approach.Furthermore,the forecast errors and noises in real-time operation are solved using a novel model-free control algorithm that can automatically track the trajectory of real-time DG output.The proposed service restoration strategy and model-free control algorithm are validated using an IEEE 123-bus test system.