Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Observer-based dynamic event-triggered control for distributed parameter systems over mobile sensor-plus-actuator networks

We develop a policy of observer-based dynamic event-triggered state feedback control for distributed parameter systems over a mobile sensor-plus-actuator network.It is assumed that the mobile sensing devices that provide spatially averaged state measurements can be used to improve state estimation in the network.For the purpose of decreasing the update frequency of controller and unnecessary sampled data transmission, an efficient dynamic event-triggered control policy is constructed.In an event-triggered system, when an error signal exceeds a specified time-varying threshold, it indicates the occurrence of a typical event.The global asymptotic stability of the event-triggered closed-loop system and the boundedness of the minimum inter-event time can be guaranteed.Based on the linear quadratic optimal regulator, the actuator selects the optimal displacement only when an event occurs.A simulation example is finally used to verify that the effectiveness of such a control strategy can enhance the system performance.

Distributed Economic MPC for Synergetic Regulation of the Voltage of an Island DC Micro-Grid

In this paper,distributed model predictive control(DMPC) for island DC micro-grids(MG) with wind/photovoltaic(PV)/battery power is proposed,which coordinates all distributed generations(DG) to stabilize the bus voltage together with the insurance of having computational efficiency under a real-time requirement.Based on the feedback of the bus voltage,the deviation of the current is dispatched to each DG according to cost over the prediction horizon.Moreover,to avoid the excessive fluctuation of the battery power,both the discharge-charge switching times and costs are considered in the model predictive control(MPC) optimization problems.A Lyapunov constraint with a time-varying steady-state is designed in each local MPC to guarantee the stabilization of the entire system.The voltage stabilization of the MG is achieved by this strategy with the cooperation of DGs.The numeric results of applying the proposed method to a MG of the Shanghai Power Supply Company shows the effectiveness of the distributed economic MPC.

Data-Driven Control of Distributed Event-Triggered Network Systems

The present paper deals with data-driven event-triggered control of a class of unknown discrete-time interconnected systems(a.k.a.network systems).To this end,we start by putting forth a novel distributed event-triggering transmission strategy based on periodic sampling,under which a model-based stability criterion for the closed-loop network system is derived,by leveraging a discrete-time looped-functional approach.Marrying the model-based criterion with a data-driven system representation recently developed in the literature,a purely data-driven stability criterion expressed in the form of linear matrix inequalities(LMIs)is established.Meanwhile,the data-driven stability criterion suggests a means for co-designing the event-triggering coefficient matrix and the feedback control gain matrix using only some offline collected state-input data.Finally,numerical results corroborate the efficacy of the proposed distributed data-driven event-triggered network system(ETS)in cutting off data transmissions and the co-design procedure.

An Optimal Control-Based Distributed Reinforcement Learning Framework for A Class of Non-Convex Objective Functionals of the Multi-Agent Network

This paper studies a novel distributed optimization problem that aims to minimize the sum of the non-convex objective functionals of the multi-agent network under privacy protection, which means that the local objective of each agent is unknown to others. The above problem involves complexity simultaneously in the time and space aspects. Yet existing works about distributed optimization mainly consider privacy protection in the space aspect where the decision variable is a vector with finite dimensions. In contrast, when the time aspect is considered in this paper, the decision variable is a continuous function concerning time. Hence, the minimization of the overall functional belongs to the calculus of variations. Traditional works usually aim to seek the optimal decision function. Due to privacy protection and non-convexity, the Euler-Lagrange equation of the proposed problem is a complicated partial differential equation.Hence, we seek the optimal decision derivative function rather than the decision function. This manner can be regarded as seeking the control input for an optimal control problem, for which we propose a centralized reinforcement learning(RL) framework. In the space aspect, we further present a distributed reinforcement learning framework to deal with the impact of privacy protection. Finally, rigorous theoretical analysis and simulation validate the effectiveness of our framework.

Differential flatness-based distributed control of underactuated robot swarms

This paper proposes a distributed control method based on the differential flatness(DF) property of robot swarms. The swarm DF mapping is established for underactuated differentially flat dynamics, according to the control objective. The DF mapping refers to the fact that the system state and input of each robot can be derived algebraically from the flat outputs of the leaders and the cooperative errors and their finite order derivatives. Based on the proposed swarm DF mapping, a distributed controller is designed. The distributed implementation of swarm DF mapping is achieved through observer design. The effectiveness of the proposed method is validated through a numerical simulation of quadrotor swarm synchronization.

Operation Control Method of Relay Protection in Flexible DC Distribution Network Compatible with Distributed Power Supply

A novel operation control method for relay protection in flexible DC distribution networks with distributed power supply is proposed to address the issue of inaccurate fault location during relay protection,leading to poor performance.The method combines a fault-tolerant fault location method based on long-term and short-term memory networks to accurately locate the fault section.Then,an operation control method for relay protection based on adaptive weight and whale optimization algorithm(WOA)is used to construct an objective function considering the shortest relay protection action time and the smallest impulse current.The adaptive weight and WOA are employed to obtain the optimal strategy for relay protection operation control,reducing the action time and impulse current.Experimental results demonstrate the effectiveness of the proposed method in accurately locating faults and improving relay protection performance.The longest operation time is reduced by 4.7023 s,and the maximum impulse current is limited to 0.3 A,effectively controlling the impact of large impulse currents and enhancing control efficiency.

Study on optimal state estimation strategy with dual distributed controllers based on Kalman filtering

Considering dual distributed controllers, a design of optimal state estimation strategy is studied for the wireless sensor and actuator network(WSAN). In particular, the optimal linear quadratic(LQ) control strategy with estimated plant state is formulated as a non-cooperative game with network-induced delays. Then, using the Kalman filter approach, an optimal estimation of the plant state is obtained based on the information fusion of the distributed controllers. Finally, an optimal state estimation strategy is derived as a linear function of the current estimated plant state and the last control strategy of multiple controllers. The effectiveness of the proposed closed-loop control strategy is verified by the simulation experiments.

Agent-Based Synthesis of Distributed Controllers for Discrete Manufacturing Systems

A method for designing real-time distributed controllers of discrete manufacturing systems is presented. The approach held is agent based;the controller strategy is distributed into several interacting agents that operate each one on a part of the manufacturing process;these agents may be distributed into several interconnected processors. The proposed method consists of a modelling methodology and software development framework that provides a generic agent architecture and communication facilities supporting the interaction among agents.

Distributed Secondary Control and Optimal Power Sharing in Microgrids

We address the control problem of microgrids and present a fully distributed control system which consists of primary controller,secondary controller,and optimal active power sharing controller.Different from the existing control structure in microgrids,all these controllers are implemented as local controllers at each distributed generator.Thus,the requirement for a central controller is obviated.The performance analysis of the proposed control systems is provided,and the finite-time convergence properties for distributed secondary frequency and voltage controllers are achieved.Moreover,the distributed control system possesses the optimal active power sharing property.In the end,a microgrid test system is investigated to validate the effectiveness of the proposed control strategies.

Distributed cooperative control of autonomous multi-agent UAV systems using smooth control

This paper addresses the cooperative control problem of multiple unmanned aerial vehicles(multi-UAV)systems.First,a new distributed consensus algorithm for second-order nonlinear multi-agent systems(MAS)is formulated under the leader-following approach.The algorithm provides smooth input signals to the agents’control channels,which avoids the chattering effect generated by the conventional sliding mode-based control protocols.Second,a new formation control scheme is developed by integrating smooth distributed consensus control protocols into the geometric pattern model to achieve three-dimensional formation tracking.The Lyapunov theory is used to prove the stability and convergence of both distributed consensus and formation controllers.The effectiveness of the proposed algorithms is demonstrated through simulation results.

Distributed Control of Multiple-Bus Microgrid With Paralleled Distributed Generators

A microgrid is hard to control due to its reduced inertia and increased uncertainties. To overcome the challenges of microgrid control, advanced controllers need to be developed.In this paper, a distributed, two-level, communication-economic control scheme is presented for multiple-bus microgrids with each bus having multiple distributed generators(DGs) connected in parallel. The control objective of the upper level is to calculate the voltage references for one-bus subsystems. The objectives of the lower control level are to make the subsystems' bus voltages track the voltage references and to enhance load current sharing accuracy among the local DGs. Firstly, a distributed consensusbased power sharing algorithm is introduced to determine the power generations of the subsystems. Secondly, a discrete-time droop equation is used to adjust subsystem frequencies for voltage reference calculations. Finally, a Lyapunov-based decentralized control algorithm is designed for bus voltage regulation and proportional load current sharing. Extensive simulation studies with microgrid models of different levels of detail are performed to demonstrate the merits of the proposed control scheme.

Leader-following formation control of multi-agent networks based on distributed observers

To investigate the leader-following formation control, in this paper we present the design problem of control protocols and distributed observers under which the agents can achieve and maintain the desired formation from any initial states, while the velocity converges to that of the virtual leader whose velocity cannot be measured by agents in real time. The two cases of switching topologies without communication delay and fixed topology with time-varying communication delay are both considered for multi-agent networks. By using the Lyapunov stability theory, the issue of stability is analysed for multi-agent systems with switching topologies. Then, by considering the time-varying communication delay, the sufficient condition is proposed for the multi-agent systems with fixed topology. Finally, two numerical examples are given to illustrate the effectiveness of the proposed leader-following formation control protocols.

NOVEL STRUCTURE FOR DISTRIBUTED NUMERICAL CONTROL

Because of an unexpected signal noise within the network or an unpredictedfault with personal computers (PCs), many problems emerge in the implementation of distributednumerical control (DNC) with PCs-based network. To solve the problems, an industrial solution ofinvolving the field-bus technology in DNC communicating area is provided. A kind of advancedField-bus, named controller area network (CAN), is originally developed to support cheap and rathersimple automotive applications. However, because of its good performance and low cost, it is alsobeing considered in automated-manufacturing and process control environments to interconnectintelligent devices, such as modem sensors and actuators. Recently it creates a new role for CANBusin DNC that brings new thinking to DNC. CAN is used as the network platform for connecting machinetools to share information with each other reliably. Additionally, thanks to also applying of'plug-in' technology and a special interface of hardware, this solution exhibits some highcompatibility with different pedigree numerical control (NC) systems, such as Fanuc, Siemens,Cincinnati and so on. In order to improve CANBus for DNC application, a communicating competitionmodel of the basic CAN protocol, called CC model, is then highlighted. This model is able to satisfythe requirements that different machine tools share the communicating bandwidth fairly when theyrun concurrently. Finally the novel view of the latest advancement in CANBus-based DNC incombination with the manufacturing paradigm is also presented.

Chattering analysis for discrete sliding mode control ofdistributed control systems

The chattering characteristic of sliding mode control isanalyzed when it is applied in distributed control systems （DCSs）.For a DCS with random time delay and packet dropout, a discreteswitching system model with time varying sampling period isconstructed based on the time delay system method. The reachinglaw based sliding mode controller is applied in the proposedsystem. The exponential stability condition in the form of linearmatrix inequality is figured out based on the multi-Lyaponov functionmethod. Then, the chattering characteristic is analyzed for theswitching system, and a chattering region related with time varyingsampling period and external disturbance is proposed. Finally, numericalexamples are given to illustrate the validity of the analysisresult.

Direct Yaw Moment Control for Distributed Drive Electric Vehicle Handling Performance Improvement

For a distributed drive electric vehicle（DDEV） driven by four in-wheel motors, advanced vehicle dynamic control methods can be realized easily because motors can be controlled independently, quickly and precisely. And direct yaw-moment control（DYC） has been widely studied and applied to vehicle stability control. Good vehicle handling performance： quick yaw rate transient response, small overshoot, high steady yaw rate gain, etc, is required by drivers under normal conditions, which is less concerned, however. Based on the hierarchical control methodology, a novel control system using direct yaw moment control for improving handling performance of a distributed drive electric vehicle especially under normal driving conditions has been proposed. The upper-loop control system consists of two parts： a state feedback controller, which aims to realize the ideal transient response of yaw rate, with a vehicle sideslip angle observer; and a steering wheel angle feedforward controller designed to achieve a desired yaw rate steady gain. Under the restriction of the effect of poles and zeros in the closed-loop transfer function on the system response and the capacity of in-wheel motors, the integrated time and absolute error（ITAE） function is utilized as the cost function in the optimal control to calculate the ideal eigen frequency and damper coefficient of the system and obtain optimal feedback matrix and feedforward matrix. Simulations and experiments with a DDEV under multiple maneuvers are carried out and show the effectiveness of the proposed method： yaw rate rising time is reduced, steady yaw rate gain is increased, vehicle steering characteristic is close to neutral steer and drivers burdens are also reduced. The control system improves vehicle handling performance under normal conditions in both transient and steady response. State feedback control instead of model following control is introduced in the control system so that the sense of control intervention to drivers is relieved.

Distributed Model Predictive Load Frequency Control of Multi-area Power System with DFIGs

Reliable load frequency control(LFC) is crucial to the operation and design of modern electric power systems. Considering the LFC problem of a four-area interconnected power system with wind turbines, this paper presents a distributed model predictive control(DMPC) based on coordination scheme.The proposed algorithm solves a series of local optimization problems to minimize a performance objective for each control area. The generation rate constraints(GRCs), load disturbance changes, and the wind speed constraints are considered. Furthermore, the DMPC algorithm may reduce the impact of the randomness and intermittence of wind turbine effectively. A performance comparison between the proposed controller with and without the participation of the wind turbines is carried out. Analysis and simulation results show possible improvements on closed–loop performance, and computational burden with the physical constraints.

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming(ADP) where only one critic neural network(NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness(UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.

Distributed Secondary Control of AC Microgrids With External Disturbances and Directed Communication Topologies:A Full-Order Sliding-Mode Approach

This paper addresses the problem of distributed secondary control for islanded AC microgrids with external disturbances.By using a full-order sliding-mode(FOSM)approach,voltage regulation and frequency restoration are achieved in finite time.For voltage regulation,a distributed observer is proposed for each distributed generator(DG)to estimate a reference voltage level.Different from some conventional observers,the reference voltage level in this paper is accurately estimated under directed communication topologies.Based on the observer,a new nonlinear controller is designed in a backstepping manner such that an FOSM surface is reached in finite time.On the surface,the voltages of DGs are regulated to the reference level in finite time.For frequency restoration,a distributed controller is further proposed such that a constructed FOSM surface is reached in finite time,on which the frequencies of DGs are restored to a reference level in finite time under directed communication topologies.Finally,case studies on a modified IEEE 37-bus test system are conducted to demonstrate the effectiveness,the robustness against load changes,and the plug-and-play capability of the proposed controllers.