Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Differential flatness-based distributed control of underactuated robot swarms

This paper proposes a distributed control method based on the differential flatness(DF) property of robot swarms. The swarm DF mapping is established for underactuated differentially flat dynamics, according to the control objective. The DF mapping refers to the fact that the system state and input of each robot can be derived algebraically from the flat outputs of the leaders and the cooperative errors and their finite order derivatives. Based on the proposed swarm DF mapping, a distributed controller is designed. The distributed implementation of swarm DF mapping is achieved through observer design. The effectiveness of the proposed method is validated through a numerical simulation of quadrotor swarm synchronization.

Optimal Cooperative Secondary Control for Islanded DC Microgrids via a Fully Actuated Approach

DC-DC converter-based multi-bus DC microgrids(MGs) in series have received much attention, where the conflict between voltage recovery and current balancing has been a hot topic. The lack of models that accurately portray the electrical characteristics of actual MGs while is controller design-friendly has kept the issue active. To this end, this paper establishes a large-signal model containing the comprehensive dynamical behavior of the DC MGs based on the theory of high-order fully actuated systems, and proposes distributed optimal control based on this. The proposed secondary control method can achieve the two goals of voltage recovery and current sharing for multi-bus DC MGs. Additionally, the simple structure of the proposed approach is similar to one based on droop control, which allows this control technique to be easily implemented in a variety of modern microgrids with different configurations. In contrast to existing studies, the process of controller design in this paper is closely tied to the actual dynamics of the MGs. It is a prominent feature that enables engineers to customize the performance metrics of the system. In addition, the analysis of the stability of the closed-loop DC microgrid system, as well as the optimality and consensus of current sharing are given. Finally, a scaled-down solar and battery-based microgrid prototype with maximum power point tracking controller is developed in the laboratory to experimentally test the efficacy of the proposed control method.

Distributed Economic MPC for Synergetic Regulation of the Voltage of an Island DC Micro-Grid

In this paper,distributed model predictive control(DMPC) for island DC micro-grids(MG) with wind/photovoltaic(PV)/battery power is proposed,which coordinates all distributed generations(DG) to stabilize the bus voltage together with the insurance of having computational efficiency under a real-time requirement.Based on the feedback of the bus voltage,the deviation of the current is dispatched to each DG according to cost over the prediction horizon.Moreover,to avoid the excessive fluctuation of the battery power,both the discharge-charge switching times and costs are considered in the model predictive control(MPC) optimization problems.A Lyapunov constraint with a time-varying steady-state is designed in each local MPC to guarantee the stabilization of the entire system.The voltage stabilization of the MG is achieved by this strategy with the cooperation of DGs.The numeric results of applying the proposed method to a MG of the Shanghai Power Supply Company shows the effectiveness of the distributed economic MPC.

Distributed Secondary Control and Optimal Power Sharing in Microgrids

We address the control problem of microgrids and present a fully distributed control system which consists of primary controller,secondary controller,and optimal active power sharing controller.Different from the existing control structure in microgrids,all these controllers are implemented as local controllers at each distributed generator.Thus,the requirement for a central controller is obviated.The performance analysis of the proposed control systems is provided,and the finite-time convergence properties for distributed secondary frequency and voltage controllers are achieved.Moreover,the distributed control system possesses the optimal active power sharing property.In the end,a microgrid test system is investigated to validate the effectiveness of the proposed control strategies.

Distributed Model Predictive Load Frequency Control of Multi-area Power System with DFIGs

Reliable load frequency control(LFC) is crucial to the operation and design of modern electric power systems. Considering the LFC problem of a four-area interconnected power system with wind turbines, this paper presents a distributed model predictive control(DMPC) based on coordination scheme.The proposed algorithm solves a series of local optimization problems to minimize a performance objective for each control area. The generation rate constraints(GRCs), load disturbance changes, and the wind speed constraints are considered. Furthermore, the DMPC algorithm may reduce the impact of the randomness and intermittence of wind turbine effectively. A performance comparison between the proposed controller with and without the participation of the wind turbines is carried out. Analysis and simulation results show possible improvements on closed–loop performance, and computational burden with the physical constraints.

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming(ADP) where only one critic neural network(NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness(UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.

Chattering analysis for discrete sliding mode control ofdistributed control systems

The chattering characteristic of sliding mode control isanalyzed when it is applied in distributed control systems （DCSs）.For a DCS with random time delay and packet dropout, a discreteswitching system model with time varying sampling period isconstructed based on the time delay system method. The reachinglaw based sliding mode controller is applied in the proposedsystem. The exponential stability condition in the form of linearmatrix inequality is figured out based on the multi-Lyaponov functionmethod. Then, the chattering characteristic is analyzed for theswitching system, and a chattering region related with time varyingsampling period and external disturbance is proposed. Finally, numericalexamples are given to illustrate the validity of the analysisresult.

Distributed Secondary Control of AC Microgrids With External Disturbances and Directed Communication Topologies:A Full-Order Sliding-Mode Approach

This paper addresses the problem of distributed secondary control for islanded AC microgrids with external disturbances.By using a full-order sliding-mode(FOSM)approach,voltage regulation and frequency restoration are achieved in finite time.For voltage regulation,a distributed observer is proposed for each distributed generator(DG)to estimate a reference voltage level.Different from some conventional observers,the reference voltage level in this paper is accurately estimated under directed communication topologies.Based on the observer,a new nonlinear controller is designed in a backstepping manner such that an FOSM surface is reached in finite time.On the surface,the voltages of DGs are regulated to the reference level in finite time.For frequency restoration,a distributed controller is further proposed such that a constructed FOSM surface is reached in finite time,on which the frequencies of DGs are restored to a reference level in finite time under directed communication topologies.Finally,case studies on a modified IEEE 37-bus test system are conducted to demonstrate the effectiveness,the robustness against load changes,and the plug-and-play capability of the proposed controllers.

Distributed Periodic Event-Triggered Optimal Control of DC Microgrids Based on Virtual Incremental Cost

This article presents a distributed periodic eventtriggered(PET)optimal control scheme to achieve generation cost minimization and average bus voltage regulation in DC microgrids.In order to accommodate the generation constraints of the distributed generators(DGs),a virtual incremental cost is firstly designed,based on which an optimality condition is derived to facilitate the control design.To meet the discrete-time(DT)nature of modern control systems,the optimal controller is directly developed in the DT domain.Afterward,to reduce the communication requirement among the controllers,a distributed event-triggered mechanism is introduced for the DT optimal controller.The event-triggered condition is detected periodically and therefore naturally avoids the Zeno phenomenon.The closed-loop system stability is proved by the Lyapunov synthesis for switched systems.The generation cost minimization and average bus voltage regulation are obtained at the equilibrium point.Finally,switch-level microgrid simulations validate the performance of the proposed optimal controller.

Resilience Against Replay Attacks:A Distributed Model Predictive Control Scheme for Networked Multi-Agent Systems

In this paper,a resilient distributed control scheme against replay attacks for multi-agent networked systems subject to input and state constraints is proposed.The methodological starting point relies on a smart use of predictive arguments with a twofold aim:1)Promptly detect malicious agent behaviors affecting normal system operations;2)Apply specific control actions,based on predictive ideas,for mitigating as much as possible undesirable domino effects resulting from adversary operations.Specifically,the multi-agent system is topologically described by a leader-follower digraph characterized by a unique leader and set-theoretic receding horizon control ideas are exploited to develop a distributed algorithm capable to instantaneously recognize the attacked agent.Finally,numerical simulations are carried out to show benefits and effectiveness of the proposed approach.

Distributed Fault-Tolerant Containment Control for Nonlinear Multi-Agent Systems Under Directed Network Topology via Hierarchical Approach

This paper investigates the distributed fault-tolerant containment control(FTCC)problem of nonlinear multi-agent systems(MASs)under a directed network topology.The proposed control framework which is independent on the global information about the communication topology consists of two layers.Different from most existing distributed fault-tolerant control(FTC)protocols where the fault in one agent may propagate over network,the developed control method can eliminate the phenomenon of fault propagation.Based on the hierarchical control strategy,the FTCC problem with a directed graph can be simplified to the distributed containment control of the upper layer and the fault-tolerant tracking control of the lower layer.Finally,simulation results are given to demonstrate the effectiveness of the proposed control protocol.

NOVEL STRUCTURE FOR DISTRIBUTED NUMERICAL CONTROL

Because of an unexpected signal noise within the network or an unpredictedfault with personal computers (PCs), many problems emerge in the implementation of distributednumerical control (DNC) with PCs-based network. To solve the problems, an industrial solution ofinvolving the field-bus technology in DNC communicating area is provided. A kind of advancedField-bus, named controller area network (CAN), is originally developed to support cheap and rathersimple automotive applications. However, because of its good performance and low cost, it is alsobeing considered in automated-manufacturing and process control environments to interconnectintelligent devices, such as modem sensors and actuators. Recently it creates a new role for CANBusin DNC that brings new thinking to DNC. CAN is used as the network platform for connecting machinetools to share information with each other reliably. Additionally, thanks to also applying of'plug-in' technology and a special interface of hardware, this solution exhibits some highcompatibility with different pedigree numerical control (NC) systems, such as Fanuc, Siemens,Cincinnati and so on. In order to improve CANBus for DNC application, a communicating competitionmodel of the basic CAN protocol, called CC model, is then highlighted. This model is able to satisfythe requirements that different machine tools share the communicating bandwidth fairly when theyrun concurrently. Finally the novel view of the latest advancement in CANBus-based DNC incombination with the manufacturing paradigm is also presented.

Neural Dynamics for Distributed Collaborative Control of Manipulators With Time Delays

Time-delay phenomena extensively exist in practical systems,e.g.,multi-agent systems,bringing negative impacts on their stabilities.This work analyzes a collaborative control problem of redundant manipulators with time delays and proposes a time-delayed and distributed neural dynamics scheme.Under assumptions that the network topology is fixed and connected and the existing maximal time delay is no more than a threshold value,it is proved that all manipulators in the distributed network are able to reach a desired motion.The proposed distributed scheme with time delays considered is converted into a time-variant optimization problem,and a neural dynamics solver is designed to solve it online.Then,the proposed neural dynamics solver is proved to be stable,convergent and robust.Additionally,the allowable threshold value of time delay that ensures the proposed scheme’s stability is calculated.Illustrative examples and comparisons are provided to intuitively prove the validity of the proposed neural dynamics scheme and solver.

Direct Yaw Moment Control for Distributed Drive Electric Vehicle Handling Performance Improvement

For a distributed drive electric vehicle（DDEV） driven by four in-wheel motors, advanced vehicle dynamic control methods can be realized easily because motors can be controlled independently, quickly and precisely. And direct yaw-moment control（DYC） has been widely studied and applied to vehicle stability control. Good vehicle handling performance： quick yaw rate transient response, small overshoot, high steady yaw rate gain, etc, is required by drivers under normal conditions, which is less concerned, however. Based on the hierarchical control methodology, a novel control system using direct yaw moment control for improving handling performance of a distributed drive electric vehicle especially under normal driving conditions has been proposed. The upper-loop control system consists of two parts： a state feedback controller, which aims to realize the ideal transient response of yaw rate, with a vehicle sideslip angle observer; and a steering wheel angle feedforward controller designed to achieve a desired yaw rate steady gain. Under the restriction of the effect of poles and zeros in the closed-loop transfer function on the system response and the capacity of in-wheel motors, the integrated time and absolute error（ITAE） function is utilized as the cost function in the optimal control to calculate the ideal eigen frequency and damper coefficient of the system and obtain optimal feedback matrix and feedforward matrix. Simulations and experiments with a DDEV under multiple maneuvers are carried out and show the effectiveness of the proposed method： yaw rate rising time is reduced, steady yaw rate gain is increased, vehicle steering characteristic is close to neutral steer and drivers burdens are also reduced. The control system improves vehicle handling performance under normal conditions in both transient and steady response. State feedback control instead of model following control is introduced in the control system so that the sense of control intervention to drivers is relieved.

Distributed model predictive control for multiagent systems with improved consistency

This paper addresses an improved distributed model predictive control （DMPC） scheme for multiagent systems with an attempt to improving its consistency. The deviation between what an agent is actually doing and what its neighbors believe that agent is doing is penalized in the cost function of each agent. At each sampling instant the compatibility constraint of each agent is set tighter than the previous sampling instant. Like the traditional approach, the performance cost is utilized as the Lyapunov function to prove closed-looped stability. The closed-loop stability is guaranteed if the weight matrix for deviation in the cost function are sufficiently large. The proposed distributed control scheme is formulated as quadratic programming with quadratic constraints. A numerical example is given to illustrate the effectiveness of the proposed scheme.

Development of Distributed Control System for Neutral Beam Injector on EAST

A distributed control system of Neutral Beam Injector （NBI） on the Experimental Advanced Superconducting Tokamak （EAST-NBI） is briefly presented in this paper. The control system is developed in accordance with the experimental operational characteristics of the EAST- NBI. The NBI control system （NBICS）, which is based on the computer network technologies and classified according to the control levels, consists of three levels： a remote monitoring layer, a server control layer, and a field control layer. The 3-layer architecture is capable of extending the system functions and upgrading devices. The timing system provides the reference clock of the synchronization and interlock for the EAST-NBI system. An interlock system ensures the safety of the experiment operators and field devices. Both of the ion sources of the beamline are designed to operate independently. This lays an important foundation for developing a control system for the second beamline on EAST. Experimental results demonstrate that the NBICS meets functional requirements of the EAST-NBI control, and makes experimental operations visual and automatic.

Distributed Control of Nonholonomic Robots Without Global Position Measurements Subject to Unknown Slippage Constraints

This paper studies the fully distributed formation control problem of multi-robot systems without global position measurements subject to unknown longitudinal slippage constraints.It is difficult for robots to obtain accurate and stable global position information in many cases,such as when indoors,tunnels and any other environments where GPS(global positioning system)is denied,thus it is meaningful to overcome the dependence on global position information.Additionally,unknown slippage,which is hard to avoid for wheeled robots due to the existence of ice,sand,or muddy roads,can not only affect the control performance of wheeled robot,but also limits the application scene of wheeled mobile robots.To solve both problems,a fully distributed finite time state observer which does not require any global position information is proposed,such that each follower robot can estimate the leader’s states within finite time.The distributed adaptive controllers are further designed for each follower robot such that the desired formation can be achieved while overcoming the effect of unknown slippage.Finally,the effectiveness of the proposed observer and control laws are verified by simulation results.

DISOPE distributed model predictive control of cascade systems with network communication

A novel distributed model predictive control scheme based on dynamic integrated system optimization and parameter estimation （DISOPE） was proposed for nonlinear cascade systems under network environment. Under the distributed control structure, online optimization of the cascade system was composed of several cascaded agents that can cooperate and exchange information via network communication. By iterating on modified distributed linear optimal control problems on the basis of estimating parameters at every iteration the correct optimal control action of the nonlinear model predictive control problem of the cascade system could be obtained, assuming that the algorithm was convergent. This approach avoids solving the complex nonlinear optimization problem and significantly reduces the computational burden. The simulation results of the fossil fuel power unit are illustrated to verify the effectiveness and practicability of the proposed algorithm.

Aperiodic Sampled-Data Control of Distributed Networked Control Systems Under Stochastic Cyber-Attacks

This paper examines the stabilization problem of a distributed networked control system under the effect of cyberattacks by employing a hybrid aperiodic triggering mechanism.The cyber-attack considered in the paper is a stochastic deception attack at the sensor-controller end. The probability of the occurrence of attack on a subsystem is represented using a random variable. A decentralized hybrid sampled-data strategy is introduced to save energy consumption and reduce the transmission load of the network. In the proposed decentralized strategy, each subsystem can decide independently whether its state should be transmitted to the controller or not. The scheme of the hybrid triggering mechanism for each subsystem composed of two stages: In the first stage, the next sampling instant is computed using a self-triggering strategy. Subsequently, in the second stage, an event-triggering condition is checked at these sampling instants and the control signal is computed only if the event-triggering condition is violated. The self-triggering condition used in the first stage is dependent on the selection of eventtriggering condition of the second stage. Finally, a comparison of the proposed approach with other triggering mechanisms existing in the literature is presented in terms of the sampling instants,transmission frequency and performance measures through simulation examples.