Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Distributed Economic MPC for Synergetic Regulation of the Voltage of an Island DC Micro-Grid

In this paper,distributed model predictive control(DMPC) for island DC micro-grids(MG) with wind/photovoltaic(PV)/battery power is proposed,which coordinates all distributed generations(DG) to stabilize the bus voltage together with the insurance of having computational efficiency under a real-time requirement.Based on the feedback of the bus voltage,the deviation of the current is dispatched to each DG according to cost over the prediction horizon.Moreover,to avoid the excessive fluctuation of the battery power,both the discharge-charge switching times and costs are considered in the model predictive control(MPC) optimization problems.A Lyapunov constraint with a time-varying steady-state is designed in each local MPC to guarantee the stabilization of the entire system.The voltage stabilization of the MG is achieved by this strategy with the cooperation of DGs.The numeric results of applying the proposed method to a MG of the Shanghai Power Supply Company shows the effectiveness of the distributed economic MPC.

Virtually coupled train set control subject to space-time separation:A distributed economic MPC approach with emergency braking configuration

The emerging virtual coupling technology aims to operate multiple train units in a Virtually Coupled Train Set(VCTS)at a minimal but safe distance.To guarantee collision avoidance,the safety distance should be calculated using the state-of-the-art space-time separation principle that separates the Emergency Braking(EB)trajectories of two successive units during the whole EB process.In this case,the minimal safety distance is usually numerically calculated without an analytic formulation.Thus,the constrained VCTS control problem is hard to address with space-time separation,which is still a gap in the existing literature.To solve this problem,we propose a Distributed Economic Model Predictive Control(DEMPC)approach with computation efficiency and theoretical guarantee.Specifically,to alleviate the computation burden,we transform implicit safety constraints into explicitly linear ones,such that the optimal control problem in DEMPC is a quadratic programming problem that can be solved efficiently.For theoretical analysis,sufficient conditions are derived to guarantee the recursive feasibility and stability of DEMPC,employing compatibility constraints,tube techniques and terminal ingredient tuning.Moreover,we extend our approach with globally optimal and distributed online EB configuration methods to shorten the minimal distance among VCTS.Finally,experimental results demonstrate the performance and advantages of the proposed approaches.

Differential flatness-based distributed control of underactuated robot swarms

This paper proposes a distributed control method based on the differential flatness(DF) property of robot swarms. The swarm DF mapping is established for underactuated differentially flat dynamics, according to the control objective. The DF mapping refers to the fact that the system state and input of each robot can be derived algebraically from the flat outputs of the leaders and the cooperative errors and their finite order derivatives. Based on the proposed swarm DF mapping, a distributed controller is designed. The distributed implementation of swarm DF mapping is achieved through observer design. The effectiveness of the proposed method is validated through a numerical simulation of quadrotor swarm synchronization.

STABILITY ANALYSIS AND COOPERATIVE CONTROL OF DISTRIBUTED MULTI-AGENT SYSTEM WITH SAMPLED COMMUNICATION

The cooperative control and stability analysis problems for the multi-agent system with sampled com- munication are investigated. Distributed state feedback controllers are adopted for the cooperation of networked agents. A theorem in the form of linear matrix inequalities（LMI） is derived to analyze the system stability. An- other theorem in the form of optimization problem subject to LMI constraints is proposed to design the controller, and then the algorithm is presented. The simulation results verify the validity and the effectiveness of the pro- posed approach.

Optimal Cooperative Secondary Control for Islanded DC Microgrids via a Fully Actuated Approach

DC-DC converter-based multi-bus DC microgrids(MGs) in series have received much attention, where the conflict between voltage recovery and current balancing has been a hot topic. The lack of models that accurately portray the electrical characteristics of actual MGs while is controller design-friendly has kept the issue active. To this end, this paper establishes a large-signal model containing the comprehensive dynamical behavior of the DC MGs based on the theory of high-order fully actuated systems, and proposes distributed optimal control based on this. The proposed secondary control method can achieve the two goals of voltage recovery and current sharing for multi-bus DC MGs. Additionally, the simple structure of the proposed approach is similar to one based on droop control, which allows this control technique to be easily implemented in a variety of modern microgrids with different configurations. In contrast to existing studies, the process of controller design in this paper is closely tied to the actual dynamics of the MGs. It is a prominent feature that enables engineers to customize the performance metrics of the system. In addition, the analysis of the stability of the closed-loop DC microgrid system, as well as the optimality and consensus of current sharing are given. Finally, a scaled-down solar and battery-based microgrid prototype with maximum power point tracking controller is developed in the laboratory to experimentally test the efficacy of the proposed control method.

A PI+R Control Scheme Based on Multi-Agent Systems for Economic Dispatch in Isolated BESSs

Battery energy storage systems(BESSs)are widely used in smart grids.However,power consumed by inner impedance and the capacity degradation of each battery unit become particularly severe,which has resulted in an increase in operating costs.The general economic dispatch(ED)algorithm based on marginal cost(MC)consensus is usually a proportional(P)controller,which encounters the defects of slow convergence speed and low control accuracy.In order to solve the distributed ED problem of the isolated BESS network with excellent dynamic and steady-state performance,we attempt to design a proportional integral(PI)controller with a reset mechanism(PI+R)to asymptotically promote MC consensus and total power mismatch towards 0 in this paper.To be frank,the integral term in the PI controller is reset to 0 at an appropriate time when the proportional term undergoes a zero crossing,which accelerates convergence,improves control accuracy,and avoids overshoot.The eigenvalues of the system under a PI+R controller is well analyzed,ensuring the regularity of the system and enabling the reset mechanism.To ensure supply and demand balance within the isolated BESSs,a centralized reset mechanism is introduced,so that the controller is distributed in a flow set and centralized in a jump set.To cope with Zeno behavior and input delay,a dwell time that the system resides in a flow set is given.Based on this,the system with input delays can be reduced to a time-delay free system.Considering the capacity limitation of the battery,a modified MC scheme with PI+R controller is designed.The correctness of the designed scheme is verified through relevant simulations.

Distributed Power Control for Energy Conservation in Hybrid Cellular Network with Device-to-Device Communication

Device-to-Device(D2D) communication has been proposed as a promising implementation of green communication to benefit the existed cellular network.In order to limit cross-tier interference while explore the gain of short-range communication,we devise a series of distributed power control(DPC) schemes for energy conservation(EC)and enhancement of radio resource utilization in the hybrid system.Firstly,a constrained opportunistic power control model is built up to take advantage of the interference avoidance methodology in the presence of service requirement and power constraint.Then,biasing scheme and admission control are added to evade ineffective power consumption and maintain the feasibility of the system.Upon feasibility,a non-cooperative game is further formulated to exploit the profit in EC with minor influence on spectral efficiency(SE).The convergence of the DPC schemes is validated and their performance is confirmed via simulation results.

Distributed Secondary Control and Optimal Power Sharing in Microgrids

We address the control problem of microgrids and present a fully distributed control system which consists of primary controller, secondary controller, and optimal active power sharing controller. Different from the existing control structure in microgrids, all these controllers are implemented as local controllers at each distributed generator. Thus, the requirement for a central controller is obviated. The performance analysis of the proposed control systems is provided, and the finite-time convergence properties for distributed secondary frequency and voltage controllers are achieved. Moreover, the distributed control system possesses the optimal active power sharing property. In the end, a microgrid test system is investigated to validate the effectiveness of the proposed control strategies. © 2014 Chinese Association of Automation.

Distributed Adaptive Cooperative Tracking of Uncertain Nonlinear Fractional-order Multi-agent Systems

In this paper, the leader-following tracking problem of fractional-order multi-agent systems is addressed. The dynamics of each agent may be heterogeneous and has unknown nonlinearities. By assumptions that the interaction topology is undirected and connected and the unknown nonlinear uncertain dynamics can be parameterized by a neural network, an adaptive learning law is proposed to deal with unknown nonlinear dynamics, based on which a kind of cooperative tracking protocols are constructed. The feedback gain matrix is obtained to solve an algebraic Riccati equation. To construct the fully distributed cooperative tracking protocols, the adaptive law is also adopted to adjust the coupling weight. With the developed control laws,we can prove that all signals in the closed-loop systems are guaranteed to be uniformly ultimately bounded. Finally, a simple simulation example is provided to illustrate the established result.

Neural Dynamics for Distributed Collaborative Control of Manipulators With Time Delays

Time-delay phenomena extensively exist in practical systems,e.g.,multi-agent systems,bringing negative impacts on their stabilities.This work analyzes a collaborative control problem of redundant manipulators with time delays and proposes a time-delayed and distributed neural dynamics scheme.Under assumptions that the network topology is fixed and connected and the existing maximal time delay is no more than a threshold value,it is proved that all manipulators in the distributed network are able to reach a desired motion.The proposed distributed scheme with time delays considered is converted into a time-variant optimization problem,and a neural dynamics solver is designed to solve it online.Then,the proposed neural dynamics solver is proved to be stable,convergent and robust.Additionally,the allowable threshold value of time delay that ensures the proposed scheme’s stability is calculated.Illustrative examples and comparisons are provided to intuitively prove the validity of the proposed neural dynamics scheme and solver.

Distributed Periodic Event-Triggered Optimal Control of DC Microgrids Based on Virtual Incremental Cost

This article presents a distributed periodic eventtriggered(PET)optimal control scheme to achieve generation cost minimization and average bus voltage regulation in DC microgrids.In order to accommodate the generation constraints of the distributed generators(DGs),a virtual incremental cost is firstly designed,based on which an optimality condition is derived to facilitate the control design.To meet the discrete-time(DT)nature of modern control systems,the optimal controller is directly developed in the DT domain.Afterward,to reduce the communication requirement among the controllers,a distributed event-triggered mechanism is introduced for the DT optimal controller.The event-triggered condition is detected periodically and therefore naturally avoids the Zeno phenomenon.The closed-loop system stability is proved by the Lyapunov synthesis for switched systems.The generation cost minimization and average bus voltage regulation are obtained at the equilibrium point.Finally,switch-level microgrid simulations validate the performance of the proposed optimal controller.

Distributed Secondary Control of AC Microgrids With External Disturbances and Directed Communication Topologies:A Full-Order Sliding-Mode Approach

This paper addresses the problem of distributed secondary control for islanded AC microgrids with external disturbances.By using a full-order sliding-mode(FOSM)approach,voltage regulation and frequency restoration are achieved in finite time.For voltage regulation,a distributed observer is proposed for each distributed generator(DG)to estimate a reference voltage level.Different from some conventional observers,the reference voltage level in this paper is accurately estimated under directed communication topologies.Based on the observer,a new nonlinear controller is designed in a backstepping manner such that an FOSM surface is reached in finite time.On the surface,the voltages of DGs are regulated to the reference level in finite time.For frequency restoration,a distributed controller is further proposed such that a constructed FOSM surface is reached in finite time,on which the frequencies of DGs are restored to a reference level in finite time under directed communication topologies.Finally,case studies on a modified IEEE 37-bus test system are conducted to demonstrate the effectiveness,the robustness against load changes,and the plug-and-play capability of the proposed controllers.

Backstepping-based distributed coordinated tracking formultiple uncertain Euler-Lagrange systems

Based on the idea of backstepping design, distributedcoordinated tracking problems under directed topology are discussedfor multiple Euler-Lagrange （EL） systems. The dynamicleader case is considered. First, with the parameter-linearity property,a distributed coordinated adaptive control scheme is proposedfor EL systems in the presence of parametric uncertainties.Then, subject to nonlinear uncertainties and external disturbances,an improved adaptive control algorithm is developed by usingneural-network （NN） approximation of nonlinear functions. Bothproposed algorithms can make tracking errors for each followerultimately bounded. The closed-loop systems are investigated byusing the combination of graph theory, Lyapunov theory, and BarbalatLemma. Numerical examples and comparisons with othermethods are provided to show the effectiveness of the proposedcontrol strategies.

Adaptive distributed formation maintenance for multiple UAVs:Exploiting proximity behavior observations

The formation maintenance of multiple unmanned aerial vehicles(UAVs)based on proximity behavior is explored in this study.Individual decision-making is conducted according to the expected UAV formation structure and the position,velocity,and attitude information of other UAVs in the azimuth area.This resolves problems wherein nodes are necessarily strongly connected and communication is strictly consistent under the traditional distributed formation control method.An adaptive distributed formation flight strategy is established for multiple UAVs by exploiting proximity behavior observations,which remedies the poor flexibility in distributed formation.This technique ensures consistent position and attitude among UAVs.In the proposed method,the azimuth area relative to the UAV itself is established to capture the state information of proximal UAVs.The dependency degree factor is introduced to state update equation based on proximity behavior.Finally,the formation position,speed,and attitude errors are used to form an adaptive dynamic adjustment strategy.Simulations are conducted to demonstrate the effectiveness and robustness of the theoretical results,thus validating the effectiveness of the proposed method.

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming(ADP) where only one critic neural network(NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness(UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.

DCS devices based non-linear process control system design for plants with distributed time-delay using particle filter

Remote control process system with distributed time-delay has attracted much attention in different fields.In this paper,non-linear remote control of a single tank process system with wireless network is considered.To deal with the distributed time-delay in a large-scale plant,the time-delay compensation controller based on DCS devices is designed by using operator theory and particle filter.Distributed control system(DCS)device is developed to monitor and control from the central monitoring room to each process.The particle filter is a probabilistic method to estimate unobservable information from observable information.First,remote control system and experimental equipment are introduced.Second,control system based on an operator theory is designed.Then,process system with distributed time-delay using particle filter is carried out.Finally,the actual experiment is conducted by using the proposed time-delay compensation controller.When estimating with the proposed method,the result is close to the case in which the distributed time-delay does not exist.The effectiveness of the proposed control system is confirmed by experiment results.

NOVEL STRUCTURE FOR DISTRIBUTED NUMERICAL CONTROL

Because of an unexpected signal noise within the network or an unpredictedfault with personal computers (PCs), many problems emerge in the implementation of distributednumerical control (DNC) with PCs-based network. To solve the problems, an industrial solution ofinvolving the field-bus technology in DNC communicating area is provided. A kind of advancedField-bus, named controller area network (CAN), is originally developed to support cheap and rathersimple automotive applications. However, because of its good performance and low cost, it is alsobeing considered in automated-manufacturing and process control environments to interconnectintelligent devices, such as modem sensors and actuators. Recently it creates a new role for CANBusin DNC that brings new thinking to DNC. CAN is used as the network platform for connecting machinetools to share information with each other reliably. Additionally, thanks to also applying of'plug-in' technology and a special interface of hardware, this solution exhibits some highcompatibility with different pedigree numerical control (NC) systems, such as Fanuc, Siemens,Cincinnati and so on. In order to improve CANBus for DNC application, a communicating competitionmodel of the basic CAN protocol, called CC model, is then highlighted. This model is able to satisfythe requirements that different machine tools share the communicating bandwidth fairly when theyrun concurrently. Finally the novel view of the latest advancement in CANBus-based DNC incombination with the manufacturing paradigm is also presented.

Resilience Against Replay Attacks:A Distributed Model Predictive Control Scheme for Networked Multi-Agent Systems

In this paper,a resilient distributed control scheme against replay attacks for multi-agent networked systems subject to input and state constraints is proposed.The methodological starting point relies on a smart use of predictive arguments with a twofold aim:1)Promptly detect malicious agent behaviors affecting normal system operations;2)Apply specific control actions,based on predictive ideas,for mitigating as much as possible undesirable domino effects resulting from adversary operations.Specifically,the multi-agent system is topologically described by a leader-follower digraph characterized by a unique leader and set-theoretic receding horizon control ideas are exploited to develop a distributed algorithm capable to instantaneously recognize the attacked agent.Finally,numerical simulations are carried out to show benefits and effectiveness of the proposed approach.

Chattering analysis for discrete sliding mode control ofdistributed control systems

The chattering characteristic of sliding mode control isanalyzed when it is applied in distributed control systems （DCSs）.For a DCS with random time delay and packet dropout, a discreteswitching system model with time varying sampling period isconstructed based on the time delay system method. The reachinglaw based sliding mode controller is applied in the proposedsystem. The exponential stability condition in the form of linearmatrix inequality is figured out based on the multi-Lyaponov functionmethod. Then, the chattering characteristic is analyzed for theswitching system, and a chattering region related with time varyingsampling period and external disturbance is proposed. Finally, numericalexamples are given to illustrate the validity of the analysisresult.