Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Hybrid Dynamic Variables-Dependent Event-Triggered Fuzzy Model Predictive Control

This article focuses on dynamic event-triggered mechanism(DETM)-based model predictive control(MPC) for T-S fuzzy systems.A hybrid dynamic variables-dependent DETM is carefully devised,which includes a multiplicative dynamic variable and an additive dynamic variable.The addressed DETM-based fuzzy MPC issue is described as a “min-max” optimization problem(OP).To facilitate the co-design of the MPC controller and the weighting matrix of the DETM,an auxiliary OP is proposed based on a new Lyapunov function and a new robust positive invariant(RPI) set that contain the membership functions and the hybrid dynamic variables.A dynamic event-triggered fuzzy MPC algorithm is developed accordingly,whose recursive feasibility is analysed by employing the RPI set.With the designed controller,the involved fuzzy system is ensured to be asymptotically stable.Two examples show that the new DETM and DETM-based MPC algorithm have the advantages of reducing resource consumption while yielding the anticipated performance.

Finite-Time Stabilization for Constrained Discrete-time Systems by Using Model Predictive Control

In this paper, a model predictive control(MPC)framework is proposed for finite-time stabilization of linear and nonlinear discrete-time systems subject to state and control constraints. The proposed MPC framework guarantees the finite-time convergence property by assigning the control horizon equal to the dimension of the overall system, and only penalizing the terminal cost in the optimization, where the stage costs are not penalized explicitly. A terminal inequality constraint is added to guarantee the feasibility and stability of the closed-loop system.Initial feasibility can be improved via augmentation. The finite-time convergence of the proposed MPC is proved theoretically,and is supported by simulation examples.

Model Predictive Control for Cascaded H-Bridge PV Inverter with Capacitor Voltage Balance

We designed an improved direct-current capacitor voltage balancing control model predictive control(MPC)for single-phase cascaded H-bridge multilevel photovoltaic(PV)inverters.Compared with conventional voltage balanc-ing control methods,the method proposed could make the PV strings of each submodule operate at their maximum power point by independent capacitor voltage control.Besides,the predicted and reference value of the grid-connected current was obtained according to the maximum power output of the maximum power point tracking.A cost function was con-structed to achieve the high-precision grid-connected control of the CHB inverter.Finally,the effectiveness of the proposed control method was verified through a semi-physical simulation platform with three submodules.

Distributed Economic MPC for Synergetic Regulation of the Voltage of an Island DC Micro-Grid

In this paper,distributed model predictive control(DMPC) for island DC micro-grids(MG) with wind/photovoltaic(PV)/battery power is proposed,which coordinates all distributed generations(DG) to stabilize the bus voltage together with the insurance of having computational efficiency under a real-time requirement.Based on the feedback of the bus voltage,the deviation of the current is dispatched to each DG according to cost over the prediction horizon.Moreover,to avoid the excessive fluctuation of the battery power,both the discharge-charge switching times and costs are considered in the model predictive control(MPC) optimization problems.A Lyapunov constraint with a time-varying steady-state is designed in each local MPC to guarantee the stabilization of the entire system.The voltage stabilization of the MG is achieved by this strategy with the cooperation of DGs.The numeric results of applying the proposed method to a MG of the Shanghai Power Supply Company shows the effectiveness of the distributed economic MPC.

Machine Learning Accelerated Real-Time Model Predictive Control for Power Systems

This paper presents a machine-learning-based speedup strategy for real-time implementation of model-predictive-control(MPC)in emergency voltage stabilization of power systems.Despite success in various applications,real-time implementation of MPC in power systems has not been successful due to the online control computation time required for large-sized complex systems,and in power systems,the computation time exceeds the available decision time used in practice by a large extent.This long-standing problem is addressed here by developing a novel MPC-based framework that i)computes an optimal strategy for nominal loads in an offline setting and adapts it for real-time scenarios by successive online control corrections at each control instant utilizing the latest measurements,and ii)employs a machine-learning based approach for the prediction of voltage trajectory and its sensitivity to control inputs,thereby accelerating the overall control computation by multiple times.Additionally,a realistic control coordination scheme among static var compensators(SVC),load-shedding(LS),and load tap-changers(LTC)is presented that incorporates the practical delayed actions of the LTCs.The performance of the proposed scheme is validated for IEEE 9-bus and 39-bus systems,with±20%variations in nominal loading conditions together with contingencies.We show that our proposed methodology speeds up the online computation by 20-fold,bringing it down to a practically feasible value(fraction of a second),making the MPC real-time and feasible for power system control for the first time.

Resilience Against Replay Attacks:A Distributed Model Predictive Control Scheme for Networked Multi-Agent Systems

In this paper,a resilient distributed control scheme against replay attacks for multi-agent networked systems subject to input and state constraints is proposed.The methodological starting point relies on a smart use of predictive arguments with a twofold aim:1)Promptly detect malicious agent behaviors affecting normal system operations;2)Apply specific control actions,based on predictive ideas,for mitigating as much as possible undesirable domino effects resulting from adversary operations.Specifically,the multi-agent system is topologically described by a leader-follower digraph characterized by a unique leader and set-theoretic receding horizon control ideas are exploited to develop a distributed algorithm capable to instantaneously recognize the attacked agent.Finally,numerical simulations are carried out to show benefits and effectiveness of the proposed approach.

Model-based Predictive Control for Spatially-distributed Systems Using Dimensional Reduction Models

In this paper, a low-dimensional multiple-input and multiple-output （MIMO） model predictive control （MPC） configuration is presented for partial differential equation （PDE） unknown spatially-distributed systems （SDSs）. First, the dimension reduction with principal component analysis （PCA） is used to transform the high-dimensional spatio-temporal data into a low-dimensional time domain. The MPC strategy is proposed based on the online correction low-dimensional models, where the state of the system at a previous time is used to correct the output of low-dimensional models. Sufficient conditions for closed-loop stability are presented and proven. Simulations demonstrate the accuracy and efficiency of the proposed methodologies.

Distributed model predictive control for multiagent systems with improved consistency

This paper addresses an improved distributed model predictive control （DMPC） scheme for multiagent systems with an attempt to improving its consistency. The deviation between what an agent is actually doing and what its neighbors believe that agent is doing is penalized in the cost function of each agent. At each sampling instant the compatibility constraint of each agent is set tighter than the previous sampling instant. Like the traditional approach, the performance cost is utilized as the Lyapunov function to prove closed-looped stability. The closed-loop stability is guaranteed if the weight matrix for deviation in the cost function are sufficiently large. The proposed distributed control scheme is formulated as quadratic programming with quadratic constraints. A numerical example is given to illustrate the effectiveness of the proposed scheme.

DISOPE distributed model predictive control of cascade systems with network communication

A novel distributed model predictive control scheme based on dynamic integrated system optimization and parameter estimation （DISOPE） was proposed for nonlinear cascade systems under network environment. Under the distributed control structure, online optimization of the cascade system was composed of several cascaded agents that can cooperate and exchange information via network communication. By iterating on modified distributed linear optimal control problems on the basis of estimating parameters at every iteration the correct optimal control action of the nonlinear model predictive control problem of the cascade system could be obtained, assuming that the algorithm was convergent. This approach avoids solving the complex nonlinear optimization problem and significantly reduces the computational burden. The simulation results of the fossil fuel power unit are illustrated to verify the effectiveness and practicability of the proposed algorithm.

Distributed Model Predictive Load Frequency Control of Multi-area Power System with DFIGs

Reliable load frequency control LFC is crucial to the operation and design of modern electric power systems. Considering the LFC problem of a four-Area interconnected power system with wind turbines, this paper presents a distributed model predictive control DMPC based on coordination scheme. The proposed algorithm solves a series of local optimization problems to minimize a performance objective for each control area. The generation rate constraints GRCs, load disturbance changes, and the wind speed constraints are considered. Furthermore, the DMPC algorithm may reduce the impact of the randomness and intermittence of wind turbine effectively. A performance comparison between the proposed controller with and without the participation of the wind turbines is carried out. Analysis and simulation results show possible improvements on closed-loop performance, and computational burden with the physical constraints. © 2014 Chinese Association of Automation.

Distributed Model Predictive Control for Networked Plant-wide Systems With Neighborhood Cooperation

For large-scale networked plant-wide systems composed by physically(or geographically) divided subsystems, only limited information is available for local controllers on account of region and communication restrictions. Concerning the optimal control problem of such subsystems, a neighbor-based distributed model predictive control(NDMPC) strategy is presented to improve the global system performance. In this scheme, the performance index of local subsystems and that of its neighbors are minimized together in the determination of the optimal control input, which makes the local control decision also beneficial to its neighboring subsystems and further contributes to improving the convergence and control performance of overall system.The stability of the closed-loop system is proved. Moreover, the parameter designing method for distributed synthesis is provided.Finally, the simulation results illustrate the main characteristics and effectiveness of the proposed control scheme.

Distributed Model Predictive Control with Actuator Saturation for Markovian Jump Linear System

This paper is concerned with the distributed model predictive control (MPC) problem for a class of discrete-time Markovian jump linear systems (MJLSs) subject to actuator saturation and polytopic uncertainty in system matrices. The global system is decomposed into several subsystems which coordinate with each other. A set of distributed controllers is designed by solving a min-max optimization problem in terms of the solutions of linear matrix inequalities (LMIs). An iterative algorithm is developed to achieve the online computation. Finally, a simulation example is employed to show the effectiveness of the proposed algorithm. © 2014 Chinese Association of Automation.

The Design of Output Feedback Distributed Model Predictive Controller for a Class of Nonlinear Systems

For a class of nonlinear systems whose states are immeasurable, when the outputs of the system are sampled asynchronously, by introducing a state observer, an output feedback distributed model predictive control algorithm is proposed. It is proved that the errors of estimated states and the actual system's states are bounded. And it is guaranteed that the estimated states of the closed-loop system are ultimately bounded in a region containing the origin. As a result, the states of the actual system are ultimately bounded. A simulation example verifies the effectiveness of the proposed distributed control method.

Distributed model predictive control based on adaptive sampling mechanism

In this work,an adaptive sampling control strategy for distributed predictive control is proposed.According to the proposed method,the sampling rate of each subsystem of the accused object is determined based on the periodic detection of its dynamic behavior and calculations made using a correlation function.Then,the optimal sampling interval within the period is obtained and sent to the corresponding sub-prediction controller,and the sampling interval of the controller is changed accordingly before the next sampling period begins.In the next control period,the adaptive sampling mechanism recalculates the sampling rate of each subsystem’s measurable output variable according to both the abovementioned method and the change in the dynamic behavior of the entire system,and this process is repeated.Such an adaptive sampling interval selection based on an autocorrelation function that measures dynamic behavior can dynamically optimize the selection of sampling rate according to the real-time change in the dynamic behavior of the controlled object.It can also accurately capture dynamic changes,meaning that each sub-prediction controller can more accurately calculate the optimal control quantity at the next moment,significantly improving the performance of distributed model predictive control(DMPC).A comparison demonstrates that the proposed adaptive sampling DMPC algorithm has better tracking performance than the traditional DMPC algorithm.

Networked Cooperative Distributed Model Predictive Control Based on State Observer

Although distributed model predictive control has caused significant attention and received many good results, the results are mostly under the assumption that the system states can be observed. However, the states are difficult to be observed in practice. In this paper, a novel distributed model predictive control is proposed based on state observer for a kind of linear discrete-time systems where states are not measured. Firstly, an output feedback control law is designed based on Lyapunov function and state observer. And the stability domain is described. Furthermore, the stability domain as a terminal constraint is added into the constraint conditions of the algorithm to make systems stable outside the stability domain. The simulation results show the effectiveness of the proposed method.

智能汽车轨迹跟踪MPC-RBF-SMC协同控制策略研究

针对自动驾驶车辆行驶过程中模型失配以及外部环境干扰导致车辆轨迹跟踪环节精确性不高的问题,提出了一种结合车辆运动学模型预测控制(MPC)、径向基(RBF)神经网络和滑模控制(SMC)的轨迹跟踪控制策略。通过建立车辆运动学MPC模型计算当前状态车辆期望横摆角速度,并将其与实际横摆角速度的偏差输入RBF-SMC控制器,利用RBF快速逼近非线性模型的特点,结合滑模控制输出前轮转角,实现车辆的横向轨迹跟踪控制。仿真结果表明,与传统的控制器相比,该方法轨迹跟踪精度显著提高,并在不同行驶工况下表现出较好的鲁棒性。

基于自适应采样周期和预测时域MPC的车辆路径跟踪控制

为解决自动驾驶车辆在低附着路面路径跟踪控制精度较低的问题,设计了一种自适应采样周期和预测时域MPC控制器。首先,结合车辆动力学模型和MPC算法设计了MPC控制器,并加入轮胎侧偏角约束;然后,分析控制器的采样周期和预测时域对控制效果的影响,提出一种综合考虑采样周期和预测时域的自适应控制策略,通过车辆前轮转向角更新采样周期,通过车速更新预测时域;最后,使用Carsim和Matlab/Simulink联合仿真平台,在低附着路面的不同车速条件下进行仿真实验。结果表明,当车速为25 km/h和45 km/h时,相较于固定控制参数MPC控制器,自适应采样周期和预测时域MPC控制器的最大横向误差分别减小140.2mm和40.8mm,其在不同车速下的路径跟踪控制精度均更高,横摆角速度和质心侧偏角均在合理范围内,车辆稳定性较好,证明所提路径跟踪控制器在低附着路面具有较高的控制精度和可行性。

MPC-based Motion Planning and Control Enables Smarter and Safer Autonomous Marine Vehicles:Perspectives and a Tutorial Survey

Autonomous marine vehicles(AMVs)have received considerable attention in the past few decades,mainly because they play essential roles in broad marine applications such as environmental monitoring and resource exploration.Recent advances in the field of communication technologies,perception capability,computational power and advanced optimization algorithms have stimulated new interest in the development of AMVs.In order to deploy the constrained AMVs in the complex dynamic maritime environment,it is crucial to enhance the guidance and control capabilities through effective and practical planning,and control algorithms.Model predictive control(MPC)has been exceptionally successful in different fields due to its ability to systematically handle constraints while optimizing control performance.This paper aims to provide a review of recent progress in the context of motion planning and control for AMVs from the perceptive of MPC.Finally,future research trends and directions in this substantial research area of AMVs are highlighted.