Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Distributed Model Predictive Load Frequency Control of Multi-area Power System with DFIGs

Reliable load frequency control(LFC) is crucial to the operation and design of modern electric power systems. Considering the LFC problem of a four-area interconnected power system with wind turbines, this paper presents a distributed model predictive control(DMPC) based on coordination scheme.The proposed algorithm solves a series of local optimization problems to minimize a performance objective for each control area. The generation rate constraints(GRCs), load disturbance changes, and the wind speed constraints are considered. Furthermore, the DMPC algorithm may reduce the impact of the randomness and intermittence of wind turbine effectively. A performance comparison between the proposed controller with and without the participation of the wind turbines is carried out. Analysis and simulation results show possible improvements on closed–loop performance, and computational burden with the physical constraints.

Resilience Against Replay Attacks:A Distributed Model Predictive Control Scheme for Networked Multi-Agent Systems

In this paper,a resilient distributed control scheme against replay attacks for multi-agent networked systems subject to input and state constraints is proposed.The methodological starting point relies on a smart use of predictive arguments with a twofold aim:1)Promptly detect malicious agent behaviors affecting normal system operations;2)Apply specific control actions,based on predictive ideas,for mitigating as much as possible undesirable domino effects resulting from adversary operations.Specifically,the multi-agent system is topologically described by a leader-follower digraph characterized by a unique leader and set-theoretic receding horizon control ideas are exploited to develop a distributed algorithm capable to instantaneously recognize the attacked agent.Finally,numerical simulations are carried out to show benefits and effectiveness of the proposed approach.

Distributed model predictive control for multiagent systems with improved consistency

This paper addresses an improved distributed model predictive control （DMPC） scheme for multiagent systems with an attempt to improving its consistency. The deviation between what an agent is actually doing and what its neighbors believe that agent is doing is penalized in the cost function of each agent. At each sampling instant the compatibility constraint of each agent is set tighter than the previous sampling instant. Like the traditional approach, the performance cost is utilized as the Lyapunov function to prove closed-looped stability. The closed-loop stability is guaranteed if the weight matrix for deviation in the cost function are sufficiently large. The proposed distributed control scheme is formulated as quadratic programming with quadratic constraints. A numerical example is given to illustrate the effectiveness of the proposed scheme.

Distributed Model Predictive Control with Actuator Saturation for Markovian Jump Linear System

This paper is concerned with the distributed model predictive control(MPC) problem for a class of discrete-time Markovian jump linear systems(MJLSs) subject to actuator saturation and polytopic uncertainty in system matrices. The global system is decomposed into several subsystems which coordinate with each other. A set of distributed controllers is designed by solving a min-max optimization problem in terms of the solutions of linear matrix inequalities(LMIs). An iterative algorithm is developed to achieve the online computation. Finally,a simulation example is employed to show the effectiveness of the proposed algorithm.

DISOPE distributed model predictive control of cascade systems with network communication

A novel distributed model predictive control scheme based on dynamic integrated system optimization and parameter estimation （DISOPE） was proposed for nonlinear cascade systems under network environment. Under the distributed control structure, online optimization of the cascade system was composed of several cascaded agents that can cooperate and exchange information via network communication. By iterating on modified distributed linear optimal control problems on the basis of estimating parameters at every iteration the correct optimal control action of the nonlinear model predictive control problem of the cascade system could be obtained, assuming that the algorithm was convergent. This approach avoids solving the complex nonlinear optimization problem and significantly reduces the computational burden. The simulation results of the fossil fuel power unit are illustrated to verify the effectiveness and practicability of the proposed algorithm.

Distributed Model Predictive Control for Networked Plant-wide Systems With Neighborhood Cooperation

For large-scale networked plant-wide systems composed by physically(or geographically) divided subsystems, only limited information is available for local controllers on account of region and communication restrictions. Concerning the optimal control problem of such subsystems, a neighbor-based distributed model predictive control(NDMPC) strategy is presented to improve the global system performance. In this scheme, the performance index of local subsystems and that of its neighbors are minimized together in the determination of the optimal control input, which makes the local control decision also beneficial to its neighboring subsystems and further contributes to improving the convergence and control performance of overall system.The stability of the closed-loop system is proved. Moreover, the parameter designing method for distributed synthesis is provided.Finally, the simulation results illustrate the main characteristics and effectiveness of the proposed control scheme.

Model-based Predictive Control for Spatially-distributed Systems Using Dimensional Reduction Models

In this paper, a low-dimensional multiple-input and multiple-output （MIMO） model predictive control （MPC） configuration is presented for partial differential equation （PDE） unknown spatially-distributed systems （SDSs）. First, the dimension reduction with principal component analysis （PCA） is used to transform the high-dimensional spatio-temporal data into a low-dimensional time domain. The MPC strategy is proposed based on the online correction low-dimensional models, where the state of the system at a previous time is used to correct the output of low-dimensional models. Sufficient conditions for closed-loop stability are presented and proven. Simulations demonstrate the accuracy and efficiency of the proposed methodologies.

The Design of Output Feedback Distributed Model Predictive Controller for a Class of Nonlinear Systems

For a class of nonlinear systems whose states are immeasurable, when the outputs of the system are sampled asynchronously, by introducing a state observer, an output feedback distributed model predictive control algorithm is proposed. It is proved that the errors of estimated states and the actual system's states are bounded. And it is guaranteed that the estimated states of the closed-loop system are ultimately bounded in a region containing the origin. As a result, the states of the actual system are ultimately bounded. A simulation example verifies the effectiveness of the proposed distributed control method.

Distributed model predictive control based on adaptive sampling mechanism

In this work,an adaptive sampling control strategy for distributed predictive control is proposed.According to the proposed method,the sampling rate of each subsystem of the accused object is determined based on the periodic detection of its dynamic behavior and calculations made using a correlation function.Then,the optimal sampling interval within the period is obtained and sent to the corresponding sub-prediction controller,and the sampling interval of the controller is changed accordingly before the next sampling period begins.In the next control period,the adaptive sampling mechanism recalculates the sampling rate of each subsystem’s measurable output variable according to both the abovementioned method and the change in the dynamic behavior of the entire system,and this process is repeated.Such an adaptive sampling interval selection based on an autocorrelation function that measures dynamic behavior can dynamically optimize the selection of sampling rate according to the real-time change in the dynamic behavior of the controlled object.It can also accurately capture dynamic changes,meaning that each sub-prediction controller can more accurately calculate the optimal control quantity at the next moment,significantly improving the performance of distributed model predictive control(DMPC).A comparison demonstrates that the proposed adaptive sampling DMPC algorithm has better tracking performance than the traditional DMPC algorithm.

Networked Cooperative Distributed Model Predictive Control Based on State Observer

Although distributed model predictive control has caused significant attention and received many good results, the results are mostly under the assumption that the system states can be observed. However, the states are difficult to be observed in practice. In this paper, a novel distributed model predictive control is proposed based on state observer for a kind of linear discrete-time systems where states are not measured. Firstly, an output feedback control law is designed based on Lyapunov function and state observer. And the stability domain is described. Furthermore, the stability domain as a terminal constraint is added into the constraint conditions of the algorithm to make systems stable outside the stability domain. The simulation results show the effectiveness of the proposed method.

基于DMPC的无信控交叉口智能网联车辆多车协同轨迹规划

为了解决智能网联自动驾驶环境下无信控十字交叉口多车协同通行的冲突问题,该文提出了一种基于分布式模型预测控制(DMPC)的多车协同轨迹规划方法。采用分布式模型预测框架实现多车协同轨迹规划的分布式独立计算,利用滚动时域预测周车轨迹实现车-车未来状态交互,基于智能网联环境车-车交互通信功能实现规划结果共享;引入道路边界约束、加速度约束与碰撞约束等车辆安全约束条件,通过二次规划求解可以安全通行十字路口的多车轨迹;基于MATLAB驾驶场景生成模块建立无信控十字交叉口环境,并在2种场景下验证了该方法的有效性。结果表明:在直行工况和左转工况下多车间最小距离分别为2.58 m和2.99 m,均满足避撞的安全距离约束,实现了多车之间的协同避撞并且能够保证通行效率。

基于视线协同和DMPC的载机-防御弹群协同主动防御制导策略

在目标-攻击弹-防御弹群(target-attacker-defenders,TADs)系统中,防御弹群通过与目标(载机)异构协同、弹群间同构协同以保护载机并降低单弹脱靶的风险。针对TADs系统在二维平面下的协同主动防御模型进行了研究,采用机/弹协同和防御弹群协同的两层制导策略。在机弹协同方面,防御弹领弹与载机进行异构协同,考虑载机及防御弹领弹的机动能力限制,采用协同视线制导律(cooperative line of sight guidance,CLOSG)分别得到载机和防御弹领弹的制导指令;在防御弹群协同方面,考虑单弹计算能力约束,拦截时间约束和加速度约束,设计出基于分布式模型预测控制(distributed model predictive control,DMPC)的算法实现弹群从弹和防御弹领弹协同同时抵达并拦截攻击弹。仿真结果表明,多防御弹协同一致拦截制导算法能够实现TADs系统中载机和防御弹群的异构协同主动防御,并实现防御弹群的一致性同时拦截,以降低单弹脱靶的风险。

Distributed Economic MPC for Synergetic Regulation of the Voltage of an Island DC Micro-Grid

In this paper,distributed model predictive control(DMPC) for island DC micro-grids(MG) with wind/photovoltaic(PV)/battery power is proposed,which coordinates all distributed generations(DG) to stabilize the bus voltage together with the insurance of having computational efficiency under a real-time requirement.Based on the feedback of the bus voltage,the deviation of the current is dispatched to each DG according to cost over the prediction horizon.Moreover,to avoid the excessive fluctuation of the battery power,both the discharge-charge switching times and costs are considered in the model predictive control(MPC) optimization problems.A Lyapunov constraint with a time-varying steady-state is designed in each local MPC to guarantee the stabilization of the entire system.The voltage stabilization of the MG is achieved by this strategy with the cooperation of DGs.The numeric results of applying the proposed method to a MG of the Shanghai Power Supply Company shows the effectiveness of the distributed economic MPC.

Consensus of Multi-Agent Systems with Input Constraints Based on Distributed Predictive Control Scheme

Consensus control of multi-agent systems has attracted compelling attentions from various scientific communities for its promising applications.This paper presents a discrete-time consensus protocol for a class of multi-agent systems with switching topologies and input constraints based on distributed predictive control scheme.The consensus protocol is not only distributed but also depends on the errors of states between agent and its neighbors.We focus mainly on dealing with the input constraints and a distributed model predictive control scheme is developed to achieve stable consensus under the condition that both velocity and acceleration constraints are included simultaneously.The acceleration constraint is regarded as the changing rate of velocity based on some reasonable assumptions so as to simplify the analysis.Theoretical analysis shows that the constrained system steered by the proposed protocol achieves consensus asymptotically if the switching interaction graphs always have a spanning tree.Numerical examples are also provided to illustrate the validity of the algorithm.

Application of distributed model predictive control based on neighborhood optimization in gauge-looper integrated system of tandem hot rolling

To solve the coupling relationship between the strip automatic gauge control and the looper control in traditional control strategy of tandem hot rolling,a distributed model predictive control(DMPC)strategy for the tandem hot rolling was explored,and a series of simulation experiments were carried out.Firstly,based on the state space analysis method,the multivariable dynamic transition process of hot strip rolling was studied,and the state space model of a gauge-looper integrated system in tandem hot rolling was established.Secondly,DMPC strategy based on neighborhood optimization was proposed,which fully considered the coupling relationship in this integrated system.Finally,a series of experiments simulating disturbances and emergency situations were completed with actual rolling data.The experimental results showed that the proposed DMPC control strategy had better performance compared with the traditional proportional-integral control and centralized model predictive control,which is applicable for the gauge-looper integrated system.

基于DMPC的多机械臂协同操作控制

针对多机械臂协作搬运中的编队稳定性和协同控制优化问题,提出了一种基于迭代线性二次型调节器(iLQR)的分布式预测控制算法(DMPC)。首先,对机械臂动力学模型中的末端接触力进行受力分析,从而得到机械臂的状态空间模型。其次,通过定义多机械臂系统的刚性编队约束来维持搬运途中编队结构的稳定。然后,将编队约束整合到DMPC的代价函数中用于建立约束最优化问题,并利用iLQR算法求解优化问题。最后,仿真实验验证了所提算法的有效性。

考虑优化调度的微电网频率恢复分布式预测控制策略

为了解决快速频率恢复和缓慢实际功率调度之间的时间尺度分离问题,提出一种考虑优化调度的微电网频率恢复分布式预测控制策略。引入的一致性策略能够在同一时间尺度上处理优化调度和频率恢复,另外提出的实际功率调度方案分布式模型预测控制解决了通信延迟、通信中断和即插即用情景,并且将下垂、实际功率传输和相位角方程作为等式约束,从而预测每个微电网的行为。此外,终端值和不等式约束有助于确定可行解空间的界,从而减少优化时间。在4种测试情景下对控制器的动态性能进行实验评估,结果证明该控制器对负载变化和通信问题具有较强的鲁棒性,并且保证了频率快速回复与高效功率调度。

基于模型预测控制的高频隔离型混合配电变压器

配电台区新型源/荷占比逐年提升,使得高/低电压、谐波放大、三相不平衡等配电终端电能质量问题凸显。提出一种基于模型预测控制的高频隔离型混合配电变压器,具备电能质量综合治理、交/直流混合配电等功能,可以满足配电终端用户高品质供用电以及新型源/荷友好接入需求。介绍了高频隔离型混合配电变压器的拓扑结构并推导了数学模型。结合拓扑结构特点,设计了连续控制集模型预测控制(CCS-MPC)策略和延时模型预测控制策略,CCS-MPC通过计算最优化占空比,实现前端变换器输出具有固定的开关频率和开关序列,减小寻优计算量。利用MATLAB/Simulink仿真软件搭建了一套10 kV/0.4 kV电压等级的高频隔离型混合配电变压器系统,验证了所提拓扑结构电能质量综合治理能力的有效性。