Reinforcement learning based adaptive control for uncertain mechanical systems with asymptotic tracking

This paper mainly focuses on the development of a learning-based controller for a class of uncertain mechanical systems modeled by the Euler-Lagrange formulation.The considered system can depict the behavior of a large class of engineering systems,such as vehicular systems,robot manipulators and satellites.All these systems are often characterized by highly nonlinear characteristics,heavy modeling uncertainties and unknown perturbations,therefore,accurate-model-based nonlinear control approaches become unavailable.Motivated by the challenge,a reinforcement learning(RL)adaptive control methodology based on the actor-critic framework is investigated to compensate the uncertain mechanical dynamics.The approximation inaccuracies caused by RL and the exogenous unknown disturbances are circumvented via a continuous robust integral of the sign of the error(RISE)control approach.Different from a classical RISE control law,a tanh(·)function is utilized instead of a sign(·)function to acquire a more smooth control signal.The developed controller requires very little prior knowledge of the dynamic model,is robust to unknown dynamics and exogenous disturbances,and can achieve asymptotic output tracking.Eventually,co-simulations through ADAMS and MATLAB/Simulink on a three degrees-of-freedom(3-DOF)manipulator and experiments on a real-time electromechanical servo system are performed to verify the performance of the proposed approach.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

Quantization and Event-Triggered Policy Design for Encrypted Networked Control

This paper proposes a novel event-driven encrypted control framework for linear networked control systems(NCSs),which relies on two modified uniform quantization policies,the Paillier cryptosystem,and an event-triggered strategy.Due to the fact that only integers can work in the Pailler cryptosystem,both the real-valued control gain and system state need to be first quantized before encryption.This is dramatically different from the existing quantized control methods,where only the quantization of a single value,e.g.,the control input or the system state,is considered.To handle this issue,static and dynamic quantization policies are presented,which achieve the desired integer conversions and guarantee asymptotic convergence of the quantized system state to the equilibrium.Then,the quantized system state is encrypted and sent to the controller when the triggering condition,specified by a state-based event-triggered strategy,is satisfied.By doing so,not only the security and confidentiality of data transmitted over the communication network are protected,but also the ciphertext expansion phenomenon can be relieved.Additionally,by tactfully designing the quantization sensitivities and triggering error,the proposed event-driven encrypted control framework ensures the asymptotic stability of the overall closedloop system.Finally,a simulation example of the secure motion control for an inverted pendulum cart system is presented to evaluate the effectiveness of the theoretical results.

Tracking Control of Multi-Agent Systems Using a Networked Predictive PID Tracking Scheme

With the rapid development of network technology and control technology,a networked multi-agent control system is a key direction of modern industrial control systems,such as industrial Internet systems.This paper studies the tracking control problem of networked multi-agent systems with communication constraints,where each agent has no information on the dynamics of other agents except their outputs.A networked predictive proportional integral derivative(PPID)tracking scheme is proposed to achieve the desired tracking performance,compensate actively for communication delays,and simplify implementation in a distributed manner.This scheme combines the past,present and predictive information of neighbour agents to form a tracking error signal for each agent,and applies the proportional,integral,and derivative of the agent tracking error signal to control each individual agent.The criteria of the stability and output tracking consensus of multi-agent systems with the networked PPID tracking scheme are derived through detailed analysis on the closed-loop systems.The effectiveness of the networked PPID tracking scheme is illustrated via an example.

Consensus of networked control multi-agent systems using a double-layer encryption scheme

This paper addresses the decentralized consensus problem for a system of multiple dynamic agents with remote controllers via networking,known as a networked control multi-agent system(NCMAS).It presents a challenging scenario where partial dynamic entities or remote control units are vulnerable to disclosure attacks,making them potentially malicious.To tackle this issue,we propose a secure decentralized control design approach employing a double-layer cryptographic strategy.This approach not only ensures that the input and output information of the benign entities remains protected from the malicious entities but also practically achieves consensus performance.The paper provides an explicit design,supported by theoretical proof and numerical verification,covering stability,steady-state error,and the prevention of computation overflow or underflow.

Fault detection and optimization for networked control systems with uncertain time-varying delay

The observer-based robust fault detection filter design and optimization for networked control systems （NOSs） with uncer- tain time-varying delays are addressed. The NCSs with uncertain time-varying delays are modeled as parameter-uncertain systems by the matrix theory. Based on the model, an observer-based residual generator is constructed and the sufficient condition for the existence of the desired fault detection filter is derived in terms of the linear matrix inequality. Furthermore, a time domain opti- mization approach is proposed to improve the performance of the fault detection system. To prevent the false alarms, a new thresh- old function is established, and the solution of the optimization problem is given by using the singular value decomposition （SVD） of the matrix. A numerical example is provided to illustrate the effectiveness of the proposed approach.

Robust H∞ output tracking control of uncertain networked control systems

This paper investigates a problem of robust output tracking control of networked control systems(NCSs) with network-induced delays, packet dropouts, parameter uncertainties and external disturbances. Firstly, an augmented model of time-delay system is proposed for networked tracking control systems. Then, considering the piecewise differentiable characteristic of time-delay, the criterion to output tracking performance analysis and controller design are derived by using an approach of free weighting matrix, reciprocally convex and cone complementarity linearization(CCL). Finally, simulation results of numerical examples show the effectiveness of the proposed approach, and illustrate the advantages of the proposed criteria which outperform previous criteria in the literature.

Delay-dependent robust stability of uncertain networked control systems with multiple state time-delays

In this paper, delay-dependent robust stability for a class of uncertain networked control systems （NCSs） with multiple state time-delays is investigated. Modeling of multi-input and multi-output （MIMO） NCSs with networkinduced delays and uncertainties through new methods are proposed. Some new stability criteria in terms of LMIs are derived by using Lyapunov stability theory combined with linear matrix inequalities （LMIs） techniques. We analyze the delay-dependent asymptotic stability and obtain maximum allowable delay bound （MADB） for the NCSs with the proposed methods. Compared with the reported results, the proposed results obtain a much less conservative MADB which are more general. Numerical example and simulation is used to illustrate the effectiveness of the proposed methods.

Robust H∞ Networked Control for Uncertain Fuzzy Systems with Time-delay

柔韧的 H 联网了控制方法因为有无常和时间的模糊系统推迟的 Takagi-Sugeno (T-S ) 被介绍。一个州的反馈控制器经由联网的控制系统(NCS ) 被设计理论。为有 H 性能的柔韧的稳定性的足够的状况被获得。在网络传播和包退学学生的导致网络的延期被分析。模拟结果显示出这个控制计划的有效性。

Security Control for Uncertain Networked Control Systems under DoS Attacks and Fading Channels

This paper characterizes the joint effects of plant uncertainty,Denial-of-Service(DoS)attacks,and fading channel on the stabilization problem of networked control systems(NCSs).It is assumed that the controller remotely controls the plant and the control input is transmitted over a fading channel.Meanwhile,considering the sustained attack cycle and frequency of DoS attacks are random,the packet-loss caused by DoS attacks is modelled by a Markov process.The sampled-data NCS is transformed into a stochastic form with Markov jump and uncertain parameter.Then,based on Lyapunov functional method,linear matrix inequality(LMI)-based sufficient conditions are presented to ensure the stability of uncertain NCSs.The main contribution of this article lies in the construction of NCSs based on DoS attacks into Markov jump system(MJS)and the joint consideration of fading channel and plant uncertainty.

H_∞Control of Uncertain Fuzzy Networked Control Systems with State Quantization

The problem of robust H∞ control for uncertain discrete-time Takagi and Sugeno (T-S) fuzzy networked control systems (NCSs) is investigated in this paper subject to state quantization. By taking into consideration network induced delays and packet dropouts, an improved model of network-based control is developed. A less conservative delay-dependent stability condition for the closed NCSs is derived by employing a fuzzy Lyapunov-Krasovskii functional. Robust H∞ fuzzy controller is constructed that guarantee asymptotic stabilization of the NCSs and expressed in LMI-based conditions. A numerical example illustrates the effectiveness of the developed technique.

Robust disturbance attenuation for uncertain nonlinear networked control systems

This paper considers the problem of robust disturbance attenuation for a class of uncertain nonlinear networked control systems. Takagi-Sugeno fuzzy models are firstly employed to describe the nonlinear plant. Markov processes are used to model the random network-induced delays and data packet dropouts. The Lyapunov-Razumikhin method has been used to derive such a controller for this class of nonlinear systems such that it is stochastically stabilizable with a disturbance attenuation level. Sufficient conditions for the existence of such a controller are derived in terms of the solvability of bilinear matrix inequalities. An iterative algorithm is proposed to change this non-convex problem into quasi-convex optimization problems, which can be solved effectively by available mathematical tools. The effectiveness of the proposed design methodology is verified by a numerical example.

Time-stamped predictive functional control for networked control systems with random delays

The random delays in a networked control system (NCS) degrade control performance and can even destabilize the control system.To deal with this problem,the time-stamped predictive functional control (PFC) algorithm is proposed,which generalizes the standard PFC algorithm to networked control systems with random delays.The algorithm uses the time-stamp method to estimate the control delay,predicts the future outputs based on a discrete time delay state space model,and drives the control law that applies to an NCS from the idea of a PFC algorithm.A networked control system was constructed based on TrueTime simulator,with which the time-stamped PFC algorithm was compared with the standard PFC algorithm.The response curves show that the proposed algorithm has better control performance.

Decentralized Control Based on FNNSMC for Interconnected Uncertain Nonlinear Systems

A new type controller, fuzzy neural networks sliding mode controller (FNNSMC), is developed for a class of large scale systems with unknown bounds of high order interconnections and disturbances. Although sliding mode control is simple and insensitive to uncertainties and disturbances, there are two main problems in the sliding mode controller (SMC): control input chattering and the assumption of known bounds of uncertainties and disturbances. The FNNSMC, which incorporates the fuzzy neural networks (FNN) and the SMC, can eliminate the chattering by using the continuous output of the FNN to replace the discontinuous sign term in the SMC. The bounds of uncertainties and disturbances are also not required in the FNNSMC design. The simulation results show that the FNNSMC has more robustness than the SMC.

A Data-Based Feedback Relearning Algorithm for Uncertain Nonlinear Systems

In this paper,a data-based feedback relearning algorithm is proposed for the robust control problem of uncertain nonlinear systems.Motivated by the classical on-policy and off-policy algorithms of reinforcement learning,the online feedback relearning(FR)algorithm is developed where the collected data includes the influence of disturbance signals.The FR algorithm has better adaptability to environmental changes(such as the control channel disturbances)compared with the off-policy algorithm,and has higher computational efficiency and better convergence performance compared with the on-policy algorithm.Data processing based on experience replay technology is used for great data efficiency and convergence stability.Simulation experiments are presented to illustrate convergence stability,optimality and algorithmic performance of FR algorithm by comparison.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Networked Control Systems:A Survey of Trends and Techniques

Networked control systems are spatially distributed systems in which the communication between sensors, actuators,and controllers occurs through a shared band-limited digital communication network. Several advantages of the network architectures include reduced system wiring, plug and play devices,increased system agility, and ease of system diagnosis and maintenance. Consequently, networked control is the current trend for industrial automation and has ever-increasing applications in a wide range of areas, such as smart grids, manufacturing systems,process control, automobiles, automated highway systems, and unmanned aerial vehicles. The modelling, analysis, and control of networked control systems have received considerable attention in the last two decades. The ‘control over networks’ is one of the key research directions for networked control systems. This paper aims at presenting a survey of trends and techniques in networked control systems from the perspective of ‘control over networks’, providing a snapshot of five control issues: sampled-data control, quantization control, networked control, event-triggered control, and security control. Some challenging issues are suggested to direct the future research.

Decentralized robust guaranteed cost control for a class of interconnected singular large-scale systems with time-delay and parameter uncertainty

The decentralized robust guaranteed cost control problem is studied for a class of interconnected singular large-scale systems with time-delay and norm-bounded time-invariant parameter uncertainty under a given quadratic cost performance function. The problem that is addressed in this study is to design a decentralized robust guaranteed cost state feedback controller such that the closed-loop system is not only regular, impulse-free and stable, but also guarantees an adequate level of performance for all admissible uncertainties. A sufficient condition for the existence of the decentralized robust guaranteed cost state feedback controllers is proposed in terms of a linear matrix inequality （LMI） via LMI approach. When this condition is feasible, the desired state feedback decentralized robust guaranteed cost controller gain matrices can be obtained. Finally, an illustrative example is provided to demonstrate the effectiveness of the proposed approach.

Global robust optimal sliding mode control for uncertain affine nonlinear systems

The problem of robustifying linear quadratic regulators （LQRs） for a class of uncertain affine nonlinear systems is considered. First, the exact linearization technique is used to transform an uncertain nonlinear system into a linear one and an optimal LQR is designed for the corresponding nominal system. Then, based on the integral sliding mode, a design approach to robustifying the optimal regulator is studied. As a result, the system exhibits global robustness to uncertainties and the ideal sliding mode dynamics is the same as that of the optimal LQR for the nominal system. A global robust optimal sliding mode control （GROSMC） is realized. Finally, a numerical simulation is demonstrated to show the effectiveness and superiority of the proposed algorithm compared with the conventional optimal LQR.

Guaranteed cost control for networked control systems

The guaranteed cost control problem for networked control systems (NCSs) is addressed under communication constraints and varying sampling rate. First of all, a simple information-scheduling scheme is presented to describe the scheduling approach of system signals in NCSs. Then, based on such a scheme and given sampling method, the design procedure in dynamic output feedback manner is also derived which renders the closed loop system to be asymptotically stable and guarantees an upper bound of the LQ performance cost function.