A new optimal adaptive backstepping control approach for nonlinear systems under deception attacks via reinforcement learning

In this paper,a new optimal adaptive backstepping control approach for nonlinear systems under deception attacks via reinforcement learning is presented in this paper.The existence of nonlinear terms in the studied system makes it very difficult to design the optimal controller using traditional methods.To achieve optimal control,RL algorithm based on critic–actor architecture is considered for the nonlinear system.Due to the significant security risks of network transmission,the system is vulnerable to deception attacks,which can make all the system state unavailable.By using the attacked states to design coordinate transformation,the harm brought by unknown deception attacks has been overcome.The presented control strategy can ensure that all signals in the closed-loop system are semi-globally ultimately bounded.Finally,the simulation experiment is shown to prove the effectiveness of the strategy.

Optimal Control of Nonlinear Systems Using Experience Inference Human-Behavior Learning

Safety critical control is often trained in a simulated environment to mitigate risk.Subsequent migration of the biased controller requires further adjustments.In this paper,an experience inference human-behavior learning is proposed to solve the migration problem of optimal controllers applied to real-world nonlinear systems.The approach is inspired in the complementary properties that exhibits the hippocampus,the neocortex,and the striatum learning systems located in the brain.The hippocampus defines a physics informed reference model of the realworld nonlinear system for experience inference and the neocortex is the adaptive dynamic programming(ADP)or reinforcement learning(RL)algorithm that ensures optimal performance of the reference model.This optimal performance is inferred to the real-world nonlinear system by means of an adaptive neocortex/striatum control policy that forces the nonlinear system to behave as the reference model.Stability and convergence of the proposed approach is analyzed using Lyapunov stability theory.Simulation studies are carried out to verify the approach.

Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints

This paper introduces an observer-based adaptive optimal control method for unknown singularly perturbed nonlinear systems with input constraints. First, a multi-Time scales dynamic neural network MTSDNN observer with a novel updating law derived from a properly designed Lyapunov function is proposed to estimate the system states. Then, an adaptive learning rule driven by the critic NN weight error is presented for the critic NN, which is used to approximate the optimal cost function. Finally, the optimal control action is calculated by online solving the Hamilton-Jacobi-Bellman HJB equation associated with the MTSDNN observer and critic NN. The stability of the overall closed-loop system consisting of the MTSDNN observer, the critic NN and the optimal control action is proved. The proposed observer-based optimal control approach has an essential advantage that the system dynamics are not needed for implementation, and only the measured input U+002F output data is needed. Moreover, the proposed optimal control design takes the input constraints into consideration and thus can overcome the restriction of actuator saturation. Simulation results are presented to confirm the validity of the investigated approach. © 2014 Chinese Association of Automation.

Global robust optimal sliding mode control for uncertain affine nonlinear systems

The problem of robustifying linear quadratic regulators （LQRs） for a class of uncertain affine nonlinear systems is considered. First, the exact linearization technique is used to transform an uncertain nonlinear system into a linear one and an optimal LQR is designed for the corresponding nominal system. Then, based on the integral sliding mode, a design approach to robustifying the optimal regulator is studied. As a result, the system exhibits global robustness to uncertainties and the ideal sliding mode dynamics is the same as that of the optimal LQR for the nominal system. A global robust optimal sliding mode control （GROSMC） is realized. Finally, a numerical simulation is demonstrated to show the effectiveness and superiority of the proposed algorithm compared with the conventional optimal LQR.

Optimal Neuro-Control Strategy for Nonlinear Systems With Asymmetric Input Constraints

In this paper,we present an optimal neuro-control scheme for continuous-time(CT)nonlinear systems with asymmetric input constraints.Initially,we introduce a discounted cost function for the CT nonlinear systems in order to handle the asymmetric input constraints.Then,we develop a Hamilton-Jacobi-Bellman equation(HJBE),which arises in the discounted cost optimal control problem.To obtain the optimal neurocontroller,we utilize a critic neural network(CNN)to solve the HJBE under the framework of reinforcement learning.The CNN's weight vector is tuned via the gradient descent approach.Based on the Lyapunov method,we prove that uniform ultimate boundedness of the CNN's weight vector and the closed-loop system is guaranteed.Finally,we verify the effectiveness of the present optimal neuro-control strategy through performing simulations of two examples.

Optimal tracking control for nonlinear large-scale systems with persistent disturbances

This paper studies the optimal control with zero steady-state error problem for nonlinear large-scale systems affected by external persistent disturbances.The nonlinear large-scale system is transformed into N nonlinear subsystems with interconnect terms.Based on the internal model principle,a disturbance compensator is constructed such that the ith subsystem with external persistent disturbances is transformed into an augmented subsystem without disturbances.According to the sensitivity approach,the optimal tracking control law for the ith nonlinear subsystem can be obtained.The optimal tracking control law for the nonlinear large-scale systems can be obtained.A numerical simulation shows that the method is effective.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

AN EFFICIENT PARALLEL PROCESSING OPTIMAL CONTROL SCHEME FOR A CLASS OF NONLINEAR COMPOSITE SYSTEMS

This article presents an efficient parallel processing approach for solving the opti- mal control problem of nonlinear composite systems. In this approach, the original high-order coupled nonlinear two-point boundary value problem （TPBVP） derived from the Pontrya- gin＇s maximum principle is first transformed into a sequence of lower-order deeoupled linear time-invariant TPBVPs. Then, an optimal control law which consists of both feedback and forward terms is achieved by using the modal series method for the derived sequence. The feedback term specified by local states of each subsystem is determined by solving a ma- trix Riccati differential equation. The forward term for each subsystem derived from its local information is an infinite sum of adjoint vectors. The convergence analysis and parallel processing capability of the proposed approach are also provided. To achieve an accurate feedforward-feedbaek suboptimal control, we apply a fast iterative algorithm with low com- putational effort. Finally, some comparative results are included to illustrate the effectiveness of the proposed approach.

Optimal control of quadratic functionals for affine nonlinear systems

In this paper we analyze the optimal control problem for a class of afflne nonlinear systems under the assumption that the associated Lie algebra is nilpotent. The Lie brackets generated by the vector fields which define the nonlinear system represent a remarkable mathematical instrument for the control of affine systems. We determine the optimal control which corresponds to the nilpotent operator of the first order. In particular, we obtain the control that minimizes the energy of the given nonlinear system. Applications of this control to bilinear systems with first order nilpotent operator are considered.

Control parameter optimal tuning method based on annealing-genetic algorithm for complex electromechanical system

A new searching algorithm named the annealing-genetic algorithm(AGA) was proposed by skillfully merging GA with SAA. It draws on merits of both GA and SAA ,and offsets their shortcomings.The difference from GA is that AGA takes objective function as adaptability function directly,so it cuts down some unnecessary time expense because of float-point calculation of function conversion.The difference from SAA is that AGA need not execute a very long Markov chain iteration at each point of temperature, so it speeds up the convergence of solution and makes no assumption on the search space,so it is simple and easy to be implemented.It can be applied to a wide class of problems.The optimizing principle and the implementing steps of AGA were expounded. The example of the parameter optimization of a typical complex electromechanical system named temper mill shows that AGA is effective and superior to the conventional GA and SAA.The control system of temper mill optimized by AGA has the optimal performance in the adjustable ranges of its parameters.

Symplectic Numerical Approach for Nonlinear Optimal Control of Systems with Inequality Constraints

This paper proposes a system representation for unifying control design and numerical calculation in nonlinear optimal control problems with inequality constraints in terms of the symplectic structure. The symplectic structure is derived from Hamiltonian systems that are equivalent to Hamilton-Jacobi equations. In the representation, the constraints can be described as an input-state transformation of the system. Therefore, it can be seamlessly applied to the stable manifold method that is a precise numerical solver of the Hamilton-Jacobi equations. In conventional methods, e.g., the penalty method or the barrier method, it is difficult to systematically assign the weights of penalty functions that are used for realizing the constraints. In the proposed method, we can separate the adjustment of weights with respect to objective functions from that of penalty functions. Furthermore, the proposed method can extend the region of computable solutions in a state space. The validity of the method is shown by a numerical example of the optimal control of a vehicle model with steering limitations.

Nonlinear adaptive optimal control for vehicle handling improvement through steer-by-wire system

A control algorithm for improving vehicle handling was proposed by applying right angle to the steering wheel,based on the nonlinear adaptive optimal control(NAOC).A nonlinear 4-DOF model was initially developed,then it was simplified to a 2-DOF model with reasonable assumptions to design observer and optimal controllers.Then a simplified model was developed for steering system.The numerical simulations were carried out using vehicle parameters for standard maneuvers in dry and wet road conditions.Moreover,the hardware in the loop method was implemented to prove the controller ability in realistic conditions.Simulation results obviously show the effectiveness of NAOC on vehicle handling and reveal that the proposed controller can significantly improve vehicle handling during severe maneuvers.

Combined indirect and direct method for adaptive fuzzy output feedback control of nonlinear system

A novel control method for a general class of nonlinear systems using fuzzy logic systems （FLSs） is presertted. Indirect and direct methods are combined to design the adaptive fuzzy output feedback controller and a high-gain observer is used to estimate the derivatives of the system output. The closed-loop system is proven to be semiglobally uniformly ultimately bounded. In addition, it is shown that if the approximation accuracy of the fuzzy logic system is high enough and the observer gain is chosen sufficiently large, an arbitrarily small tracking error can be achieved. Simulation results verify the effectiveness of the newly designed scheme and the theoretical discussion.

ASUBOPTIMAL DESIGN OF H∞DECENTRALIZED STATE FEEDBACK CONTROL FOR INTERCONNEC TED SYSTEMS

In the paper, the problem of H∞ decentralized state feedback control for largescale systems is described. An algorithm is proposed which uses the method of a feasible direction matrix. The algorithm only requires the solution of an algebraic Riccati equation (ARE) and makes the H∞norm of the closedloop transfer function matrix from disturbance inputs to controlled outputs less than a given constant which ensure the stability of the overall controlled system at each iteration. The given example shows that the convergence of the algorithm is satisfactory.

Solving the Optimal Control Problems of Nonlinear Duffing Oscillators By Using an Iterative Shape Functions Method

In the optimal control problem of nonlinear dynamical system,the Hamiltonian formulation is useful and powerful to solve an optimal control force.However,the resulting Euler-Lagrange equations are not easy to solve,when the performance index is complicated,because one may encounter a two-point boundary value problem of nonlinear differential algebraic equations.To be a numerical method,it is hard to exactly preserve all the specified conditions,which might deteriorate the accuracy of numerical solution.With this in mind,we develop a novel algorithm to find the solution of the optimal control problem of nonlinear Duffing oscillator,which can exactly satisfy all the required conditions for the minimality of the performance index.A new idea of shape functions method(SFM)is introduced,from which we can transform the optimal control problems to the initial value problems for the new variables,whose initial values are given arbitrarily,and meanwhile the terminal values are determined iteratively.Numerical examples confirm the high-performance of the iterative algorithms based on the SFM,which are convergence fast,and also provide very accurate solutions.The new algorithm is robust,even large noise is imposed on the input data.

Low-Cost Embedded Controller for Complex Control Systems

Because of limited resource of embedded platforms, the computational complexity of advanced control algorithms raises significant challenges for the use of embedded systems in complex control field. A Scilab/Scicos based embedded controller is developed on which various control software can be easily modeled, simulated, implemented, and evaluated to meet the ever-expanding requirements of industrial control applications. Built on the Cirrus Logic EP9315 ARM systems-on-chip board, this embedded controller is possible to develop complex embedded control systems that employ advanced control strategies in a rapid and cost-efficient fashion. Due to the free and open source nature of the software packages used, the cost of the embedded controller is minimized.

A Novel Evolutionary-Fuzzy Control Algorithm for Complex Systems

This paper presents an adaptive fuzzy control scheme based on modified genetic algorithm. In the control scheme, genetic algorithm is used to optimze the nonlinear quantization functions of the controller and some key parameters of the adaptive control algorithm. Simulation results show that this control scheme has satisfactory performance in MIMO systems, chaotic systems and delay systems.

STOCHASTIC OPTIMAL CONTROL OF STRONGLY NONLINEAR SYSTEMS UNDER WIDE-BAND RANDOM EXCITATION WITH ACTUATOR SATURATION

A bounded optimal control strategy for strongly non-linear systems under non-white wide-band random excitation with actuator saturation is proposed. First, the stochastic averaging method is introduced for controlled strongly non-linear systems under wide-band random excitation using generalized harmonic functions. Then, the dynamical programming equation for the saturated control problem is formulated from the partially averaged Itō equation based on the dynamical programming principle. The optimal control consisting of the unbounded optimal control and the bounded bang-bang control is determined by solving the dynamical programming equation. Finally, the response of the optimally controlled system is predicted by solving the reduced Fokker-Planck-Kolmogorov （FPK） equation associated with the completed averaged Itō equation. An example is given to illustrate the proposed control strategy. Numerical results show that the proposed control strategy has high control effectiveness and efficiency and the chattering is reduced significantly comparing with the bang-bang control strategy.

Optimal control strategies for stochastically excited quasi partially integrable Hamiltonian systems

In this paper two different control strategies designed to alleviate the response of quasi partially integrable Hamiltonian systems subjected to stochastic excitation are proposed. First, by using the stochastic averaging method for quasi partially integrable Hamiltonian systems, an n-DOF controlled quasi partially integrable Hamiltonian system with stochastic excitation is converted into a set of partially averaged It^↑o stochastic differential equations. Then, the dynamical programming equation associated with the partially averaged It^↑o equations is formulated by applying the stochastic dynamical programming principle. In the first control strategy, the optimal control law is derived from the dynamical programming equation and the control constraints without solving the dynamical programming equation. In the second control strategy, the optimal control law is obtained by solving the dynamical programming equation. Finally, both the responses of controlled and uncontrolled systems are predicted through solving the Fokker-Plank-Kolmogorov equation associated with fully averaged It^↑o equations. An example is worked out to illustrate the application and effectiveness of the two proposed control strategies.