A new optimal adaptive backstepping control approach for nonlinear systems under deception attacks via reinforcement learning

In this paper,a new optimal adaptive backstepping control approach for nonlinear systems under deception attacks via reinforcement learning is presented in this paper.The existence of nonlinear terms in the studied system makes it very difficult to design the optimal controller using traditional methods.To achieve optimal control,RL algorithm based on critic–actor architecture is considered for the nonlinear system.Due to the significant security risks of network transmission,the system is vulnerable to deception attacks,which can make all the system state unavailable.By using the attacked states to design coordinate transformation,the harm brought by unknown deception attacks has been overcome.The presented control strategy can ensure that all signals in the closed-loop system are semi-globally ultimately bounded.Finally,the simulation experiment is shown to prove the effectiveness of the strategy.

An optimal adaptive H-infinity tracking control design via wavelet network

In this paper, an optimal adaptive H-infinity tracking control design method via wavelet network for a class of uncertain nonlinear systems with external disturbances is proposed to achieve H-infinity tracking performance. First, an alternate tracking error and a performance index with respect to the tracking error and the control effort are introduced in order to obtain better performance, especially, in reducing the cost of the control effort in the case of small attenuation levels. Next, H-infinity tracking performance, which attenuates the influence of both wavelet network approximation error and external disturbances on the modified tracking error, is formulated. Our results indicate that a small attenuation level does not lead to a large control signal. The proposed method insures an optimal trade-off between the amplitude of control signals and the performance of tracking errors. An example is given to illustrate the design efficiency.

Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints

This paper introduces an observer-based adaptive optimal control method for unknown singularly perturbed nonlinear systems with input constraints. First, a multi-Time scales dynamic neural network MTSDNN observer with a novel updating law derived from a properly designed Lyapunov function is proposed to estimate the system states. Then, an adaptive learning rule driven by the critic NN weight error is presented for the critic NN, which is used to approximate the optimal cost function. Finally, the optimal control action is calculated by online solving the Hamilton-Jacobi-Bellman HJB equation associated with the MTSDNN observer and critic NN. The stability of the overall closed-loop system consisting of the MTSDNN observer, the critic NN and the optimal control action is proved. The proposed observer-based optimal control approach has an essential advantage that the system dynamics are not needed for implementation, and only the measured input U+002F output data is needed. Moreover, the proposed optimal control design takes the input constraints into consideration and thus can overcome the restriction of actuator saturation. Simulation results are presented to confirm the validity of the investigated approach. © 2014 Chinese Association of Automation.

Global robust optimal sliding mode control for uncertain affine nonlinear systems

The problem of robustifying linear quadratic regulators （LQRs） for a class of uncertain affine nonlinear systems is considered. First, the exact linearization technique is used to transform an uncertain nonlinear system into a linear one and an optimal LQR is designed for the corresponding nominal system. Then, based on the integral sliding mode, a design approach to robustifying the optimal regulator is studied. As a result, the system exhibits global robustness to uncertainties and the ideal sliding mode dynamics is the same as that of the optimal LQR for the nominal system. A global robust optimal sliding mode control （GROSMC） is realized. Finally, a numerical simulation is demonstrated to show the effectiveness and superiority of the proposed algorithm compared with the conventional optimal LQR.

Nonlinear adaptive optimal control for vehicle handling improvement through steer-by-wire system

A control algorithm for improving vehicle handling was proposed by applying right angle to the steering wheel,based on the nonlinear adaptive optimal control(NAOC).A nonlinear 4-DOF model was initially developed,then it was simplified to a 2-DOF model with reasonable assumptions to design observer and optimal controllers.Then a simplified model was developed for steering system.The numerical simulations were carried out using vehicle parameters for standard maneuvers in dry and wet road conditions.Moreover,the hardware in the loop method was implemented to prove the controller ability in realistic conditions.Simulation results obviously show the effectiveness of NAOC on vehicle handling and reveal that the proposed controller can significantly improve vehicle handling during severe maneuvers.

On stochastic optimal control of partially observable nonlinear quasi Hamiltonian systems

A stochastic optimal control strategy for partially observable nonlinear quasi Hamiltonian systems is proposed. The optimal control forces consist of two parts. The first part is determined by the conditions under which the stochastic optimal control problem of a partially observable nonlinear system is converted into that of a completely observable linear system. The second part is determined by solving the dynamical programming equation derived by applying the stochastic averaging method and stochastic dynamical programming principle to the completely observable linear control system. The response of the optimally controlled quasi Hamiltonian system is predicted by solving the averaged Fokker-Planck-Kolmogorov equation associated with the optimally controlled completely observable linear system and solving the Riccati equation for the estimated error of system states. An example is given to illustrate the procedure and effectiveness of the proposed control strategy.

Optimal Delayed Control of Nonlinear Vibration Resonances of Single Degree of Freedom System

The primary resonance of a single-degree-of-freedom(SDOF)system subjected to a harmonic excitation is mitigated by the method of optimal time-delay feedback control.The stable regions of the time delays and feedback gains are obtained from the stable conditions of eigenvalue equation.Attenuation ratio is applied for evaluating the performance of the vibration control by taking aproportion of peak amplitude of primary resonance for the suspension system with or without controllers.Taking the attenuation ratio as the objective function and the stable regions of the time delays and feedback gains as constrains,the optimal feedback gains are determined by using minimum optimal method.Finally,simulation examples are also presented.

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming(ADP) where only one critic neural network(NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness(UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.

STOCHASTIC OPTIMAL CONTROL OF STRONGLY NONLINEAR SYSTEMS UNDER WIDE-BAND RANDOM EXCITATION WITH ACTUATOR SATURATION

A bounded optimal control strategy for strongly non-linear systems under non-white wide-band random excitation with actuator saturation is proposed. First, the stochastic averaging method is introduced for controlled strongly non-linear systems under wide-band random excitation using generalized harmonic functions. Then, the dynamical programming equation for the saturated control problem is formulated from the partially averaged Itō equation based on the dynamical programming principle. The optimal control consisting of the unbounded optimal control and the bounded bang-bang control is determined by solving the dynamical programming equation. Finally, the response of the optimally controlled system is predicted by solving the reduced Fokker-Planck-Kolmogorov （FPK） equation associated with the completed averaged Itō equation. An example is given to illustrate the proposed control strategy. Numerical results show that the proposed control strategy has high control effectiveness and efficiency and the chattering is reduced significantly comparing with the bang-bang control strategy.

Optimal Neuro-Control Strategy for Nonlinear Systems With Asymmetric Input Constraints

In this paper,we present an optimal neuro-control scheme for continuous-time(CT)nonlinear systems with asymmetric input constraints.Initially,we introduce a discounted cost function for the CT nonlinear systems in order to handle the asymmetric input constraints.Then,we develop a Hamilton-Jacobi-Bellman equation(HJBE),which arises in the discounted cost optimal control problem.To obtain the optimal neurocontroller,we utilize a critic neural network(CNN)to solve the HJBE under the framework of reinforcement learning.The CNN's weight vector is tuned via the gradient descent approach.Based on the Lyapunov method,we prove that uniform ultimate boundedness of the CNN's weight vector and the closed-loop system is guaranteed.Finally,we verify the effectiveness of the present optimal neuro-control strategy through performing simulations of two examples.

STOCHASTIC OPTIMAL VIBRATION CONTROL OF PARTIALLY OBSERVABLE NONLINEAR QUASI HAMILTONIAN SYSTEMS WITH ACTUATOR SATURATION

An optimal vibration control strategy for partially observable nonlinear quasi Hamiltonian systems with actuator saturation is proposed. First,a controlled partially observable non-linear system is converted into a completely observable linear control system of finite dimension based on the theorem due to Charalambous and Elliott. Then the partially averaged It stochastic differential equations and dynamical programming equation associated with the completely observable linear system are derived by using the stochastic averaging method and stochastic dynamical programming principle,respectively. The optimal control law is obtained from solving the final dynamical programming equation. The results show that the proposed control strategy has high control effectiveness and control effciency.

Solving the Optimal Control Problems of Nonlinear Duffing Oscillators By Using an Iterative Shape Functions Method

In the optimal control problem of nonlinear dynamical system,the Hamiltonian formulation is useful and powerful to solve an optimal control force.However,the resulting Euler-Lagrange equations are not easy to solve,when the performance index is complicated,because one may encounter a two-point boundary value problem of nonlinear differential algebraic equations.To be a numerical method,it is hard to exactly preserve all the specified conditions,which might deteriorate the accuracy of numerical solution.With this in mind,we develop a novel algorithm to find the solution of the optimal control problem of nonlinear Duffing oscillator,which can exactly satisfy all the required conditions for the minimality of the performance index.A new idea of shape functions method(SFM)is introduced,from which we can transform the optimal control problems to the initial value problems for the new variables,whose initial values are given arbitrarily,and meanwhile the terminal values are determined iteratively.Numerical examples confirm the high-performance of the iterative algorithms based on the SFM,which are convergence fast,and also provide very accurate solutions.The new algorithm is robust,even large noise is imposed on the input data.

Optimal tracking control for nonlinear large-scale systems with persistent disturbances

This paper studies the optimal control with zero steady-state error problem for nonlinear large-scale systems affected by external persistent disturbances.The nonlinear large-scale system is transformed into N nonlinear subsystems with interconnect terms.Based on the internal model principle,a disturbance compensator is constructed such that the ith subsystem with external persistent disturbances is transformed into an augmented subsystem without disturbances.According to the sensitivity approach,the optimal tracking control law for the ith nonlinear subsystem can be obtained.The optimal tracking control law for the nonlinear large-scale systems can be obtained.A numerical simulation shows that the method is effective.

Stability and optimal control for delayed rumor-spreading model with nonlinear incidence over heterogeneous networks

On the multilingual online social networks of global information sharing,the wanton spread of rumors has an enormous negative impact on people's lives.Thus,it is essential to explore the rumor-spreading rules in multilingual environment and formulate corresponding control strategies to reduce the harm caused by rumor propagation.In this paper,considering the multilingual environment and intervention mechanism in the rumor-spreading process,an improved ignorants–spreaders-1–spreaders-2–removers(I2SR)rumor-spreading model with time delay and the nonlinear incidence is established in heterogeneous networks.Firstly,based on the mean-field equations corresponding to the model,the basic reproduction number is derived to ensure the existence of rumor-spreading equilibrium.Secondly,by applying Lyapunov stability theory and graph theory,the global stability of rumor-spreading equilibrium is analyzed in detail.In particular,aiming at the lowest control cost,the optimal control scheme is designed to optimize the intervention mechanism,and the optimal control conditions are derived using the Pontryagin's minimum principle.Finally,some illustrative examples are provided to verify the effectiveness of the theoretical results.The results show that optimizing the intervention mechanism can effectively reduce the densities of spreaders-1 and spreaders-2 within the expected time,which provides guiding insights for public opinion managers to control rumors.

Discrete-Time Nonlinear Stochastic Optimal Control Problem Based on Stochastic Approximation Approach

In this paper, a computational approach is proposed for solving the discrete-time nonlinear optimal control problem, which is disturbed by a sequence of random noises. Because of the exact solution of such optimal control problem is impossible to be obtained, estimating the state dynamics is currently required. Here, it is assumed that the output can be measured from the real plant process. In our approach, the state mean propagation is applied in order to construct a linear model-based optimal control problem, where the model output is measureable. On this basis, an output error, which takes into account the differences between the real output and the model output, is defined. Then, this output error is minimized by applying the stochastic approximation approach. During the computation procedure, the stochastic gradient is established, so as the optimal solution of the model used can be updated iteratively. Once the convergence is achieved, the iterative solution approximates to the true optimal solution of the original optimal control problem, in spite of model-reality differences. For illustration, an example on a continuous stirred-tank reactor problem is studied, and the result obtained shows the applicability of the approach proposed. Hence, the efficiency of the approach proposed is highly recommended.

AN EFFICIENT PARALLEL PROCESSING OPTIMAL CONTROL SCHEME FOR A CLASS OF NONLINEAR COMPOSITE SYSTEMS

This article presents an efficient parallel processing approach for solving the opti- mal control problem of nonlinear composite systems. In this approach, the original high-order coupled nonlinear two-point boundary value problem （TPBVP） derived from the Pontrya- gin＇s maximum principle is first transformed into a sequence of lower-order deeoupled linear time-invariant TPBVPs. Then, an optimal control law which consists of both feedback and forward terms is achieved by using the modal series method for the derived sequence. The feedback term specified by local states of each subsystem is determined by solving a ma- trix Riccati differential equation. The forward term for each subsystem derived from its local information is an infinite sum of adjoint vectors. The convergence analysis and parallel processing capability of the proposed approach are also provided. To achieve an accurate feedforward-feedbaek suboptimal control, we apply a fast iterative algorithm with low com- putational effort. Finally, some comparative results are included to illustrate the effectiveness of the proposed approach.

Symplectic multi-level method for solving nonlinear optimal control problem

By converting an optimal control problem for nonlinear systems to a Hamiltonian system,a symplecitc-preserving method is proposed.The state and costate variables are approximated by the Lagrange polynomial.The state variables at two ends of the time interval are taken as independent variables.Based on the dual variable principle,nonlinear optimal control problems are replaced with nonlinear equations.Furthermore,in the implementation of the symplectic algorithm,based on the 2N algorithm,a multilevel method is proposed.When the time grid is refined from low level to high level,the initial state and costate variables of the nonlinear equations can be obtained from the Lagrange interpolation at the low level grid to improve efficiency.Numerical simulations show the precision and the efficiency of the proposed algorithm in this paper.

Robust Optimal Output Tracking Control of A Midwater Trawl System Based on T-S Fuzzy Nonlinear Model

A robust optimal output tracking control method for a midwater trawl system is investigated based on T-S fuzzy nonlinear model. A simplified nonlinear mathematical model is first employed to represent a midwater trawl system, and then a T-S fuzzy model is adopted to approximate the nonlinear system. Since the strong nonlinearities and the external disturbance of the trawling system, a mixed H2/H∞ fuzzy output tracking control strategy via T-S fuzzy system is proposed to regulate the trawl depth to follow a desired trajectory. The trawl depth can be regulated by adjusting the winch velocity automatically and the tracking error can be minimized according to the robust optimal criterion. In order to validate the proposed control method, a computer simulation is conducted. The simulation results indicate that the proposed fuzzy robust optimal controller make the trawl net rapidly follow the desired trajectory under the model uncertainties and the extemal disturbance caused by wave and current.

Optimal control of quadratic functionals for affine nonlinear systems

In this paper we analyze the optimal control problem for a class of afflne nonlinear systems under the assumption that the associated Lie algebra is nilpotent. The Lie brackets generated by the vector fields which define the nonlinear system represent a remarkable mathematical instrument for the control of affine systems. We determine the optimal control which corresponds to the nilpotent operator of the first order. In particular, we obtain the control that minimizes the energy of the given nonlinear system. Applications of this control to bilinear systems with first order nilpotent operator are considered.