Reinforcement learning based adaptive control for uncertain mechanical systems with asymptotic tracking

This paper mainly focuses on the development of a learning-based controller for a class of uncertain mechanical systems modeled by the Euler-Lagrange formulation.The considered system can depict the behavior of a large class of engineering systems,such as vehicular systems,robot manipulators and satellites.All these systems are often characterized by highly nonlinear characteristics,heavy modeling uncertainties and unknown perturbations,therefore,accurate-model-based nonlinear control approaches become unavailable.Motivated by the challenge,a reinforcement learning(RL)adaptive control methodology based on the actor-critic framework is investigated to compensate the uncertain mechanical dynamics.The approximation inaccuracies caused by RL and the exogenous unknown disturbances are circumvented via a continuous robust integral of the sign of the error(RISE)control approach.Different from a classical RISE control law,a tanh(·)function is utilized instead of a sign(·)function to acquire a more smooth control signal.The developed controller requires very little prior knowledge of the dynamic model,is robust to unknown dynamics and exogenous disturbances,and can achieve asymptotic output tracking.Eventually,co-simulations through ADAMS and MATLAB/Simulink on a three degrees-of-freedom(3-DOF)manipulator and experiments on a real-time electromechanical servo system are performed to verify the performance of the proposed approach.

Adaptive Event-Triggered Time-Varying Output Group Formation Containment Control of Heterogeneous Multiagent Systems

In this paper,a class of time-varying output group formation containment control problem of general linear hetero-geneous multiagent systems(MASs)is investigated under directed topology.The MAS is composed of a number of tracking leaders,formation leaders and followers,where two different types of leaders are used to provide reference trajectories for movement and to achieve certain formations,respectively.Firstly,compen-sators are designed whose states are estimations of tracking lead-ers,based on which,a controller is developed for each formation leader to accomplish the expected formation.Secondly,two event-triggered compensators are proposed for each follower to evalu-ate the state and formation information of the formation leaders in the same group,respectively.Subsequently,a control protocol is designed for each follower,utilizing the output information,to guide the output towards the convex hull generated by the forma-tion leaders within the group.Next,the triggering sequence in this paper is decomposed into two sequences,and the inter-event intervals of these two triggering conditions are provided to rule out the Zeno behavior.Finally,a numerical simulation is intro-duced to confirm the validity of the proposed results.

Adaptive Optimal Output Regulation of Interconnected Singularly Perturbed Systems With Application to Power Systems

This article studies the adaptive optimal output regulation problem for a class of interconnected singularly perturbed systems(SPSs) with unknown dynamics based on reinforcement learning(RL).Taking into account the slow and fast characteristics among system states,the interconnected SPS is decomposed into the slow time-scale dynamics and the fast timescale dynamics through singular perturbation theory.For the fast time-scale dynamics with interconnections,we devise a decentralized optimal control strategy by selecting appropriate weight matrices in the cost function.For the slow time-scale dynamics with unknown system parameters,an off-policy RL algorithm with convergence guarantee is given to learn the optimal control strategy in terms of measurement data.By combining the slow and fast controllers,we establish the composite decentralized adaptive optimal output regulator,and rigorously analyze the stability and optimality of the closed-loop system.The proposed decomposition design not only bypasses the numerical stiffness but also alleviates the high-dimensionality.The efficacy of the proposed methodology is validated by a load-frequency control application of a two-area power system.

Identification of time-varying system and energy-based optimization of adaptive control in seismically excited structure

The combination of structural health monitoring and vibration control is of great importance to provide components of smart structures.While synthetic algorithms have been proposed,adaptive control that is compatible with changing conditions still needs to be used,and time-varying systems are required to be simultaneously estimated with the application of adaptive control.In this research,the identification of structural time-varying dynamic characteristics and optimized simple adaptive control are integrated.First,reduced variations of physical parameters are estimated online using the multiple forgetting factor recursive least squares(MFRLS)method.Then,the energy from the structural vibration is simultaneously specified to optimize the control force with the identified parameters to be operational.Optimization is also performed based on the probability density function of the energy under the seismic excitation at any time.Finally,the optimal control force is obtained by the simple adaptive control(SAC)algorithm and energy coefficient.A numerical example and benchmark structure are employed to investigate the efficiency of the proposed approach.The simulation results revealed the effectiveness of the integrated online identification and optimal adaptive control in systems.

Distributed Adaptive Output Consensus of Unknown Heterogeneous Non-Minimum Phase Multi-Agent Systems

This article addresses the leader-following output consensus problem of heterogeneous linear multi-agent systems with unknown agent parameters under directed graphs.The dynamics of followers are allowed to be non-minimum phase with unknown arbitrary individual relative degrees.This is contrary to many existing works on distributed adaptive control schemes where agent dynamics are required to be minimum phase and often of the same relative degree.A distributed adaptive pole placement control scheme is developed,which consists of a distributed observer and an adaptive pole placement control law.It is shown that under the proposed distributed adaptive control scheme,all signals in the closed-loop system are bounded and the outputs of all the followers track the output of the leader asymptotically.The effectiveness of the proposed scheme is demonstrated by one practical example and one numerical example.

Event-Triggered Asymmetric Bipartite Consensus Tracking for Nonlinear Multi-Agent Systems Based on Model-Free Adaptive Control

In this paper,an asymmetric bipartite consensus problem for the nonlinear multi-agent systems with cooperative and antagonistic interactions is studied under the event-triggered mechanism.For the agents described by a structurally balanced signed digraph,the asymmetric bipartite consensus objective is firstly defined,assigning the agents'output to different signs and module values.Considering with the completely unknown dynamics of the agents,a novel event-triggered model-free adaptive bipartite control protocol is designed based on the agents'triggered outputs and an equivalent compact form data model.By utilizing the Lyapunov analysis method,the threshold of the triggering condition is obtained.Subsequently,the asymptotic convergence of the tracking error is deduced and a sufficient condition is obtained based on the contraction mapping principle.Finally,the simulation example further demonstrates the effectiveness of the protocol.

Adaptive Uniform Performance Control of Strict-Feedback Nonlinear Systems With Time-Varying Control Gain

In this paper,we present a novel adaptive performance control approach for strict-feedback nonparametric systems with unknown time-varying control coefficients,which mainly includes the following steps.Firstly,by introducing several key transformation functions and selecting the initial value of the time-varying scaling function,the symmetric prescribed performance with global and semi-global properties can be handled uniformly,without the need for control re-design.Secondly,to handle the problem of unknown time-varying control coefficient with an unknown sign,we propose an enhanced Nussbaum function(ENF)bearing some unique properties and characteristics,with which the complex stability analysis based on specific Nussbaum functions as commonly used is no longer required.Thirdly,by utilizing the core-function information technique,the nonparametric uncertainties in the system are gracefully handled so that no approximator is required.Furthermore,simulation results verify the effectiveness and benefits of the approach.

An Adaptive Neuro-Fuzzy Inference System to Improve Fractional Order Controller Performance

The design and analysis of a fractional order proportional integral deri-vate(FOPID)controller integrated with an adaptive neuro-fuzzy inference system(ANFIS)is proposed in this study.Afirst order plus delay time plant model has been used to validate the ANFIS combined FOPID control scheme.In the pro-posed adaptive control structure,the intelligent ANFIS was designed such that it will dynamically adjust the fractional order factors(λandµ)of the FOPID(also known as PIλDµ)controller to achieve better control performance.When the plant experiences uncertainties like external load disturbances or sudden changes in the input parameters,the stability and robustness of the system can be achieved effec-tively with the proposed control scheme.Also,a modified structure of the FOPID controller has been used in the present system to enhance the dynamic perfor-mance of the controller.An extensive MATLAB software simulation study was made to verify the usefulness of the proposed control scheme.The study has been carried out under different operating conditions such as external disturbances and sudden changes in input parameters.The results obtained using the ANFIS-FOPID control scheme are also compared to the classical fractional order PIλDµand conventional PID control schemes to validate the advantages of the control-lers.The simulation results confirm the effectiveness of the ANFIS combined FOPID controller for the chosen plant model.Also,the proposed control scheme outperformed traditional control methods in various performance metrics such as rise time,settling time and error criteria.

Robust adaptive neuro-fuzzy control of uncertain nonholonomic systems

In this paper, we present an adaptive neuro-fuzzy controller design for a class of uncertain nonholonomic systems in the perturbed chained form with unknown virtual control coefficients and strong drift nonlinearities. The robust adaptive neuro-fuzzy control laws are developed using state scaling and backstepping. Semiglobal uniform ultimate bound-edness of all the signals in the closed-loop are guaranteed, and the system states are proven to converge to a small neigh-borhood of zero. The control performance of the closed-loop system is guaranteed by appropriately choosing the design parameters. By using fuzzy logic approximation, the proposed control is free of control singularity problem. An adaptive control-based switching strategy is proposed to overcome the uncontrollability problem associated with x 0 (t 0 ) = 0.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Noise-tolerate and adaptive coefficient zeroing neural network for solving dynamic matrix square root

The solving of dynamic matrix square root(DMSR)problems is frequently encountered in many scientific and engineering fields.Although the original zeroing neural network is powerful for solving the DMSR,it cannot vanish the influence of the noise perturbations,and its constant-coefficient design scheme cannot accelerate the convergence speed.Therefore,a noise-tolerate and adaptive coefficient zeroing neural network(NTACZNN)is raised to enhance the robust noise immunity performance and accelerate the conver-gence speed simultaneously.Then,the global convergence and robustness of the pro-posed NTACZNN are theoretically analysed under an ideal environment and noise-perturbed circumstances.Furthermore,some illustrative simulation examples are designed and performed in order to substantiate the efficacy and advantage of the NTACZNN for the DMSR problem solution.Compared with some existing ZNNs,the proposed NTACZNN possesses advanced performance in terms of noise tolerance,solution accuracy,and convergence rate.

Adaptive Neuro-fuzzy Controller Design for Non-affine Nonlinear Systems

An adaptive neuro-fuzzy control is investigated for a class of non-affine nonlinear systems.To do so,rigorous description and quantification of the approximation error of the neuro-fuzzy controller are firstly discussed.Applying this result and Lyapunov stability theory,a novel updating algorithm to adapt the weights,centers,and widths of the neuro-fuzzy controller is presented.Consequently,the proposed design method is able to guarantee the stability of the closed-loop system and the convergence of the tracking error.Simulation results illustrate the effectiveness of the proposed adaptive neuro-fuzzy control scheme.

Dynamic Constraint-Driven Event-Triggered Control of Strict-Feedback Systems Without Max/Min Values on Irregular Constraints

This work proposes an event-triggered adaptive control approach for a class of uncertain nonlinear systems under irregular constraints.Unlike the constraints considered in most existing papers,here the external irregular constraints are considered and a constraints switching mechanism(CSM)is introduced to circumvent the difficulties arising from irregular output constraints.Based on the CSM,a new class of generalized barrier functions are constructed,which allows the control results to be independent of the maximum and minimum values(MMVs)of constraints and thus extends the existing results.Finally,we proposed a novel dynamic constraint-driven event-triggered strategy(DCDETS),under which the stress on signal transmission is reduced greatly and no constraints are violated by making a dynamic trade-off among system state,external constraints,and inter-execution intervals.It is proved that the system output is driven to close to the reference trajectory and the semi-global stability is guaranteed under the proposed control scheme,regardless of the external irregular output constraints.Simulation also verifies the effectiveness and benefits of the proposed method.

Noise-Tolerant ZNN-Based Data-Driven Iterative Learning Control for Discrete Nonaffine Nonlinear MIMO Repetitive Systems

Aiming at the tracking problem of a class of discrete nonaffine nonlinear multi-input multi-output(MIMO) repetitive systems subjected to separable and nonseparable disturbances, a novel data-driven iterative learning control(ILC) scheme based on the zeroing neural networks(ZNNs) is proposed. First, the equivalent dynamic linearization data model is obtained by means of dynamic linearization technology, which exists theoretically in the iteration domain. Then, the iterative extended state observer(IESO) is developed to estimate the disturbance and the coupling between systems, and the decoupled dynamic linearization model is obtained for the purpose of controller synthesis. To solve the zero-seeking tracking problem with inherent tolerance of noise,an ILC based on noise-tolerant modified ZNN is proposed. The strict assumptions imposed on the initialization conditions of each iteration in the existing ILC methods can be absolutely removed with our method. In addition, theoretical analysis indicates that the modified ZNN can converge to the exact solution of the zero-seeking tracking problem. Finally, a generalized example and an application-oriented example are presented to verify the effectiveness and superiority of the proposed process.

Characteristics Prediction Method of Electro-hydraulic Servo Valve Based on Rough Set and Adaptive Neuro-fuzzy Inference System

Synthesis characteristics of the electro-hydraulic servo valve are key factors to determine eligibility of the hydraulic production. Testing all synthesis characteristics of the electro-hydraulic servo valve after assembling leads to high repair rate and reject rate, so accurate prediction for the synthesis characteristics in the industrial production is particular important in decreasing the repair rate and the reject rate of the product. However, the research in forecasting synthesis characteristics of the electro-hydraulic servo valve is rare. In this work, a hybrid prediction method was proposed based on rough set（RS） and adaptive neuro-fuzzy inference system（ANFIS） in order to predict synthesis characteristics of electro-hydraulic servo valve. Since the geometric factors affecting the synthesis characteristics of the electro-hydraulic servo valve are from workers＇ experience, the inputs of the prediction method are uncertain. RS-based attributes reduction was used as the preprocessor, and then the exact geometric factors affecting the synthesis characteristics of the electro-hydraulic servo valve were obtained. On the basis of the exact geometric factors, ANFIS was used to build the final prediction model. A typical electro-hydraulic servo valve production was used to demonstrate the proposed prediction method. The prediction results showed that the proposed prediction method was more applicable than the artificial neural networks（ANN） in predicting the synthesis characteristics of electro-hydraulic servo valve, and the proposed prediction method was a powerful tool to predict synthesis characteristics of the electro-hydraulic servo valve. Moreover, with the use of the advantages of RS and ANFIS, the highly effective forecasting framework in this study can also be applied to other problems involving synthesis characteristics forecasting.

Adaptive Output-feedback Regulation for Nonlinear Delayed Systems Using Neural Network

A novel adaptive neural network （NN） output-feedback regulation algorithm for a class of nonlinear time-varying timedelay systems is proposed. Both the designed observer and controller are independent of time delay. Different from the existing results, where the upper bounding functions of time-delay terms are assumed to be known, we only use an NN to compensate for all unknown upper bounding functions without that assumption. The proposed design method is proved to be able to guarantee semi-global uniform ultimate boundedness of all the signals in the closed system, and the system output is proved to converge to a small neighborhood of the origin. The simulation results verify the effectiveness of the control scheme.

Adaptive Fuzzy Observer Backstepping Control for a Class of Uncertain Nonlinear Systems with Unknown Time-delay

In this paper, a new adaptive fuzzy backstepping control approach is developed for a class of nonlinear systems with unknown time-delay and unmeasured states. Using fuzzy logic systems to approximate the unknown nonlinear functions, a fuzzy state observer is designed for estimating the unmeasured states. On the basis of the state observer and applying the backstepping technique, an adaptive fuzzy observer control approach is developed. The main features of the proposed adaptive fuzzy control approach not only guarantees that all the signals of the closed-loop system are semiglobally uniformly ultimately bounded, but also contain less adaptation parameters to be updated on-line. Finally, simulation results are provided to show the effectiveness of the proposed approach.

Observer-based Adaptive Iterative Learning Control for Nonlinear Systems with Time-varying Delays

An observer-based adaptive iterative learning control （AILC） scheme is developed for a class of nonlinear systems with unknown time-varying parameters and unknown time-varying delays. The linear matrix inequality （LMI） method is employed to design the nonlinear observer. The designed controller contains a proportional-integral-derivative （PID） feedback term in time domain. The learning law of unknown constant parameter is differential-difference-type, and the learning law of unknown time-varying parameter is difference-type. It is assumed that the unknown delay-dependent uncertainty is nonlinearly parameterized. By constructing a Lyapunov-Krasovskii-like composite energy function （CEF）, we prove the boundedness of all closed-loop signals and the convergence of tracking error. A simulation example is provided to illustrate the effectiveness of the control algorithm proposed in this paper.

Robust adaptive fuzzy tracking control for a class of strict-feedback nonlinear systems based on backstepping technique

In this paper, the robust adaptive fuzzy tracking control problem is discussed for a class of perturbed strict-feedback nonlinear systems. The fuzzy logic systems in Mamdani type are used to approximate unknown nonlinear functions. A design scheme of the robust adaptive fuzzy controller is proposed by use of the backstepping technique. The proposed controller guarantees semi-global uniform ultimate boundedness of all the signals in the derived closed-loop system and achieves the good tracking performance. The possible controller singularity problem which may occur in some existing adaptive control schemes with feedback linearization techniques can be avoided. In addition, the number of the on-line adaptive parameters is not more than the order of the designed system. Finally, two simulation examples are used to demonstrate the effectiveness of the proposed control scheme.

Adaptive Variable Structure Control of MIMO Nonlinear Systems with Time-varying Delays and Unknown Dead-zones

In this paper, adaptive variable structure neural control is presented for a class of uncertain multi-input multi-output （MIMO） nonlinear systems with state time-varying delays and unknown nonlinear dead-zones. The unknown time-varying delay uncer- tainties are compensated for using appropriate Lyapunov-Krasovskii functionals in the design. The approach removes the assumption of linear function outside the deadband without necessarily constructing a dead-zone inverse as an added contribution. By utilizing the integral-type Lyapunov function and introducing an adaptive compensation term for the upper bound of the residual and optimal approximation error as well as the dead-zone disturbance, the closed-loop control system is proved to be semi-globally uniformly ultimately bounded. In addition, a modified adaptive control algorithm is given in order to avoid the high-frequency chattering phenomenon. Simulation results demonstrate the effectiveness of the approach.