Reinforcement learning based adaptive control for uncertain mechanical systems with asymptotic tracking

This paper mainly focuses on the development of a learning-based controller for a class of uncertain mechanical systems modeled by the Euler-Lagrange formulation.The considered system can depict the behavior of a large class of engineering systems,such as vehicular systems,robot manipulators and satellites.All these systems are often characterized by highly nonlinear characteristics,heavy modeling uncertainties and unknown perturbations,therefore,accurate-model-based nonlinear control approaches become unavailable.Motivated by the challenge,a reinforcement learning(RL)adaptive control methodology based on the actor-critic framework is investigated to compensate the uncertain mechanical dynamics.The approximation inaccuracies caused by RL and the exogenous unknown disturbances are circumvented via a continuous robust integral of the sign of the error(RISE)control approach.Different from a classical RISE control law,a tanh(·)function is utilized instead of a sign(·)function to acquire a more smooth control signal.The developed controller requires very little prior knowledge of the dynamic model,is robust to unknown dynamics and exogenous disturbances,and can achieve asymptotic output tracking.Eventually,co-simulations through ADAMS and MATLAB/Simulink on a three degrees-of-freedom(3-DOF)manipulator and experiments on a real-time electromechanical servo system are performed to verify the performance of the proposed approach.

Adaptive Event-Triggered Time-Varying Output Group Formation Containment Control of Heterogeneous Multiagent Systems

In this paper,a class of time-varying output group formation containment control problem of general linear hetero-geneous multiagent systems(MASs)is investigated under directed topology.The MAS is composed of a number of tracking leaders,formation leaders and followers,where two different types of leaders are used to provide reference trajectories for movement and to achieve certain formations,respectively.Firstly,compen-sators are designed whose states are estimations of tracking lead-ers,based on which,a controller is developed for each formation leader to accomplish the expected formation.Secondly,two event-triggered compensators are proposed for each follower to evalu-ate the state and formation information of the formation leaders in the same group,respectively.Subsequently,a control protocol is designed for each follower,utilizing the output information,to guide the output towards the convex hull generated by the forma-tion leaders within the group.Next,the triggering sequence in this paper is decomposed into two sequences,and the inter-event intervals of these two triggering conditions are provided to rule out the Zeno behavior.Finally,a numerical simulation is intro-duced to confirm the validity of the proposed results.

Adaptive Optimal Output Regulation of Interconnected Singularly Perturbed Systems With Application to Power Systems

This article studies the adaptive optimal output regulation problem for a class of interconnected singularly perturbed systems(SPSs) with unknown dynamics based on reinforcement learning(RL).Taking into account the slow and fast characteristics among system states,the interconnected SPS is decomposed into the slow time-scale dynamics and the fast timescale dynamics through singular perturbation theory.For the fast time-scale dynamics with interconnections,we devise a decentralized optimal control strategy by selecting appropriate weight matrices in the cost function.For the slow time-scale dynamics with unknown system parameters,an off-policy RL algorithm with convergence guarantee is given to learn the optimal control strategy in terms of measurement data.By combining the slow and fast controllers,we establish the composite decentralized adaptive optimal output regulator,and rigorously analyze the stability and optimality of the closed-loop system.The proposed decomposition design not only bypasses the numerical stiffness but also alleviates the high-dimensionality.The efficacy of the proposed methodology is validated by a load-frequency control application of a two-area power system.

Robust adaptive leaderless consensus of unknown non-minimum phase linear multi-agent systems subject to disturbances and/or unmodeled dynamics

This article investigates the problem of robust adaptive leaderless consensus for heterogeneous uncertain nonminimumphase linear multi-agent systems over directed communication graphs. Each agent is assumed tobe of unknown nominal dynamics and also subject to external disturbances and/or unmodeled dynamics. Anovel distributed robust adaptive control strategy is proposed. It is shown that the robust adaptive leaderlessconsensus problem is solved with the proposed control strategy under some sufficient conditions. Two examplesare provided to demonstrate the efficacy of the proposed control strategy.

Identification of time-varying system and energy-based optimization of adaptive control in seismically excited structure

The combination of structural health monitoring and vibration control is of great importance to provide components of smart structures.While synthetic algorithms have been proposed,adaptive control that is compatible with changing conditions still needs to be used,and time-varying systems are required to be simultaneously estimated with the application of adaptive control.In this research,the identification of structural time-varying dynamic characteristics and optimized simple adaptive control are integrated.First,reduced variations of physical parameters are estimated online using the multiple forgetting factor recursive least squares(MFRLS)method.Then,the energy from the structural vibration is simultaneously specified to optimize the control force with the identified parameters to be operational.Optimization is also performed based on the probability density function of the energy under the seismic excitation at any time.Finally,the optimal control force is obtained by the simple adaptive control(SAC)algorithm and energy coefficient.A numerical example and benchmark structure are employed to investigate the efficiency of the proposed approach.The simulation results revealed the effectiveness of the integrated online identification and optimal adaptive control in systems.

Distributed Adaptive Output Consensus of Unknown Heterogeneous Non-Minimum Phase Multi-Agent Systems

This article addresses the leader-following output consensus problem of heterogeneous linear multi-agent systems with unknown agent parameters under directed graphs.The dynamics of followers are allowed to be non-minimum phase with unknown arbitrary individual relative degrees.This is contrary to many existing works on distributed adaptive control schemes where agent dynamics are required to be minimum phase and often of the same relative degree.A distributed adaptive pole placement control scheme is developed,which consists of a distributed observer and an adaptive pole placement control law.It is shown that under the proposed distributed adaptive control scheme,all signals in the closed-loop system are bounded and the outputs of all the followers track the output of the leader asymptotically.The effectiveness of the proposed scheme is demonstrated by one practical example and one numerical example.

Adaptive Uniform Performance Control of Strict-Feedback Nonlinear Systems With Time-Varying Control Gain

In this paper,we present a novel adaptive performance control approach for strict-feedback nonparametric systems with unknown time-varying control coefficients,which mainly includes the following steps.Firstly,by introducing several key transformation functions and selecting the initial value of the time-varying scaling function,the symmetric prescribed performance with global and semi-global properties can be handled uniformly,without the need for control re-design.Secondly,to handle the problem of unknown time-varying control coefficient with an unknown sign,we propose an enhanced Nussbaum function(ENF)bearing some unique properties and characteristics,with which the complex stability analysis based on specific Nussbaum functions as commonly used is no longer required.Thirdly,by utilizing the core-function information technique,the nonparametric uncertainties in the system are gracefully handled so that no approximator is required.Furthermore,simulation results verify the effectiveness and benefits of the approach.

Time/Space and Inertial System Reconsidered Based on the Adaptive Dynamical View

We have succeeded in 2-slit interference simulation by assuming that a travelling particle interacts with its environment, getting information on the environmental condition according to the adaptive dynamics by Ohya, thus proposed the possibility that the entanglement comes from the interaction with the environment (Ando et al., 2023). This concept means that there should be no isolated or inertial system other than our unique universe space. Taking this message into account and assuming that the signal velocity is constant against our unique universe space, we reconsidered the inertial system and relativity theory by Galilei and Einstein and found several misunderstandings and errors. Time delay and Lorentz shrinkage were derived similarly to the prediction by special relativity theory, but Lorentz transformation and 4-dimensional time/space view were not. They must have implicitly and unconsciously assumed that any signals transfer information without giving any influences to any systems different from our adaptive dynamical view. We propose that their relativity theories should be reinterpreted in view of adaptive dynamics.

Event-Triggered Asymmetric Bipartite Consensus Tracking for Nonlinear Multi-Agent Systems Based on Model-Free Adaptive Control

In this paper,an asymmetric bipartite consensus problem for the nonlinear multi-agent systems with cooperative and antagonistic interactions is studied under the event-triggered mechanism.For the agents described by a structurally balanced signed digraph,the asymmetric bipartite consensus objective is firstly defined,assigning the agents'output to different signs and module values.Considering with the completely unknown dynamics of the agents,a novel event-triggered model-free adaptive bipartite control protocol is designed based on the agents'triggered outputs and an equivalent compact form data model.By utilizing the Lyapunov analysis method,the threshold of the triggering condition is obtained.Subsequently,the asymptotic convergence of the tracking error is deduced and a sufficient condition is obtained based on the contraction mapping principle.Finally,the simulation example further demonstrates the effectiveness of the protocol.

Robust adaptive dynamic surface control for nonlinear uncertain systems

We propose a new method for robust adaptive backstepping control of nonlinear systems with parametric uncertainties and disturbances in the strict feedback form. The method is called dynamic surface control. Traditional backstepping algorithms require repeated differentiations of the modelled nonlinearities. The addition of n first order low pass filters allows the algorithm to be implemented without differentiating any model nonlinearities, thus ending the complexity arising due to the 'explosion of terms' that makes other methods difficult to implement in practice. The combined robust adaptive backstepping/first order filter system is proved to be semiglobally asymptotically stable for sufficiently fast filters by a singular perturbation approach. The simulation results demonstrate the feasibility and effectiveness of the controller designed by the method.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Adaptive Nonlinear PID Control and Rule-Based Compensation for Systems with Backlash

The control of dynamic nonlinear systems with unknown backlash was considered. By using an efficient approach to estimate the unknown backlash parameters, a rule? based backlash compensator was presented for canceling the effect of backlash. Adaptive nonlinear PID controller together with rule? based backlash compensator was developed and a satisfactory tracking performance was achieved. Simulation results demonstrated the effectiveness of the proposed method.

Noise-tolerate and adaptive coefficient zeroing neural network for solving dynamic matrix square root

The solving of dynamic matrix square root(DMSR)problems is frequently encountered in many scientific and engineering fields.Although the original zeroing neural network is powerful for solving the DMSR,it cannot vanish the influence of the noise perturbations,and its constant-coefficient design scheme cannot accelerate the convergence speed.Therefore,a noise-tolerate and adaptive coefficient zeroing neural network(NTACZNN)is raised to enhance the robust noise immunity performance and accelerate the conver-gence speed simultaneously.Then,the global convergence and robustness of the pro-posed NTACZNN are theoretically analysed under an ideal environment and noise-perturbed circumstances.Furthermore,some illustrative simulation examples are designed and performed in order to substantiate the efficacy and advantage of the NTACZNN for the DMSR problem solution.Compared with some existing ZNNs,the proposed NTACZNN possesses advanced performance in terms of noise tolerance,solution accuracy,and convergence rate.

Dynamic Constraint-Driven Event-Triggered Control of Strict-Feedback Systems Without Max/Min Values on Irregular Constraints

This work proposes an event-triggered adaptive control approach for a class of uncertain nonlinear systems under irregular constraints.Unlike the constraints considered in most existing papers,here the external irregular constraints are considered and a constraints switching mechanism(CSM)is introduced to circumvent the difficulties arising from irregular output constraints.Based on the CSM,a new class of generalized barrier functions are constructed,which allows the control results to be independent of the maximum and minimum values(MMVs)of constraints and thus extends the existing results.Finally,we proposed a novel dynamic constraint-driven event-triggered strategy(DCDETS),under which the stress on signal transmission is reduced greatly and no constraints are violated by making a dynamic trade-off among system state,external constraints,and inter-execution intervals.It is proved that the system output is driven to close to the reference trajectory and the semi-global stability is guaranteed under the proposed control scheme,regardless of the external irregular output constraints.Simulation also verifies the effectiveness and benefits of the proposed method.

Noise-Tolerant ZNN-Based Data-Driven Iterative Learning Control for Discrete Nonaffine Nonlinear MIMO Repetitive Systems

Aiming at the tracking problem of a class of discrete nonaffine nonlinear multi-input multi-output(MIMO) repetitive systems subjected to separable and nonseparable disturbances, a novel data-driven iterative learning control(ILC) scheme based on the zeroing neural networks(ZNNs) is proposed. First, the equivalent dynamic linearization data model is obtained by means of dynamic linearization technology, which exists theoretically in the iteration domain. Then, the iterative extended state observer(IESO) is developed to estimate the disturbance and the coupling between systems, and the decoupled dynamic linearization model is obtained for the purpose of controller synthesis. To solve the zero-seeking tracking problem with inherent tolerance of noise,an ILC based on noise-tolerant modified ZNN is proposed. The strict assumptions imposed on the initialization conditions of each iteration in the existing ILC methods can be absolutely removed with our method. In addition, theoretical analysis indicates that the modified ZNN can converge to the exact solution of the zero-seeking tracking problem. Finally, a generalized example and an application-oriented example are presented to verify the effectiveness and superiority of the proposed process.

ADAPTIVE TRACKING CONTROL FOR A CLASS OF NONLINEAR COMPOSITE SYSTEMS *

In this paper, the problem of adaptive tracking control for a class of nonlinear large scale systems with unknown parameters entering linearly is discussed. Based on the theory of input output linearization of nonlinear systems, direct adaptive control schemes are presented to achieve bounded tracking. The proposed control schemes are robust with respect to the uncertainties in interconnection structure as well as subsystem dynamics. A numerical example is given to illustrate the efficiency of this method.

Event-triggered robust cooperative output regulation for a class of linear multi-agent systems with an unknown exosystem

This paper investigates the robust cooperative output regulation problem for a class of heterogeneousuncertain linear multi-agent systems with an unknown exosystem via event-triggered control (ETC). By utilizingthe internal model approach and the adaptive control technique, a distributed adaptive internal model isconstructed for each agent. Then, based on this internal model, a fully distributed ETC strategy composed ofa distributed event-triggered adaptive output feedback control law and a distributed dynamic event-triggeringmechanism is proposed, in which each agent updates its control input at its own triggering time instants. It isshown that under the proposed ETC strategy, the robust cooperative output regulation problem can be solvedwithout requiring either the global information associated with the communication topology or the bounds ofthe uncertain or unknown parameters in each agent and the exosystem. A numerical example is provided toillustrate the effectiveness of the proposed control strategy.

Generalized projective synchronization of chaotic systems via adaptive learning control

In this paper, a learning control approach is applied to the generalized projective synchronisation （GPS） of different chaotic systems with unknown periodically time-varying parameters. Using the Lyapunov--Krasovskii functional stability theory, a differential-difference mixed parametric learning law and an adaptive learning control law are constructed to make the states of two different chaotic systems asymptotically synchronised. The scheme is successfully applied to the generalized projective synchronisation between the Lorenz system and Chen system. Moreover, numerical simulations results are used to verify the effectiveness of the proposed scheme.

Adaptive Robust Control for a Class of Uncertain MIMO Non-affine Nonlinear Systems

In this paper, the adaptive robust tracking control scheme is proposed for a class of multi-input and multi-output (MIMO) non-affine systems with uncertain structure and parameters, unknown control direction and unknown external disturbance based on backstepping technique. The MIMO non-affine system is first transformed into a time-varying system with strict feedback structure using the mean value theorem, and then the bounded time-varying parameters are estimated by adaptive algorithms with projection. To handle the possible 'controller singularity' problem caused by unknown control direction, a Nussbaum function is employed, and the dynamic surface control (DSC) method is applied to solve the problem of 'explosion of complexity' in backstepping control. It is proved that the proposed control scheme can guarantee that all signals of the closed-loop system are bounded through Lyapunov stability theorem and decoupled backstepping method. Simulation results are presented to illustrate the effectiveness of the proposed control scheme. © 2014 Chinese Association of Automation.

Adaptive Output-feedback Regulation for Nonlinear Delayed Systems Using Neural Network

A novel adaptive neural network （NN） output-feedback regulation algorithm for a class of nonlinear time-varying timedelay systems is proposed. Both the designed observer and controller are independent of time delay. Different from the existing results, where the upper bounding functions of time-delay terms are assumed to be known, we only use an NN to compensate for all unknown upper bounding functions without that assumption. The proposed design method is proved to be able to guarantee semi-global uniform ultimate boundedness of all the signals in the closed system, and the system output is proved to converge to a small neighborhood of the origin. The simulation results verify the effectiveness of the control scheme.