Reinforcement learning based adaptive control for uncertain mechanical systems with asymptotic tracking

This paper mainly focuses on the development of a learning-based controller for a class of uncertain mechanical systems modeled by the Euler-Lagrange formulation.The considered system can depict the behavior of a large class of engineering systems,such as vehicular systems,robot manipulators and satellites.All these systems are often characterized by highly nonlinear characteristics,heavy modeling uncertainties and unknown perturbations,therefore,accurate-model-based nonlinear control approaches become unavailable.Motivated by the challenge,a reinforcement learning(RL)adaptive control methodology based on the actor-critic framework is investigated to compensate the uncertain mechanical dynamics.The approximation inaccuracies caused by RL and the exogenous unknown disturbances are circumvented via a continuous robust integral of the sign of the error(RISE)control approach.Different from a classical RISE control law,a tanh(·)function is utilized instead of a sign(·)function to acquire a more smooth control signal.The developed controller requires very little prior knowledge of the dynamic model,is robust to unknown dynamics and exogenous disturbances,and can achieve asymptotic output tracking.Eventually,co-simulations through ADAMS and MATLAB/Simulink on a three degrees-of-freedom(3-DOF)manipulator and experiments on a real-time electromechanical servo system are performed to verify the performance of the proposed approach.

Adaptive Trajectory Tracking Control for Nonholonomic Wheeled Mobile Robots:A Barrier Function Sliding Mode Approach

The trajectory tracking control performance of nonholonomic wheeled mobile robots(NWMRs)is subject to nonholonomic constraints,system uncertainties,and external disturbances.This paper proposes a barrier function-based adaptive sliding mode control(BFASMC)method to provide high-precision,fast-response performance and robustness for NWMRs.Compared with the conventional adaptive sliding mode control,the proposed control strategy can guarantee that the sliding mode variables converge to a predefined neighborhood of origin with a predefined reaching time independent of the prior knowledge of the uncertainties and disturbances bounds.Another advantage of the proposed algorithm is that the control gains can be adaptively adjusted to follow the disturbances amplitudes thanks to the barrier function.The benefit is that the overestimation of control gain can be eliminated,resulting in chattering reduction.Moreover,a modified barrier function-like control gain is employed to prevent the input saturation problem due to the physical limit of the actuator.The stability analysis and comparative experiments demonstrate that the proposed BFASMC can ensure the prespecified convergence performance of the NWMR system output variables and strong robustness against uncertainties/disturbances.

Multiple model PHD filter for tracking sharply maneuvering targets using recursive RANSAC based adaptive birth estimation

An algorithm to track multiple sharply maneuvering targets without prior knowledge about new target birth is proposed. These targets are capable of achieving sharp maneuvers within a short period of time, such as drones and agile missiles.The probability hypothesis density (PHD) filter, which propagates only the first-order statistical moment of the full target posterior, has been shown to be a computationally efficient solution to multitarget tracking problems. However, the standard PHD filter operates on the single dynamic model and requires prior information about target birth distribution, which leads to many limitations in terms of practical applications. In this paper,we introduce a nonzero mean, white noise turn rate dynamic model and generalize jump Markov systems to multitarget case to accommodate sharply maneuvering dynamics. Moreover, to adaptively estimate newborn targets’information, a measurement-driven method based on the recursive random sampling consensus (RANSAC) algorithm is proposed. Simulation results demonstrate that the proposed method achieves significant improvement in tracking multiple sharply maneuvering targets with adaptive birth estimation.

Robust Space-Time Adaptive Track-Before-Detect Algorithm Based on Persymmetry and Symmetric Spectrum

Underwater monopulse space-time adaptive track-before-detect method,which combines space-time adaptive detector(STAD)and the track-before-detect algorithm based on dynamic programming(DP-TBD),denoted as STAD-DP-TBD,can effectively detect low-speed weak targets.However,due to the complexity and variability of the underwater environment,it is difficult to obtain sufficient secondary data,resulting in a serious decline in the detection and tracking performance,and leading to poor robustness of the algorithm.In this paper,based on the adaptive matched filter(AMF)test and the RAO test,underwater monopulse AMF-DP-TBD algorithm and RAO-DP-TBD algorithm which incorporate persymmetry and symmetric spectrum,denoted as PSAMF-DP-TBD and PS-RAO-DP-TBD,are proposed and compared with the AMF-DP-TBD algorithm and RAO-DP-TBD algorithm based on persymmetry array,denoted as P-AMF-DP-TBD and P-RAO-DP-TBD.The simulation results show that the four methods can work normally with sufficient secondary data and slightly insufficient secondary data,but when the secondary data is severely insufficient,the P-AMF-DP-TBD and P-RAO-DP-TBD algorithms has failed while the PSAMF-DP-TBD and PS-RAO-DP-TBD algorithms still have good detection and tracking capabilities.

Mobility-Aware Adaptive Beam Tracking for Vehicles in Mm Wave Communication Networks

The millimeter wave(mm Wave)is a potential solution for high data rate communication due to its availability of large bandwidth.However,it is challenging to perform beam tracking in vehicular mm Wave communication systems due to high mobility and narrow beams.In this paper,an adaptive beam tracking algorithm is proposed to improve the network throughput performance while reducing the training signal overhead.In particular,based on the mobility prediction at base station(BS),a novel frame structure with dynamic bundled timeslot is designed.Moreover,an actor-critic reinforcement learning based algorithm is proposed to obtain the joint optimization of both beam width and the number of bundled timeslots,which makes the beam tracking adapt to the changing environment.Simulation results demonstrate that,compared with the traditional full scan and Kalman filter based beam tracking algorithms,our proposed algorithm can improve the time-averaged throughput by 11.34%and 24.86%respectively.With the newly designed frame structure,it also outperforms beam tracking with conventional frame structure,especially in scenarios with large range of vehicle speeds.

Adaptive saturated tracking control for solid launch vehicles in ascending based on differential inclusion stabilization

The large-range uncertainties of specific impulse,mass flow per second,aerodynamic coefficients and atmospheric density during rapid turning in solid launch vehicles(SLVs) ascending leads to the deviation of the actual trajectory from the reference one.One of the traditional trajectory tracking methods is to observe the uncertainties by Extended State Observer(ESO) and then modify the control commands.However,ESO cannot accurately estimate the uncertainties when the uncertainty ranges are large,which reduces the guidance accuracy.This paper introduces differential inclusion(DI) and designs a controller to solve the large-range parameter uncertainties problem.When above uncertainties have large ranges,it can be combined with the ascent dynamic equation and described as a DI system in the mathematical form of a set.If the DI system is stabilized,all the subsets are stabilized.Different from the traditional controllers,the parameters of the designed controller are calculated by the uncertain boundaries.Therefore,the controller can solve the problem of large-range parameter uncertainties of in ascending.Firstly,the ascent deviation system is obtained by linearization along the reference trajectory.The trajectory tracking system with engine parameters and aerodynamic uncertainties is described as an ascent DI system with respect to state deviation,which is called DI system.A DI adaptive saturation tracking controller(DIAST) is proposed to stabilize the DI system.Secondly,an improved barrier Lyapunov function(named time-varying tangent-log barrier Lyapunov function) is proposed to constrain the state deviations.Compared with traditional barrier Lyapunov function,it can dynamically adjust the boundary of deviation convergence,which improve the convergence rate and accuracy of altitude,velocity and LTIA deviation.In addition,the correction amplitudes of angle of attack(AOA) and angle of sideslip(AOS) need to be limited in order to guarantee that the overload constraint is not violated during actual flight.In this paper,a fixed time adaptive saturation compensation auxiliary system is designed to shorten the saturation time and accelerate the convergence rate,which eliminates the adverse effects caused by the saturation.Finally,it is proved that the state deviations are ultimately uniformly bounded under the action of DIAST controller.Simulation results show that the DI ascent tracking system is stabilized within the given uncertainty boundary values.The feasible bounds of uncertainty is broadened compared with Integrated Guidance and Control algorithm.Compared with Robust Gain-Scheduling Control method,the robustness to the engine parameters are greatly improved and the control variable is smoother.

Event-Triggered Asymmetric Bipartite Consensus Tracking for Nonlinear Multi-Agent Systems Based on Model-Free Adaptive Control

In this paper,an asymmetric bipartite consensus problem for the nonlinear multi-agent systems with cooperative and antagonistic interactions is studied under the event-triggered mechanism.For the agents described by a structurally balanced signed digraph,the asymmetric bipartite consensus objective is firstly defined,assigning the agents'output to different signs and module values.Considering with the completely unknown dynamics of the agents,a novel event-triggered model-free adaptive bipartite control protocol is designed based on the agents'triggered outputs and an equivalent compact form data model.By utilizing the Lyapunov analysis method,the threshold of the triggering condition is obtained.Subsequently,the asymptotic convergence of the tracking error is deduced and a sufficient condition is obtained based on the contraction mapping principle.Finally,the simulation example further demonstrates the effectiveness of the protocol.

Event-based performance guaranteed tracking control for constrained nonlinear system via adaptive dynamic programming method

An optimal tracking control problem for a class of nonlinear systems with guaranteed performance and asymmetric input constraints is discussed in this paper.The control policy is implemented by adaptive dynamic programming(ADP)algorithm under two event-based triggering mechanisms.It is often challenging to design an optimal control law due to the system deviation caused by asymmetric input constraints.First,a prescribed performance control technique is employed to guarantee the tracking errors within predetermined boundaries.Subsequently,considering the asymmetric input constraints,a discounted non-quadratic cost function is introduced.Moreover,in order to reduce controller updates,an event-triggered control law is developed for ADP algorithm.After that,to further simplify the complexity of controller design,this work is extended to a self-triggered case for relaxing the need for continuous signal monitoring by hardware devices.By employing the Lyapunov method,the uniform ultimate boundedness of all signals is proved to be guaranteed.Finally,a simulation example on a mass–spring–damper system subject to asymmetric input constraints is provided to validate the effectiveness of the proposed control scheme.

Practical Prescribed Time Tracking Control With Bounded Time-Varying Gain Under Non-Vanishing Uncertainties

This paper investigates the prescribed-time control(PTC) problem for a class of strict-feedback systems subject to non-vanishing uncertainties. The coexistence of mismatched uncertainties and non-vanishing disturbances makes PTC synthesis nontrivial. In this work, a control method that does not involve infinite time-varying gain is proposed, leading to a practical and global prescribed time tracking control solution for the strict-feedback systems, in spite of both the mismatched and nonvanishing uncertainties. Different from methods based on control switching to avoid the issue of infinite control gain that involves control discontinuity at the switching point, in our method a softening unit is exclusively included to ensure the continuity of the control action. Furthermore, in contrast to most existing prescribed-time control works where the control scheme is only valid on a finite time interval, in this work, the proposed control scheme is valid on the entire time interval. In addition, the prior information on the upper or lower bound of gi is not in need,enlarging the applicability of the proposed method. Both the theoretical analysis and numerical simulation confirm the effectiveness of the proposed control algorithm.

Practical prescribed-time fuzzy tracking control for uncertain nonlinear systems with time-varying actuators faults

The paper investigates the practical prescribed-time fuzzy tracking control problem for a category of nonlinear system subject to time-varying actuator faults.The presence of unknown nonlinear dynamics and actuator faults makes achieving tracking control within a prescribed-time challenging.To tackle this issue,we propose a novel practical prescribed-time fuzzy tracking control strategy,which is independent of the initial state of the system and does not rely on precise modeling of the system and actuators.We apply the approximation capabilities of fuzzy logic systems to handle the unknown nonlinear functions and unidentified actuator faults in the system.The piecewise controller and adaptive law constructed based on piecewise prescribed time-varying function and backstepping technique method establish the theoretical framework of practical prescribed-time tracking control,and extend the range of prescribed-time tracking control to infinity.Regardless of the initial conditions,the proposed control strategy can guarantee that all signals remain uniformly bounded within the practical prescribed time in the presence of unknown nonlinear item and time-varying actuator faults.Simulation example is presented to demonstrate the effectiveness of the proposed control strategy.

Adaptive Trajectory Tracking Control for a Nonholonomic Mobile Robot

As one of the core issues of the mobile robot motion control, trajectory tracking has received extensive attention. At present, the solution of the problem only takes kinematic or dynamic model into account separately, so that the presented strategy is difficult to realize satisfactory tracking quality in practical application. Considering the unknown parameters of two models, this paper presents an adaptive controller for solving the trajectory tracking problem of a mobile robot. Firstly, an adaptive kinematic controller utilized to generate the command of velocity is designed based on Backstepping method. Then, in order to make the real velocity of mobile robot reach the desired velocity asymptotically, a dynamic adaptive controller is proposed adopting reference model and Lyapunov stability theory. Finally, through simulating typical trajectories including circular trajectory, fold line and parabola trajectory in normal and perturbed cases, the results illustrate that the control scheme can solve the tracking problem effectively. The proposed control law, which can tune the kinematic and dynamic model parameters online and overcome external disturbances, provides a novel method for improving trajectory tracking performance of the mobile robot.

Adaptive Robust Motion Trajectory Tracking Control of Pneumatic Cylinders with LuGre Model-based Friction Compensation

Friction compensation is particularly important for motion trajectory tracking control of pneumatic cylinders at low speed movement. However, most of the existing model-based friction compensation schemes use simple classical models, which are not enough to address applications with high-accuracy position requirements. Furthermore, the friction force in the cylinder is time-varying, and there exist rather severe unmodelled dynamics and unknown disturbances in the pneumatic system. To deal with these problems effectively, an adaptive robust controller with LuGre model-based dynamic friction compensation is constructed. The proposed controller employs on-line recursive least squares estimation（RLSE） to reduce the extent of parametric uncertainties, and utilizes the sliding mode control method to attenuate the effects of parameter estimation errors, unmodelled dynamics and disturbances. In addition, in order to realize LuGre model-based friction compensation, the modified dual-observer structure for estimating immeasurable friction internal state is developed. Therefore, a prescribed motion tracking transient performance and final tracking accuracy can be guaranteed. Since the system model uncertainties are unmatched, the recursive backstepping design technology is applied. In order to solve the conflicts between the sliding mode control design and the adaptive control design, the projection mapping is used to condition the RLSE algorithm so that the parameter estimates are kept within a known bounded convex set. Finally, the proposed controller is tested for tracking sinusoidal trajectories and smooth square trajectory under different loads and sudden disturbance. The testing results demonstrate that the achievable performance of the proposed controller is excellent and is much better than most other studies in literature. Especially when a 0.5 Hz sinusoidal trajectory is tracked, the maximum tracking error is 0.96 mm and the average tracking error is 0.45 mm. This paper constructs an adaptive robust controller which can compensate the friction force in the cylinder.

Adaptive multi-feature tracking in particle swarm optimization based particle filter framework

This paper proposes a particle swarm optimization（PSO） based particle filter（PF） tracking framework,the embedded PSO makes particles move toward the high likelihood area to find the optimal position in the state transition stage,and simultaneously incorporates the newest observations into the proposal distribution in the update stage.In the proposed approach,likelihood measure functions involving multiple features are presented to enhance the performance of model fitting.Furthermore,the multi-feature weights are self-adaptively adjusted by a PSO algorithm throughout the tracking process.There are three main contributions.Firstly,the PSO algorithm is fused into the PF framework,which can efficiently alleviate the particles degeneracy phenomenon.Secondly,an effective convergence criterion for the PSO algorithm is explored,which can avoid particles getting stuck in local minima and maintain a greater particle diversity.Finally,a multi-feature weight self-adjusting strategy is proposed,which can significantly improve the tracking robustness and accuracy.Experiments performed on several challenging public video sequences demonstrate that the proposed tracking approach achieves a considerable performance.

Multiple-target tracking with adaptive sampling intervals for phased-array radar

A novel adaptive sampling interval algorithm for multitarget tracking is presented. This algorithm which is based on interacting multiple models incorporates the grey relational grade （GRG） into the particle swarm optimization （PSO）. Firstly, the desired tracking accuracy is set for each target. Secondly, sampling intervals are selected as particles, and then the advantage of the GRG is taken as the measurement function for resource management. Meanwhile, the fitness value of the PSO is used to measure the difference between desired tracking accuracy and estimated tracking accuracy. Finally, it is suggested that the radar should track the target whose prediction value of the next sampling interval is the smallest. Simulations show that the proposed method improves both the tracking accuracy and tracking efficiency of the phased-array radar.

Robust adaptive tracking control of robotic systems with uncertainties

To deal with the uncertainty factors of robotic systems, a robust adaptive tracking controller is proposed. The knowledge of the uncertainty factors is assumed to be unidentified; the proposed controller can guarantee robustness to parametric and dynamics uncertainties and can also reject any bounded, immeasurable disturbances entering the system. The stability of the proposed controller is proven by the Lyapunov method. The proposed controller can easily be implemented and the stability of the closed system can be ensured; the tracking error and adaptation parameter error are uniformly ultimately bounded （UUB）. Finally, some simulation examples are utilized to illustrate the control performance.

Adaptive Backstepping Tracking Control of a 6-DOF Unmanned Helicopter

This paper presents an adaptive backstepping control design for a class of unmanned helicopters with parametric uncertainties. The control objective is to let the helicopter track some pre-defined position and yaw trajectories. In order to facilitate the control design, we divide the helicopter s dynamic model into three subsystems. The proposed controller combines the backstepping method with online parameter update laws to achieve the control objective. The global asymptotical stability(GAS) of the closed-loop system is proved by a Lyapunov based stability analysis. Numerical simulations demonstrate that the controller can achieve good tracking performance in the presence of parametric uncertainties.

Parallel Control for Optimal Tracking via Adaptive Dynamic Programming

This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems.Unlike existing optimal state feedback control,the control input of the optimal parallel control is introduced into the feedback system.However,due to the introduction of control input into the feedback system,the optimal state feedback control methods can not be applied directly.To address this problem,an augmented system and an augmented performance index function are proposed firstly.Thus,the general nonlinear system is transformed into an affine nonlinear system.The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically.It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function.Moreover,an adaptive dynamic programming(ADP)technique is utilized to implement the optimal parallel tracking control using a critic neural network(NN)to approximate the value function online.The stability analysis of the closed-loop system is performed using the Lyapunov theory,and the tracking error and NN weights errors are uniformly ultimately bounded(UUB).Also,the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals.Finally,the effectiveness of the developed optimal parallel control method is verified in two cases.

H_∞tracking design for a class of decentralized nonlinear systems via fuzzy adaptive observer

A novel H∞ tracking-based decentralized indirect adaptive output feedback fuzzy controller for a class of uncertain large-scale nonlinear systems is developed. By virtue of the proper filtering of the observation error dynamics, the observer-based decentralized indirect adaptive fuzzy control scheme is presented for a class of large-scale nonlinear systems using the combination of H∞ tracking technique, a fuzzy adaptive observer and fuzzy inference systems. The output feedback and adaptation mechanisms are both robust and implementable indeed owing to their freedom from the unavailable observation error vector. All the signals of the closed-loop largescale system are guaranteed to stay uniformly bounded and the output errors take on H∞ tracking performance. Simulation results substantiate the effectiveness of the proposed scheme.

Trajectory tracking of powered parafoil based on characteristic model based all-coefficient adaptive control

One of the primary difficulties in using powered parafoil(PPF) systems is the lack of effective trajectory tracking controllers since the trajectory tracking control is the essential operation for PPF to accomplish autonomous tasks. The characteristic model(CM) based all-coefficient adaptive control(ACAC) designed for PPF systems in horizontal and vertical trajectory control is proposed. The method is easy to use and convenient to adjust and test. Just a few parameters are adapted during the control process. In application, vertical and horizontal CMs are designed and ACAC controllers are constructed to control vertical altitude and horizontal trajectory of PPF based on the proposed CMs, respectively. Result analysis of different simulations shows that the applied ACAC control method is effective for trajectory tracking of the PPF systems and the approach guarantees the transient performance of the PPF systems with better disturbance rejection ability.

Discounted Iterative Adaptive Critic Designs With Novel Stability Analysis for Tracking Control

The core task of tracking control is to make the controlled plant track a desired trajectory.The traditional performance index used in previous studies cannot eliminate completely the tracking error as the number of time steps increases.In this paper,a new cost function is introduced to develop the value-iteration-based adaptive critic framework to solve the tracking control problem.Unlike the regulator problem,the iterative value function of tracking control problem cannot be regarded as a Lyapunov function.A novel stability analysis method is developed to guarantee that the tracking error converges to zero.The discounted iterative scheme under the new cost function for the special case of linear systems is elaborated.Finally,the tracking performance of the present scheme is demonstrated by numerical results and compared with those of the traditional approaches.