Event-based performance guaranteed tracking control for constrained nonlinear system via adaptive dynamic programming method

An optimal tracking control problem for a class of nonlinear systems with guaranteed performance and asymmetric input constraints is discussed in this paper.The control policy is implemented by adaptive dynamic programming(ADP)algorithm under two event-based triggering mechanisms.It is often challenging to design an optimal control law due to the system deviation caused by asymmetric input constraints.First,a prescribed performance control technique is employed to guarantee the tracking errors within predetermined boundaries.Subsequently,considering the asymmetric input constraints,a discounted non-quadratic cost function is introduced.Moreover,in order to reduce controller updates,an event-triggered control law is developed for ADP algorithm.After that,to further simplify the complexity of controller design,this work is extended to a self-triggered case for relaxing the need for continuous signal monitoring by hardware devices.By employing the Lyapunov method,the uniform ultimate boundedness of all signals is proved to be guaranteed.Finally,a simulation example on a mass–spring–damper system subject to asymmetric input constraints is provided to validate the effectiveness of the proposed control scheme.

Hybrid TDOA/FDOA and track optimization of UAV swarm based on A-optimality

The source location based on the hybrid time difference of arrival(TDOA)/frequency difference of arrival(FDOA) is a basic problem in wireless sensor networks, and the layout of sensors in the hybrid TDOA/FDOA positioning will greatly affect the accuracy of positioning. Using unmanned aerial vehicle(UAV) as base stations, by optimizing the trajectory of the UAV swarm, an optimal positioning configuration is formed to improve the accuracy of the target position and velocity estimation. In this paper, a hybrid TDOA/FDOA positioning model is first established, and the positioning accuracy of the hybrid TDOA/FDOA under different positioning configurations and different measurement errors is simulated by the geometric dilution of precision(GDOP) factor. Second, the Cramer-Rao lower bound(CRLB) matrix of hybrid TDOA/FDOA location under different moving states of the target is derived theoretically, the objective function of the track optimization is obtained, and the track of the UAV swarm is optimized in real time. The simulation results show that the track optimization effectively improves the accuracy of the target position and velocity estimation.

Approximate Optimal Tracking Control for Near-Surface AUVs with Wave Disturbances

This paper considers the optimal trajectory tracking control problem for near-surface autonomous underwater vehicles(AUVs) in the presence of wave disturbances. An approximate optimal tracking control(AOTC) approach is proposed. Firstly, a six-degrees-of-freedom(six-DOF) AUV model with its body-fixed coordinate system is decoupled and simplified and then a nonlinear control model of AUVs in the vertical plane is given. Also, an exosystem model of wave disturbances is constructed based on Hirom approximation formula. Secondly, the time-parameterized desired trajectory which is tracked by the AUV's system is represented by the exosystem. Then, the coupled two-point boundary value(TPBV) problem of optimal tracking control for AUVs is derived from the theory of quadratic optimal control. By using a recently developed successive approximation approach to construct sequences, the coupled TPBV problem is transformed into a problem of solving two decoupled linear differential sequences of state vectors and adjoint vectors. By iteratively solving the two equation sequences, the AOTC law is obtained, which consists of a nonlinear optimal feedback item, an expected output tracking item, a feedforward disturbances rejection item, and a nonlinear compensatory term. Furthermore, a wave disturbances observer model is designed in order to solve the physically realizable problem. Simulation is carried out by using the Remote Environmental Unit(REMUS) AUV model to demonstrate the effectiveness of the proposed algorithm.

TDOA and track optimization of UAV swarm based on D-optimality

To solve the problem of time difference of arrival(TDOA)positioning and tracking of targets by the unmanned aerial vehicles(UAV)swarm in future air combat,this paper adopts the TDOA positioning method and uses time difference sensors of the UAV swarm to locate target radiation sources.Firstly,a TDOA model for the target is set up for the UAV swarm under the condition that the error variance varies with the received signal-to-noise ratio.The accuracy of the positioning error is analyzed by geometric dilution of precision(GDOP).The D-optimality criterion of the positioning model is theoretically derived.The target is positioned and settled,and the maximum value of the Fisher information matrix determinant is used as the optimization objective function to optimize the track of the UAV in real time.Simulation results show that the track optimization improves the positioning accuracy and stability of the UAV swarm to the target.

Optimal tracking control for automatic transmission shift process

In order to improve the shift quality, a linear quadratic optimal tracking control algorithm for automatic transmission shift process is proposed. The dynamic equations of the shift process are derived using a Lagrange method. And a powertrain model is built in the Matlab/Simulink and veri- fied by the measurements. Considering the shift jerk and friction loss during the shift process, the tracking trajectories of the turbine speed and output shaft speed are defined. Furthermore, the linear quadratic optimal tracking control performance index is proposed. Based on the Pontryagin＇ s mini- mum principle, the optimal control law of the shift process is presented. Finally, the simulation study of the 1 - 2 upshift process under different load conditions is carried out with the powertrain model. The simulation results demonstrate that the shift jerk and friction loss can be significantly re- duced by applying the proposed optimal tracking control method.

Optimal tracking control for linear time-delay large-scale systems with persistent disturbances

An optimal tracking control （OTC） problem for linear time-delay large-scale systems affected by external persistent disturbances is investigated. Based on the internal model principle, a disturbance compensator is constructed. The system with persistent disturbances is transformed into an augmented system without persistent disturbances. The original OTC problem of linear time-delay system is transformed into a sequence of linear two- point boundary value （TPBV） problems by introducing a sensitivity parameter and expanding Maclaurin series around it. By solving an OTC law of the augmented system, the OTC law of the original system is obtained. A numerical simulation is provided to illustrate the effectiveness of the proposed method.

Optimal tracking control for nonlinear large-scale systems with persistent disturbances

This paper studies the optimal control with zero steady-state error problem for nonlinear large-scale systems affected by external persistent disturbances.The nonlinear large-scale system is transformed into N nonlinear subsystems with interconnect terms.Based on the internal model principle,a disturbance compensator is constructed such that the ith subsystem with external persistent disturbances is transformed into an augmented subsystem without disturbances.According to the sensitivity approach,the optimal tracking control law for the ith nonlinear subsystem can be obtained.The optimal tracking control law for the nonlinear large-scale systems can be obtained.A numerical simulation shows that the method is effective.

Off-policy integral reinforcement learning optimal tracking control for continuous-time chaotic systems

This paper estimates an off-policy integral reinforcement learning（IRL） algorithm to obtain the optimal tracking control of unknown chaotic systems. Off-policy IRL can learn the solution of the HJB equation from the system data generated by an arbitrary control. Moreover, off-policy IRL can be regarded as a direct learning method, which avoids the identification of system dynamics. In this paper, the performance index function is first given based on the system tracking error and control error. For solving the Hamilton–Jacobi–Bellman（HJB） equation, an off-policy IRL algorithm is proposed.It is proven that the iterative control makes the tracking error system asymptotically stable, and the iterative performance index function is convergent. Simulation study demonstrates the effectiveness of the developed tracking control method.

Approximation-error-ADP-based optimal tracking control for chaotic systems with convergence proof

In this paper, an optimal tracking control scheme is proposed for a class of discrete-time chaotic systems using the approximation-error-based adaptive dynamic programming （ADP） algorithm. Via the system transformation, the optimal tracking problem is transformed into an optimal regulation problem, and then the novel optimal tracking control method is proposed. It is shown that for the iterative ADP algorithm with finite approximation error, the iterative performance index functions can converge to a finite neighborhood of the greatest lower bound of all performance index functions under some convergence conditions. Two examples are given to demonstrate the validity of the proposed optimal tracking control scheme for chaotic systems.

Best tracking performance for integrator and dead time plant

The optimal tracking performance for integrator and dead time plant in the case where plant uncertainty and control energy constraints are to be considered jointly is inrestigated. Firstly, an average cost function of the tracking error and the plant input energy over a class of stochastic model errors are defined. Then, we obtain an internal model controller design method that minimizes the average performance and further studies optimal tracking performance for integrator and dead time plant in the simultaneous presence of plant uncertainty and control energy constraint. The results can be used to evaluate optimal tracking performance and control energy in practical designs.

OPTIMAL TRACK SEEKING CONTROL OF DUAL-STAGE ACTUATOR FOR HIGH DENSITY HARD DISK DRIVES

Based on generalized the variation method, by introducing Hamilton function and Lagrange multiplier, this paper proposed a linear quadratic optimal control strategy for an incomplete controllable system with fixed terminal state and time. Applying the proposed optimal control to the simple two-input dual-stage actuator magnetic head positioning system with three degrees-of-freedom, the simulation results show that the system has no residual vibration at the terminal position and time, which can reduce the total access time during head positioning process. To verify the validation of the optimal control strategy of three degrees-of-freedom spring-mass models in actual magnetic head positioning of hard disk drives, a finite element model of an actual magnetic head positioning system is presented. Substituting the optimal control force from simple three degrees-of-freedom spring-mass models into the finite element model, the simulation results show that the magnetic head also has no residual vibration at the end of track-to-track travel. That is to say, the linear quadratic optimal control technique based on simple two-input dual- stage actuator system with three degrees-of-freedom proposed in this paper is of high reliability for the industrial application of an actual magnetic head positioning system.

Output neighboring-extremal and NCO tracking control in the optimal operation of CSTR

A new optimizing framework of process operation is proposed to deal with optimizing op- eration of continuous stirred tank reactor （CSTR）. The optimization framework includes two layers： the first layer, necessary condition of optimally （NCO） tracking controller, calculates the optimal set-point of the process; and the second layer, output neighboring-extremal controller, calculates the input values of the controlled plant. The algorithm design and convergent analysis of output neighboring-extremal controller are discussed emphatically, and in the case of existing parametric uncertainty, the approach is shown to converge to the optimum atmost in two iterations. At last the approach is illustrated by simulation results for a dynamic CSTR.

Development of a Bias Compensating Q-Learning Controller for a Multi-Zone HVAC Facility

We present the development of a bias compensating reinforcement learning(RL)algorithm that optimizes thermal comfort(by minimizing tracking error)and control utilization(by penalizing setpoint deviations)in a multi-zone heating,ventilation,and air-conditioning(HVAC)lab facility subject to unmeasurable disturbances and unknown dynamics.It is shown that the presence of unmeasurable disturbance results in an inconsistent learning equation in traditional RL controllers leading to parameter estimation bias(even with integral action support),and in the extreme case,the divergence of the learning algorithm.We demonstrate this issue by applying the popular Q-learning algorithm to linear quadratic regulation(LQR)of a multi-zone HVAC environment and showing that,even with integral support,the algorithm exhibits bias issue during the learning phase when the HVAC disturbance is unmeasurable due to unknown heat gains,occupancy variations,light sources,and outside weather changes.To address this difficulty,we present a bias compensating learning equation that learns a lumped bias term as a result of disturbances(and possibly other sources)in conjunction with the optimal control parameters.Experimental results show that the proposed scheme not only recovers the bias-free optimal control parameters but it does so without explicitly learning the dynamic model or estimating the disturbances,demonstrating the effectiveness of the algorithm in addressing the above challenges.

Adaptive learning-based optimal tracking control system design and analysis of a disturbed nonlinear hypersonic vehicle model

We propose an adaptive learning-based optimal control scheme for height-velocity control models considering model uncertainties and external disturbances of hypersonic winged-cone vehicles.The longitudinal nonlinear model is first established and transformed into the control-oriented error equations,and the control scheme is organized by a steady-compensation combination.To overcome and eliminate the impact of model uncertainties and external disturbances,an adaptive radial basis function neural network(RBFNN)is designed by a q-gradient approach.Taking the height-velocity error system with estimated uncertainties into account,the adaptive learning-based optimal tracking control(ALOTC)scheme is proposed by combining the critic-only adaptive dynamic programming(ADP)framework and parameter optimization of system settling time.Furthermore,a novel weight update law is proposed to satisfy the online iteration requirements,and the algorithm convergence and closedloop stability are discussed by the Lyapunov theory.Finally,four simulation cases are provided to prove the effectiveness,accuracy,and robustness of the proposed scheme for the hypersonic longitudinal control system.

Neural-network-based stochastic linear quadratic optimal tracking control scheme for unknown discrete-time systems using adaptive dynamic programming

In this paper,a stochastic linear quadratic optimal tracking scheme is proposed for unknown linear discrete-time(DT)systems based on adaptive dynamic programming(ADP)algorithm.First,an augmented system composed of the original system and the command generator is constructed and then an augmented stochastic algebraic equation is derived based on the augmented system.Next,to obtain the optimal control strategy,the stochastic case is converted into the deterministic one by system transformation,and then an ADP algorithm is proposed with convergence analysis.For the purpose of realizing the ADP algorithm,three back propagation neural networks including model network,critic network and action network are devised to guarantee unknown system model,optimal value function and optimal control strategy,respectively.Finally,the obtained optimal control strategy is applied to the original stochastic system,and two simulations are provided to demonstrate the effectiveness of the proposed algorithm.

A novel statistically tracked particle swarm optimization method for automatic generation control

Particle swarm optimization(PSO)is one of the popular stochastic optimization based on swarm intelligence algorithm.This simple and promising algorithm has applications in many research fields.In PSO,each particle can adjust its‘flying’according to its own flying experience and its companions’flying experience.This paper proposes a new PSO variant,called the statistically tracked PSO,which uses group statistical characteristics to update the velocity of the particle after certain iterations,thus avoiding localminima and helping particles to explore global optimum with an improved convergence.The performance of the proposed algorithm is tested on a deregulated automatic generation control problem in power systems and encouraging results are obtained.