摘要
In this tutorial paper,the finite-horizon optimal adaptive regulation of linear and nonlinear dynamic systems with unknown system dynamics is presented in a forward-in-time manner using adaptive dynamic programming(ADP).An adaptive estimator(AE)is introduced with the idea of Q-learning to relax the requirement of system dynamics in the case of linear system,while neural network-based identifier is utilised for nonlinear systems.The time-varying nature of the solution to the Bellman/Hamilton–Jacobi–Bellman equation is handled by utilising a time-dependent basis function,while the terminal constraint is incorporated as part of the update law of the AE/Identifier in solving the optimal feedback control.Utilising an initial admissible control,the proposed optimal regulation scheme of the uncertain linear and nonlinear system yields a forward-in-time and online solution without using value and/or policy iterations.An adaptive observer is utilised for linear systems in order to relax the need for state availability so that the optimal adaptive control design depends only on the reconstructed states.Finally,the optimal control is covered for nonlinear-networked control systems where in the feedback loop is closed via a communication network.Effectiveness of the proposed approach is verified by simulation results.The end result is a variant of a roll-out scheme in ADP wherein an initial admissible policy is selected as the base policy and the control policy is enhanced using a one-time policy improvement at each sampling interval.
