基于策略迭代的脉冲系统最优控制

Discrete-Time Impulsive Optimal Control on Policy Iteration

针对离散时间非线性系统的最优脉冲控制问题,提出了一种基于策略迭代(PI)的自适应动态规划(ADP)算法.首先引入脉冲区间集的约束条件,将系统转换为离散时间非线性脉冲控制系统,并根据哈密顿-雅可比-贝尔曼方程得到脉冲控制下的最优性能指标函数.其次提出了一种基于PI的ADP算法解决了脉冲系统最优控制问题,并给出了脉冲系统的收敛性分析.相比于值迭代(VI)算法,PI在保证系统稳定的同时收敛速度更快.然后提出了一种策略评估算法,放宽了PI算法的初始条件,解决了初始值选取困难的问题.最后通过仿真实例验证了该算法的有效性.

An adaptive dynamic programming(ADP)algorithm based on strategy iteration(PI)is proposed for optimal pulse control of discrete-time nonlinear systems.Firstly,the system is transformed into a discrete-time nonlinear pulse control system by introducing the constraint of pulse interval set,and the optimal performance index function under pulse control is obtained according to the Hamilton-Jacobi-Bellman equation.Secondly,an ADP algorithm based on PI is proposed to solve the optimal control problem of the pulse system,and the convergence analysis of the pulse system is given.Compared with the value iteration(VI)algorithm,PI converges faster while ensuring system stability.Then a strategy evaluation algorithm is proposed,which relaxes the initial conditions of PI algorithm and solves the difficult problem of initial value selection.Finally,a simulation example is given to verify the effectiveness of the proposed algorithm.