带状态约束的事件触发积分强化学习控制

Event-triggered Integral Reinforcement Learning Control with State Constraints

为克服全状态对称约束以及控制策略频繁更新的局限,针对一类具有部分动力学未知的仿射非线性连续系统的最优控制问题,提出一种带状态约束的事件触发积分强化学习的控制器设计方法;该方法是一种基于数据的在线策略迭代方法;引入系统转换将带有全状态约束的系统转化为不含约束的系统;基于事件触发机制以及积分强化学习算法,通过交替执行系统转换、策略评估、策略改进,最终系统在满足全状态约束的情况下,代价函数以及控制策略将分别收敛于最优值,并能降低控制策略的更新频率;此外,利用李雅普诺夫函数对系统的稳定性进行严格的分析;通过单连杆机械臂的仿真实验说明算法的可行性。

In order to overcome the limitations of full-state symmetry constraints and frequent update of control policy,aimed at an optimal control problem for a class of affine nonlinear continuous systems with partially unknown dynamics,a controller design method for the event-triggered integral reinforcement learning with state constraints is proposed.The method is a data-based online policy iteration approach.Firstly,the system transformation is introduced to transform the constrained system into the unconstrained system.Next,based on the event triggering mechanism and integral reinforcement learning algorithm,the system transformation,policy evaluation,and policy improvement are alternately operated.Finally,the system will satisfy that the full-state constraints,cost function and control policy will converge to the optimal values respectively,and it can reduce the update frequency of the control policy.In addition,the system stability is strictly analyzed by constructing the Lyapunov function.The simulation experiment of the single-link robotic arm verifies the effectiveness of the proposed algorithm.