基于强化学习的部分线性离散时间系统的最优输出调节

Optimal Output Regulation of Partially Linear Discrete-time Systems Using Reinforcement Learning

针对同时具有线性外部干扰与非线性不确定性下的离散时间部分线性系统的最优输出调节问题,提出了仅利用在线数据的基于强化学习的数据驱动控制方法.首先,该问题可拆分为一个受约束的静态优化问题和一个动态规划问题,第一个问题可以解出调节器方程的解.第二个问题可以确定出控制器的最优反馈增益.然后,运用小增益定理证明了存在非线性不确定性离散时间部分线性系统的最优输出调节问题的稳定性.针对传统的控制方法需要准确的系统模型参数用来解决这两个优化问题,提出了一种数据驱动离线策略更新算法,该算法仅使用在线数据找到动态规划问题的解.然后,基于动态规划问题的解,利用在线数据为静态优化问题提供了最优解.最后,仿真结果验证了该方法的有效性.

A data-driven control method only using online data based on reinforcement learning is proposed for the optimal output regulation problem of discrete-time partially linear systems with both linear disturbance and nonlinear uncertainties.First,the problem can be split into a constrained static optimization problem and a dynamic one.The solution of the first problem is corresponding to the solution of the regulator equation.The second can determine the optimal feedback gain of the controller.Then the small-gain theorem is used to prove the stability of the optimal output regulation problem of discrete-time partially linear systems with nonlinear uncertainties.The traditional control method needs the dynamics of the system to solve the two problems.But for this problem,a data-driven off-policy algorithm is proposed using only the measured data to find the solution of the dynamic optimization problem.Then,based on the solution of the dynamic one,the solution of the static optimization problem can be found only using data online.Finally,simulation results verify the effectiveness of the proposed method.