基于Q学习算法的随机离散时间系统的随机线性二次最优追踪控制

Stochastic linear quadratic optimal tracking control for stochastic discrete time systems based on Q-learning

针对随机线性离散时间系统,利用Q学习算法求解无限时域的随机线性二次最优追踪控制(SLQT)问题.首先,假设通过命令生成器生成追踪所需的参考信号,并建立一个由原随机系统和参考轨迹系统组成的增广系统,把最优追踪问题转化为最优调节问题的形式.其次,为了在线求解随机系统的最优追踪问题,将随机系统转为确定性系统,并根据增广系统定义随机线性二次最优追踪控制的Q函数,在无需知道系统模型参数的情况下在线求解增广随机代数方程(GSAE).再次,证明了Q学习算法和增广随机代数方程的等价性,给出了Q学习算法实现步骤.最后,给出一个仿真实例说明Q学习算法的有效性.

For stochastic linear discrete time systems,a Q-learning algorithm is proposed in this paper to solve the stochastic linear quadratic optimal tracking control problem in the infinite time domain.First,it is assumed that the reference signal required for tracking is generated by the command generator,and an augmented system consisting of the original stochastic system and the reference trajectory system is established,then the optimal tracking problem is transformed into an optimal regulation problem.Second,in order to solve the optimal tracking problem online,the stochastic system is transformed into a deterministic one,the Q function of stochastic linear quadratic optimal tracking control is defined according to the augmented system,and the augmented stochastic algebraic equation is solved online without knowing the parameters of the system model.Third,the equivalence between the Q-learning algorithm and the augmented stochastic algebraic equation is proved,and the implementation steps of the Q-learning algorithm are given.Finally,a simulation example is given to illustrate the effectiveness of the proposed Q-learning algorithm.