基于非策略Q学习方法的两个个体优化控制

Two-player Optimization Control Based on Off-policy Q-learning Algorithm

针对线性离散时间系统的非零和博弈问题,提出一种非策略Q学习算法。首先,提出非零和博弈优化问题,并且严格证明根据每个个体性能指标定义的值函数为线性二次型。然后,基于动态规划和Q学习方法,给出非策略Q学习算法,得到非零和博弈的近似最优解,实现系统的全局纳什均衡。此算法不要求系统模型参数已知,完全利用可测数据学习纳什均衡解。最后,算例仿真验证了方法的有效性。

In this paper,an off-policy Q-learning algorithm is proposed for solving non-zero sum game problems of linear discrete-time systems.First,the non-zero sum game problem is proposed,and the value function defined according to the performance index of each player is strictly proved to be linear quadratic.Then,based on the dynamic programming and Q-learning method,an off-policy Q-learning algorithm is developed,and the approximate optimal solution of the non-zero sum game is obtained to realize the global Nash equilibrium of the system.This algorithm does not require the system model parameters to be known a priori,and fully utilizes measurable data to learn the Nash equilibrium solution.Finally,the simulation results are given to show the effectiveness of the proposed method.