基于博弈论及Q学习的多Agent协作追捕算法

Multi-agent collaborative pursuit algorithm based on game theory and Q-learning

多Agent协作追捕问题是多Agent协调与协作研究中的一个典型问题。针对具有学习能力的单逃跑者追捕问题,提出了一种基于博弈论及Q学习的多Agent协作追捕算法。首先,建立协作追捕团队,并构建协作追捕的博弈模型;其次,通过对逃跑者策略选择的学习,建立逃跑者有限的Step-T累积奖赏的运动轨迹,并把运动轨迹调整到追捕者的策略集中;最后,求解协作追捕博弈得到Nash均衡解,每个Agent执行均衡策略完成追捕任务。同时,针对在求解中可能存在多个均衡解的问题,加入了虚拟行动行为选择算法来选择最优的均衡策略。C#仿真实验表明,所提算法能够有效地解决障碍环境中单个具有学习能力的逃跑者的追捕问题,实验数据对比分析表明该算法在同等条件下的追捕效率要优于纯博弈或纯学习的追捕算法。

The multi-agent collaborative pursuit problem is a typical problem in the multi-agent coordination and collaboration research.Aiming at the pursuit problem of single escaper with learning ability,a multi-agent collaborative pursuit algorithm based on game theory and Q-learning was proposed.Firstly,a cooperative pursuit team was established and a game model of cooperative pursuit was built.Secondly,through the learning of the escaper’s strategy choices,the trajectory of the escaper’s limited Step-T cumulative reward was established,and the trajectory was adjusted to the pursuer’s strategy set.Finally,the Nash equilibrium solution was obtained by solving the cooperative pursuit game,and the equilibrium strategy was executed by each agent to complete the pursuit task.At the same time,in order to solve the problem that there may be multiple equilibrium solutions,the virtual action behavior selection algorithm was added to select the optimal equilibrium strategy.C#simulation experiments show that,the proposed algorithm can effectively solve the pursuit problem of single escaper with learning ability in the obstacle environment,and the comparative analysis of experimental data shows that the pursuit efficiency of the algorithm under the same conditions is better than that of pure game or pure learning.