利用团队间的影响解决多智能体强化学习中的奖励冲突

Solving reward conflict in multi-agent reinforcement learning by using the influence between teams

本文提出了一种基于智能体间相互作用的MARL学习框架,称为IC,以解决MARL中稀疏奖励环境导致智能体产生冲突的问题。IC的主要功能是根据智能体间的高斯核函数大小赋予不同的值,计算出智能体的影响矩阵,并将影响矩阵的核范数作为额外奖励引入到目标函数中,以提高智能体探索性能以及团队之间的协作能力。实验结果表明,IC可以显著提高智能体间的协作能力,并在稀疏奖励环境中加速智能体对最优策略的学习。这是首次在MARL中尝试利用智能体之间的相互影响来促进智能体的探索能力。

This article proposes a MARL learning framework based on the interaction between agents,called IC,to solve the problem of conflicts between agents caused by sparse reward environments in MARL.The main function of IC is to assign different values based on the Gaussian kernel function size between agents,calculate the influence matrix of agents,and introduce the kernel norm of the influence matrix as an additional reward into the objective function to improve the exploration performance of agents and the collaboration ability between teams.The experimental results indicate that IC can significantly improve the collaboration ability between agents and accelerate their learning of optimal strategies in a sparse reward environment.This is the first attempt in MARL to utilize the mutual influence between agents to promote their exploration ability.