基于强化学习的多智能体协作实现

Multi-agent cooperation based on reinforcement learning

基于马尔科夫过程的强化学习作为一种在线学习方式,能够很好地应用于单智能体环境中。但是由于强化学习理论的限制,在多智能体系统中马尔科夫过程模型不再适用,因此强化学习不能直接用于多智能体的协作学习问题。本文提出了多智能体协作的两层强化学习方法。该方法主要通过在单个智能体中构筑两层强化学习单元来实现。第一层强化学习单元负责学习智能体的联合任务协作策略,第二层强化学习单元负责学习在本智能体看来是最有效的行动策略。所提出的方法应用于3个智能体协作抬起圆形物体的计算机模拟中,结果表明所提出的方法比采用传统强化学习方法的智能体协作得更好。

Reinforcement learning based on Markov decision process is a way of on-line learning, which can be applied to single agent environment. However, due to the theoretical limitation that it assumes that an environment is Markovian, traditional reinforcement learning algorithms cannot be applied directly to multi-agent system. In this paper, a two-layer reinforcement learning method for multi-agent cooperation is presented. The proposed method is realized by adding two-layer reinforcement learning units to every agent. The first layer is for learning global cooperation strategy, and the second layer is for learning efficient action policy in one's own view. An experiment that three agents raise a disk-like object cooperatively has been done. Results show that the cooperative performance with the presented method is better than that using traditional reinforcement learning.