竞争与合作视角下的多Agent强化学习研究进展

RECENT PROCESS AND PROSPECT OF MULTI-AGENT REINFORCEMENT LEARNING UNDER THE PERSPECTIVE OF COMPETITION AND COOPERATION

随着深度学习和强化学习研究取得长足的进展,多Agent强化学习已成为解决大规模复杂序贯决策问题的通用方法。为了推动该领域的发展,从竞争与合作的视角收集并总结近期相关的研究成果。该文介绍单Agent强化学习;分别介绍多Agent强化学习的基本理论框架——马尔可夫博弈以及扩展式博弈,并重点阐述了其在竞争、合作和混合三种场景下经典算法及其近期研究进展;讨论多Agent强化学习面临的核心挑战——环境的不稳定性,并通过一个例子对其解决思路进行总结与展望。

With the rapid development of deep learning and reinforcement learning,multi-agent reinforcement learning(MARL)has become a common approach to solve the large scale complex sequential decision-making problem.In order to promote the development of this field,this paper collects and reviews recent research results from the perspective of competition and cooperation.This paper introduced deep reinforcement learning and introduced the basic theoretical framework of MARL-Markov game and extensive game,and especially emphasized the reinforcement learning algorithms developed recently in three scenarios of competition,cooperation and mixture.This paper discussed the core challenge of MARL that was non-stationary of the environment,and an example was given to summarize and prospect its solutions.