面向Ad-Hoc协作的局部观测重建方法

Local observation reconstruction for Ad-Hoc cooperation

在多智能体强化学习的研究中,如何进行Ad-Hoc协作,也就是说如何适应种类和数量变化的队友,是一个关键问题。现有方法或者有很强的先验知识假设,或者使用硬编码的规则进行合作,缺乏通用性,无法泛化到更一般的Ad-Hoc协作场景。为解决该问题,提出一种面向Ad-Hoc协作的局部观测重建算法,利用注意力机制和采样网络对局部观测进行重建,使得算法认识到并充分利用不同局面中的高维状态表征,实现了在Ad-Hoc协作场景下的零样本泛化。在星际争霸微操环境和Ad-Hoc协作场景上与代表性算法的性能进行对比与分析,验证了算法的有效性。

In recent years,multi-agent reinforcement learning has received a lot of attention from researchers.In the study of multi-agent reinforcement learning,the question of how to perform ad-hoc cooperation,i.e.,how to adapt to a changing variety and number of teammates,is a key problem.Existing methods either have strong prior knowledge assumptions or use hard-coded protocols for cooperation,which lack generality and can not be generalized to more general ad-hoc cooperation scenarios.To address this problem,this paper proposes a local observation reconstruction algorithm for ad-hoc cooperation,which uses attention mechanisms and sampling networks to reconstruct local observations,enabling the algorithm to recognize and make full use of high-dimensional state representations in different situations and achieve zero-shot generalization in ad-hoc cooperation scenarios.In this paper,the performance of the algorithm is compared and analyzed with representative algorithms on the StarCraft micromanagement environment and ad-hoc cooperation scenarios to verify the effectiveness of the algorithm.