Cooperative and Competitive Multi-Agent Systems:From Optimization to Games

Multi-agent systems can solve scientific issues related to complex systems that are difficult or impossible for a single agent to solve through mutual collaboration and cooperation optimization.In a multi-agent system,agents with a certain degree of autonomy generate complex interactions due to the correlation and coordination,which is manifested as cooperative/competitive behavior.This survey focuses on multi-agent cooperative optimization and cooperative/non-cooperative games.Starting from cooperative optimization,the studies on distributed optimization and federated optimization are summarized.The survey mainly focuses on distributed online optimization and its application in privacy protection,and overviews federated optimization from the perspective of privacy protection me-chanisms.Then,cooperative games and non-cooperative games are introduced to expand the cooperative optimization problems from two aspects of minimizing global costs and minimizing individual costs,respectively.Multi-agent cooperative and non-cooperative behaviors are modeled by games from both static and dynamic aspects,according to whether each player can make decisions based on the information of other players.Finally,future directions for cooperative optimization,cooperative/non-cooperative games,and their applications are discussed.

Decision Making in Team-Adversary Games with Combinatorial Action Space

The team-adversary game simulates many real-world scenarios in which a team of agents competes cooperatively against an adversary.However,decision-making in this type of game is a big challenge since the joint action space of the team is combinatorial and exponentially related to the number of team members.It also hampers the existing equilibrium finding algorithms from solving team-adversary games efficiently.To solve this issue caused by the combinatorial action space,we propose a novel framework based on Counterfactual Regret Minimization(CFR)framework:CFR-MIX.Firstly,we propose a new strategy representation to replace the traditional joint action strategy by using the individual action strategies of all the team members,which can significantly reduce the strategy space.To maintain the cooperation between team members,a strategy consistency relationship is proposed.Then,we transform the consistency relationship of the strategy to the regret consistency for computing the equilibrium strategy with the new strategy representation under the CFR framework.To guarantee the regret consistency relationship,a product-form decomposition method over cumulative regret values is proposed.To implement this decomposition method,our CFR-MIX framework employs a mixing layer under the CFR framework to get the final decision strategy for the team,i.e.,the Nash equilibrium strategy.Finally,we conduct experiments on games in different domains.Extensive results show that CFR-MIX significantly outperforms state-of-the-art algorithms.We hope it can help the team make decisions in large-scale team-adversary games.