Cooperative and Competitive Multi-Agent Systems:From Optimization to Games

Multi-agent systems can solve scientific issues related to complex systems that are difficult or impossible for a single agent to solve through mutual collaboration and cooperation optimization.In a multi-agent system,agents with a certain degree of autonomy generate complex interactions due to the correlation and coordination,which is manifested as cooperative/competitive behavior.This survey focuses on multi-agent cooperative optimization and cooperative/non-cooperative games.Starting from cooperative optimization,the studies on distributed optimization and federated optimization are summarized.The survey mainly focuses on distributed online optimization and its application in privacy protection,and overviews federated optimization from the perspective of privacy protection me-chanisms.Then,cooperative games and non-cooperative games are introduced to expand the cooperative optimization problems from two aspects of minimizing global costs and minimizing individual costs,respectively.Multi-agent cooperative and non-cooperative behaviors are modeled by games from both static and dynamic aspects,according to whether each player can make decisions based on the information of other players.Finally,future directions for cooperative optimization,cooperative/non-cooperative games,and their applications are discussed.

The Coalition Cooperative Game Method and Its Application in Multi-objective Optimization Design

This paper proposes a multi-objective optimization design method based on the coalition cooperative game theory where the three design goals have been seen as three game players. By calculating the affecting factors and fuzzy clustering, the design variables are divided into different strategic spaces which belong to each player, then it constructs a payoff function based on the coalition mechanism. Each game player takes its own revenue function as a target and obtains the best strategy versus other players. The best strategies of all players consist of the strategy permutation of a round game and it obtains the final game solutions through multi-round games according to the convergence criterion. A multi-objective optimization example of the luff mechanism of compensative sheave block shows the effectiveness of the coalition cooperative game method.

THE OPTIMAL CONTROL FOR PROMOTING THE COOPERATION IN EVOLUTION GAME GENERATED BY PRISONER'S DILEMMA

Natural selection opposes the evolution of cooperation unless specific mechanisms are at work in Prisoner＇s Dilemma. By taking advantage of the modern control theory, the controller design is discussed and the optimal control is designed for promoting cooperation based on the recent advances in mechanisms for the evolution of cooperation. Two con- trol strategies are proposed： compensation control strategy for the cooperator when playing against a defector and reward control strategy for cooperator when playing against a coop- erator. The feasibility and effectiveness of these control strategies for promoting cooperation in different stages are analyzed. The reward for cooperation can＇t prevent defection from being evolutionary stable strategy （ESS）. On the other hand, compensation for the coopera- tor can＇t prevent defection from emerging and sustaining. By considering the effect and the cost, an optimal control scheme with constraint on the admissible control set is put forward. By analyzing the special nonlinear system of replicator dynamics, the exact analytic solution of the optimal control scheme is obtained based on the maximum principle. Finally, the effectiveness of the proposed method is illustrated by examples.

Equilibrium solution of two enterprises cooperative game

Based on the actual experience of cooperation in the supply chain, the Nash solution of two enterprises cooperative games is given. Not only is the solution unique, but it is also stable, and neither side has the capability to deviate the allocation of interests from the equilibrium point. If some firm tries to withdraw from cooperation or threaten to use other particular strategy, the negotiations are likely to achieve the distribution by the threat game; The calculating method of the choice of the optimal bargaining base point and the corresponding optimal pay-off vector are given.

Invasive Species Control Based on a Cooperative Game

We develop a long-term dynamic model for controlling invasive species using the theory of cooperative games. The model is applied to control of invasive buffelgrass in the Arizona desert, which directly competes with indigenous species and can increase wildfire risk. Interest groups care about damages to three threatened resources: saguaro, cactus, riparian vegetation, and buildings. The model optimally allocates labor and a budget to protect these resources by controlling the buffelgrass population over a multi-period planning horizon. The solution is based on computing the Shapley values for the interest groups. A homeowner strategy of creating defensible space around structures to protect against wildfire affords less protection to the other resources. A similar result holds for protection of saguaros, which are also spatially concentrated. Under the optimal solution, groups caring about spatially-dispersed, riparian vegetation would compensate homeowners and groups caring about saguaros for a reallocation of resources toward greater protection of dispersed vegetation. Results highlight the importance of the spatial configuration of players and the resources they wish to protect in invasive species control problems.

Game theory approach to optimal capital cost allocation in pollution control

This paper tries to integrate game theory, a very useful tool to resolve conflict phenomena, with optimal capital cost allocation issue in total emission control. First the necessity of allocating optimal capital costs fairly and reasonably among polluters in total emission control was analyzed. Then the possibility of applying game theory to the issue of the optimal capital cost allocation was expounded. Next the cooperative N person game model of the optimal capital cost allocation and its solution ways including method based on Shapley value, least core method, weak least core methods, proportional least core method, CGA method, MCRS method and so on were delineated. Finally through application of these methods it was concluded that to apply game theory in the optimal capital cost allocation issue is helpful to implement the total emission control planning schemes successfully, to control pollution effectively, and to ensure sustainable development.

Improved Performance for a Heterogeneous Satellite-Cooperative Network with Best Relay Node Selection

Nowadays, hybrid satellite-terrestrial cooperative network has emerged as a key technology to provide a great variety of communication services. The deployment of this network will improve coverage and capacity in remote areas. Despite the benefits of this network, by increasing the number of users, communication efficiency based on interference management is a major challenge in satellite-based system. Also, the direct links between satellite system and the terrestrial equipment do not always have desirable channel condition. In order to avoid serious throughput degradation, choosing a cooperative relay node is very important. In this paper, Stackelberg game is exploited for interference management that is raised by satellites in down link over terrestrial equipment. Then, for interference management between ground station and relay node with other mobile users, CVX is used to allocate optimum power. Also, the best relay node in this structure is selected based on the harmonic mean function. Thus, the performance of the heterogeneous satellite-cooperative network is investigated based on three benchmarks, namely, successful transmission, energy consumption and outage probability. Finally, the simulation results showed the effect of proposed system model on the performance of next generation satellite networks.

考虑负荷空间均衡和充电站合作博弈的快速充电定价机制

为了平衡快速充电负荷空间分布,提高电动汽车用户充电满意度,提出考虑负荷空间均衡和充电站合作博弈的快速充电动态定价机制。首先,通过实际订单数据逆地理编码得到出行概率转移矩阵,结合路网模型和速度-流量模型建立电动汽车充电负荷时空分布模型;然后,考虑用户主观意愿,建立计及用户偏好的充电站选择决策模型,模拟用户对于充电站选择的理性决策行为;最后,以充电站节点负荷均衡性为目标,建立区域内快速充电站合作模式下的动态定价博弈模型,通过迭代算法求解均衡值,并利用Shapley值法对合作联盟所得剩余收益进行分配。仿真实例表明,所提快速充电站动态定价策略能有效降低充电站节点的负荷方差,提高充电桩利用率和充电站收益,并减小用户排队时间,提高用户满意度。

非合作博弈背景下基于BSA的配电网优化重构

为缓解分布式电源大规模接入对配电网安全稳定运行的影响,提出一种考虑分布式电源输出功率的不确定性的有源配电网优化重构方法.首先,采用非合作博弈理论研究电网调度人员与“大自然”之间的博弈关系,将配电网系统中光伏单元的不确定性视为“大自然”博弈方;其次,以有功网损、负荷均衡度、电压偏差最小为目标函数,建立有源配电网优化重构模型,通过回溯搜索算法(backtracking search algorithm,BSA)进行迭代求解,得到最优重构方案;最后,在IEEE33节点系统进行仿真分析,验证模型的正确性及求解算法的有效性.研究结果表明,相较传统重构方法,本文方法更充分考虑了分布式电源输出功率的不确定性,并且在最恶劣的情况发生时,得到的重构策略能够使配电网系统的有功网损、负荷均衡度、电压偏差指标分别降低0.31%、0.59%、0.48%.

考虑多重不确定性与电碳耦合交易的多微网合作博弈优化调度

双碳背景下,构建低碳运行的能源系统是实现“双碳”目标的重要方向与实施路径。为了促进多微网系统内部能源的本地消纳以及低碳经济运行,该文对不确定性环境下多微网系统的合作运行及电碳耦合交易展开研究。首先,对于每个微网,构建了电转气碳捕集系统相耦合的热电联产运行模式,基于地方碳交易市场和阶梯碳交易机制,提出了多能源微网的低碳运行模型;其次,考虑到新能源发电和电力市场电价都存在不确定性的实际情况,采用机会约束和鲁棒优化的方法,以降低不确定性影响;再次,基于纳什谈判理论,建立了多个微网电碳耦合的合作博弈模型,各微网主体可同时参与到上级能源市场和地方能源市场中进行电能和碳排放配额的交易;最后,将非凸的合作博弈问题分解为两个线性可求解的子问题,进一步采用交替方向乘子法对问题进行求解。通过算例验证,该文所提方法可以有效提升各微网经济效益并减少碳排放。

考虑居民用户可调节潜力的负荷聚合商日前投标决策优化模型研究

负荷聚合商能够充分挖掘居民用户资源参与需求响应,深化居民用电可调节潜力量化分析对负荷精准调控与负荷聚合商模式智能推广具有重要作用。首先,基于居民用电负荷调控差异性,考虑居民典型用电电器综合满意度计量,构建居民柔性负荷可调节潜力分析模型;然后,考虑负荷聚合商参与市场交易的运营模式,构建基于非合作博弈理论的日前投标决策优化模型;最后,通过算例仿真验证模型的有效性及适用性。结果表明该模型能够兼顾调控潜力,有效支撑负荷聚合商最优投标决策,为负荷聚合商推广下统筹资源优化调控提供借鉴与参考。

基于解雇威胁的股东与经理人非合作微分博弈模型研究

本文针对存在解雇威胁时公司股东与项目经理人之间的非合作问题,构建了两阶段连续时间下双方进行非合作博弈的微分对策模型。通过求解HJB方程得到了该博弈的均衡结果,运用“逆向归纳法”推导出经理人最优投资决策的表达形式,最后从理论推导和数值模拟两个方面分析了“解雇机制”对于经理人最优投资决策和最优努力的影响。研究表明:(1)当企业实物资本所占比例超过某一阈值时,初始资本的增加以及解雇概率的降低会提高经理人的投资积极性;当该比例低于阈值时,初始资本的增加会对经理人最优投资比例的选择产生负向影响,但随着解雇风险的上升,经理人会提高投资比例。(2)经理人所选择的再投资比例,会随着一定范围内企业实物资本所占比重的增加而逐渐降低。(3)解雇机制在参数设置满足约束条件的企业中能够有效激励经理人努力工作。

基于直觉模糊联盟合作博弈的我国主要流域水资源优化配置研究

我国水资源地区分布不均,呈现南多北少的特点。随着人口增长和社会经济的不断发展,水资源需求量日益增加。为了提高水资源的综合利用效率,在完成水资源初始配置后迫切需要开展水资源跨区调配实现二次优化配置。如何选择水资源调配合作对象,如何兼顾公平与效率,如何共享合作收益,成为水资源再配置需要解决的核心问题。本文提出将直觉模糊联盟合作博弈运用到水资源优化配置问题中,通过构建基于清晰联盟合作博弈和模糊联盟合作博弈的水资源优化配置模型及合作收益函数,运用直觉模糊集、区间Choquet积分、连续有序加权平均算子等方法,推导出基于直觉模糊联盟合作博弈的水资源优化配置模型及合作收益函数,运用直觉模糊Shapley值求解最优解。选取我国南方地区的长江流域和北方地区的黄河、海河和淮河流域等水资源一级区作为研究对象,选择工业用水作为调配用水,将工业用水收益率高低作为水资源配置“优先原则”,比较分析三种不同的模型下我国主要流域水资源调配的合作对象及合作收益分配方式。结果表明:①各地区参与水资源二次配置,都能取得相较于不合作更好的合作收益;②参与直觉模糊合作联盟获得的收益全部都高于参与模糊合作联盟获得的收益;③在直觉模糊合作联盟中,不应盲目追求联盟的规模,应根据地区自身实际情况决定。以上研究结果表明,直觉模糊联盟合作博弈方法为解决跨区域流域水资源配置提供了一种新的解决方案,这将为我国主要流域开展跨区域水资源调配,优化水资源配置提供科学决策依据。

发电商合作博弈策略下跨区输电电价优化方法

由于负荷中心与可再生资源在地理位置上分布的不均匀,更大范围的资源合理配置需要跨区输电,而随着电力市场的深化发展,参与跨区电力市场出清的发电商将可能出现通过合作博弈策略行使市场力以争取自身利益最大的行为,此举可能导致跨区输电通道利用率降低,进而影响线路准许收入的稳定回收。为应对上述问题,提出了一种电力市场环境下考虑参与跨区出清的区外发电商合作博弈策略的跨区输电电价三层优化模型,上层为以准许收入稳定回收为目标的输电电价优化模型,中层为由区外发电商组成的跨区出清利益联盟的以联盟收益最大为目标的报量决策模型,下层为计及区外发电商参与的区内电力市场出清模型。为求解所提三层模型,采用了一种改进列和约束生成算法(C&CG)算法,将模型分解为主、子问题迭代求解,其中对于作为双层模型的主、子问题采用卡罗需-库恩-塔克条件(KKT)和大M法转化为等效单层模型后进行求解,而主、子问题模型中的双线性项则采用强对偶定理及变量离散参数化的方法进行线性转化以便于求解。最后通过PJM-5节点系统验证了所提三层优化模型能较好地体现区外发电商因合作博弈策略组成的跨区出清利益联盟的策略性决策和市场力行为,令输电电价的制定能在考虑区外发电商合作博弈策略影响下,实现线路准许收入的稳定回收。

双碳目标下基于Stackelberg和合作博弈的虚拟电厂双层优化调度

在中国的双碳目标下,大规模新能源发电接入虚拟电厂时,将导致电力市场交易中的高额惩罚成本。为此,设计了虚拟电厂参与日前市场竞标的双层博弈模型,以双层收益最大化为目标对虚拟电厂进行调度优化。首先,利用场景生成与削减技术考虑风力发电商和电动汽车聚合商的不确定性,设计了由风力发电商、电动汽车聚合商和燃气轮机合作组成的虚拟电厂下层电量竞标模型,以及虚拟电厂控制中心的上层电价竞标模型。其次,基于两个子模型之间的主从递阶关系,利用卡罗需库-恩塔-克(KKT)条件将双层模型转化为一个混合互补问题,通过求解获得优化方案,并基于Shapley值法对合作方进行利润分配。数值算例表明,设计的模型能有效减少碳排放和降低风电惩罚成本,同时增加参与者的收益。

基于逃逸角的多ASV微分博弈协同围捕方法

针对多个自主水面航行器(ASV)围捕单个主动逃逸的对抗性目标问题,利用微分博弈理论建立了多ASV协同围捕问题博弈模型,在含有距离项的支付函数中引入由逃逸角构成的合围项,从而降低目标中途逃逸的概率;然后将围捕问题转换为求解可实现策略的优化问题,利用粒子群优化(PSO)算法求解满足纳什均衡的最优策略,仿真和湖上试验结果均证明了基于PSO的微分博弈围捕算法的有效性。

基于合作博弈的网联车辆合流序列优化策略

为提升道路合流区交通通行效率,提出了一种基于合作博弈的网联车辆合流序列优化决策算法。首先,构建包含油耗、舒适度的多优化目标联合成本函数,通过分析多人合作博弈问题推导出合流区全局最小成本策略;然后,将多车合作博弈分解为多个两车合作博弈,预测不同策略的最小合流成本,利用成本矩阵法求得两车博弈的帕累托有效解,确定最优合流序列;最后,在三车道合并至一车道合流场景下,MATLAB仿真结果表明,上述优化合流序列能有效降低油耗,缩短出行时间,提高乘客的舒适性。

基于合作博弈的风-光-电氢微网容量配置

当风电场、光伏电站、电氢混合储能系统分属不同的投资者时,微网容量配置存在微网整体运行最优与各投资者自身运行最优的矛盾,同时风、光出力的不确定性也会影响容量配置。针对上述问题,提出一种考虑风、光不确定性,基于合作博弈的风-光-电氢微网容量配置方法。首先,依据有序聚类和K-均值聚类提取出风-光-负荷典型月场景;其次综合考虑影响微网运行的因素,构建基于不同投资者的经济模型,在典型月场景下,分析完全合作博弈、部分合作博弈、非合作博弈模式下各投资者与联盟的收益关系;最后,以月净收益最高为优化目标,利用改进灰狼优化算法配置投资者装机容量,并基于shapley值分配合作联盟投资者收益。根据算例结果和关键参数灵敏度分析可知,微网在完全合作博弈模式下兼顾经济性与低碳性,且各投资者利益分配合理;电价、风光补贴价格的波动对各投资者影响不同。

投资界面延伸下电网接入工程投资博弈策略

为切实降低用户办电成本,电网投资界面被要求延伸至用户红线,而其延伸成本如何在政府、电网、用户多主体间分配尚缺乏统一合理的标准。基于此,首先阐述电网投资界面延伸情况及投资主体利益诉求,并对电网接入工程投资成本进行分析;其次,以最大化各主体投资比例与其最不理想比例差距为目标,构建Nash-Harsanyi讨价还价博弈模型,通过初始投资比例、风险分担比例、合作协同能力量化谈判力水平,为电力接入费用寻求公平合理的政企共担投资策略;最后,以某经济开发区电网接入工程为例进行算例分析,结果表明,电网、用户、政府投资比例由0.65∶0.14∶0.21优化为0.61∶0.12∶0.27,三者满意度均得以提升,其中电网企业由于较高的谈判力得到最大程度提升,这有助于推动电网投资界面延伸工作,进一步优化营商环境。

基于层级结构的空−地协同预设时间最优容错控制

研究了发生执行器故障的无人机−无人车异构编队系统的层级预设时间最优编队控制问题.以保容错性能和收敛速度的优化控制为研究主线,以层级控制、图博弈理论和预设时间控制为技术基础,构建了一种预设时间最优容错控制算法.虚拟层设计了基于一致性跟踪误差和能量消耗的二次型性能指标函数,借助耦合哈密顿−雅克比−贝尔曼(Hanmilton-Jacobi-Bellman,HJB)方程和强化学习求解近似最优控制策略,实现多智能体的同步最优控制和交互纳什均衡.实际控制层基于最优信号并利用滑模控制和自适应技术,设计了预设时间容错跟踪控制器,实现对最优编队轨迹的有限时间跟踪.在保证全局收敛时间完全不依赖于系统的初始状态和控制器参数的同时,也有效实现对执行器故障参数的逼近.最后,通过仿真实验验证了所提控制策略的有效性.