Value Iteration-Based Cooperative Adaptive Optimal Control for Multi-Player Differential Games With Incomplete Information

This paper presents a novel cooperative value iteration(VI)-based adaptive dynamic programming method for multi-player differential game models with a convergence proof.The players are divided into two groups in the learning process and adapt their policies sequentially.Our method removes the dependence of admissible initial policies,which is one of the main drawbacks of the PI-based frameworks.Furthermore,this algorithm enables the players to adapt their control policies without full knowledge of others’ system parameters or control laws.The efficacy of our method is illustrated by three examples.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

Dynamic Multi-team Antagonistic Games Model with Incomplete Information and Its Application to Multi-UAV

At present, the studies on multi-team antagonistic games(MTAGs) are still in the early stage, because this complicated problem involves not only incompleteness of information and conflict of interests, but also selection of antagonistic targets.Therefore, based on the previous researches, a new framework is proposed in this paper, which is dynamic multi-team antagonistic games with incomplete information(DMTAGII) model.For this model, the corresponding concept of perfect Bayesian Nash equilibrium(PBNE) is established and the existence of PBNE is also proved. Besides, an interactive iteration algorithm is introduced according to the idea of the best response for solving the equilibrium. Then, the scenario of multiple unmanned aerial vehicles(UAVs) against multiple military targets is studied to solve the problems of tactical decision making based on the DMTAGII model. In the process of modeling, the specific expressions of strategy, status and payoff functions of the games are considered, and the strategy is coded to match the structure of genetic algorithm so that the PBNE can be solved by combining the genetic algorithm and the interactive iteration algorithm.Finally, through the simulation the feasibility and effectiveness of the DMTAGII model are verified. Meanwhile, the calculated equilibrium strategies are also found to be realistic, which can provide certain references for improving the autonomous ability of UAV systems.

Security Defence Policy Selection Method Using the Incomplete Information Game Model

Nowadays, the network defence policy selection using game model of incomplete information ignores the type of the defender, which quantifies cost simply, resulting in unreasonable defence policies selection. Aiming at the problem, we use Bayesian game theory to model the active defence policy selection. We take the types of both the attacker and the defender into consideration. Besides, the traditional quantization method is enhanced. Then, we calculate the equilibrium of static Bayesian game. Based on the analysis of the equilibrium, we select the optimal defence policy through the prediction for attackers' actions. The paper calculates the defence effectiveness of defence policies and provides a defence policies selection algorithm. Ultimately, we present an example to verify the effectiveness of the method and model proposed in the paper.

Threat sequencing of multiple UCAVs with incomplete information based on game theory

The threat sequencing of multiple unmanned combat air vehicles(UCAVs) is a multi-attribute decision-making(MADM)problem. In the threat sequencing process of multiple UCAVs,due to the strong confrontation and high dynamics of the air combat environment, the weight coefficients of the threat indicators are usually time-varying. Moreover, the air combat data is difficult to be obtained accurately. In this study, a threat sequencing method of multiple UCAVs is proposed based on game theory by considering the incomplete information. Firstly, a zero-sum game model of decision maker( D) and nature(N)with fuzzy payoffs is established to obtain the uncertain parameters which are the weight coefficient parameters of the threat indicators and the interval parameters of the threat matrix. Then,the established zero-sum game with fuzzy payoffs is transformed into a zero-sum game with crisp payoffs(matrix game) to solve. Moreover, a decision rule is addressed for the threat sequencing problem of multiple UCAVs based on the obtained uncertain parameters. Finally, numerical simulation results are presented to show the effectiveness of the proposed approach.

Strategy Selection for Moving Target Defense in Incomplete Information Game

As a core component of the network,web applications have become one of the preferred targets for attackers because the static configuration of web applications simplifies the exploitation of vulnerabilities by attackers.Although the moving target defense(MTD)has been proposed to increase the attack difficulty for the attackers,there is no solo approach can cope with different attacks;in addition,it is impossible to implement all these approaches simultaneously due to the resource limitation.Thus,the selection of an optimal defense strategy based on MTD has become the focus of research.In general,the confrontation of two players in the security domain is viewed as a stochastic game,and the reward matrices are known to both players.However,in a real security confrontation,this scenario represents an incomplete information game.Each player can only observe the actions performed by the opponent,and the observed actions are not completely accurate.To accurately describe the attacker’s reward function to reach the Nash equilibrium,this work simulated and updated the strategy selection distribution of the attacker by observing and investigating the strategy selection history of the attacker.Next,the possible rewards of the attacker in each confrontation via the observation matrix were corrected.On this basis,the Nash-Q learning algorithm with reward quantification was proposed to select the optimal strategy.Moreover,the performances of the Minimax-Q learning algorithm and Naive-Q learning algorithm were compared and analyzed in the MTD environment.Finally,the experimental results showed that the strategy selection algorithm can enable defenders to select a more reasonable defensive strategy and achieve the maximum possible reward.

HYPERGAMES AND BAYESIAN GAMES:A THEORETICAL COMPARISON OF THE MODELS OF GAMES WITH INCOMPLETE INFORMATION

现在的学习与不完全的信息讨论在比赛的二个独立地发达的模型之间的关系， hypergames (班尼特， 1977 ) 并且贝叶斯的比赛(Harsanyi， 1967 ) 。作者首先证明任何 hypergame 罐头自然地以以一个统一方法的贝叶斯的比赛是 reformulated。转变过程被称为 hypergame 的贝叶斯的表示。作者然后证明为 hypergames 定义的一些平衡概念在某种意义上等价于为贝叶斯的比赛的那些。而且，作者基于建议分析小心地讨论每个模型怎么应该根据分析器目的被使用。

A Tree Formulation for Signaling Games with Noise

The paper provides an analysis of a sender-receiver sequential signaling game. The private information of the sender is transmitted with noise by a Machine, i.e. does not always correctly reflect the state of nature. Hence, a truthful revelation by the sender of his information does not necessarily imply that the signal he sends is correct. Also, the receiver can take a correct action even if the sender transmits an incorrect signal. The payoffs of the two players depend on their combined actions. Perfect Bayesian Equilibria which can result from different degrees of noise is analysed. The Bayesian updating of probabilities is explained. The fixed point theorem which makes the connection with the idea of rational expectations in economics is calculated. Given a number of equilibria, we comment on the most credible one on the basis of the implied payoffs for both players. The equilibrium signals are an example of the formation of a language convention discussed by D. Lewis.

Public Opinion Dissemination with Incomplete Information on Social Network: A Study Based on the Infectious Diseases Model and Game Theory

Given the fragmentation of public opinion dissemination and the lag of network users’cognition,the paper analyzes public opinion dissemination with incomplete information,which can provide reference for us to control and guide the spread of public opinion.Based on the derivative and secondary radiation of public opinion dissemination with incomplete information,the Susceptible-Susceptible-Infected-Recovered-Recovered-Infected(SSIRR-I)model is proposed.Given the interaction between users,the Deffuant opinion dynamics model and evolutionary game theory are introduced to simulate the public opinion game between dissemination and immune nodes.Finally,the numerical simulation and results analysis are given.The results reveal that the rate of opinion convergence significantly affects disseminating public opinion,which is positively correlated with the promotion effect of the dissemination node and negatively correlated with the suppression effect of the immune node of public opinion dissemination.Derivative and secondary radiations have different effects on public opinion dissemination in the early stage,but promote public opinion dissemination in the later stage.The dominant immune nodes have an apparent inhibitory effect on the spread of public opinion;nevertheless,they cannot block the dissemination of public opinion.

基于不完全信息博弈的云制造群智能优化方法

为解决云制造过程中云平台经营方与需求方之间的不完全信息以及相互竞争制约的关系导致制造服务难以抉择的问题,提出了一种基于不完全信息博弈模型的云制造群智能优化方法。以各自理性追求自身收益函数最大化为目标,针对需求方与云平台之间的利益竞争关系建立了基于不完全信息的静态博弈模型,并提出了需求方与云平台之间的竞争规则,通过海萨尼转换引入自然,将其转换为完全信息下的动态博弈得到贝叶斯扩展式,并证明了贝叶斯纳什均衡的存在性和唯一性。提出了一种基于高斯函数与扰动策略更新的粒子群算法对上述模型进行求解。仿真结果表明:改进算法相对其他算法有较快的收敛速度与较高的云制造系统总收益,不完全信息博弈模型能够兼顾不同类型的需求方提高云制造系统的总收益。

不完全信息Epsilon纳什均衡的航天器末端追逃博弈策略

针对不完全信息下的航天器末端追逃问题,提出了一种满足Epsilon纳什均衡的微分博弈控制策略。首先,建立了完全信息下的有限时间追逃纳什均衡策略对,并将其作为目标航天器实际采取的控制策略,使目标航天器掌握博弈进程的完全信息,进而获得更好的逃逸性能。在此基础上,考虑拦截航天器不能获取目标控制矩阵信息的态势,设计了基于广义卡尔曼滤波的行为学习信息估计算法,使拦截器能够对目标的不完全信息进行估计,并提出了不完全信息下的末端追逃博弈控制策略。经过理论分析,证明了所设计的不完全信息下微分博弈策略对满足Epsilon纳什均衡。最后,仿真结果表明该算法可以有效估计目标的不完全信息,确保拦截器能够在有限时间内快速拦截目标。

基于不完全信息的工业园区多主体需求响应博弈策略研究

随着我国电力市场化改革进程的推进与发展,在工业园区层面如何攫取各响应主体最大利润,以引导用户积极参与需求响应成为亟待解决的新问题。基于响应流程中市场信息披露有限且用户与聚合商自由缔结合约的情形,本文提出了一种聚合商视角下的最佳博弈策略及针对用户的收益分配方式。考虑工业园区内用户的利益关系,建立以不完全信息贝叶斯博弈作为上层、以合作博弈作为下层的多主体双层博弈架构;其次以最大化自身利益为目标,采用改进的虚拟遗憾最小化思想结合启发式算法求解博弈均衡,达成园区内的稳定合作关系。最后通过算例进行验证,分别设置了不同场景下申报策略和收益分配两方面的多个对比模型,结果表明本文所提方法得出的策略能够提高用户收益,且利益分配机制的合理选择能够在提高用户需求响应积极性方面起到促进作用。

非完全信息下协作式入侵检测系统检测库配置研究

本文研究了有限时间下非完全信息协作式入侵检测系统(IDS)的检测库配置。针对各入侵检测系统面对不同类型攻击时的最优检测库的配置以及检测库分配的矛盾,提出了一种双层检测库配置方法。第1层研究的是各攻击者的策略制定以及对应入侵检测系统的最优检测库配置方案;第2层研究的是如何解决多个入侵检测系统加载同一检测库的矛盾,给出了基于策略共享的集中式分配方法。第1层研究又可分为2步解决:第1步针对攻击者无法获知系统的真实状态、与入侵检测系统之间存在着信息不对称的问题,提出构建一个基于信念的随机博弈框架,通过后向递归算法求解,并证明该解就是博弈的稳态纳什均衡(SNE)策略;第2步通过求解混合Markov决策过程得到各入侵检测系统的最优检测库配置方案。仿真结果表明,本文所提方法能有效地得到协作式入侵检测系统面对不同类型攻击时的最优检测库配置方案。

基于不完全信息视角下的社会“内卷”化现象博弈分析

近年来,“内卷”一词在学校与职场中频繁出现,“内卷”现象已成为现代社会中一个不可忽视的问题。基于演化博弈方法分别构建传统情况下与保障机制下社会内卷博弈模型,探究各参与主体的均衡选择,并进一步利用Matlab进行数值仿真。研究结果发现:(1)传统情况下,社会内卷博弈系统将朝着(卷,卷)的最劣稳定状态演化。随着时间的推移,整个社会处于一种爆炸式竞争的状态,这种状态很容易引发社会问题。(2)保障机制下,社会内卷博弈系统存在四种演化情况,当且仅当满足特定条件时,博弈系统会收敛到(不卷,不卷)的帕累托最优状态。总体来说,在信息不透明的情况下,竞争者为了将自己的生存概率最大化,会有意或无意地让所有参与者进入“内卷”状态,并且会自我加强、自我循环。

Optimal Active Defence Using Dynamic Multi-Stage Signalling Game

Nowadays, security defence of network uses the game theory, which mostly applies complete information game model or even the static game model. To get closer to the actual network and defend actively, we propose a network attack-defence game model by using signalling game, which is modelled in the way of dynamic and incomplete information. We improve the traditional attack-defence strategies quantization method to meet the needs of the network signalling game model. Moreover, we give the calculation of the game equilibrium and analyse the optimal defence scheme. Finally, we analyse and verify effectiveness of the model and method through a simulation experiment.

A Hierarchical Game Model for Computation Sharing in Smart Buildings

Recently,initiatives to integrate Internet of Things(IoT)technologies into smart buildings have attracted extensive attention for improving the performance of buildings and the comfort of occupants.However,the amount of data generated by IoT devices remains a challenge to the building management systems(BMSs)in terms of intensity and complexity.Different from cloud computing and edge computing,we propose a computation sharing architecture in smart buildings to incentivize idle computing devices(ICDs,sellers)to offload computational tasks for the BMS(buyer).In this paper,we design a hierarchical game model,consisting of a Stackelberg game and a Cournot game,to achieve a dynamic increase in computational capacity for the BMS.To guarantee the utility of BMS and ICDs,the Stackelberg game model is built to analyze the interactions between BMS and ICDs.Then,the Cournot game model is presented to formulate the internal competition among multiple ICDs.Under the premise of the subgame perfect Nash equilibrium,the BMS can quote the optimal pricing strategy,and the ICDs can share the corresponding optimal amount of computing resources.Finally,the simulation results show that the BMS’s computational capacity is enhanced on-demand,and each participant in the game obtains maximal utility.

Multi-Team Bertrand Game with Heterogeneous Players

In this paper, we proposed a general form of a multi-team Bertrand game. Then, we studied a two-team Bertrand game, each team consists of two firms, with heterogeneous strategies among teams and homogeneous strategies among players. We find the equilibrium solutions and the conditions of their local stability. Numerical simulations were used to illustrate the complex behaviour of the proposed model, such as period doubling bifurcation and chaos. Finally, we used the feedback control method to control the model.

Insurance Plan for Service Assurance in Cloud Computing Market with Incomplete Information

Nowadays,cloud computing has been identified as new opportunities for migrating to the expected agility,reuse,and adaptive capabilities that can support the ever changing IT trends and requirements.Unfortunately,the rapid evolution of those technologies also comes with open issues such as security,privacy,integrity,quality of services,and their possible detrimental consequences.In this work,the concept of insurance is introduced to compensate the cloud computing customers when encountering those failures if service providers(SPs)have insurance purchased.Particularly,we consider the situation when the insurer is unable to see the system failure risk levels of the SPs,which is usually seen as an incomplete information market,in contrast with the optimal situation in a complete information market.First,an insurance-based cloud computing architecture is proposed to build a monetary credit system in which the cloud computing SP pays a premium for a certain coverage to the insurer.Subsequently,problem is formulated to solve the optimal insurance plan in complete and incomplete information markets,together with detail analysis of insurance policies in both cases are provided.Furthermore,simulation results show the properties of the two insurance plans and parameters that affect the design of the insurance plan.

On Incentive Strategy with a kind of Incomplete Information

The designing of incentive strategy when the follower’s objective functions have Parameters unknown to the leader is investigated in this paper. A desinging approach named IncentiveStrategy with Unknown but Bounded error (ISUBB) is proposed. A simple example is given to explain the use of ISUBS.