Evolutionary game dynamics of combining two different aspiration-driven update rules in structured populations

In evolutionary games,most studies on finite populations have focused on a single updating mechanism.However,given the differences in individual cognition,individuals may change their strategies according to different updating mechanisms.For this reason,we consider two different aspiration-driven updating mechanisms in structured populations:satisfied-stay unsatisfied shift(SSUS)and satisfied-cooperate unsatisfied defect(SCUD).To simulate the game player’s learning process,this paper improves the particle swarm optimization algorithm,which will be used to simulate the game player’s strategy selection,i.e.,population particle swarm optimization(PPSO)algorithms.We find that in the prisoner’s dilemma,the conditions that SSUS facilitates the evolution of cooperation do not enable cooperation to emerge.In contrast,SCUD conditions that promote the evolution of cooperation enable cooperation to emerge.In addition,the invasion of SCUD individuals helps promote cooperation among SSUS individuals.Simulated by the PPSO algorithm,the theoretical approximation results are found to be consistent with the trend of change in the simulation results.

A Monte Carlo Neural Fictitious Self-Play approach to approximate Nash Equilibrium in imperfect-information dynamic games

Solving the optimization problem to approach a Nash Equilibrium point plays an important role in imperfect information games,e.g.,StarCraft and poker.Neural Fictitious Self-Play(NFSP)is an effective algorithm that learns approximate Nash Equilibrium of imperfect-information games from purely self-play without prior domain knowledge.However,it needs to train a neural network in an off-policy manner to approximate the action values.For games with large search spaces,the training may suffer from unnecessary exploration and sometimes fails to converge.In this paper,we propose a new Neural Fictitious Self-Play algorithm that combines Monte Carlo tree search with NFSP,called MC-NFSP,to improve the performance in real-time zero-sum imperfect-information games.With experiments and empirical analysis,we demonstrate that the proposed MC-NFSP algorithm can approximate Nash Equilibrium in games with large-scale search depth while the NFSP can not.Furthermore,we develop an Asynchronous Neural Fictitious Self-Play framework(ANFSP).It uses asynchronous and parallel architecture to collect game experience and improve both the training efficiency and policy quality.The experiments with th e games with hidden state information(Texas Hold^m),and the FPS(firstperson shooter)games demonstrate effectiveness of our algorithms.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

EVOLUTION OF DYNAMIC GAMES AND BACKWARD INDUCTION

As an efficient method of solving subgame-perfect Nash equilibrium,the backward induction is analyzed from an evolutionary point of view in this paper,replacing a player with a population and turning a game into a population game,which shows that equilibrium of a perfect information game is the unique evolutionarily stable outcome for dynamic models in the limit.

Evolutionary game dynamics of combining the Moran and imitation processes

One of the assumptions of previous research in evolutionary game dynamics is that individuals use only one rule to update their strategy. In reality, an individual's strategy update rules may change with the environment, and it is possible for an individual to use two or more rules to update their strategy. We consider the case where an individual updates strategies based on the Moran and imitation processes, and establish mixed stochastic evolutionary game dynamics by combining both processes. Our aim is to study how individuals change strategies based on two update rules and how this affects evolutionary game dynamics. We obtain an analytic expression and properties of the fixation probability and fixation times(the unconditional fixation time or conditional average fixation time) associated with our proposed process. We find unexpected results. The fixation probability within the proposed model is independent of the probabilities that the individual adopts the imitation rule update strategy. This implies that the fixation probability within the proposed model is equal to that from the Moran and imitation processes. The one-third rule holds in the proposed mixed model. However, under weak selection, the fixation times are different from those of the Moran and imitation processes because it is connected with the probability that individuals adopt an imitation update rule. Numerical examples are presented to illustrate the relationships between fixation times and the probability that an individual adopts the imitation update rule, as well as between fixation times and selection intensity. From the simulated analysis, we find that the fixation time for a mixed process is greater than that of the Moran process, but is less than that of the imitation process. Moreover, the fixation times for a cooperator in the proposed process increase as the probability of adopting an imitation update increases; however, the relationship becomes more complex than a linear relationship.

Non-Cooperative Differential Game Based Energy Consumption Control for Dynamic Demand Response in Smart Grid

In this paper, we conduct research on the dynamic demand response problem in smart grid to control the energy consumption. The objective of the energy consumption control is constructed based on differential game, as the dynamic of each users’ energy state in smart gird can be described based on a differential equation. Concept of electricity sharing is introduced to achieve load shift of main users from the high price hours to the low price hours. Nash equilibrium is given based on the Hamilton equation and the effectiveness of the proposed model is verified based on the numerical simulation results.

Optimal Active Defence Using Dynamic Multi-Stage Signalling Game

Nowadays, security defence of network uses the game theory, which mostly applies complete information game model or even the static game model. To get closer to the actual network and defend actively, we propose a network attack-defence game model by using signalling game, which is modelled in the way of dynamic and incomplete information. We improve the traditional attack-defence strategies quantization method to meet the needs of the network signalling game model. Moreover, we give the calculation of the game equilibrium and analyse the optimal defence scheme. Finally, we analyse and verify effectiveness of the model and method through a simulation experiment.

Power source–power grid coordinated typhoon defense strategy based on multiagent dynamic game theory

A power source–power grid coordinated typhoon defense strategy is proposed in this study to minimize the cost of power grid anti-typhoon reinforcement measures and improve defense efficiency.It is based on multiagent dynamic game theory.This strategy regards a typhoon as a rational gamer that always causes the greatest damage.Together with the grid planner and black start unit(BSU)planner,it forms a multiagent defense–attack–defense dynamic game model naturally.The model is adopted to determine the optimal reinforcements for the transmission lines,black start power capacity,and location.Typhoon Hato,which struck a partial coastal area in Guangdong province in China in 2017,was adopted to formulate a step-by-step model of a typhoon attacking coastal area power systems.The results were substituted into the multiagent defense–attack–defense dynamic game model to obtain the optimal transmission line reinforcement positions,as well as optimal BSU capacity and geographic positions.An effective typhoon defense strategy and minimum load shedding were achieved,demonstrating the feasibility and correctness of the proposed strategy.The related theories and methods of this study have positive significance for the prevention of uncertain large-scale natural disasters.

A Dynamic Distributed Spectrum Allocation Mechanism Based on Game Model in Fog Radio Access Networks

With the explosive growth of highspeed wireless data demand and the number of mobile devices, fog radio access networks(F-RAN) with multi-layer network structure becomes a hot topic in recent research. Meanwhile, due to the rapid growth of mobile communication traffic, high cost and the scarcity of wireless resources, it is especially important to develop an efficient radio resource management mechanism. In this paper, we focus on the shortcomings of resource waste, and we consider the actual situation of base station dynamic coverage and user requirements. We propose a spectrum pricing and allocation scheme based on Stackelberg game model under F-RAN framework, realizing the allocation of resource on demand. This scheme studies the double game between the users and the operators, as well as between the traditional operators and the virtual operators, maximizing the profits of the operators. At the same time, spectrum reuse technology is adopted to improve the utilization of network resource. By analyzing the simulation results, it is verified that our proposed scheme can not only avoid resource waste, but also effectively improve the operator's revenue efficiency and overall network resource utilization.

Value Iteration-Based Cooperative Adaptive Optimal Control for Multi-Player Differential Games With Incomplete Information

This paper presents a novel cooperative value iteration(VI)-based adaptive dynamic programming method for multi-player differential game models with a convergence proof.The players are divided into two groups in the learning process and adapt their policies sequentially.Our method removes the dependence of admissible initial policies,which is one of the main drawbacks of the PI-based frameworks.Furthermore,this algorithm enables the players to adapt their control policies without full knowledge of others’ system parameters or control laws.The efficacy of our method is illustrated by three examples.

Parallel Computation of the Noncooperative Equilibria for Dynamic Games

A method of the parallel computation of the linear quadratic non cooperative dynamic games problem is proposed. The Lyapunov function is introduced, through which the form adapted to parallel computation of the open loop Nash equilibrium strategies is gi

System dynamics of behaviour-evolutionary mix-game models

In real financial markets there are two kinds of traders： one is fundamentalist, and the other is a trend-follower. The mix-game model is proposed to mimic such phenomena. In a mix-game model there are two groups of agents： Group 1 plays the majority game and Group 2 plays the minority game. In this paper, we investigate such a case that some traders in real financial markets could change their investment behaviours by assigning the evolutionary abilities to agents： if the winning rates of agents are smaller than a threshold, they will join the other group; and agents will repeat such an evolution at certain time intervals. Through the simulations, we obtain the following findings： （i） the volatilities of systems increase with the increase of the number of agents in Group 1 and the times of behavioural changes of all agents; （ii） the performances of agents in both groups and the stabilities of systems become better if all agents take more time to observe their new investment behaviours; （iii） there are two-phase zones of market and non-market and two-phase zones of evolution and non-evolution; （iv） parameter configurations located within the cross areas between the zones of markets and the zones of evolution are suited for simulating the financial markets.

Research on Operation Optimization of Energy Storage Power Station and Integrated Energy Microgrid Alliance Based on Stackelberg Game

With the development of renewable energy technologies such as photovoltaics and wind power,it has become a research hotspot to improve the consumption rate of new energy and reduce energy costs through the deployment of energy storage.To solve the problem of the interests of different subjects in the operation of the energy storage power stations(ESS)and the integrated energy multi-microgrid alliance(IEMA),this paper proposes the optimization operation method of the energy storage power station and the IEMA based on the Stackelberg game.In the upper layer,ESS optimizes charging and discharging decisions through a dynamic pricing mechanism.In the lower layer,IEMA optimizes the output of various energy conversion coupled devices within the IEMA,as well as energy interaction and demand response(DR),based on the energy interaction prices provided by ESS.The results demonstrate that the optimization strategy proposed in this paper not only effectively balances the benefits of the IEMA and ESS but also enhances energy consumption rates and reduces IEMA energy costs.

Dynamic Multi-team Antagonistic Games Model with Incomplete Information and Its Application to Multi-UAV

At present, the studies on multi-team antagonistic games (MTAGs) are still in the early stage, because this complicated problem involves not only incompleteness of information and conflict of interests, but also selection of antagonistic targets. Therefore, based on the previous researches, a new framework is proposed in this paper, which is dynamic multi-team antagonistic games with incomplete information (DMTAGII) model. For this model, the corresponding concept of perfect Bayesian Nash equilibrium (PBNE) is established and the existence of PBNE is also proved. Besides, an interactive iteration algorithm is introduced according to the idea of the best response for solving the equilibrium. Then, the scenario of multiple unmanned aerial vehicles (UAVs) against multiple military targets is studied to solve the problems of tactical decision making based on the DMTAGII model. In the process of modeling, the specific expressions of strategy, status and payoff functions of the games are considered, and the strategy is coded to match the structure of genetic algorithm so that the PBNE can be solved by combining the genetic algorithm and the interactive iteration algorithm. Finally, through the simulation the feasibility and effectiveness of the DMTAGII model are verified. Meanwhile, the calculated equilibrium strategies are also found to be realistic, which can provide certain references for improving the autonomous ability of UAV systems. © 2014 Chinese Association of Automation.

Certainty of patient survival with reference to game information dynamic model

This paper is concerned with Hepato-Cellular Carcinoma (HCC) patients treated naturopathic agents. Patients treated with ≥4 agents survived significantly longer than patients treated with ≤3 agents. The great effect is seen in patients treated with at least 4 agents that include Cordyceps sinensis. This greater certainty of patient survival without toxic side effects is significant benefit comparing with the conventional therapy. Treatment of HCC with a regimen of ≥4 agents prepared from natural products is associated with greater certainty of patient survival in a substantial portion of patients. The information dynamic model for certainty of patient survival is derived based on fluid mechanics, where a series of approximate solutions of the flow between two parallel flat walls, one of which is at rest, the other is suddenly accelerated from the rest to a constant velocity are used. The kinetic energy of certainty of patient survival decreases with increasing time, while the potential energy increases with increasing time. Total mechanical energy of patients treated with 4 or more agents is smaller than that treated with 3 or fewer agents. The kinetic energy (potential energy) of patients treated with 4 or more agents decreases (increases) more slower than the kinetic energy (potential energy) of patients treated with 3 or fewer agents.

A Network Analysis of Skill Game Dynamics

Casino games can be classified in two main categories, i.e. skill games and gambling. Notably, the former refers to games whose outcome is affected by the strategies of players, the latter to those games whose outcome is completely random. For instance, lotteries are easily recognized as pure gambling, while some variants of Poker (e.g. Texas Hold’em) are usually considered as skill games. In both cases, the theory of probability constitutes the mathematical framework for studying their dynamics, despite their classification. Here, it is worth to consider that when games entail the competition between many players, the structure of interactions can acquire a relevant role. For instance, some games as Bingo are not characterized by this kind of interactions, while other games as Poker, show a network structure, i.e. players interact each other and have the opportunity to share or exchange information. In this paper, we analyze the dynamics of a population composed of two species, i.e. strong and weak agents. The former represents expert players, while the latter beginners, i.e. non-expert ones. Here, pair-wise interactions are based on a very simple game, whose outcome is affected by the nature of the involved agents. In doing so, expert agents have a higher probability to succeed when playing with weak agents, while the success probability is equal when two agents of the same kind face each other. Numerical simulations are performed considering a population arranged in different topologies like regular graphs and in scale-free networks. This choice allows to model dynamics that we might observe on online game platforms. Further aspects as the adaptability of agents are taken into account, e.g. the possibility to improve (i.e. to becomean expert). Results show that complex topologies represent a strong opportunity for experts and a risk for both kinds of agents.

A Three-Stage Stochastic Dynamic Pricing Game Model Affected by New Products into the Market

In real-life marketing, a common phenomenon is that the prices of current product will have been cut down even the new product has not gone into market yet. Thus, it is very important for merchant to set the strategy which can make the excepted revenue maximum. So, this paper constructs a three-stage stochastic dynamic pricing game model for analyzing the influence of the uncertainty of entry timing of the new products on pricing of products being sold. By analyzing of the pricing strategy, there are big differences in the predictions of new product going into market between merchant and customers;the merchant will adopt cutting price for promotion strategy to reduce negative influence of the new products on the demand of the products sold now. Otherwise, the merchant will adopt the strategy of maximizing current period’s profit.

Dynamics of a Bertrand Duopoly Game of the Greek Oil Market and Application of the d-Backtest Method

This work presents the complexity that emerges in a Bertrand duopoly between two companies in the Greek oil market, one of which is semi-public and the other is private. The game uses linear demand functions for differentiated products from the existing literature and asymmetric cost functions that arose after approaches using the published financial reports of the two oil companies (Hellenic Petroleum and Motor Oil). The game is based on the assumption of homogeneous players who are characterized by bounded rationality and follow an adjustment mechanism. The players’ decisions for each time period are expressed by two difference equations. A dynamical analysis of the game’s discrete dynamical system is made by finding the equilibrium positions and studying their stability. Numerical simulations include bifurcation diagrams and strange attractors. Lyapunov numbers’ graphs and sensitivity analysis in initial conditions prove the algebraic results and reveal the complexity and chaotic behavior of the system focusing on the two parameters k1 and k2 (speed of adjustment for each player). The d-Backtest method is applied through which an attempt is made to control the chaos that appears outside the stability space in order to return to the locally asymptotically stable Nash equilibrium for the system.

Dynamical Systems Theory Compared to Game Theory: The Case of the Salamis’s Battle

In this paper, we present an innovative non–linear, discrete, dynamical system trying to model the historic battle of Salamis between Greeks and Persians. September 2020 marks the anniversary of the 2500 years that have passed since this famous naval battle which took place in late September 480 B.C. The suggested model describes very well the most effective strategic behavior between two participants during a battle (or in a war). Moreover, we compare the results of the Dynamical Systems analysis to Game Theory, considering this conflict as a “war game”.

Exploring the dynamic evolutionary mechanism of game model on the protection of traditional villages

With the rapid improvement of urbanization and industrialization in countries around the world,how to effectively solve the rapid demise of traditional villages is a social dilemma faced by all countries,which is why a series of relevant protection regulations have been promulgated in different historical periods.However,the formulation of relevant policies is still not scientific,universal,and long-term.In this study,we constructed an evolutionary game model of local governments and residents based on the evolutionary game theory(EGT),which is used to explore the evolutionary stability strategy(ESS)and stability conditions of stakeholders under the premise of mutual influence and restriction.Besides,the study also included the analysis about the impacts of different influence factors on the evolution tendency of the game model.At the same time,numerical simulation examples were used to verify the theoretical results and three crucial conclusions have been drawn.Firstly,the strategic evolution of stakeholders is a dynamic process of continuous adjustment and optimization,and its results and speed show consistent interdependence.Secondly,the decision-making of stakeholders mainly depends on the basic cost,and the high cost of investment is not conducive to the protection of traditional villages.Thirdly,the dynamic evolutionary mechanism composed of different influence factors will have an impact on the direction and speed of decision-making of stakeholders,which provides the basis for them to effectively restrict the decision-making of each other.This study eliminates the weaknesses of existing research approaches and provides scientific and novel ideas for the protection of traditional villages,which can contribute to the formulation and improvement of the relevant laws and regulations.