Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

Power source–power grid coordinated typhoon defense strategy based on multiagent dynamic game theory

A power source–power grid coordinated typhoon defense strategy is proposed in this study to minimize the cost of power grid anti-typhoon reinforcement measures and improve defense efficiency.It is based on multiagent dynamic game theory.This strategy regards a typhoon as a rational gamer that always causes the greatest damage.Together with the grid planner and black start unit(BSU)planner,it forms a multiagent defense–attack–defense dynamic game model naturally.The model is adopted to determine the optimal reinforcements for the transmission lines,black start power capacity,and location.Typhoon Hato,which struck a partial coastal area in Guangdong province in China in 2017,was adopted to formulate a step-by-step model of a typhoon attacking coastal area power systems.The results were substituted into the multiagent defense–attack–defense dynamic game model to obtain the optimal transmission line reinforcement positions,as well as optimal BSU capacity and geographic positions.An effective typhoon defense strategy and minimum load shedding were achieved,demonstrating the feasibility and correctness of the proposed strategy.The related theories and methods of this study have positive significance for the prevention of uncertain large-scale natural disasters.

Evolutionary game dynamics of combining two different aspiration-driven update rules in structured populations

In evolutionary games,most studies on finite populations have focused on a single updating mechanism.However,given the differences in individual cognition,individuals may change their strategies according to different updating mechanisms.For this reason,we consider two different aspiration-driven updating mechanisms in structured populations:satisfied-stay unsatisfied shift(SSUS)and satisfied-cooperate unsatisfied defect(SCUD).To simulate the game player’s learning process,this paper improves the particle swarm optimization algorithm,which will be used to simulate the game player’s strategy selection,i.e.,population particle swarm optimization(PPSO)algorithms.We find that in the prisoner’s dilemma,the conditions that SSUS facilitates the evolution of cooperation do not enable cooperation to emerge.In contrast,SCUD conditions that promote the evolution of cooperation enable cooperation to emerge.In addition,the invasion of SCUD individuals helps promote cooperation among SSUS individuals.Simulated by the PPSO algorithm,the theoretical approximation results are found to be consistent with the trend of change in the simulation results.

Bounded Rationality Based Multi-VPP Trading in Local Energy Markets:A Dynamic Game Approach with Different Trading Targets

It is expected that multiple virtual power plants(multi-VPPs)will join and participate in the future local energy market(LEM).The trading behaviors of these VPPs needs to be carefully studied in order to maximize the benefits brought to the local energy market operator(LEMO)and each VPP.We propose a bounded rationality-based trading model of multiVPPs in the local energy market by using a dynamic game approach with different trading targets.Three types of power bidding models for VPPs are first set up with different trading targets.In the dynamic game process,VPPs can also improve the degree of rationality and then find the most suitable target for different requirements by evolutionary learning after considering the opponents’bidding strategies and its own clustered resources.LEMO would decide the electricity buying/selling price in the LEM.Furthermore,the proposed dynamic game model is solved by a hybrid method consisting of an improved particle swarm optimization(IPSO)algorithm and conventional largescale optimization.Finally,case studies are conducted to show the performance of the proposed model and solution approach,which may provide some insights for VPPs to participate in the LEM in real-world complex scenarios.

A Monte Carlo Neural Fictitious Self-Play approach to approximate Nash Equilibrium in imperfect-information dynamic games

Solving the optimization problem to approach a Nash Equilibrium point plays an important role in imperfect information games,e.g.,StarCraft and poker.Neural Fictitious Self-Play(NFSP)is an effective algorithm that learns approximate Nash Equilibrium of imperfect-information games from purely self-play without prior domain knowledge.However,it needs to train a neural network in an off-policy manner to approximate the action values.For games with large search spaces,the training may suffer from unnecessary exploration and sometimes fails to converge.In this paper,we propose a new Neural Fictitious Self-Play algorithm that combines Monte Carlo tree search with NFSP,called MC-NFSP,to improve the performance in real-time zero-sum imperfect-information games.With experiments and empirical analysis,we demonstrate that the proposed MC-NFSP algorithm can approximate Nash Equilibrium in games with large-scale search depth while the NFSP can not.Furthermore,we develop an Asynchronous Neural Fictitious Self-Play framework(ANFSP).It uses asynchronous and parallel architecture to collect game experience and improve both the training efficiency and policy quality.The experiments with th e games with hidden state information(Texas Hold^m),and the FPS(firstperson shooter)games demonstrate effectiveness of our algorithms.

On Incentive Equlibria for Dynamic Games

The problem of incentive equilibria for dynamic games proposed in Ref.1 is considered.A sufficient condition and a necessary condition concerning the existence of incentive equilibria are proposed. As an illustrative example, the incentive equilibria for linear-quadratic games are discussed.

Dynamic Multi-team Antagonistic Games Model with Incomplete Information and Its Application to Multi-UAV

At present, the studies on multi-team antagonistic games(MTAGs) are still in the early stage, because this complicated problem involves not only incompleteness of information and conflict of interests, but also selection of antagonistic targets.Therefore, based on the previous researches, a new framework is proposed in this paper, which is dynamic multi-team antagonistic games with incomplete information(DMTAGII) model.For this model, the corresponding concept of perfect Bayesian Nash equilibrium(PBNE) is established and the existence of PBNE is also proved. Besides, an interactive iteration algorithm is introduced according to the idea of the best response for solving the equilibrium. Then, the scenario of multiple unmanned aerial vehicles(UAVs) against multiple military targets is studied to solve the problems of tactical decision making based on the DMTAGII model. In the process of modeling, the specific expressions of strategy, status and payoff functions of the games are considered, and the strategy is coded to match the structure of genetic algorithm so that the PBNE can be solved by combining the genetic algorithm and the interactive iteration algorithm.Finally, through the simulation the feasibility and effectiveness of the DMTAGII model are verified. Meanwhile, the calculated equilibrium strategies are also found to be realistic, which can provide certain references for improving the autonomous ability of UAV systems.

A Class of General Transformation of Characteristic Functions in Dynamic Games

The transformation of characteristic functions is an effective way to avoid time-inconsistency of cooperative solutions in dynamic games.There are several forms on the transformation of characteristic functions.In this paper,a class of general transformation of characteristic functions is proposed.It can lead to the time-consistency of cooperative solutions and guarantee that the irrational-behaviorproof conditions hold true.To illustrate the theory,an example of dynamic game on a tree is given.

On the finite horizon Nash equilibrium solution in the differential game approach to formation control

The solvability of the coupled Riccati differential equations appearing in the differential game approach to the formation control problem is vital to the finite horizon Nash equilibrium solution.These equations(if solvable)can be solved numerically by using the terminal value and the backward iteration.To investigate the solvability and solution of these equations the formation control problem as the differential game is replaced by a discrete-time dynamic game.The main contributions of this paper are as follows.First,the existence of Nash equilibrium controls for the discretetime formation control problem is shown.Second,a backward iteration approximate solution to the coupled Riccati differential equations in the continuous-time differential game is developed.An illustrative example is given to justify the models and solution.

Dynamic Resource Pricing and Allocation in Multilayer Satellite Network

The goal of delivering high-quality service has spurred research of 6G satellite communication networks.The limited resource-allocation problem has been addressed by next-generation satellite communication networks,especially multilayer networks with multiple low-Earth-orbit(LEO)and nonlow-Earth-orbit(NLEO)satellites.In this study,the resource-allocation problem of a multilayer satellite network consisting of one NLEO and multiple LEO satellites is solved.The NLEO satellite is the authorized user of spectrum resources and the LEO satellites are unauthorized users.The resource allocation and dynamic pricing problems are combined,and a dynamic gamebased resource pricing and allocation model is proposed to maximize the market advantage of LEO satellites and reduce interference between LEO and NLEO satellites.In the proposed model,the resource price is formulated as the dynamic state of the LEO satellites,using the resource allocation strategy as the control variable.Based on the proposed dynamic game model,an openloop Nash equilibrium is analyzed,and an algorithm is proposed for the resource pricing and allocation problem.Numerical simulations validate the model and algorithm.

Study on group air to ground attack-defends hierarchical dynamic decision-making

As to oppositional, multi-objective and hierarchical characteristic of air formation to ground attackdefends campaign, and using dynamic space state model of military campaign, this article establishes a principal and subordinate hierarchical interactive decision-making way, the Nash-Stackelberg-Nash model, to solve the problems in military operation, and find out the associated best strategy in hierarchical dynamic decision-making. The simulating result indicate that when applying the model to air formation to ground attack-defends decision-making system, it can solve the problems of two hierarchies, dynamic oppositional decision-making favorably, and reach preferable effect in battle. It proves that the model can provide an effective way for analyzing a battle,

Study on Game Theory of Social Law Enforcement

Based on the basis of the two stage dynamic game of complete information and purely tactful perfect equilibrium theory, the non cooperative gaming between the police department and the criminals is analyzed. Dynamic game can be proved to forecast and explain potential tactful choices of the police department and the criminals at various stages, so as to analyze the essence of the law enforcement by the theoretical models.

Irrational-Behavior-Proof Conditions for Stochastic Games over Event Trees

In this paper,the irrational-behavior-proof conditions in a class of stochastic dynamic games over event trees are presented.Four kinds of irrational-behavior-proof conditions are proposed by the imputation distribution procedure,and their relationships are discussed.More specific properties for the general transformation of characteristic functions are developed,based on which,the irrational-behavior-proof conditions are proved to be true in a transformed cooperative game.

联保贷款违约传染机制研究

本文基于动态博弈模型,对联保贷款履约激励机制进行分析。研究发现:①在社会惩罚有效的情况下,联保制度有利于提高借款人还款积极性,否则,联保制度对贷款履约激励机制具有负效应;②联保贷款违约传染机制具有较高的运行效率。因此,确保联保贷款组员之间社会惩罚的有效性,加强联保贷款客户筛选工作、提升违约威慑可置信度、增强借款人债权追索能力是提高联保贷款运行效率的重要手段。

Hawk and pigeon's intelligence for UAV swarm dynamic combat game via competitive learning pigeon-inspired optimization

For unmanned aerial vehicle(UAV)swarm dynamic combat,swarm antagonistic motion control and attack target allocation are extremely challenging sub-tasks.In this paper,the competitive learning pigeon-inspired optimization(CLPIO)algorithm is proposed to handle the cooperative dynamic combat problem,which integrates the distributed swarm antagonistic motion and centralized attack target allocation.Moreover,the threshold trigger strategy is presented to switch two sub-tasks.To seek a feasible and optimal combat scheme,a dynamic game approach combined with hawk grouping mechanism and situation assessment between sub-groups is designed to guide the solution of the optimal attack scheme,and the model of swarm antagonistic motion imitating pigeon’s intelligence is proposed to form a confrontation situation.The analysis of the CLPIO algorithm shows its convergence in theory and the comparison with the other four metaheuristic algorithms shows its superiority in solving the mixed Nash equilibrium problem.Finally,numerical simulation verifis that the proposed methods can provide an effective combat scheme in the set scenario.

Evaluation of global navigation satellite system spoofing efficacy

The spoofing capability of Global Navigation Satellite System(GNSS)represents an important confrontational capability for navigation security,and the success of planned missions may depend on the effective evaluation of spoofing capability.However,current evaluation systems face challenges arising from the irrationality of previous weighting methods,inapplicability of the conventional multi-attribute decision-making method and uncertainty existing in evaluation.To solve these difficulties,considering the validity of the obtained results,an evaluation method based on the game aggregated weight model and a joint approach involving the grey relational analysis and technique for order preference by similarity to an ideal solution(GRA-TOPSIS)are firstly proposed to determine the optimal scheme.Static and dynamic evaluation results under different schemes are then obtained via a fuzzy comprehensive assessment and an improved dynamic game method,to prioritize the deceptive efficacy of the equipment accurately and make pointed improvement for its core performance.The use of judging indicators,including Spearman rank correlation coefficient and so on,combined with obtained evaluation results,demonstrates the superiority of the proposed method and the optimal scheme by the horizontal comparison of different methods and vertical comparison of evaluation results.Finally,the results of field measurements and simulation tests show that the proposed method can better overcome the difficulties of existing methods and realize the effective evaluation.

Discrete-time dynamic graphical games:model-free reinforcement learning solution

This paper introduces a model-free reinforcement learning technique that is used to solve a class of dynamic games known as dynamic graphical games. The graphical game results from to make all the agents synchronize to the state of a command multi-agent dynamical systems, where pinning control is used generator or a leader agent. Novel coupled Bellman equations and Hamiltonian functions are developed for the dynamic graphical games. The Hamiltonian mechanics are used to derive the necessary conditions for optimality. The solution for the dynamic graphical game is given in terms of the solution to a set of coupled Hamilton-Jacobi-Bellman equations developed herein. Nash equilibrium solution for the graphical game is given in terms of the solution to the underlying coupled Hamilton-Jacobi-Bellman equations. An online model-free policy iteration algorithm is developed to learn the Nash solution for the dynamic graphical game. This algorithm does not require any knowledge of the agents＇ dynamics. A proof of convergence for this multi-agent learning algorithm is given under mild assumption about the inter-connectivity properties of the graph. A gradient descent technique with critic network structures is used to implement the policy iteration algorithm to solve the graphical game online in real-time.

A Dynamic Multi-Player Bargaining Game with Veto Players

This paper studies the effect of veto right on players’income in multi-player dynamic bargaining game.Based on a basic multi-person dynamic bargaining model generalized by the Rubinstein’s two-person alternating-offer bargaining model,the authors construct a dynamic multi-player bargaining game with veto players by adding a constraint to its negotiation process,which is obtained by studying the influence of exercising the veto right exercised by veto players.The authors emphatically describe the strategic game form of this dynamic bargaining game and study its equilibrium,then we analyze the relationship between the minimum acceptable payoff of the veto players and the equilibrium income.The research shows that veto right may increase the benefits of veto players and decrease the benefits of non-veto players.Veto players will not affect the players’benefits and the form of equilibrium when the minimum acceptable payoff of every veto player is relatively low.When the minimum acceptable payoff of the veto player is high enough,he can only get the minimum acceptable payoff,and his benefit increases as his minimum acceptable payoff increases.In this case,the veto player has intention to obtain more resources by presenting a higher minimum acceptable payoff.

Dynamic multicast sharing protection algorithm based on fuzzy game in multi-domain optical network

Aiming at the problem of dynamic multicast service protection in multi-domain optical network, this paper proposes a dynamic multicast sharing protection algorithm based on fuzzy game in multi-domain optical network. The algorithm uses the minimum cost spanning tree strategy and fuzzy game theory. First, it virtualizes two planes to calculate the multicast tree and the multicast protection tree respectively. Then, it performs a fuzzy game to form a cooperative alliance to optimize the path composition of each multicast tree. Finally, it generates a pair of optimal multicast work tree and multicast protection tree for dynamic multicast services. The time complexity of the algorithm is O(k3 m2 n), where n represents the number of nodes in the networks, k represents the number of dynamic multicast requests, and m represents the number of destination nodes for each multicast request. The experimental results show that the proposed algorithm reduces significantly the blocking rate of dynamic multicast services, and improves the utilization of optical network resources within a certain number of dynamic multicast request ranges.

Research on pricing game strategy for load-balancing in VANET

In vehicle Ad-hoc netwok （VANET）, traffic load is often unevenly distributed among access points （APs）. Such load imbalance hampers the network from fully utilizing the network capacity. To alleviate such imbalance, the paper introduces a novel pricing game model. The research scene is at the intersection when the traffic light is green. As vehicles are highly mobile and the network typology changes dynamically, the paper divides the green light time into equal slots and calculates APs＇ prices with the presented pricing game in each time slot. The whole process is a repeated game model. The final equilibrium solution set is APs＇ pricing strategy, and the paper claim that this equilibrium solution set can affect vehicles＇ selection and ensure APs＇ load-balancing. Simulation results based on a realistic vehicular traffic model demonstrate the effectiveness of the game method.