Value Iteration-Based Cooperative Adaptive Optimal Control for Multi-Player Differential Games With Incomplete Information

This paper presents a novel cooperative value iteration(VI)-based adaptive dynamic programming method for multi-player differential game models with a convergence proof.The players are divided into two groups in the learning process and adapt their policies sequentially.Our method removes the dependence of admissible initial policies,which is one of the main drawbacks of the PI-based frameworks.Furthermore,this algorithm enables the players to adapt their control policies without full knowledge of others’ system parameters or control laws.The efficacy of our method is illustrated by three examples.

How do international agreement constraints affect marine litter management strategies?A research based on differential game

Based on differential game theory,the decision-making problem of two homogeneous countries facing transboundary marine litter governance is studied.On the basis of assuming that the input of marine litter is an exogenous variable,the focus is on reducing the accumulation of marine litter through cleanup and transfer processing by both parties.Considering the constant and increasing input of marine litter,in the framework of international agreement constraints,the analysis of the game behavior of the players in the marine litter governance under the open-loop strategy(in the case of agreement constraints)and the Markov strategy(in the case of no agreement constraints)was compared and analyzed.The research results show that when the direct pollution cost of marine litter is high enough,both sides of the game adopt an open-loop strategy that complies with the constraints of the agreement,which can reduce the accumulation of marine litter and improve the environmental quality.However,when there is a high initial accumulation of marine litter,the Markov strategy without protocol constraints will be better than the open-loop strategy.In the case that marine litter does not need to be transferred,there will be no difference between the two sides of the game adopting the Markov strategy and adopting the open-loop strategy on the equilibrium growth path.

VIRTUAL TARGET DIFFERENTIAL GAME MIDCOURSE GUIDANCE LAW FOR HYPERSONIC CRUISE MISSILE BASED ON NEURAL NETWORK

For the high altitude cruising flight phase of a hypersonic cruise missile （HCM）, a relative motion mod- el between the missile and the target is established by defining virtual target and combining the theory of the dif- ferential geometry with missile motion equations. Based on the model, the motion between the missile and the tar- get is considered as a single target differential game problem, and a new open-loop differential game midcourse guidance law （DGMGL） is deduced by solving the corresponding Hamiltonian Function. Meanwhile, a new struc- ture of a closed-loop DGMGL is presented and the training data for back propagation neural network （BPNN） are designed. By combining the theory of BPNN with the open-loop DGMGL obtained above, the law intelligence is realized. Finally, simulation is carried out and the validity of the law is testified.

Differential game strategy in three-player evasion and pursuit scenarios

A conflict of three players, including an attacker, a defender, and a target with bounded control is discussed based on the differential game theories in which the target and the defender use an optimal pursuit strategy. The current approach chooses the miss distance as the outcome of the conflict. Different optimal guidance laws are investigated, and feasible conditions are analyzed for the attacker to accomplish an attacking task. For some given conditions, the attacker cannot intercept the target by only using a one-to-one optimal pursuit guidance law; thus, a guidance law for the attacker to reach a critical safe value is investigated.Specifically, the guidance law is divided into two parts. Before the engagement time between the defender and the attacker, the attacker uses this derived guidance law to guarantee that the evasion distance from the defender is safe, and that the zero-effort-miss(ZEM) distance between the attacker and the target is the smallest.After that engagement time, the attacker uses the optimal one-toone guidance law to accomplish the pursuit task. The advantages and limited conditions of these derived guidance laws are also investigated by using nonlinear simulations.

A Stackelberg Differential Game Based Bandwidth Allocation in Satellite Communication Network

In this paper, a Stackelberg differential game based approach is proposed to solve the bandwidth allocation problems in satellite communication network. All the satellites are divided into two groups, one has high download requirements, and the other one has low download requirements. Each satellites group has its own controller for bandwidth allocation, and can get payments from the satellites for the allocated resources. The relationships between the controllers and satellites are formed as a Stackelberg game. In our model, differential equation is introduced to describe the bandwidth dynamics for the whole satellite communication network. Combine the differential equation and Stackelberg game together, we can formulate the bandwidth allocation problems in satellite communication network as a Stackelber differential game. The solutions to the proposed game is solved based the Bellman dynamic equations. Numerical simulations are given to prove the effeteness and correctness of the proposed approach.

A Non-Cooperative Differential Game-Based Security Model in Fog Computing

Fog computing is a new paradigm providing network services such as computing, storage between the end users and cloud. The distributed and open structure are the characteristics of fog computing, which make it vulnerable and very weak to security threats. In this article, the interaction between vulnerable nodes and malicious nodes in the fog computing is investigated as a non-cooperative differential game. The complex decision making process is reviewed and analyzed. To solve the game, a fictitious play-based algorithm is which the vulnerable node and the malicious nodes reach a feedback Nash equilibrium. We attain optimal strategy of energy consumption with Qo S guarantee for the system, which are conveniently operated and suitable for fog nodes. The system simulation identifies the propagation of malicious nodes. We also determine the effects of various parameters on the optimal strategy. The simulation results support a theoretical foundation to limit malicious nodes in fog computing, which can help fog service providers make the optimal dynamic strategies when different types of nodes dynamically change their strategies.

On the finite horizon Nash equilibrium solution in the differential game approach to formation control

The solvability of the coupled Riccati differential equations appearing in the differential game approach to the formation control problem is vital to the finite horizon Nash equilibrium solution.These equations(if solvable)can be solved numerically by using the terminal value and the backward iteration.To investigate the solvability and solution of these equations the formation control problem as the differential game is replaced by a discrete-time dynamic game.The main contributions of this paper are as follows.First,the existence of Nash equilibrium controls for the discretetime formation control problem is shown.Second,a backward iteration approximate solution to the coupled Riccati differential equations in the continuous-time differential game is developed.An illustrative example is given to justify the models and solution.

Resource Allocation for Network Security Risk Assessment：A Non-Cooperative Differential Game Based Approach

In this paper, we propose a non-cooperative differential game theory based resource allocation approach for the network security risk assessment. For the risk assessment, the resource will be used for risk assess, including response cost and response negative cost. The whole assessment process is considered as a differential game for optimal resource control. The proposed scheme can be obtained through the Nash Equilibrium. It is proved that the game theory based algorithm is applicable and the optimal resource level can be achieved based on the proposed algorithm.

DETERMINING THE DISCRIMINATING DOMAIN FOR HYBRID LINEAR DIFFERENTIAL GAME WITH TWO PLAYERS AND TWO TARGETS

This paper studies a bounded discriminating domain for hybrid linear differential game with two players and two targets using viability theory. First of all, we prove that the convex hull of a closed set is also a discriminating domain if the set is a discriminating domain. Secondly, in order to determine that a bounded polyhedron is a discriminating domain, we give a result that it only needs to verify that the extreme points of the polyhedron meet the viability conditions. The difference between our result and the existing ones is that our result just needs to verify the finite points （extreme points） and the existing ones need to verify all points in the bounded polyhedron.

Resource Allocation in Edge-Computing Based Wireless Networks Based on Differential Game and Feedback Control

In this paper,we have proposed a differential game model to optimally solve the resource allocation problems in the edge-computing based wireless networks.In the proposed model,a wireless network with one cloud-computing center(CC)and lots of edge services providers(ESPs)is investigated.In order to provide users with higher services quality,the ESPs in the proposed wireless network should lease the computing resources from the CC and the CC can allocate its idle cloud computing resource to the ESPs.We will try to optimally allocate the edge computing resources between the ESPs and CC using the differential game and feedback control.Based on the proposed model,the ESPs can choose the amount of computing resources from the CC using feedback control,which is affected by the unit price of computing resources controlled by the CC.In the simulation part,the optimal allocated resources for users’services are obtained based on the Nash equilibrium of the proposed differential game.The effectiveness and correctness of the proposed scheme is also verified through the numerical simulations and results.

NUMERICAL METHOD BASED ON HAMILTON SYSTEM AND SYMPLECTIC ALGORITHM TO DIFFERENTIAL GAMES

The resolution of differential games often concerns the difficult problem of two points border value （TPBV）, then ascribe linear quadratic differential game to Hamilton system. To Hamilton system, the algorithm of symplectic geometry has the merits of being able to copy the dynamic structure of Hamilton system and keep the measure of phase plane. From the viewpoint of Hamilton system, the symplectic characters of linear quadratic differential game were probed; as a try, Symplectic-Runge-Kutta algorithm was presented for the resolution of infinite horizon linear quadratic differential game. An example of numerical calculation was given, and the result can illuminate the feasibility of this method. At the same time, it embodies the fine conservation characteristics of symplectic algorithm to system energy.

Non-Cooperative Differential Game Based Load Balancing Algorithm in Radio-Over-Fibre System

Load balancing plays a critical role in a cellular network. As one kind of cellular network, Radio-over-Fibre （RoF） system can provide ubiquitous high data-rate transmissions, which has attracted many attentions, but it also suffer load unbalancing problem. In order to improve the system performance, in this paper, we propose a novel loading balance scheme in RoF system based on differential game theory. The scheme formulates the load allocated to each RAP （Radio Access Point） as a Nasb Equilibrium, using non-cooperative differential game to obtain the optical load allocation of each RAP. The simulations performed show that the non-cooperative differential game algorithm is applicable and the optimal load solution can be achieved.

Stochastic Differential Game Model for Decentralized Power Control with Wisdom Regulation Factor in Wearable Audio-Oriented BodyNets

In this study, aiming at the characteristics of randomness and dynamics in Wearable Audiooriented BodyNets （WA-BodyNets）, stochastic differential game theory is applied to the investigation of the problem of transmitted power control inconsumer electronic devices. First, astochastic differential game model is proposed for non-cooperative decentralized uplink power control with a wisdom regulation factor over WA-BodyNets with a onehop star topology.This model aims to minimize the cost associated with the novel payoff function of a player, for which two cost functions are defined： functions of inherent power radiation and accumulated power radiation darmge. Second, the feedback Nash equilibrium solution of the proposed model and the constraint of the Quality of Service （QoS） requirement of the player based on the SIR threshold are derived by solving the Fleming-Bellman-Isaacs partial differential equations. Furthermore, the Markov property of the optimal feedback strategies in this model is verified.The simulation results show that the proposed game model is effective and feasible for controlling the transmitted power of WA-BodyNets.

A QoS Guarantee Service Serving Model in LEO Satellite Networks Based on Differential Game

Low Earth Orbit （LEO） satellites networks can provide multimedia service and plays an increasingly important role in the exploitation of space. However, one of the challenges in LEO satellites networks is that the services are suffered from high symbol error, limited storage space and limited available energy. To analyze the performance of the service in LEO satellites networks, a model, based on differential game, is proposed for satisfying the QoS requirements of multimedia applications. The controller of our model is the transmitting rate and the object is to maximize the payoff depending on the error symbol rate, the available energy, the bandwidth and the process ability so as to guarantee the QoS service. In order to solve our built model, we use the Bellman theorem to make formulas on the trance of the optimal transmitting rate. Furthermore, simulation results verify that the service can be maximized by using our derived transmitting rate trance.

A simplified differential game model for the optimal choice of tax rate

A classical problem on optimal choice of tax rate from the perspective of differential game approach is studied. Under some appropriate assumptions on the profit and utility functions, the open-loop Stackelberg equilibrium solution which is time- dependent is obtained. Result shows that 1) the optimal strategies derived from differential game and traditional unilateral optimal control approaches are different; 2) both marginal profit rate and the market rate of interest have great effect on the equilibrium solution; and 3) the government should think about the firm’s potential reaction when selecting tax rates and the timing of taxation.

A Cloud-Assisted Malware Detection and Suppression Framework for Wireless Multimedia System in IoT Based on Dynamic Differential Game

As a novel dynamic network service infrastructure, Internet of Things （IoT） has gained remarkable popularity with obvious su- periorities in the interoperability and real-time communication. Despite of the convenience in collecting information to provide the decision basis for the users, the vulnerability of embed- ded sensor nodes in multimedia devices makes the malware propagation a growing serious problem, which would harm the security of devices and their users financially and physi- cally in wireless multimedia system （WMS）. Therefore, many researches related to the mal- ware propagation and suppression have been proposed to protect the topology and system security of wireless multimedia network. In these studies, the epidemic model is of great significance to the analysis of malware prop- agation. Considering the cloud and state tran- sition of sensor nodes, a cloud-assisted model for malware detection and the dynamic differ- ential game against malware propagation are proposed in this paper. Firstly, a SVM based malware detection model is constructed with the data sharing at the security platform in the cloud. Then the number of malware-infected nodes with physical infectivity to susceptible nodes is calculated precisely based on the at- tributes of WMS transmission. Then the statetransition among WMS the modified epidemic devices is defined by model. Furthermore, a dynamic differential game and target cost function are successively derived for the Nash equilibrium between malware and WMS sys- tem. On this basis, a saddle-point malware de- tection and suppression algorithm is presented depending on the modified epidemic model and the computation of optimal strategies. Nu- merical results and comparisons show that the proposed algorithm can increase the utility of WMS efficiently and effectively.

Open Loop Saddle Point on Linear Quadratic Stochastic Differential Games

In this paper, we deal with one kind of two-player zero-sum linear quadratic stochastic differential game problem. We give the existence of an open loop saddle point if and only if the lower and upper values exist.

Differential Game Model of Resource Extraction with Continuous and Dynamic Updating

This paper is devoted to a new class of differential games with continuous and dynamic updating.The direct application of resource extraction in a case of dynamic and continuous updating is considered.It is proved that the optimal control(cooperative strategies)and feedback Nash equilibrium strategies uniformly converge to the corresponding strategies in the game model with continuous updating as the number of updating instants converges to infinity.Similar results are presented for an optimal trajectory(cooperative trajectory),equilibrium trajectory and corresponding payoffs.

Equilibrium Strategy of the Pursuit-Evasion Game in Three-Dimensional Space

The pursuit-evasion game models the strategic interaction among players, attracting attention in many realistic scenarios, such as missile guidance, unmanned aerial vehicles, and target defense. Existing studies mainly concentrate on the cooperative pursuit of multiple players in two-dimensional pursuit-evasion games. However, these approaches can hardly be applied to practical situations where players usually move in three-dimensional space with a three-degree-of-freedom control. In this paper,we make the first attempt to investigate the equilibrium strategy of the realistic pursuit-evasion game, in which the pursuer follows a three-degree-of-freedom control, and the evader moves freely. First, we describe the pursuer's three-degree-of-freedom control and the evader's relative coordinate. We then rigorously derive the equilibrium strategy by solving the retrogressive path equation according to the Hamilton-Jacobi-Bellman-Isaacs(HJBI) method, which divides the pursuit-evasion process into the navigation and acceleration phases. Besides, we analyze the maximum allowable speed for the pursuer to capture the evader successfully and provide the strategy with which the evader can escape when the pursuer's speed exceeds the threshold. We further conduct comparison tests with various unilateral deviations to verify that the proposed strategy forms a Nash equilibrium.

Linear Quadratic Leader-Follower Stochastic Differential Games:Closed-Loop Solvability

In this paper,a leader-follower stochastic differential game is studied for a linear stochastic differential equation with quadratic cost functionals.The coefficients in the state equation and the weighting matrices in the cost functionals are all deterministic.Closed-loop strategies are introduced,which require to be independent of initial states;and such a nature makes it very useful and convenient in applications.The follower first solves a stochastic linear quadratic optimal control problem,and his optimal closed-loop strategy is characterized by a Riccati equation,together with an adapted solution to a linear backward stochastic differential equation.Then the leader turns to solve a stochastic linear quadratic optimal control problem of a forward-backward stochastic differential equation,necessary conditions for the existence of the optimal closed-loop strategy for the leader is given by a Riccati equation.Some examples are also given.