Value Iteration-Based Cooperative Adaptive Optimal Control for Multi-Player Differential Games With Incomplete Information

This paper presents a novel cooperative value iteration(VI)-based adaptive dynamic programming method for multi-player differential game models with a convergence proof.The players are divided into two groups in the learning process and adapt their policies sequentially.Our method removes the dependence of admissible initial policies,which is one of the main drawbacks of the PI-based frameworks.Furthermore,this algorithm enables the players to adapt their control policies without full knowledge of others’ system parameters or control laws.The efficacy of our method is illustrated by three examples.

How do international agreement constraints affect marine litter management strategies?A research based on differential game

Based on differential game theory,the decision-making problem of two homogeneous countries facing transboundary marine litter governance is studied.On the basis of assuming that the input of marine litter is an exogenous variable,the focus is on reducing the accumulation of marine litter through cleanup and transfer processing by both parties.Considering the constant and increasing input of marine litter,in the framework of international agreement constraints,the analysis of the game behavior of the players in the marine litter governance under the open-loop strategy(in the case of agreement constraints)and the Markov strategy(in the case of no agreement constraints)was compared and analyzed.The research results show that when the direct pollution cost of marine litter is high enough,both sides of the game adopt an open-loop strategy that complies with the constraints of the agreement,which can reduce the accumulation of marine litter and improve the environmental quality.However,when there is a high initial accumulation of marine litter,the Markov strategy without protocol constraints will be better than the open-loop strategy.In the case that marine litter does not need to be transferred,there will be no difference between the two sides of the game adopting the Markov strategy and adopting the open-loop strategy on the equilibrium growth path.

Resource Allocation in Edge-Computing Based Wireless Networks Based on Differential Game and Feedback Control

In this paper,we have proposed a differential game model to optimally solve the resource allocation problems in the edge-computing based wireless networks.In the proposed model,a wireless network with one cloud-computing center(CC)and lots of edge services providers(ESPs)is investigated.In order to provide users with higher services quality,the ESPs in the proposed wireless network should lease the computing resources from the CC and the CC can allocate its idle cloud computing resource to the ESPs.We will try to optimally allocate the edge computing resources between the ESPs and CC using the differential game and feedback control.Based on the proposed model,the ESPs can choose the amount of computing resources from the CC using feedback control,which is affected by the unit price of computing resources controlled by the CC.In the simulation part,the optimal allocated resources for users’services are obtained based on the Nash equilibrium of the proposed differential game.The effectiveness and correctness of the proposed scheme is also verified through the numerical simulations and results.

VIRTUAL TARGET DIFFERENTIAL GAME MIDCOURSE GUIDANCE LAW FOR HYPERSONIC CRUISE MISSILE BASED ON NEURAL NETWORK

For the high altitude cruising flight phase of a hypersonic cruise missile （HCM）, a relative motion mod- el between the missile and the target is established by defining virtual target and combining the theory of the dif- ferential geometry with missile motion equations. Based on the model, the motion between the missile and the tar- get is considered as a single target differential game problem, and a new open-loop differential game midcourse guidance law （DGMGL） is deduced by solving the corresponding Hamiltonian Function. Meanwhile, a new struc- ture of a closed-loop DGMGL is presented and the training data for back propagation neural network （BPNN） are designed. By combining the theory of BPNN with the open-loop DGMGL obtained above, the law intelligence is realized. Finally, simulation is carried out and the validity of the law is testified.

LINEAR QUADRATIC NONZERO-SUM DIFFERENTIAL GAMES WITH RANDOM JUMPS

The existence and uniqueness of the solutions for one kind of forward-backward stochastic differential equations with Brownian motion and Poisson process as the noise source were given under the monotone conditions.Then these results were applied to nonzero-sum differential games with random jumps to get the explicit form of the open-loop Nash equilibrium point by the solution of the forward-backward stochastic differential equations.

通信限制下多DG接入配网的新型电流差动保护

差动保护是解决分布式电源(distributed generation, DG)接入下配电网保护难题的有效手段,然而配网现有通信建设水平很大程度上制约了差动保护的构建。首先,融合电流相位与相量差动保护思想,同时将电流相位信息进行变换,实现通信信息降容。应用递推最小二乘法实现短路电流相位参数的快速估计,提出了短路电流相位修正策略,以应对CT饱和与直流分量对保护的干扰。最后,提出适用于含多DG配网的新型电流差动保护构建原则与保护判据。基于PSACD/EMTDC平台搭建仿真模型,验证了所提出的电流相位参数预测方法的精度与新型差动保护的性能。算例结果表明:所提电流相位参数预测方法速度快、精度高,具有良好的抗CT饱和能力;所提新型差动保护原理具有良好的性能。

Differential game strategy in three-player evasion and pursuit scenarios

A conflict of three players, including an attacker, a defender, and a target with bounded control is discussed based on the differential game theories in which the target and the defender use an optimal pursuit strategy. The current approach chooses the miss distance as the outcome of the conflict. Different optimal guidance laws are investigated, and feasible conditions are analyzed for the attacker to accomplish an attacking task. For some given conditions, the attacker cannot intercept the target by only using a one-to-one optimal pursuit guidance law; thus, a guidance law for the attacker to reach a critical safe value is investigated.Specifically, the guidance law is divided into two parts. Before the engagement time between the defender and the attacker, the attacker uses this derived guidance law to guarantee that the evasion distance from the defender is safe, and that the zero-effort-miss(ZEM) distance between the attacker and the target is the smallest.After that engagement time, the attacker uses the optimal one-toone guidance law to accomplish the pursuit task. The advantages and limited conditions of these derived guidance laws are also investigated by using nonlinear simulations.

Non-Cooperative Differential Game Based Energy Consumption Control for Dynamic Demand Response in Smart Grid

In this paper, we conduct research on the dynamic demand response problem in smart grid to control the energy consumption. The objective of the energy consumption control is constructed based on differential game, as the dynamic of each users’ energy state in smart gird can be described based on a differential equation. Concept of electricity sharing is introduced to achieve load shift of main users from the high price hours to the low price hours. Nash equilibrium is given based on the Hamilton equation and the effectiveness of the proposed model is verified based on the numerical simulation results.

LINEAR QUADRATIC NONZERO-SUM DIFFERENTIAL GAMES WITH RANDOM JUMPS

The existence and uniqueness of the solutions for one kind of forward- backward stochastic differential equations with Brownian motion and Poisson process as the noise source were given under the monotone conditions. Then these results were applied to nonzero-sum differential games with random jumps to get the explicit form of the open-loop Nash equilibrium point by the solution of the forward-backward stochastic differential equations.

A Non-Cooperative Differential Game-Based Security Model in Fog Computing

Fog computing is a new paradigm providing network services such as computing, storage between the end users and cloud. The distributed and open structure are the characteristics of fog computing, which make it vulnerable and very weak to security threats. In this article, the interaction between vulnerable nodes and malicious nodes in the fog computing is investigated as a non-cooperative differential game. The complex decision making process is reviewed and analyzed. To solve the game, a fictitious play-based algorithm is which the vulnerable node and the malicious nodes reach a feedback Nash equilibrium. We attain optimal strategy of energy consumption with Qo S guarantee for the system, which are conveniently operated and suitable for fog nodes. The system simulation identifies the propagation of malicious nodes. We also determine the effects of various parameters on the optimal strategy. The simulation results support a theoretical foundation to limit malicious nodes in fog computing, which can help fog service providers make the optimal dynamic strategies when different types of nodes dynamically change their strategies.

On the finite horizon Nash equilibrium solution in the differential game approach to formation control

The solvability of the coupled Riccati differential equations appearing in the differential game approach to the formation control problem is vital to the finite horizon Nash equilibrium solution.These equations(if solvable)can be solved numerically by using the terminal value and the backward iteration.To investigate the solvability and solution of these equations the formation control problem as the differential game is replaced by a discrete-time dynamic game.The main contributions of this paper are as follows.First,the existence of Nash equilibrium controls for the discretetime formation control problem is shown.Second,a backward iteration approximate solution to the coupled Riccati differential equations in the continuous-time differential game is developed.An illustrative example is given to justify the models and solution.

Equilibrium Strategy of the Pursuit-Evasion Game in Three-Dimensional Space

The pursuit-evasion game models the strategic interaction among players, attracting attention in many realistic scenarios, such as missile guidance, unmanned aerial vehicles, and target defense. Existing studies mainly concentrate on the cooperative pursuit of multiple players in two-dimensional pursuit-evasion games. However, these approaches can hardly be applied to practical situations where players usually move in three-dimensional space with a three-degree-of-freedom control. In this paper,we make the first attempt to investigate the equilibrium strategy of the realistic pursuit-evasion game, in which the pursuer follows a three-degree-of-freedom control, and the evader moves freely. First, we describe the pursuer's three-degree-of-freedom control and the evader's relative coordinate. We then rigorously derive the equilibrium strategy by solving the retrogressive path equation according to the Hamilton-Jacobi-Bellman-Isaacs(HJBI) method, which divides the pursuit-evasion process into the navigation and acceleration phases. Besides, we analyze the maximum allowable speed for the pursuer to capture the evader successfully and provide the strategy with which the evader can escape when the pursuer's speed exceeds the threshold. We further conduct comparison tests with various unilateral deviations to verify that the proposed strategy forms a Nash equilibrium.

A Stackelberg Differential Game Based Bandwidth Allocation in Satellite Communication Network

In this paper, a Stackelberg differential game based approach is proposed to solve the bandwidth allocation problems in satellite communication network. All the satellites are divided into two groups, one has high download requirements, and the other one has low download requirements. Each satellites group has its own controller for bandwidth allocation, and can get payments from the satellites for the allocated resources. The relationships between the controllers and satellites are formed as a Stackelberg game. In our model, differential equation is introduced to describe the bandwidth dynamics for the whole satellite communication network. Combine the differential equation and Stackelberg game together, we can formulate the bandwidth allocation problems in satellite communication network as a Stackelber differential game. The solutions to the proposed game is solved based the Bellman dynamic equations. Numerical simulations are given to prove the effeteness and correctness of the proposed approach.

Differential Game Based Link Resource Management for Next Generation Optical Network

In this paper, we investigate the link resource management problem for optical networks, to achieve the resource cost during the information transmission. We use the differential game to formulate the cost control problem for the link resource management, to minimize the resource allocation cost functions, which dynamic behaviours are described by differential equations. Each link controls its transmission bandwidth based on the Nash equilibriums of the differential game. The effectiveness of the proposed model is given through numerical simulations.

Infinite Horizon LQ Zero-Sum Stochastic Differential Games with Markovian Jumps

This paper studies a class of continuous-time two person zero-sum stochastic differential games characterized by linear It?’s differential equation with state-dependent noise and Markovian parameter jumps. Under the assumption of stochastic stabilizability, necessary and sufficient condition for the existence of the optimal control strategies is presented by means of a system of coupled algebraic Riccati equations via using the stochastic optimal control theory. Furthermore, the stochastic H∞ control problem for stochastic systems with Markovian jumps is discussed as an immediate application, and meanwhile, an illustrative example is presented.

Privacy Protection Algorithm for the Internet of Vehicles Based on Local Differential Privacy and Game Model

In recent years,with the continuous advancement of the intelligent process of the Internet of Vehicles(IoV),the problem of privacy leakage in IoV has become increasingly prominent.The research on the privacy protection of the IoV has become the focus of the society.This paper analyzes the advantages and disadvantages of the existing location privacy protection system structure and algorithms,proposes a privacy protection system structure based on untrusted data collection server,and designs a vehicle location acquisition algorithm based on a local differential privacy and game model.The algorithm first meshes the road network space.Then,the dynamic game model is introduced into the game user location privacy protection model and the attacker location semantic inference model,thereby minimizing the possibility of exposing the regional semantic privacy of the k-location set while maximizing the availability of the service.On this basis,a statistical method is designed,which satisfies the local differential privacy of k-location sets and obtains unbiased estimation of traffic density in different regions.Finally,this paper verifies the algorithm based on the data set of mobile vehicles in Shanghai.The experimental results show that the algorithm can guarantee the user’s location privacy and location semantic privacy while satisfying the service quality requirements,and provide better privacy protection and service for the users of the IoV.

DETERMINING THE DISCRIMINATING DOMAIN FOR HYBRID LINEAR DIFFERENTIAL GAME WITH TWO PLAYERS AND TWO TARGETS

This paper studies a bounded discriminating domain for hybrid linear differential game with two players and two targets using viability theory. First of all, we prove that the convex hull of a closed set is also a discriminating domain if the set is a discriminating domain. Secondly, in order to determine that a bounded polyhedron is a discriminating domain, we give a result that it only needs to verify that the extreme points of the polyhedron meet the viability conditions. The difference between our result and the existing ones is that our result just needs to verify the finite points （extreme points） and the existing ones need to verify all points in the bounded polyhedron.

Resource Allocation for Network Security Risk Assessment：A Non-Cooperative Differential Game Based Approach

In this paper, we propose a non-cooperative differential game theory based resource allocation approach for the network security risk assessment. For the risk assessment, the resource will be used for risk assess, including response cost and response negative cost. The whole assessment process is considered as a differential game for optimal resource control. The proposed scheme can be obtained through the Nash Equilibrium. It is proved that the game theory based algorithm is applicable and the optimal resource level can be achieved based on the proposed algorithm.

NUMERICAL METHOD BASED ON HAMILTON SYSTEM AND SYMPLECTIC ALGORITHM TO DIFFERENTIAL GAMES

The resolution of differential games often concerns the difficult problem of two points border value （TPBV）, then ascribe linear quadratic differential game to Hamilton system. To Hamilton system, the algorithm of symplectic geometry has the merits of being able to copy the dynamic structure of Hamilton system and keep the measure of phase plane. From the viewpoint of Hamilton system, the symplectic characters of linear quadratic differential game were probed; as a try, Symplectic-Runge-Kutta algorithm was presented for the resolution of infinite horizon linear quadratic differential game. An example of numerical calculation was given, and the result can illuminate the feasibility of this method. At the same time, it embodies the fine conservation characteristics of symplectic algorithm to system energy.