Distributed Nash Equilibrium Seeking Strategies Under Quantized Communication

This paper is concerned with distributed Nash equi librium seeking strategies under quantized communication. In the proposed seeking strategy, a projection operator is synthesized with a gradient search method to achieve the optimization o players' objective functions while restricting their actions within required non-empty, convex and compact domains. In addition, a leader-following consensus protocol, in which quantized informa tion flows are utilized, is employed for information sharing among players. More specifically, logarithmic quantizers and uniform quantizers are investigated under both undirected and connected communication graphs and strongly connected digraphs, respec tively. Through Lyapunov stability analysis, it is shown that play ers' actions can be steered to a neighborhood of the Nash equilib rium with logarithmic and uniform quantizers, and the quanti fied convergence error depends on the parameter of the quan tizer for both undirected and directed cases. A numerical exam ple is given to verify the theoretical results.

Nash Equilibrium of a Fixed-Sum Two-Player Game

It is well established that Nash equilibrium exists within the framework of mixed strategies in strategic-form non-cooperative games. However, finding the Nash equilibrium generally belongs to the class of problems known as PPAD (Polynomial Parity Argument on Directed graphs), for which no polynomial-time solution methods are known, even for two-player games. This paper demonstrates that in fixed-sum two-player games (including zero-sum games), the Nash equilibrium forms a convex set, and has a unique expected payoff. Furthermore, these equilibria are Pareto optimal. Additionally, it is shown that the Nash equilibrium of fixed-sum two-player games can theoretically be found in polynomial time using the principal-dual interior point method, a solution method of linear programming.

A Penalty Approach for Generalized Nash Equilibrium Problem

The generalized Nash equilibrium problem （GNEP） is a generalization of the standard Nash equilibrium problem （NEP）, in which both the utility function and the strategy space of each player depend on the strategies chosen by all other players. This problem has been used to model various problems in applications. However, the convergent solution algorithms are extremely scare in the literature. In this paper, we present an incremental penalty method for the GNEP, and show that a solution of the GNEP can be found by solving a sequence of smooth NEPs. We then apply the semismooth Newton method with Armijo line search to solve latter problems and provide some results of numerical experiments to illustrate the proposed approach.

Distributed Nash Equilibrium Seeking for General Networked Games With Bounded Disturbances

This paper is concerned with anti-disturbance Nash equilibrium seeking for games with partial information.First,reduced-order disturbance observer-based algorithms are proposed to achieve Nash equilibrium seeking for games with firstorder and second-order players,respectively.In the developed algorithms,the observed disturbance values are included in control signals to eliminate the influence of disturbances,based on which a gradient-like optimization method is implemented for each player.Second,a signum function based distributed algorithm is proposed to attenuate disturbances for games with secondorder integrator-type players.To be more specific,a signum function is involved in the proposed seeking strategy to dominate disturbances,based on which the feedback of the velocity-like states and the gradients of the functions associated with players achieves stabilization of system dynamics and optimization of players'objective functions.Through Lyapunov stability analysis,it is proven that the players'actions can approach a small region around the Nash equilibrium by utilizing disturbance observerbased strategies with appropriate control gains.Moreover,exponential(asymptotic)convergence can be achieved when the signum function based control strategy(with an adaptive control gain)is employed.The performance of the proposed algorithms is tested by utilizing an integrated simulation platform of virtual robot experimentation platform(V-REP)and MATLAB.

Fully Distributed Nash Equilibrium Seeking for High-Order Players With Actuator Limitations

This paper explores the problem of distributed Nash equilibrium seeking in games, where players have limited knowledge on other players' actions. In particular, the involved players are considered to be high-order integrators with their control inputs constrained within a pre-specified region. A linear transformation for players' dynamics is firstly utilized to facilitate the design of bounded control inputs incorporating multiple saturation functions. By introducing consensus protocols with adaptive and time-varying gains, the unknown actions for players are distributively estimated. Then, a fully distributed Nash equilibrium seeking strategy is exploited, showcasing its remarkable properties: (1) ensuring the boundedness of control inputs;(2) avoiding any global information/parameters;and (3) allowing the graph to be directed. Based on Lyapunov stability analysis, it is theoretically proved that the proposed distributed control strategy can lead all the players' actions to the Nash equilibrium. Finally, an illustrative example is given to validate effectiveness of the proposed method.

Fuzzy Nash equilibrium of fuzzy n-person non-cooperative game

The fuzzy non-cooperative game with fuzzy payoff function is studied. Based on fuzzy set theory with game theory, the fuzzy Nash equilibrium of fuzzy non-cooperative games is proposed. Most of researchers rank fuzzy number by its center of gravity or by the real number with its maximal membership. By reducing fuzzy number into a real number, we lose much fuzzy information that should be kept during the operations between fuzzy numbers. The fuzzy quantities or alternatives are ordered directly by Yuan＇s binary fuzzy ordering relation. In doing so, the existence of fuzzy Nash equilibrium for fuzzy non-cooperative games is shown based on the utility function and the crisp Nash theorem. Finally, an illustrative example in traffic flow patterns of equilibrium is given in order to show the detailed calculation process of fuzzy Nash equilibrium.

On the finite horizon Nash equilibrium solution in the differential game approach to formation control

The solvability of the coupled Riccati differential equations appearing in the differential game approach to the formation control problem is vital to the finite horizon Nash equilibrium solution.These equations(if solvable)can be solved numerically by using the terminal value and the backward iteration.To investigate the solvability and solution of these equations the formation control problem as the differential game is replaced by a discrete-time dynamic game.The main contributions of this paper are as follows.First,the existence of Nash equilibrium controls for the discretetime formation control problem is shown.Second,a backward iteration approximate solution to the coupled Riccati differential equations in the continuous-time differential game is developed.An illustrative example is given to justify the models and solution.

Generalized Quantum Games with Nash Equilibrium

We define generalized quantum games by introducing the coherent payoff operators and propose a simple scheme to illustrate it.The scheme is implemented with a single spin qubit system and a two-entangled-qubit system.The Nash Equilibrium Theorem is proved for the models.

A Statistical Analysis of Games with No Certain Nash Equilibrium Make Many Results Doubtful

Some games may have a Nash equilibrium if the parameters (e.g. probabilities for success) take certain values but no equilibrium for other values. So there is a transition from Nash equilibrium to no Nash equilibrium in parameter space. However, in real games in business and economics, the input parameters are not given. They are typically observed in several similar occasions of the past. Therefore they have a distribution and the average is used. Even if the averages are in an area of Nash equilibrium, some values may be outside making the entire result meaningless. As the averages are sometimes just guessed, the distribution cannot be known. The main focus of this article is to show this effect in an example, and to explain the surprising result by topological explanations. We give an example of two players having three strategies each (e.g. player and keeper in penalty shooting) where we demonstrate the effect explicitly. As the transition of Nash equilibrium to no equilibrium is sharp, there may be a special form of chaos which we suggest to call topological chaos.

An Extension of One-Period Nash Equilibrium Model in Non-Life Insurance Markets

This paper deals with an extension of the one-period model in non-life insurance markets (cf. [1]) by using a transition probability matrix depending on some economic factors. We introduce a multi-period model and in each period the solvency constraints will be updated. Moreover, the model has the inactive state including some uninsured population. Similar results on the existence of premium equilibrium and sensitivity analysis for this model are presented and illustrated by numerical results.

Properties of Nash Equilibrium Retail Prices in Contract Model with a Supplier, Multiple Retailers and Price-Dependent Demand

Recently, price contract models between suppliers and retailers, with stochastic demand have been analyzed based on well-known newsvendor problems. In Bernstein and Federgruen [6], they have analyzed a contract model with single supplier and multiples retailers and price dependent demand, where retailers compete on retail prices. Each retailer decides a number of products he procures from the supplier and his retail price to maximize his own profit. This is achieved after giving the wholesale and buy-back prices, which are determined by the supplier as the supplier’s profit is maximized. Bernstein and Federgruen have proved that the retail prices become a unique Nash equilibrium solution under weak conditions on the price dependent distribution of demand. The authors, however, have not mentioned the numerical values and proprieties on these retail prices, the number of products and their individual and overall profits. In this paper, we analyze the model numerically. We first indicate some numerical problems with respect to theorem of Nash equilibrium solutions, which Bernstein and Federgruen proved, and we show their modified results. Then, we compute numerically Nash equilibrium prices, optimal wholesale and buy-back prices for the supplier’s and retailers’ profits, and supply chain optimal retailers’ prices. We also discuss properties on relation between these values and the demand distribution.

Distributed Nash Equilibrium Seeking on Compact Action Sets over Jointly Strongly Connected Switching Networks

This paper studies the distributed Nash equilibrium seeking(DNES)problem for games whose action sets are compact and whose network graph is switching satisfying the jointly strongly connected condition.To keep the actions of all players in their action sets all the time,one has to resort to the projected gradient-based method.Under the assumption that the unique Nash equilibrium is the unique equilibrium of the pseudogradient system,an algorithm is proposed that is able to exponentially find the Nash equilibrium.Further,the authors also consider the distributed Nash equilibrium seeking problem for games whose actions are governed by high-order integrator dynamics and belong to some compact sets.Two examples are used to illustrate the proposed approach.

A distributed normalized Nash equilibrium seeking algorithm for power allocation among micro-grids

In this paper, a power allocation problem based on the Cournot game and generalized Nash game is proposed. After integrating dynamic average consensus algorithm and distributed projection neural network through singular perturbation systems, a normalized Nash equilibrium seeking algorithm is presented to solve the proposed power allocation problem in a distributed way.Combine Lyapunov stability with the singular perturbation analysis, the convergence of the proposed algorithm is analyzed. A simulation on IEEE 118-bus confirms that the proposed distributed algorithm can adjust the power allocation according to different situations, while keeping the optimal solution within the feasible set.

Distributed heavy-ball algorithm of Nash equilibrium seeking for aggregative games

In this paper,we consider the Nash equilibrium(NE)seeking problem for aggregative games and design a distributed heavy-ball algorithm to solve it.This algorithm has faster convergence rate than the well-known distributed first-order algorithms for aggregative games.In order to seek the NE,each player needs to exchange information with its neighbours as well as a cen-tral aggregation.For aggregative games,the aggregative term can be either linear or nonlinear in this paper.Furthermore,we consider the generalised Nash equilibrium seeking problem for aggregative games by taking into account the linear coupled constraints among players,and modify our initial algorithm to include game constraints.

Approximating Nash Equilibrium in Day-ahead Electricity Market Bidding with Multi-agent Deep Reinforcement Learning

In this paper,a day-ahead electricity market bidding problem with multiple strategic generation company(GEN-CO)bidders is studied.The problem is formulated as a Markov game model,where GENCO bidders interact with each other to develop their optimal day-ahead bidding strategies.Considering unobservable information in the problem,a model-free and data-driven approach,known as multi-agent deep deterministic policy gradient(MADDPG),is applied for approximating the Nash equilibrium(NE)in the above Markov game.The MAD-DPG algorithm has the advantage of generalization due to the automatic feature extraction ability of the deep neural networks.The algorithm is tested on an IEEE 30-bus system with three competitive GENCO bidders in both an uncongested case and a congested case.Comparisons with a truthful bidding strategy and state-of-the-art deep reinforcement learning methods including deep Q network and deep deterministic policy gradient(DDPG)demonstrate that the applied MADDPG algorithm can find a superior bidding strategy for all the market participants with increased profit gains.In addition,the comparison with a conventional-model-based method shows that the MADDPG algorithm has higher computational efficiency,which is feasible for real-world applications.

A Monte Carlo Neural Fictitious Self-Play approach to approximate Nash Equilibrium in imperfect-information dynamic games

Solving the optimization problem to approach a Nash Equilibrium point plays an important role in imperfect information games,e.g.,StarCraft and poker.Neural Fictitious Self-Play(NFSP)is an effective algorithm that learns approximate Nash Equilibrium of imperfect-information games from purely self-play without prior domain knowledge.However,it needs to train a neural network in an off-policy manner to approximate the action values.For games with large search spaces,the training may suffer from unnecessary exploration and sometimes fails to converge.In this paper,we propose a new Neural Fictitious Self-Play algorithm that combines Monte Carlo tree search with NFSP,called MC-NFSP,to improve the performance in real-time zero-sum imperfect-information games.With experiments and empirical analysis,we demonstrate that the proposed MC-NFSP algorithm can approximate Nash Equilibrium in games with large-scale search depth while the NFSP can not.Furthermore,we develop an Asynchronous Neural Fictitious Self-Play framework(ANFSP).It uses asynchronous and parallel architecture to collect game experience and improve both the training efficiency and policy quality.The experiments with th e games with hidden state information(Texas Hold^m),and the FPS(firstperson shooter)games demonstrate effectiveness of our algorithms.

The Computation of Nash Equilibrium in Fashion Games via Semi-Tensor Product Method

Using the semi-tensor product of matrices, this paper investigates the computation of purestrategy Nash equilibrium （PNE） for fashion games, and presents several new results. First, a formal fashion game model on a social network is given. Second, the utility function of each player is converted into an Mgebraic form via the semi-tensor product of matrices, based on which the case of two-strategy fashion game is studied and two methods are obtained for the case to verify the existence of PNE. Third, the multi-strategy fashion game model is investigated and an algorithm is established to find all the PNEs for the general case. Finally, two kinds of optimization problems, that is, the so-called social welfare and normalized satisfaction degree optimization problems are investigated and two useful results are given. The study of several illustrative examples shows that the new results obtained in this paper are effective.

Comparing Competition Equilibrium with Nash Equilibrium in Electric Power Market

The Nash equilibrium and competition equilibrium have been widely studied in the electric power market up to now.In this paper,it is explained that the Nash equilibrium can be achieved by using marginal cost pricing and the com-petition equilibrium can be performed by using accounting cost pricing based on the model of the power market system.The comparison between the Nash equilibrium and competition equilibrium indicates that surplus and unfair allocation of market benefits may be obtained by the Nash equilibrium,and the competition equilibrium realizes the optimization in economics with maximum market efficiency and fairness for market benefit allocations while the optimization in mathematics is achieved by the Nash equilibrium.There is sameness between the Nash equilibrium and competition equilibrium at the point when the power network characteristics are disregarded.The case study is made on a IEEE 30-bus system,and the calculation results indicate that it is the key issue to perform the competition equilibrium by using accounting cost pricing.

Distributed best response dynamics for Nash equilibrium seeking in potential games

In this paper,we consider distributed Nash equilibrium(NE)seeking in potential games over a multi-agent network,where each agent can not observe the actions of all its rivals.Based on the best response dynamics,we design a distributed NE seeking algorithm by incorporating the non-smooth finite-time average tracking dynamics,where each agent only needs to know its own action and exchange information with its neighbours through a communication graph.We give a sufficient condition for the Lipschitz continuity of the best response mapping for potential games,and then prove the convergence of the proposed algorithm based on the Lyapunov theory.Numerical simulations are given to verify the resultandillustrate the effectiveness of the algorithm.

Nash equilibrium seeking with prescribed performance

In this work,we study a Nash equilibrium(NE)seeking problem for strongly monotone non-cooperative games with prescribed performance.Unlike general NE seeking algorithms,the proposed prescribed-performance NE seeking laws ensure that the convergence error evolves within a predefined region.Thus,the settling time,convergence rate,and maximum overshoot of the algorithm can be guaranteed.First,we develop a second-order Newton-like algorithm that can guarantee prescribed performance and asymptotically converge to the NE of the game.Then,we develop a first-order gradient-based algorithm.To remove some restrictions on this first-order algorithm,we propose two discontinuous dynamical system-based algorithms using tools from non-smooth analysis and adaptive control.We study the special case in optimization problems.Then,we investigate the robustness of the algorithms.It can be proven that the proposed algorithms can guarantee asymptotic convergence to the Nash equilibrium with prescribed performance in the presence of bounded disturbances.Furthermore,we consider a second-order dynamical system solution.The simulation results verify the effectiveness and efficiency of the algorithms,in terms of their convergence rate and disturbance rejection ability.