In this paper,the optimal consensus control problem is investigated for heterogeneous linear multi-agent systems(MASs)with spanning tree condition based on game theory and rein-forcement learning.First,the graphical m...In this paper,the optimal consensus control problem is investigated for heterogeneous linear multi-agent systems(MASs)with spanning tree condition based on game theory and rein-forcement learning.First,the graphical minimax game algebraic Riccati equation(ARE)is derived by converting the consensus problem into a zero-sum game problem between each agent and its neighbors.The asymptotic stability and minimax validation of the closed-loop systems are proved theoretically.Then,a data-driven off-policy reinforcement learning algorithm is proposed to online learn the optimal control policy without the information of the system dynamics.A certain rank condition is established to guarantee the convergence of the proposed algorithm to the unique solution of the ARE.Finally,the e®ectiveness of the proposed method is demonstrated through a numerical simulation.展开更多
Minimax algorithm and machine learning technologies have been studied for decades to reach an ideal optimization in game areas such as chess and backgammon. In these fields, several generations try to optimize the cod...Minimax algorithm and machine learning technologies have been studied for decades to reach an ideal optimization in game areas such as chess and backgammon. In these fields, several generations try to optimize the code for pruning and effectiveness of evaluation function. Thus, there are well-armed algorithms to deal with various sophisticated situations in gaming occasion. However, as a traditional zero-sum game, Connect-4 receives less attention compared with the other members of its zero-sum family using traditional minimax algorithm. In recent years, new generation of heuristics is created to address this problem based on research conclusions, expertise and gaming experiences. However, this paper mainly introduced a self-developed heuristics supported by well-demonstrated result from researches and our own experiences which fighting against the available version of Connect-4 system online. While most previous works focused on winning algorithms and knowledge based approaches, we complement these works with analysis of heuristics. We have conducted three experiments on the relationship among functionality, depth of searching and number of features and doing contrastive test with sample online. Different from the sample based on summarized experience and generalized features, our heuristics have a basic concentration on detailed connection between pieces on board. By analysing the winning percentages when our version fights against the online sample with different searching depths, we find that our heuristics with minimax algorithm is perfect on the early stages of the zero-sum game playing. Because some nodes in the game tree have no influence on the final decision of minimax algorithm, we use alpha-beta pruning to decrease the number of meaningless node which greatly increases the minimax efficiency. During the contrastive experiment with the online sample, this paper also verifies basic characters of the minimax algorithm including depths and quantity of features. According to the experiment, these two characters can both effect the decision for each step and none of them can be absolutely in charge. Besides, we also explore some potential future issues in Connect-4 game optimization such as precise adjustment on heuristic values and inefficiency pruning on the search tree.展开更多
In economics, buyers and sellers are usually the main sides in a market. Game theory can perfectly model decisions behind each “player” and calculate an outcome that benefits both sides. However, the use of game the...In economics, buyers and sellers are usually the main sides in a market. Game theory can perfectly model decisions behind each “player” and calculate an outcome that benefits both sides. However, the use of game theory is not lim-ited to economics. In this paper, I will introduce the mathematical model of general sum game, solutions and theorems surrounding game theory, and its real life applications in many different scenarios.展开更多
The energy dissipation rate is an important concept in the theory of turbulence. Doering-Constantin's variational principle characterizes the upper bounds (maxi- mum) of the time-averaged rate of viscous energy dis...The energy dissipation rate is an important concept in the theory of turbulence. Doering-Constantin's variational principle characterizes the upper bounds (maxi- mum) of the time-averaged rate of viscous energy dissipation. In the present study, an optimization theoretical point of view was adopted to recast Doering-Constantin's formu- lation into a minimax principle for the energy dissipation of an incompressible shear flow. Then, the Kakutani minimax theorem in the game theory is applied to obtain a set of conditions, under which the maximization and the minimization in the minimax principle are commutative. The results explain the spectral constraint of Doering-Constantin, and confirm the equivalence between Doering-Constantin's variational principle and Howard- Busse's statistical turbulence theory.展开更多
Minimax state estimation is discussed for uncerttain systems with L2 bounded constraint. A dtaity relation-equality is introduced to estimate terminal state variabes x(T) by measurable outputs . hawing a game theory, ...Minimax state estimation is discussed for uncerttain systems with L2 bounded constraint. A dtaity relation-equality is introduced to estimate terminal state variabes x(T) by measurable outputs . hawing a game theory, opti-mal estimation leads to a simple solution. LQL control scheme, is further discussed to make it rational in the actual application.展开更多
In this paper,a communication model in cognitive radios is developed and uses machine learning to learn the dynamics of jamming attacks in cognitive radios.It is designed further to make their transmission decision th...In this paper,a communication model in cognitive radios is developed and uses machine learning to learn the dynamics of jamming attacks in cognitive radios.It is designed further to make their transmission decision that automati-cally adapts to the transmission dynamics to mitigate the launched jamming attacks.The generative adversarial learning neural network(GALNN)or genera-tive dynamic neural network(GDNN)automatically learns with the synthesized training data(training)with a generator and discriminator type neural networks that encompass minimax game theory.The elimination of the jamming attack is carried out with the assistance of the defense strategies and with an increased detection rate in the generative adversarial network(GAN).The GDNN with game theory is designed to validate the channel condition with the cross entropy loss function and back-propagation algorithm,which improves the communica-tion reliability in the network.The simulation is conducted in NS2.34 tool against several performance metrics to reduce the misdetection rate and false alarm rates.The results show that the GDNN obtains an increased rate of successful transmis-sion by taking optimal actions to act as a defense mechanism to mislead the jam-mer,where the jammer makes high misclassification errors on transmission dynamics.展开更多
A minimax optimal control strategy for quasi-Hamiltonian systems with bounded parametric and/or external disturbances is proposed based on the stochastic averaging method and stochastic differential game. To conduct t...A minimax optimal control strategy for quasi-Hamiltonian systems with bounded parametric and/or external disturbances is proposed based on the stochastic averaging method and stochastic differential game. To conduct the system energy control,the partially averaged It stochastic differential equations for the energy processes are first derived by using the stochastic averaging method for quasi-Hamiltonian systems. Combining the above equations with an appropriate performance index,the proposed strategy is searching for an optimal worst-case controller by solving a stochastic differential game problem. The worst-case disturbances and the optimal controls are obtained by solving a Hamilton-Jacobi-Isaacs(HJI) equation. Numerical results for a controlled and stochastically excited Duffing oscillator with uncertain disturbances exhibit the efficacy of the proposed control strategy.展开更多
基金supported by the National Natural Science Foundation of China under Grant Nos.61803032 and 61873031。
文摘In this paper,the optimal consensus control problem is investigated for heterogeneous linear multi-agent systems(MASs)with spanning tree condition based on game theory and rein-forcement learning.First,the graphical minimax game algebraic Riccati equation(ARE)is derived by converting the consensus problem into a zero-sum game problem between each agent and its neighbors.The asymptotic stability and minimax validation of the closed-loop systems are proved theoretically.Then,a data-driven off-policy reinforcement learning algorithm is proposed to online learn the optimal control policy without the information of the system dynamics.A certain rank condition is established to guarantee the convergence of the proposed algorithm to the unique solution of the ARE.Finally,the e®ectiveness of the proposed method is demonstrated through a numerical simulation.
文摘Minimax algorithm and machine learning technologies have been studied for decades to reach an ideal optimization in game areas such as chess and backgammon. In these fields, several generations try to optimize the code for pruning and effectiveness of evaluation function. Thus, there are well-armed algorithms to deal with various sophisticated situations in gaming occasion. However, as a traditional zero-sum game, Connect-4 receives less attention compared with the other members of its zero-sum family using traditional minimax algorithm. In recent years, new generation of heuristics is created to address this problem based on research conclusions, expertise and gaming experiences. However, this paper mainly introduced a self-developed heuristics supported by well-demonstrated result from researches and our own experiences which fighting against the available version of Connect-4 system online. While most previous works focused on winning algorithms and knowledge based approaches, we complement these works with analysis of heuristics. We have conducted three experiments on the relationship among functionality, depth of searching and number of features and doing contrastive test with sample online. Different from the sample based on summarized experience and generalized features, our heuristics have a basic concentration on detailed connection between pieces on board. By analysing the winning percentages when our version fights against the online sample with different searching depths, we find that our heuristics with minimax algorithm is perfect on the early stages of the zero-sum game playing. Because some nodes in the game tree have no influence on the final decision of minimax algorithm, we use alpha-beta pruning to decrease the number of meaningless node which greatly increases the minimax efficiency. During the contrastive experiment with the online sample, this paper also verifies basic characters of the minimax algorithm including depths and quantity of features. According to the experiment, these two characters can both effect the decision for each step and none of them can be absolutely in charge. Besides, we also explore some potential future issues in Connect-4 game optimization such as precise adjustment on heuristic values and inefficiency pruning on the search tree.
文摘In economics, buyers and sellers are usually the main sides in a market. Game theory can perfectly model decisions behind each “player” and calculate an outcome that benefits both sides. However, the use of game theory is not lim-ited to economics. In this paper, I will introduce the mathematical model of general sum game, solutions and theorems surrounding game theory, and its real life applications in many different scenarios.
基金supported by the National Natural Science Foundation of China (No.10772103)the Shanghai Leading Academic Discipline Project (No.Y0103)
文摘The energy dissipation rate is an important concept in the theory of turbulence. Doering-Constantin's variational principle characterizes the upper bounds (maxi- mum) of the time-averaged rate of viscous energy dissipation. In the present study, an optimization theoretical point of view was adopted to recast Doering-Constantin's formu- lation into a minimax principle for the energy dissipation of an incompressible shear flow. Then, the Kakutani minimax theorem in the game theory is applied to obtain a set of conditions, under which the maximization and the minimization in the minimax principle are commutative. The results explain the spectral constraint of Doering-Constantin, and confirm the equivalence between Doering-Constantin's variational principle and Howard- Busse's statistical turbulence theory.
文摘Minimax state estimation is discussed for uncerttain systems with L2 bounded constraint. A dtaity relation-equality is introduced to estimate terminal state variabes x(T) by measurable outputs . hawing a game theory, opti-mal estimation leads to a simple solution. LQL control scheme, is further discussed to make it rational in the actual application.
文摘In this paper,a communication model in cognitive radios is developed and uses machine learning to learn the dynamics of jamming attacks in cognitive radios.It is designed further to make their transmission decision that automati-cally adapts to the transmission dynamics to mitigate the launched jamming attacks.The generative adversarial learning neural network(GALNN)or genera-tive dynamic neural network(GDNN)automatically learns with the synthesized training data(training)with a generator and discriminator type neural networks that encompass minimax game theory.The elimination of the jamming attack is carried out with the assistance of the defense strategies and with an increased detection rate in the generative adversarial network(GAN).The GDNN with game theory is designed to validate the channel condition with the cross entropy loss function and back-propagation algorithm,which improves the communica-tion reliability in the network.The simulation is conducted in NS2.34 tool against several performance metrics to reduce the misdetection rate and false alarm rates.The results show that the GDNN obtains an increased rate of successful transmis-sion by taking optimal actions to act as a defense mechanism to mislead the jam-mer,where the jammer makes high misclassification errors on transmission dynamics.
基金the National Natural Science Foundation of China (No. 10772159)the Specialized Research Fund for DoctorProgram of Higher Education of China (No. 20060335125) theNatural Science Foundation of Zhejiang Province (No. Y607087),China
文摘A minimax optimal control strategy for quasi-Hamiltonian systems with bounded parametric and/or external disturbances is proposed based on the stochastic averaging method and stochastic differential game. To conduct the system energy control,the partially averaged It stochastic differential equations for the energy processes are first derived by using the stochastic averaging method for quasi-Hamiltonian systems. Combining the above equations with an appropriate performance index,the proposed strategy is searching for an optimal worst-case controller by solving a stochastic differential game problem. The worst-case disturbances and the optimal controls are obtained by solving a Hamilton-Jacobi-Isaacs(HJI) equation. Numerical results for a controlled and stochastically excited Duffing oscillator with uncertain disturbances exhibit the efficacy of the proposed control strategy.