Self organizing optimization and phase transition in reinforcement learning minority game system

Whether the complex game system composed of a large number of artificial intelligence(AI)agents empowered with reinforcement learning can produce extremely favorable collective behaviors just through the way of agent self-exploration is a matter of practical importance.In this paper,we address this question by combining the typical theoretical model of resource allocation system,the minority game model,with reinforcement learning.Each individual participating in the game is set to have a certain degree of intelligence based on reinforcement learning algorithm.In particular,we demonstrate that as AI agents gradually becomes familiar with the unknown environment and tries to provide optimal actions to maximize payoff,the whole system continues to approach the optimal state under certain parameter combinations,herding is effectively suppressed by an oscillating collective behavior which is a self-organizing pattern without any external interference.An interesting phenomenon is that a first-order phase transition is revealed based on some numerical results in our multi-agents system with reinforcement learning.In order to further understand the dynamic behavior of agent learning,we define and analyze the conversion path of belief mode,and find that the self-organizing condensation of belief modes appeared for the given trial and error rates in the AI system.Finally,we provide a detection method for period-two oscillation collective pattern emergence based on the Kullback–Leibler divergence and give the parameter position where the period-two appears.

Load-Balance Mechanism for WSNs Based on Minority Game with Dormancy Strategy

Load-balance is an important issue in wireless sensor networks (WSNs), especially in WSNs with hierarchical structure. Energy consumed unevenly will bring the production of hot spots. Hot spot will cause WSNs to divide some unconnected sub-networks and shorten the lifetime of WSNs. To tackle this problem, a load-balance mechanism is proposed based on minority game (MG) with dormancy strategy. This mechanism can cause the rich behaviors of cooperation , prolong lifetime of WSNs, and keep energy consumed evenly. This dormancy mechanism can save energy of nodes by keeping in sleep temperately . Simulation results show that the proposed strategy can efficiently enhance the lifetime of cluster and the lifetime of whole WSNs.

Chaotic-periodic transition in a two-sided minority game

Phase transitions are being used increasingly to probe the collective behaviors of social human systems. In this study, we propose a different way of investigating such transitions in a human system by establishing a two-sided minority game model. A new type of agents who can actively transfer resources are added to our artificial bipartite resource-allocation market. The degree of deviation from equilibria is characterized by the entropy-like quantity of market complexity. Under different threshold values, Qth, two phases are found by calculating the exponents of the associated power spectra. For large values of Qth, the general motion of strategies for the agents is relatively periodic whereas for low values of Qth, the motion becomes chaotic. The transition occurs abruptly at a critical value of Qty. Our simulation results were also tested based on human experiments. The results of this study suggest that a chaotic-periodic transition related to the quantity of market information should exist in most bipartite markets, thereby allowing better control of such a transi- tion and providing a better understanding of the endogenous emergence of business cycles from the perspective of quantum mechanics.

Ethnic Minority Games Come To Guizhou

Ateam of barefoot young peoplet ake to the water on 7.5-meter-long bamboo poles and maintain their stability with ease as they glide across the Hongfeng Lake in southwest China’s Guizhou Province.But unlike local merchants who long agou sed large pieces of bamboo as makeshift floats to get their wares to market,thesey oung people are professional,full-timea thletes in the ethnic minority sport of single bamboo drifting.

Memory-Based Boolean Game and Self-Organized Phenomena on Networks

We study a memory-based Boolean game （MBBG） taking place on a regular ring, wherein each agent acts according to its local optimal states of the last M time steps recorded in memory, and the agents in the minority are rewarded. One free parameter p between 0 and 1 is introduced to denote the strength of the agent willing to make a decision according to its memory. It is found that giving proper willing strength p, the MBBG system can spontaneously evolve to a state of performance better than the random game; while for larger p, the herd behaviour emerges to reduce the system profit. By analysing the dependence of dynamics of the system on the memory capacity M, we find that a higher memory capacity favours the emergence of the better performance state, and effectively restrains the herd behaviour, thus increases the system profit. Considering the high cost of long-time memory, the enhancement of memory capacity for restraining the herd behaviour is also discussed, and M =5 is suggested to be a good choice.

System dynamics of behaviour-evolutionary mix-game models

In real financial markets there are two kinds of traders： one is fundamentalist, and the other is a trend-follower. The mix-game model is proposed to mimic such phenomena. In a mix-game model there are two groups of agents： Group 1 plays the majority game and Group 2 plays the minority game. In this paper, we investigate such a case that some traders in real financial markets could change their investment behaviours by assigning the evolutionary abilities to agents： if the winning rates of agents are smaller than a threshold, they will join the other group; and agents will repeat such an evolution at certain time intervals. Through the simulations, we obtain the following findings： （i） the volatilities of systems increase with the increase of the number of agents in Group 1 and the times of behavioural changes of all agents; （ii） the performances of agents in both groups and the stabilities of systems become better if all agents take more time to observe their new investment behaviours; （iii） there are two-phase zones of market and non-market and two-phase zones of evolution and non-evolution; （iv） parameter configurations located within the cross areas between the zones of markets and the zones of evolution are suited for simulating the financial markets.

Dynamic behaviours of mix-game model and its application

In this paper a minority game （MG） is modified by adding into it some agents who play a majority game. Such a game is referred to as a mix-game. The highlight of this model is that the two groups of agents in the mix-game have different bounded abilities to deal with historical information and to count their own performance. Through simulations, it is found that the local volatilities change a lot by adding some agents who play the majority game into the MG, and the change of local volatilities greatly depends on different combinations of historical memories of the two groups. Furthermore, the analyses of the underlying mechanisms for this finding are made. The applications of mix-game mode are also given as an example.

A Minor Player in the Global Game

The U.S.Department of Treasury’s decision to postpone its mid-April report labeling China a "currency manipulator" was all but expected.Beijing Review reporters Chen Wen and Liu

Recent progress in econophysics： Chaos, leverage, and business cycles as revealed by agent-based modeling and human experiments

Agent-based modeling and controlled human experiments serve as two fundamental research methods in the field of econophysics. Agent-based modeling has been in development for over 20 years, but how to design virtual agents with high levels of human-like ＂intelligence＂ remains a challenge. On the other hand, experimental econophysics is an emerging field; however, there is a lack of experience and paradigms related to the field. Here, we review some of the most recent research results obtained through the use of these two methods concerning financial problems such as chaos, leverage, and business cycles. We also review the principles behind assessments of agents＇ intelligence levels, and some relevant designs for human experiments. The main theme of this review is to show that by combining theory, agent-based modeling, and controlled human experiments, one can garner more reliable and credible results on account of a better verification of theory; accordingly, this way, a wider range of economic and financial problems and phenomena can be studied.