Multi-User MmWave Beam Tracking via Multi-Agent Deep Q-Learning

Beamforming is significant for millimeter wave multi-user massive multi-input multi-output systems.In the meanwhile,the overhead cost of channel state information and beam training is considerable,especially in dynamic environments.To reduce the overhead cost,we propose a multi-user beam tracking algorithm using a distributed deep Q-learning method.With online learning of users’moving trajectories,the proposed algorithm learns to scan a beam subspace to maximize the average effective sum rate.Considering practical implementation,we model the continuous beam tracking problem as a non-Markov decision process and thus develop a simplified training scheme of deep Q-learning to reduce the training complexity.Furthermore,we propose a scalable state-action-reward design for scenarios with different users and antenna numbers.Simulation results verify the effectiveness of the designed method.

Privacy Preserving Demand Side Management Method via Multi-Agent Reinforcement Learning

The smart grid utilizes the demand side management technology to motivate energy users towards cutting demand during peak power consumption periods, which greatly improves the operation efficiency of the power grid. However, as the number of energy users participating in the smart grid continues to increase, the demand side management strategy of individual agent is greatly affected by the dynamic strategies of other agents. In addition, the existing demand side management methods, which need to obtain users’ power consumption information,seriously threaten the users’ privacy. To address the dynamic issue in the multi-microgrid demand side management model, a novel multi-agent reinforcement learning method based on centralized training and decentralized execution paradigm is presented to mitigate the damage of training performance caused by the instability of training experience. In order to protect users’ privacy, we design a neural network with fixed parameters as the encryptor to transform the users’ energy consumption information from low-dimensional to high-dimensional and theoretically prove that the proposed encryptor-based privacy preserving method will not affect the convergence property of the reinforcement learning algorithm. We verify the effectiveness of the proposed demand side management scheme with the real-world energy consumption data of Xi’an, Shaanxi, China. Simulation results show that the proposed method can effectively improve users’ satisfaction while reducing the bill payment compared with traditional reinforcement learning(RL) methods(i.e., deep Q learning(DQN), deep deterministic policy gradient(DDPG),QMIX and multi-agent deep deterministic policy gradient(MADDPG)). The results also demonstrate that the proposed privacy protection scheme can effectively protect users’ privacy while ensuring the performance of the algorithm.

Working-memory training improves developmental dyslexia in Chinese children

Although plasticity in the neural system underlies working memory, and working memory can be improved by training, there is thus far no evidence that children with developmental dyslexia can benefit from working-memory training. In the present study, thirty dyslexic children aged 8-11 years were recruited from an elementary school in Wuhan, China. They received working-memory training including training in visuospatial memory, verbal memory, and central executive tasks. The difficulty of the tasks was adjusted based on the performance of each subject, and the training sessions lasted 40 minutes per day, for 5 weeks. The results showed that working-memory training significantly enhanced performance on the nontrained working memory tasks such as the visuospatial, the verbal domains, and central executive tasks in children with developmental dyslexia. More importantly, the visual rhyming task and reading fluency task were also significantly improved by training. Progress on working memory measures was related to changes in reading skills. These experimental findings indicate that working memory is a pivotal factor in reading development among children with developmental dyslexia, and interventions to improve working memory may help dyslexic children to become more proficient in reading.