Privacy Preserving Demand Side Management Method via Multi-Agent Reinforcement Learning

The smart grid utilizes the demand side management technology to motivate energy users towards cutting demand during peak power consumption periods, which greatly improves the operation efficiency of the power grid. However, as the number of energy users participating in the smart grid continues to increase, the demand side management strategy of individual agent is greatly affected by the dynamic strategies of other agents. In addition, the existing demand side management methods, which need to obtain users’ power consumption information,seriously threaten the users’ privacy. To address the dynamic issue in the multi-microgrid demand side management model, a novel multi-agent reinforcement learning method based on centralized training and decentralized execution paradigm is presented to mitigate the damage of training performance caused by the instability of training experience. In order to protect users’ privacy, we design a neural network with fixed parameters as the encryptor to transform the users’ energy consumption information from low-dimensional to high-dimensional and theoretically prove that the proposed encryptor-based privacy preserving method will not affect the convergence property of the reinforcement learning algorithm. We verify the effectiveness of the proposed demand side management scheme with the real-world energy consumption data of Xi’an, Shaanxi, China. Simulation results show that the proposed method can effectively improve users’ satisfaction while reducing the bill payment compared with traditional reinforcement learning(RL) methods(i.e., deep Q learning(DQN), deep deterministic policy gradient(DDPG),QMIX and multi-agent deep deterministic policy gradient(MADDPG)). The results also demonstrate that the proposed privacy protection scheme can effectively protect users’ privacy while ensuring the performance of the algorithm.

Hierarchical under frequency load shedding scheme for inter-connected power systems

Severe disturbances in a power network can cause the system frequency to exceed the safe operating range.As the last defensive line for system emergency control,under frequency load shedding(UFLS)is an important method for preventing a wide range of frequency excursions.This paper proposes a hierarchical UFLS scheme of“centralized real-time decision-making and decentralized real-time control”for inter-connected systems.The centralized decision-layer of the scheme takes into account the importance of the load based on the equivalent transformation of kinetic energy(KE)and potential energy(PE)in the transient energy function(TEF),while the load PE is used to determine the load shedding amount(LSA)allocation in different loads after faults in real-time.At the same time,the influence of inertia loss is considered in the calculation of unbalanced power,and the decentralized control center is used to implement the one-stage UFLS process to compensate for the unbalanced power.Simulations are carried out on the modified New England 10-generator 39-bus system and 197-bus system in China to verify the performance of the proposed scheme.The results show that,compared with other LSA allocation indicators,the proposed alloca-tion indicators can achieve better fnadir and td.At the same time,compared with other multi-stage UFLS schemes,the proposed scheme can obtain the maximum fnadir with a smaller LSA in scenarios with high renewable energy sources(RES)penetration.