Optimal Secondary Control of Islanded AC Microgrids with Communication Time-delay Based on Multi-agent Deep Reinforcement Learning

In this paper,an optimal secondary control strategy is proposed for islanded AC microgrids considering communi-cation time-delays.The proposed method is designed based on the data-driven principle,which consists of an offine training phase and online application phase.For offline training,each control agent is formulated by a deep neural network(DNN)and trained based on a multi-agent deep reinforcement learning(MA-DRL)framework.A deep deterministic policy gradient(DDPG)algorithm is improved and applied to search for an optimal policy of the secondary control,where a global cost function is developed to evaluate the overall control performance.In addition,the communication time-delay is introduced in the system to enrich training scenarios,which aims to solve the time-delay problem in the secondary control.For the online stage,each controller is deployed in a distributed way which only requires local and neighboring information for each DG.Based on this,the well-trained controllers can provide optimal solutions under load variations,and communication time-delays for online applications.Several case studies are conducted to validate the feasibility and stability of the proposed secondary control.Index Terms-Communication time-delay,global cost function,islanded AC microgrid,multi-agent deep reinforcement learning(MA-DRL),secondary control.