Distributed Secondary Control Strategy Based on Q-learning and Pinning Control for Droop-controlled Microgrids

A distributed secondary control(DSC) strategy that combines Q-learning and pinning control is originally proposed to achieve a fully optimal DSC for droop-controlled microgrids(MGs). It takes advantages of cross-fusion of the two algorithms to realize the high efficiency and self-adaptive control in MGs. It has the following advantages. Firstly, it adopts the advantages of reinforcement learning in autonomous learning control and intelligent decision-making, driving the action value of pinning control for feedback adaptive correction. Secondly, only a small part of points selected as pinned points needs to be controlled and pre-learned, hence the actual control problem is transformed into a synchronous tracking problem and the installation number of controllers is further reduced.Thirdly, the pinning matrix can be modified to adapt to plugand-play operation under the distributed control architecture.Finally, the effectiveness and versatility of the proposed strategy are demonstrated with a typical droop-controlled MG model.