多微网系统端网协同分布式实时智能优化

Collaborative distributed real-time intelligent optimization of multi-microgrid system

多微网互联对于提升微网稳定性和促进可再生能源消纳具有重要作用,对此,考虑多微网系统网内多智能终端协同和网间功率互济需求,提出一种基于多智能体强化学习的端-网实时协同优化算法.该算法可自适应源荷不确定性和系统拓扑变化,实时产生网间能量互济和网内智能终端协同优化策略.首先建立端-网协同优化系统结构和优化模型;其次建立基于多智能体马尔科夫决策过程的强化学习模型,进而提出基于多智能体近端策略优化算法(MAPPO)的分布式协同优化算法;然后考虑调节过程中功率平衡约束,设计一种新的功率平衡反馈信号,能够有效避免功率不平衡现象的出现;最后针对3个典型场景进行仿真,结果表明各微电网无需全局信息便可得到准确的近似全局最优解.将所提出方法分别与状态完全观测和状态部分观测的分布式强化学习算法进行对比,结果表明所提出的方法既能获得良好的协同优化效果,又能满足实时优化对于算法效率的需求.

Interconnection between multiple microgrids plays an important role on enhancing stability of microgrids and renewable energy utilization.Considering the needs of collaborative operation of intelligent terminals and power mutual assistance between microgrids,a real-time collaborative terminal-microgrid optimization algorithm based on multi-agent reinforcement learning is proposed.The proposed algorithm can flexibly adapt to the system uncertainties and topology changes,and optimizes energy interaction between microgrids and collaboration among intelligent terminals.Firstly,the structure of the collaborative terminal-microgird optimization system is established,and the corresponding optimization model is formulated.Then,a multi-agent Markov decision process based reinforcement learning model is proposed.Furthermore,a distributed collaborative optimization algorithm based on the multi-agent proximal policy optimization(MAPPO)algorithm is proposed.Considering the power balance constraint,a novel power balance feedback is designed to effectively avoid the occurrence of power imbalance.Simulations are conducted under three typical scenarios,and the results show that approximate global optimal solutions can be obtained without global state observation.What is more,the proposed algorithm is compared with the distributed reinforcement learning algorithms with fully state observation and partial state observation,respectively.Results show that the proposed method can achieve good collaborative optimization results and meet the requirements on compute efficiency of real-time optimization.