基于强化学习算法的微电网优化策略

Optimization Strategy for Microgrid Based on Reinforcement Learning Algorithm

分布式能源具有小规模波动和间歇性的特点,导致微电网运行策略难以制定。微电网有效集成多种分布式能源和外部电网,多能源微电网管理正成为一项非常复杂的任务。针对该问题,提出一种在负荷需求、可再生能源和储能设备等综合因素影响下的微电网实时优化运行策略。该策略首先基于强化学习框架,将微电网运行问题建模为马尔可夫决策过程,然后以最小化微电网电压波动和运行损耗为目的构建微电网策略优化模型。为有效利用微电网的互联结构,在近端策略优化算法的基础上,设计一种图注意力近端策略优化算法(graph attention proximal policy optimization,GT-PPO),该算法使用注意力机制和图神经网络学习微电网节点的相关性,以学习各类环境下不同时段多能源微电网最优调度策略。最后,采用改进的IEEE 33节点、IEEE 118节点两种规格的微电网进行仿真实验。实验结果表明,该优化策略可以实现微电网的实时优化,且结果优于传统的近端策略优化(proximal policy optimization,PPO)算法和双深度Q网络(double deep Q network,DDQN)算法。

Distributed energy has the characteristics of small-scale fluctuations and intermittency,making it difficult to formulate operational strategies for microgrids.As an effective way to integrate multiple distributed energy sources and external grids,multi-energy microgrid management is becoming a very complex task.A microgrid real-time optimal operation strategy was proposed under the influence of comprehensive factors such as load demand,renewable energy sources and energy storage devices.Firstly,based on the reinforcement learning framework,the microgrid operation problem was modeled as a Markov decision process,and then a microgrid strategy optimization model was constructed with the aim of minimizing voltage fluctuations and operational losses in the microgrid.In order to effectively utilize the interconnection structure of the distribution network,a graph attention proximal policy optimization(GT-PPO)algorithm was designed on the basis of the proximal policy optimization algorithm.This algorithm uses an attention mechanism and a graph neural network to learn the correlation of distribution network nodes to formulate the optimal scheduling strategy for multi-energy distribution networks at different times under various environments.Simulation experiments were conducted using two specifications of the improved IEEE 33 node and IEEE 118 node distribution networks.The experimental results show that the optimization strategy can achieve real-time optimization of microgrids,and the results are better than the traditional proximal policy optimization(PPO)algorithm and double deep Q network(DDQN)algorithm.