基于深度强化学习的微网储能系统控制策略研究

Control Strategy of Microgrid Energy Storage System Based on Deep Reinforcement Learning

微网作为新兴的能源管理形式,近年来发展迅速,为保障微网系统能安全、稳定、经济的运行,为其提供合理的能量调度策略是关键。微网根据运行模式的不同,可分为并网微网和孤岛微网两大类。该文以并网微网为研究对象,应用Simulink仿真技术,按照恒功率控制(PQ控制)原理,搭建了一个包含外部电源、光伏发电、储能,以及负荷的微网系统。然后以此仿真系统为基础,结合深度强化学习中的双重深度Q学习算法,以最小化微网24h从外电网取电费用为目标,在满足微网系统电压偏差、功率平衡以及储能的荷电状态等约束下,以储能的实时充放电功率为控制变量,训练得到储能控制的优化策略。并通过实验验证与传统方法进行对比,分别从定性角度分析了储能充放电策略的合理性,和从定量角度展示了该文所提方法在优化购电费用上的有效性。

As a new form of energy management,a microgrid has developed rapidly in recent years.In order to ensure the safe,stable and economical operation of a microgrid system,it is important to provide it with reasonable energy scheduling strategy.According to their operating modes,micro-grids can be divided into two categories:grid-connected microgrids and island microgrids.This paper takes the grid-connected microgrids as the object,applies the Simulink simulation technology to build a microgrid system including an external power supply,photovoltaic power generation,energy storage,and load according to the principle of constant power control(PQ control).Based on this simulation system,this microgrid system is combined with the double deep Q network algorithmfirstly.Then it is trained to get an optimization strategy of energy storage control problem,the goal of which is to minimize the 24-hour electricity cost while meeting the voltage deviation of the microgrid,power balance and the constraint of state of charge of the energy storage.Through experimental verification,the rationality of the energy storage strategy is analyzed from a qualitative perspective,and the effectiveness of the method proposed in this paper is demonstrated from a quantitative perspective.