含储能系统的配电网电压调节深度强化学习算法

Deep Reinforcement Learning Algorithm of Voltage Regulation in Distribution Network with Energy Storage System

通过在配电网末端接入用于系统调压等辅助服务的储能系统,能有效应对可再生能源的高度间歇性以及负荷需求波动导致的系统电压运行水平问题。文章将电池储能的运行建模为马尔可夫决策过程,考虑其后续调控能力,提出了一种含储能系统的配电网电压调节深度强化学习(deep reinforcement learning,DRL)算法,通过内嵌一个Q深度神经网络来逼近储能最佳动作价值,以解决状态空间过大的问题。储能荷电状态(state of charge,SOC)、可再生能源预测出力以及负荷水平组成状态特征向量作为Q网络的输入,输出提高电压运行水平的最优离散化充放电动作,并通过回放策略来训练。相比传统方法,所提方案基于学习而无需显式的不确定性模型,且计算效率较高。最后在TensorFlow架构下利用MATPOWER对IEEE 33节点配网系统进行了分析,证明了所提出方法的有效性。

Energy storage system connected with the end of distribution network,which is used for auxiliary services such as system voltage regulation,can effectively deal w ith the problem of voltage fluctuation caused by intermittent distributed renewable energies and the fluctuation of load demand.In this paper,the operation of energy-storage battery is modeled as a Markov Decision Making process.Considering its subsequent regulation ability,an intelligent control strategy based on deep reinforcement learning(DRL)is proposed.By embedding a Q deep neural netw ork to approach the optimal action value,the problem of too large state space can be solved.The state vector composed of the state of charge(SOC),the predicted output of renewable energy and the load level is used as the input of Q network,and the optimal discrete charge and discharge action is output,which is trained by replay strategy.Compared with the traditional method,the proposed method is based on learning without explicit uncertainty model,and the calculation efficiency is high.Finally,the IEEE 33-node distribution network system is analyzed by using M ATPOWER in TtensorFlow,and the effectiveness of the proposed method is proved.