现货市场环境下基于深度强化学习的光储联合电站储能系统最优运行方法

Deep Reinforcement Learning Based Optimal Energy Storage System Operation of Photovoltaic Power Stations With Energy Storage in Power Market

光伏-储能联合电站不仅能够有效减少光伏实时出力偏差,也是一种能够提供电能量和调频辅助服务的潜在市场主体。为实现上述3类目标,与光伏出力协同的储能电量调度策略至关重要,然而目前大部分光储联合电站的储能电量调度策略无法同时协调降低光伏实时出力偏差和参与电能量与调频辅助服务市场3种决策;另一方面,电力现货市场价格与调频信号不确定性及储能电量调度策略将光储联合电站储能运行优化问题转化为一个随机动态非凸优化问题,现有相关研究大部分利用随机场景法或智能算法处理非凸优化,所获得的储能运行方案存在一定的局限性,且难以根据实时数据动态制定运行方案。因此,提出一种现货市场环境下基于DQN(deep Q-network)的光储联合电站储能系统优化运行方法,该方法克服了非凸优化难题,结合所提储能电量闭环调度策略能够实现光储联合电站在考虑偏差考核成本、电能量收益、调频辅助服务收益下的储能系统小时级动态优化运行,进而最大化光-储联合电站的经济收益。测试算例通过实际市场数据验证了所提方法的可行性和有效性。

Photovoltaic power stations with energy storage(PV-ES) can not only effectively reduce the real-time variation of the PV’s output,but also be a potential market entity that provides with electric energy and regulation ancillary services.To achieve the above three goals,the ES scheduling strategy should be coordinated with the PV’s outputs.However,at present,most ES scheduling strategies of the PV-ES do not simultaneously coordinate with the reduction of the PV’s real-time output variation and participate in the energy and regulation ancillary service market.On the other hand,the uncertainties,such as the power market prices,or the frequency regulation signals,combining with the ES scheduling strategy,are turning the ES operation problem of PV-ES into a stochastic dynamic non-convex optimal problem.Most of the existing relevant studies use the stochastic scene method or the intelligent algorithm to deal with the non-convex optimization,resulting in the limitation of ES operation and the difficulties of dynamically formulating the operation scheme of ES according to the real-time data.To close the gap,a Deep Q-network based optimal ES operation strategy of PV-ES in power market is proposed.This proposed closed-loop capacity scheduling strategy of ES can not only cope with the non-convex problem,but also autonomously schedule the ES hourly capacity and obtain the maximum of PV-ES profits considering the cost of deviation.