基于深度强化学习的自适应不确定性经济调度

Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning

当风电、光伏等间歇性电源大规模接入电力系统时,为应对其出力的不确定性,电力系统经济调度模型需建立在对不确定性建模的基础上,建模精确度将直接影响调度结果的精确度。但当系统同时包含风电、光伏和负荷复杂的不确定性时,对系统整体不确定性进行精确建模显得尤为困难。针对这一问题,引入深度强化学习中深度确定性策略梯度算法,避免对复杂的不确定性进行建模,利用其与环境交互、根据反馈学习改进策略的机制,自适应不确定性的变化。为确保算法适用性,进行了模型泛化方法的设计,针对算法稳定性问题进行了感知-学习比例调整和改进经验回放的机制设计。算例结果表明,所提方法能在自适应系统不确定性的基础上,实现任意场景下的电力系统动态经济调度。

When the large-scale intermittent generation sources including wind and photovoltaic generation connects to the power system, in order to cope with their uncertainty, the economic dispatch model for power system needs to be built based on the uncertainty modeling. The accuracy of modeling will directly affect the accuracy of the dispatch results. However, when considering the complex uncertainty of both load and intermittent generation sources such as wind and photovoltaic generation, it is particularly difficult to accurately model the overall uncertainty of the system. In view of this problem, the deep deterministic policy gradient(DDPG) algorithm in the deep reinforcement learning is introduced. The work of uncertainty modeling is avoided. Instead,the uncertainty is adapted by the DDPG algorithm relied on the mechanism of interacting with the environment and improving the strategy based on feedbacks. In order to guarantee the applicability of the algorithm, the generalization method for the DDPG model is proposed. Aiming at the stability problem of the algorithm, two mechanisms are designed, including perception-learning ratio adjustment and experience replay improvement. The result of example shows that the dynamic economic dispatch problem of power systems in any scenario can be solved by the proposed method based on adapting the system uncertainty.