面向储能辅助火电机组一次调频的深度强化学习控制策略

Deep Reinforcement Learning Control Strategy for Primary Frequency Regulation of Energy Storage Assisted Thermal Power Units

为充分挖掘火储联合系统中储能单元的一次调频潜力,同时减少火电机组调频频度,提出了一种基于双延迟深度确定性策略梯度算法的储能辅助火电机组一次调频控制策略。搭建了典型含有火储联合调频系统的区域电网一次调频模型;以提高电网频率控制效果、维持储能荷电状态(SOC)平稳、减少火电机组调频动作次数等多项目标建立优化问题,综合考虑电网运行状态及调频动作的约束,将储能辅助火电机组一次调频问题建模为一个马尔可夫决策过程;使用双延迟深度确定性策略梯度算法进行优化问题求解。典型调频任务场景的仿真结果表明,与传统火储联合调频控制算法相比,所提出的储能辅助火电机组智能调频控制策略可使电网频率偏差均值降低约10.4%、火电机组的调频动作次数减少约56.2%,验证了该策略能在有效释放火储联合系统的调频潜力,同时大幅减少火电机组一次调频频次。

In order to fully tap the potential of primary frequency modulation in the energy storage unit of the thermal-storage combined system and reduce the frequency regulation of thermal power units,a twin delayed deep deterministic policy gradient algorithm based energy storage assisted primary frequency regulation control strategy for thermal power unit is proposed.A typical regional power grid primary frequency regulation model containing thermal-storage combined frequency regulation system is established.Optimization problems with multiple objectives such as improving the frequency control effect of the power grid,maintaining a stable state of charge(SOC),and reducing the frequency regulation actions of thermal power unit are set.Taking into account the constraints of power grid operation status and frequency regulation actions,the primary frequency regulation problem of thermal power unit assisted by energy storage is modeled as a Markov decision process.Twin delayed deep deterministic policy gradient algorithm is used for optimization problem solving.The simulation results of typical frequency regulation task scenarios show that compared with traditional frequency regulation system control algorithms,the proposed energy storage assisted intelligent frequency regulation control strategy for thermal power units can reduce the average frequency difference of the power grid by about 10.4%,and reduce the frequency regulation actions of thermal power unit by about 56.2%.This verifies that the strategy can effectively release the frequency regulation potential of the thermal-storage combined system while significantly reducing frequency regulation of thermal power units.