基于多步回溯Q(λ)学习的互联电网随机最优CPS控制

Stochastic Optimal CPS Control for Interconnected Power Grids Using Multi-Step Backtrack Q(λ) Learning

针对非马尔可夫环境下火电占优的互联电网AGC控制策略,引入随机最优控制中Q(λ)学习的"后向估计"原理,可有效解决火电机组大时滞环节带来的延时回报问题。本文以CPS1/CPS2滚动平均值为状态输入,将CPS评价指标与松弛目标根据线性加权原则转化为MDP奖励函数,从长期的角度提出一种在线反馈学习结构的随机最优CPS控制。统计性仿真试验表明,所提CPS控制具有较强的适应性和动态性能,在保证CPS合格率基础上能有效减少调度端的平均发令次数和反调次数。同时,该策略提供了一种可通过修正松弛因子在线调整AGC系统的"松弛度",可降低发电成本及机组磨损,从而实现CPS松弛控制。

This paper presents the application of multi-step backtrack Q（？） learning based on stochastic optimal control to effectively solve the long time-delay link for thermal plants under Non-Markovian environment.The moving averages of CPS1/CPS2 are used as the state input,and the CPS control and relaxed control objectives are formulated as MDP reward function by means of linear weighted aggregative approach.The optimal CPS control methodology open avenues to on-line feedback learning rule to maximize the long-run discounted reward.Statistic experiments show that the Q（？） controllers can enhance obviously the robustness and dynamic performance of AGC systems,and reduce the number of pulses and pulse reversals while the CPS compliances are ensured.The proposed strategy also provides a convenient means for controlling the degree of compliance and relaxation by online tune relaxation factors to implement the desirable CPS relaxed control.