基于CPSS平行系统懒惰强化学习算法的实时发电调控

Lazy Reinforcement Learning Through Parallel Systems and Social System for Real-time Economic Generation Dispatch and Control

为解决电力系统中存在的多种时间尺度下经济调度和发电控制的协同问题,即长时间尺度下优化,短时间尺度下优化和实时控制的问题,本文提出了一种统一时间尺度的实时经济发电调度和控制框架,并为该框架提出了懒惰强化学习方法(Lazy reinforcement learning, LRL).该方法将懒惰控制器引入以人工社会–计算实验–平行执行和社会系统为基础的强化学习中,使得机组组合,经济调度,自动发电控制和发电命令调配的问题有机结合在一起,取代过去传统的发电控制框架.为了减少仿真所需的真实时间,平行系统包含多个虚拟系统和一个真实系统.仿真实验比较了懒惰学习算法,松弛人工网络以及4 608种组合常规发电控制算法在IEEE新英格兰10机39节点仿真系统的控制效果.实验表明,懒惰强化学习方法的控制效果最优.仿真结果验证了懒惰强化学习方法在基于ACP和社会系统的REG框架下具有有效性和可行性.

To mitigate the coordinated problem of multi-time scale economic dispatch and generation control in power systems, i.e., long term time scale optimization, short term time scale optimization, and real-time control, a real-time economic generation dispatch and control(REG) framework with a unified time scale is designed in this paper. With a lazy operator employed into reinforcement learning based on artificial societies - computational experiments - parallel execution(ACP) and social system, a lazy reinforcement learning(LRL) is proposed for the REG framework, which, being an alternative to conventional generation control framework, combines unit commitment, economic dispatch, automatic generation control, and generation command dispatch. To reduce the time of simulations, parallel systems which contain multiple virtual systems and a real system, are built. Compared with 4 608 combined conventional generation control algorithms and relaxed artificial neural network in the simulation of IEEE 10-generator 39-bus New-England power system,the LRL obtains the best optimal control performance. Simulation results have verified the effectiveness and feasibility of the proposed LRL based on ACP and social system for the REG framework.