基于探索感知思维深度强化学习的自动发电控制

Automatic Generation Control Based on Deep Reinforcement Learning With Exploration Awareness

减少碳排放的重要途径之一是大规模引入新能源。然而,随着大规模风、光、电动汽车等新能源和分布式能源接入复杂互联电网,给电网带来严重的随机扰动问题。该文从自动发电控制角度,探索了一种动作探索感知思维的深度强化学习算法,即DDQN-AD。通过将神经网络的预测机制作为强化学习的动作选择机制,同时引入具有动作探索感知思维的AD策略,将区域控制误差与碳排放作为综合奖励函数,来获取强随机环境下的最优控制策略,进而解决分布式能源大规模接入电网所带来的随机扰动问题。对改进的IEEE标准两区域LFC模型,以及多区域多能生态系统模型进行仿真,结果显示DDQN-AD与已有的多种智能算法相比,具有更优的动态性能和在线学习能力,能够获得区域最优控制,减少碳排放。

One of the important ways to reduce carbon emissions is to introduce new energy on a large scale. However,with the access of large-scale wind, light, electric vehicles and other new energy sources and distributed energy to the complex grid, it will bring serious random disturbance problems to the grid. From the perspective of automatic generation control, a deep reinforcement learning algorithm with action exploration awareness was explored, namely DDQN-AD. By using the prediction mechanism of deep neural network as the action selection mechanism of reinforcement learning, and introducing the AD strategy with exploration-awareness, the regional control error and carbon emission were used as comprehensive reward functions to obtain the optimal strategies in a variety of random environments. In order to solve the problem of random disturbance caused by the large-scale access of new energy and distributed energy to the power grid, the simulation of the improved IEEE standard two-area load frequency control model and the multi-region pluripotent ecosystem model was carried out. From the result, it shows that DDQN-AD has greater online learning ability and better dynamic performance compared with many existing intelligent algorithms, and it can obtain regional optimal control and reduce carbon emissions.