基于深度强化学习的电网前瞻调度智能决策架构及关键技术初探

Architecture and Key Technologies of Intelligent Decision-making of Power Grid Look-ahead Dispatch Based on Deep Reinforcement Learning

随着新型电力系统建设的快速推进,电网运行方式不确定性增加,调度对象类型/数量指数级增长,当前基于物理模型的电网调度计划存在优化决策计算速度慢、耗时长以及应对多重不确定场景适应性不够等问题,特别是日内阶段仍经常依赖调度员人工调控。为此,该文结合电网前瞻调度的时序滚动优化、多元对象决策、调度多目标构建等实际特点,提出基于深度强化学习的电网前瞻调度智能决策功能架构,分析离线训练模块、在线决策模块和效果评估模块3部分的具体实现;并在适用于电网前瞻调度的深度强化学习算法、学习样本效率提升、调度多目标奖励函数设计、拓扑改变情形下的迁移学习和前瞻调度效果评估等关键技术方面进行了初步探索,基于IEEE30节点算例验证了所提算法和技术的有效性。最后,探讨了电网前瞻调度智能决策需进一步研究的问题。

With the increasing uncertainty of power grid operation and the exponential growth of types/numbers of dispatch objects,the current power grid dispatch plan based on physical model has some problems,such as slow calculation speed,long time-consuming,and insufficient adaptability to multiple uncertain scenarios.Therefore,combined with time sequence rolling optimization,multiple variables decision,and multi-objective construction of power grid look-ahead dispatch,this paper proposed an intelligent decision function architecture of power grid look-ahead dispatch based on deep reinforcement learning,and analyzed the concrete implementation of three parts:offline training module,online decision-making module,and effect evaluation module.Some key supporting technologies were particularly discussed,such as deep reinforcement learning algorithm,learning sample efficiency improvement,multi-objective construction and reward function design,migration learning under topology change,dispatch effect evaluation,etc.The effectiveness of the proposed algorithm and technology has been verified by an IEEE30 node example.Finally,the problems that needed to be further studied were discussed.