基于深度强化学习的含智能软开关配电网电压控制方法

Voltage Control Method of Distribution Network with Soft Open Point Based on Deep Reinforcement Learning

大量分布式新能源接入给配电网运行带来了电压越限和网损增加等一系列问题。提出了一种基于多智能体强化学习的无模型电压控制策略,通过协调光伏逆变器、分布式储能和智能软开关以降低网损、消除电压越限。针对传统电压控制策略对配电网精确的模型参数依赖性强的问题,提出了基于高斯过程回归的潮流替代模型,通过多智能体与潮流替代模型交互实现无模型的离线训练和在线应用。同时提出了一种基于随机加权三重Q学习的多智能体深度强化学习算法,能够进一步降低柔性演员-评论家算法的高低估误差,提升算法探索能力和收敛结果。最后在IEEE33节点系统上的仿真结果,验证了所提方法在解决配电网分布式电压优化控制问题上的有效性。

The widespread integration of distributed renewable energy sources has brought a series of problems to the operation of distribution networks,including voltage violations and increase in network losses.This paper proposes a model-free voltage control strategy based on multi-agent reinforcement learning.By coordinating photovoltaic inverters,distributed energy storages,and soft open points,the strategy aims to reduce network losses and eliminate voltage viola-tions.To tackle the problem that traditional voltage control strategies have strong dependence on accurate distribution network model parameters,a power flow surrogate model based on Gaussian process regression is proposed.The model enables offline training and online application through interactions between multi-agents and the power flow surrogate model.Additionally,a multi-agent deep reinforcement learning algorithm based on random weighted triple Q-learning is proposed to further reduce the overestimation and underestimation errors of the soft actor-critic algorithm.The proposed method improves the algorithm exploration capability and results quality.Finally,simulation results on the IEEE 33-node system verify the effectiveness of the proposed method in solving the distributed voltage control problem of distribution networks.