基于反事实多智能体强化学习和有功无功协同控制的配电网电压优化

Active and reactive power coordinated optimal voltage control of a distribution network based on counterfactual multi-agent reinforcement learning

大量分布式电源的接入使配电网的结构与控制方式发生改变。针对分布式电源间歇性和波动性引起的电压越限问题,通过调节系统中无功潮流与有功潮流的分布来维持配电网的电压稳定。提出了一种基于反事实多智能体策略梯度(counterfactual multi-agent policy gradients, COMA)算法的配电网电压协同优化方法,通过反事实基线解决了多智能体强化学习中的“信度分配”问题,实现有功出力设备和无功补偿设备的联合优化调度。智能体通过局部观测值选定动作,减轻系统的通信压力,且不依赖精确的潮流模型,以实现配电网的实时优化控制。通过改进的IEEE33节点系统和141节点系统验证了所提算法的可行性与有效性。并与经典算法的控制效果进行比较,进一步证明所提算法在配电网电压优化控制方面的性能优势。

The integration of a significant number of distributed generators has altered the structure and control methods in distribution networks.To address the voltage stability issues caused by the intermittency and fluctuation of distributed generators,this paper proposes the stabilization of the distribution network voltage by adjusting the distribution of reactive and active power flows within the system.A distribution network voltage coordinated optimization method is proposed based on the counterfactual multi-agent policy gradients(COMA)algorithm.The proposed method can use a counterfactual baseline to resolve the“credit assignment”challenge in multi-agent reinforcement learning,enabling the joint optimization scheduling of active power generation and reactive power compensation devices.Agents select actions based on local observations,thereby reducing the system’s communication load and eliminating the dependency on precise flow models,to achieve real-time optimization control of distribution networks.The feasibility and effectiveness of the proposed algorithm are demonstrated by using the improved IEEE33-node system and 141-node system.Compared with the classic control algorithms,the proposed method has further performance advantages in the voltage optimization and control problems for distribution networks.