基于多主体博弈和强化学习的多微网系统协同优化研究

Collaborative Optimization of Multi-microgrid System Based on Multi-agent Game and Reinforcement Learning

面向分属于不同投资和运营主体的多个冷热电联供型微网构成的多微网系统,该文提出了一种基于多主体博弈的多微网系统协同优化方法,通过博弈论建立多微网系统的协同优化模型,实现各个微网的利益均衡。针对该模型Nash均衡求解困难的问题,提出了一种改进的Nash-Q学习算法。该算法采用深度神经网络来拟合Nash-Q学习算法中的价值函数,不仅有效解决了Nash-Q学习算法直接应用于复杂环境时遇到的维数灾难问题,并且保证了算法的后效性,能快速完成合理有效的在线优化。实验结果表明,相较于传统数学规划方法和贪婪算法,改进的Nash-Q学习算法能够学习到Nash均衡策略,实现各微网间电能互补,降低各微网的运行成本,验证了所提模型和算法的有效性。

Aiming at the multi-microgrid system composed of multiple microgrids with combined cooling,heating and power,which belong to different investment and operating entities,this paper proposes a collaborative optimization for multi-microgrid systems based on the multi-agent game.By using the game theory,a collaborative optimization model for multi-microgrid system is established to achieve the balance of the interests of all the microgrids.For the difficulty of solving the Nash equilibrium of the model,an improved Nash-Q learning algorithm is proposed.In this algorithm the deep neural network is used to fit the value function in the Nash-Q learning algorithm.Then the dimension disaster problem is solved when Nash-Q learning algorithm is directly applied to complex environment,and also the aftereffect of the algorithm is ensured,which quickly completes the reasonable and effective online optimization.The experimental results show that compared with the traditional mathematical programming and the greedy algorithm,the improved Nash-Q learning algorithm learns well the Nash equilibrium strategy,realizes the power complementarity among microgrids,reduces the operation cost of microgrids,which verifies the effectiveness of the proposed model and the algorithm.