基于内嵌增广拉格朗日函数Q-learning方法的虚拟电厂市场博弈策略

Strategies of Market Game Behavior of Virtual Power Plants Based on Q-learning With Augmented Lagrange Function

聚合了多品类能源资源的虚拟电厂为多主体参与下的竞争电力市场增添了较多不确定性。为了刻画虚拟电厂在市场博弈行为中的特点,探寻更高效的电力市场交易机制,提出了一种含虚拟电厂的电力市场古诺博弈模型,并证明了其纳什均衡解的唯一存在性。由于传统优化算法难以准确模拟现实中发电商之间的竞争过程,而适用于不完全信息博弈的Q-learning算法又难以处理复杂的约束条件,提出了基于多智体框架下Q-learning算法内嵌拉格朗日函数的联合求解算法,对含虚拟电厂的电力市场博弈模型进行求解。算例分别采用遗传算法与所提算法对模型进行仿真测试,结果表明,在电能需求总量恒定且统一出清模式的电力市场中,虚拟电厂较传统电厂有着优越的资源整合与盈利能力,且所提算法能够获得更为稳定的收敛结果。

In the electricity market with multiple participants,virtual power plants(VPP)integrated with different energy resources have been faced with considerable uncertainties in the energy trading mechanism.In order to show the characteristics of VPPs in the electricity market and explore more efficient transaction mechanisms,this paper proposes a Cournot game model of electricity market with VPPs and proves the unique existence of its Nash Equilibrium(NE).Furthermore,since it is difficult for the traditional optimization algorithms to accurately simulate the competition process between the power generation companies,and it is also difficult for the Q-learning algorithm which is suitable for incomplete information games to deal with the complex constraints,this paper proposes a Q-learning algorithm with augmented Lagrange function based on a multi-agent framework to solve the proposed game model.Finally,this paper uses genetic algorithm and the algorithm proposed in this paper to simulate the presented model.The results show that in the electricity market with given demand and unified clearing mechanism,a VPP has superior capacity of resource integration and profitability to the traditional power plants,and the algorithm proposed in this paper can obtain more stable convergence results.